Skip to content

Commit

Permalink
Added formatting (#260)
Browse files Browse the repository at this point in the history
* addd formatting settings

* formatting

* aded formatting to workflows
  • Loading branch information
torfjelde authored Jun 6, 2023
1 parent 88d9c66 commit dd8a24b
Show file tree
Hide file tree
Showing 45 changed files with 975 additions and 771 deletions.
2 changes: 2 additions & 0 deletions .JuliaFormatter.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
style="blue"
format_markdown=true
38 changes: 38 additions & 0 deletions .github/workflows/Format.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Format

on:
push:
branches:
- master
pull_request:
branches:
- master
merge_group:
types: [checks_requested]

concurrency:
# Skip intermediate builds: always.
# Cancel intermediate builds: only if it is a pull request build.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ startsWith(github.ref, 'refs/pull/') }}

jobs:
format:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: julia-actions/setup-julia@latest
with:
version: 1
- name: Format code
run: |
using Pkg
Pkg.add(; name="JuliaFormatter", uuid="98e50ef6-434e-11e9-1051-2b60c6c9e899")
using JuliaFormatter
format("."; verbose=true)
shell: julia --color=yes {0}
- uses: reviewdog/action-suggester@v1
if: github.event_name == 'pull_request'
with:
tool_name: JuliaFormatter
fail_on_error: true
17 changes: 11 additions & 6 deletions docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,18 @@ using Bijectors
# Doctest setup
DocMeta.setdocmeta!(Bijectors, :DocTestSetup, :(using Bijectors); recursive=true)

makedocs(
sitename = "Bijectors",
format = Documenter.HTML(),
modules = [Bijectors],
pages = ["Home" => "index.md", "Transforms" => "transforms.md", "Distributions.jl integration" => "distributions.md", "Examples" => "examples.md"],
makedocs(;
sitename="Bijectors",
format=Documenter.HTML(),
modules=[Bijectors],
pages=[
"Home" => "index.md",
"Transforms" => "transforms.md",
"Distributions.jl integration" => "distributions.md",
"Examples" => "examples.md",
],
strict=false,
checkdocs=:exports,
)

deploydocs(repo = "github.com/TuringLang/Bijectors.jl.git", push_preview=true)
deploydocs(; repo="github.com/TuringLang/Bijectors.jl.git", push_preview=true)
7 changes: 5 additions & 2 deletions docs/src/distributions.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
## Basic usage

Other than the `logpdf_with_trans` methods, the package also provides a more composable interface through the `Bijector` types. Consider for example the one from above with `Beta(2, 2)`.

```julia
julia> using Random; Random.seed!(42);
julia> using Random;
Random.seed!(42);

julia> using Bijectors; using Bijectors: Logit
julia> using Bijectors;
using Bijectors: Logit;

julia> dist = Beta(2, 2)
Beta{Float64}=2.0, β=2.0)
Expand Down
22 changes: 11 additions & 11 deletions docs/src/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@ using Bijectors
```

## Univariate ADVI example

But the real utility of `TransformedDistribution` becomes more apparent when using `transformed(dist, b)` for any bijector `b`. To get the transformed distribution corresponding to the `Beta(2, 2)`, we called `transformed(dist)` before. This is simply an alias for `transformed(dist, bijector(dist))`. Remember `bijector(dist)` returns the constrained-to-constrained bijector for that particular `Distribution`. But we can of course construct a `TransformedDistribution` using different bijectors with the same `dist`. This is particularly useful in something called _Automatic Differentiation Variational Inference (ADVI)_.[2] An important part of ADVI is to approximate a constrained distribution, e.g. `Beta`, as follows:
1. Sample `x` from a `Normal` with parameters `μ` and `σ`, i.e. `x ~ Normal(μ, σ)`.
2. Transform `x` to `y` s.t. `y ∈ support(Beta)`, with the transform being a differentiable bijection with a differentiable inverse (a "bijector")

1. Sample `x` from a `Normal` with parameters `μ` and `σ`, i.e. `x ~ Normal(μ, σ)`.
2. Transform `x` to `y` s.t. `y ∈ support(Beta)`, with the transform being a differentiable bijection with a differentiable inverse (a "bijector")

This then defines a probability density with same _support_ as `Beta`! Of course, it's unlikely that it will be the same density, but it's an _approximation_. Creating such a distribution becomes trivial with `Bijector` and `TransformedDistribution`:

Expand All @@ -16,7 +18,7 @@ dist = Beta(2, 2)
b = bijector(dist) # (0, 1) → ℝ
b⁻¹ = inverse(b) # ℝ → (0, 1)
td = transformed(Normal(), b⁻¹) # x ∼ 𝓝(0, 1) then b(x) ∈ (0, 1)
x = rand(rng, td) # ∈ (0, 1)
x = rand(rng, td) # ∈ (0, 1)
```

It's worth noting that `support(Beta)` is the _closed_ interval `[0, 1]`, while the constrained-to-unconstrained bijection, `Logit` in this case, is only well-defined as a map `(0, 1) → ℝ` for the _open_ interval `(0, 1)`. This is of course not an implementation detail. `` is itself open, thus no continuous bijection exists from a _closed_ interval to ``. But since the boundaries of a closed interval has what's known as measure zero, this doesn't end up affecting the resulting density with support on the entire real line. In practice, this means that
Expand All @@ -29,25 +31,22 @@ inverse(td.transform)(rand(rng, td))
will never result in `0` or `1` though any sample arbitrarily close to either `0` or `1` is possible. _Disclaimer: numerical accuracy is limited, so you might still see `0` and `1` if you're lucky._

## Multivariate ADVI example

We can also do _multivariate_ ADVI using the `Stacked` bijector. `Stacked` gives us a way to combine univariate and/or multivariate bijectors into a singe multivariate bijector. Say you have a vector `x` of length 2 and you want to transform the first entry using `Exp` and the second entry using `Log`. `Stacked` gives you an easy and efficient way of representing such a bijector.

```@repl advi
using Bijectors: SimplexBijector
# Original distributions
dists = (
Beta(),
InverseGamma(),
Dirichlet(2, 3)
);
dists = (Beta(), InverseGamma(), Dirichlet(2, 3));
# Construct the corresponding ranges
ranges = [];
idx = 1;
for i = 1:length(dists)
for i in 1:length(dists)
d = dists[i]
push!(ranges, idx:idx + length(d) - 1)
push!(ranges, idx:(idx + length(d) - 1))
global idx
idx += length(d)
Expand All @@ -74,6 +73,7 @@ sum(y[3:4]) ≈ 1.0
```

## Normalizing flows

A very interesting application is that of _normalizing flows_.[1] Usually this is done by sampling from a multivariate normal distribution, and then transforming this to a target distribution using invertible neural networks. Currently there are two such transforms available in Bijectors.jl: `PlanarLayer` and `RadialLayer`. Let's create a flow with a single `PlanarLayer`:

```@setup normalizing-flows
Expand Down Expand Up @@ -144,7 +144,7 @@ f = NLLObjective(reconstruct, MvNormal(2, 1), xs);
# Train using gradient descent.
ε = 1e-3;
for i = 1:100
for i in 1:100
∇s = Zygote.gradient(f, θs...)
θs = map(θs, ∇s) do θ, ∇
θ - ε .* ∇
Expand Down
39 changes: 20 additions & 19 deletions docs/src/index.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,32 @@
# Bijectors.jl

This package implements a set of functions for transforming constrained random variables (e.g. simplexes, intervals) to Euclidean space. The 3 main functions implemented in this package are the `link`, `invlink` and `logpdf_with_trans` for a number of distributions. The distributions supported are:
1. `RealDistribution`: `Union{Cauchy, Gumbel, Laplace, Logistic, NoncentralT, Normal, NormalCanon, TDist}`,
2. `PositiveDistribution`: `Union{BetaPrime, Chi, Chisq, Erlang, Exponential, FDist, Frechet, Gamma, InverseGamma, InverseGaussian, Kolmogorov, LogNormal, NoncentralChisq, NoncentralF, Rayleigh, Weibull}`,
3. `UnitDistribution`: `Union{Beta, KSOneSided, NoncentralBeta}`,
4. `SimplexDistribution`: `Union{Dirichlet}`,
5. `PDMatDistribution`: `Union{InverseWishart, Wishart}`, and
6. `TransformDistribution`: `Union{T, Truncated{T}} where T<:ContinuousUnivariateDistribution`.

1. `RealDistribution`: `Union{Cauchy, Gumbel, Laplace, Logistic, NoncentralT, Normal, NormalCanon, TDist}`,
2. `PositiveDistribution`: `Union{BetaPrime, Chi, Chisq, Erlang, Exponential, FDist, Frechet, Gamma, InverseGamma, InverseGaussian, Kolmogorov, LogNormal, NoncentralChisq, NoncentralF, Rayleigh, Weibull}`,
3. `UnitDistribution`: `Union{Beta, KSOneSided, NoncentralBeta}`,
4. `SimplexDistribution`: `Union{Dirichlet}`,
5. `PDMatDistribution`: `Union{InverseWishart, Wishart}`, and
6. `TransformDistribution`: `Union{T, Truncated{T}} where T<:ContinuousUnivariateDistribution`.

All exported names from the [Distributions.jl](https://github.com/TuringLang/Bijectors.jl) package are reexported from `Bijectors`.

Bijectors.jl also provides a nice interface for working with these maps: composition, inversion, etc.
The following table lists mathematical operations for a bijector and the corresponding code in Bijectors.jl.

| Operation | Method | Automatic |
|:------------------------------------:|:-----------------:|:-----------:|
| `b ↦ b⁻¹` | `inverse(b)` ||
| `(b₁, b₂) ↦ (b₁ ∘ b₂)` | `b₁ ∘ b₂` ||
| `(b₁, b₂) ↦ [b₁, b₂]` | `stack(b₁, b₂)` ||
| `x ↦ b(x)` | `b(x)` | × |
| `y ↦ b⁻¹(y)` | `inverse(b)(y)` | × |
| `x ↦ log|det J(b, x)|` | `logabsdetjac(b, x)` | AD |
| `x ↦ b(x), log|det J(b, x)|` | `with_logabsdet_jacobian(b, x)` ||
| `p ↦ q := b_* p` | `q = transformed(p, b)` ||
| `y ∼ q` | `y = rand(q)` ||
| `p ↦ b` such that `support(b_* p) = ℝᵈ` | `bijector(p)` ||
| `(x ∼ p, b(x), log|det J(b, x)|, log q(y))` | `forward(q)` ||
| Operation | Method | Automatic |
|:-------------------------------------------:|:-------------------------------:|:---------:|
| `b ↦ b⁻¹` | `inverse(b)` ||
| `(b₁, b₂) ↦ (b₁ ∘ b₂)` | `b₁ ∘ b₂` ||
| `(b₁, b₂) ↦ [b₁, b₂]` | `stack(b₁, b₂)` ||
| `x ↦ b(x)` | `b(x)` | × |
| `y ↦ b⁻¹(y)` | `inverse(b)(y)` | × |
| `x ↦ log|det J(b, x)|` | `logabsdetjac(b, x)` | AD |
| `x ↦ b(x), log|det J(b, x)|` | `with_logabsdet_jacobian(b, x)` ||
| `p ↦ q := b_* p` | `q = transformed(p, b)` ||
| `y ∼ q` | `y = rand(q)` ||
| `p ↦ b` such that `support(b_* p) = ℝᵈ` | `bijector(p)` ||
| `(x ∼ p, b(x), log|det J(b, x)|, log q(y))` | `forward(q)` ||

In this table, `b` denotes a `Bijector`, `J(b, x)` denotes the Jacobian of `b` evaluated at `x`, `b_*` denotes the [push-forward](https://www.wikiwand.com/en/Pushforward_measure) of `p` by `b`, and `x ∼ p` denotes `x` sampled from the distribution with density `p`.

Expand Down
6 changes: 3 additions & 3 deletions docs/src/transforms.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
## Usage

A very simple example of a "bijector"/diffeomorphism, i.e. a differentiable transformation with a differentiable inverse, is the `exp` function:
- The inverse of `exp` is `log`.
- The derivative of `exp` at an input `x` is simply `exp(x)`, hence `logabsdetjac` is simply `x`.

- The inverse of `exp` is `log`.
- The derivative of `exp` at an input `x` is simply `exp(x)`, hence `logabsdetjac` is simply `x`.

```@repl usage
using Bijectors
Expand Down Expand Up @@ -100,4 +101,3 @@ Bijectors.OrderedBijector
Bijectors.NamedTransform
Bijectors.NamedCoupling
```

Loading

0 comments on commit dd8a24b

Please sign in to comment.