Composing models with latent and to_distribution

[sunxd3]

#696 introduced several new abstractions for model composition in DynamicPPL.jl:

ReturnedModelWrapper

Wraps a model to treat it as a distribution over its return values instead of its parameters.
Example:

@model function demo()
    x ~ Normal()
    y = x + 1
    return y
end

# ReturnedModelWrapper(demo()) represents distribution over y

Distributional

• Sampleable <: Distributional:
  • Can be sampled from
  • May not support logpdf computation

Extensions (To be implemented)

• latent: (vis-a-vis returned)
  • Would view model in terms of its parameters rather than returns
  • Example: For above model, would give distribution over x
• Another subtype of Distributional that supports sampling and computing logp
• A high level user facing interface to_distribution that would create proper probability distributions with models
  • Could potentially work with both returned and latent

Integration with NamedTupleVariate

Supporting NamedTupleVariate (#801) would give us a natural interface for to_distribution:

• Allowing model to be viewed as distributions over model parameters and their values.
• Notice ProductNamedTupleDistribution is not sufficient, because it assumes independence
• Limitation is that not all DynamicPPL models can use this interface, as some require the more complex VarName to value mapping rather than simple NamedTuples

[sunxd3]

related issues: #755, TuringLang/Turing.jl#2428, TuringLang/Turing.jl#2425

[sunxd3]

related TuringLang/Turing.jl#2485