plot_dist_comparison for Turing models

[mateuszbaran]

Hi!

Could you add an example of usage of plot_dist_comparison with Turing models? It appears that I need to pass sampled priors in inference data but it's not clear how to do that.

[sethaxen]

Once you have an InferenceData with both posterior and prior groups, you can just pass this directly to plot_dist_comparison, so I think your questions is how to get both groups from Turing? There's a worked example of this in the Quickstart: https://arviz-devs.github.io/ArviZ.jl/stable/quickstart/#Plotting-with-MCMCChains.jl's-Chains-objects-produced-by-Turing.jl (it also shows how to get prior and posterior predictions as well as log likelihoods).

[sethaxen]

e.g., using the idata created in the Quickstart:

plot_dist_comparison(idata; var_names=[:μ, :τ])

plot_dist_comparison(idata; coords=Dict(:school => ["Choate", "Deerfield"]), kind="observed")

[mateuszbaran]

Thanks, that helped. One suggestion I have here (OK, maybe two) is to give a warning or something when a user wants to use plot_dist_comparison on inference data that has no prior, and maybe mention plot_dist_comparison somewhere around there: https://arviz-devs.github.io/ArviZ.jl/stable/quickstart/#Additional-information-in-Turing.jl for easier discoverability.

[sethaxen]

give a warning or something when a user wants to use plot_dist_comparison on inference data that has no prior

Good suggestion! That should probably be handled by the underlying Python function. I'll open an issue there.

maybe mention plot_dist_comparison somewhere around there: https://arviz-devs.github.io/ArviZ.jl/stable/quickstart/#Additional-information-in-Turing.jl for easier discoverability.

I don't know about this. The goal of the Quickstart is to show how to get outputs of Julia PPLs into the InferenceData format for use with ArviZ, with just a few toy examples of what you can then do. From there all plotting regardless of PPLs is the same, and the number of supported plots is large. Rather than add another plot to the Quickstart, perhaps it would be better to add an example of plot_dist_comparison to the gallery, which currently is missing an example.

[mateuszbaran]

Good suggestion! That should probably be handled by the underlying Python function. I'll open an issue there.

Thanks!

I don't know about this. The goal of the Quickstart is to show how to get outputs of Julia PPLs into the InferenceData format for use with ArviZ, with just a few toy examples of what you can then do. From there all plotting regardless of PPLs is the same, and the number of supported plots is large. Rather than add another plot to the Quickstart, perhaps it would be better to add an example of plot_dist_comparison to the gallery, which currently is missing an example.

Sure, that's an even better idea.

[sethaxen]

Actually, the prior doesn't seem to be required to use plot_dist_comparison e.g.

julia> using ArviZ

julia> idata = load_arviz_data("centered_eight")
InferenceData with groups:
	> posterior
	> posterior_predictive
	> sample_stats
	> prior
	> observed_data

julia> idata2 = InferenceData(posterior=idata.posterior)
InferenceData with groups:
	> posterior

julia> plot_dist_comparison(idata2; var_names=["mu"])

It's not a very useful plot, but no errors.

julia> ArviZ.arviz_version()
v"0.11.2"

julia> using Pkg; Pkg.status()
     Project ArviZ v0.5.0
      Status `~/projects/ArviZ.jl/Project.toml`
  [8f4d0f93] Conda v1.5.1
  [a93c6f00] DataFrames v0.22.6
  [d9ec5142] NamedTupleTools v0.13.7
  [438e738f] PyCall v1.92.2
  [d330b81b] PyPlot v2.9.0
  [ae029012] Requires v1.1.3
  [2913bbd2] StatsBase v0.33.4
  [d6f4376e] Markdown
  [3fa0cd96] REPL

Can you provide a failing example?

[mateuszbaran]

There was no actual error and I've got a very similar plot but at the first glance it wasn't obvious to me that the lack of samples from prior in InferenceData is the problem. It's the kind of problem that you only experience the first time you try to do that and then it's clear why it didn't work.