time integrated flux diagnostics

[rgknox]

Description:

This adds a site-level diagnostic that compares the total amount of mass in live vegetation (plants plus seeds) to the total integrated fluxes in and out of that pool over the life of the simulation. The same is true for the litter pool. History output for the bias on this comparison is added as well, to see if the bias grows over time.

Collaborators:

@JessicaNeedham

Expectation of Answer Changes:

This does have a bug fix, so some diagnostics related to litter fluxes may be corrected.

Checklist

If this is your first time contributing, please read the CONTRIBUTING document.

All checklist items must be checked to enable merging this pull request:

Contributor

• The in-code documentation has been updated with descriptive comments
• The documentation has been assessed to determine if updates are necessary

Integrator

• FATES PASS/FAIL regression tests were run
• Evaluation of test results for answer changes was performed and results provided

Documentation

• Technical Note update:
• User's Guide update:

Test Results:

CTSM (or) E3SM (specify which) test hash-tag:

CTSM (or) E3SM (specify which) baseline hash-tag:

FATES baseline hash-tag:

Test Output:

[rgknox]

The bias between integrated flux and live-vegetation and litter states are looking pretty good now, see plots below, the scale on liveveg and litter is e-8 and e-14 respectively.

I had to move the calculation of the flux integration to later in the call sequence so that termination mortality that occurs after vegetation dynamics is encapsulated. I'll also run a test with CNP to see if this was the cause of #1218 .

[mpaiao]

Thanks for addressing the mass conservation assessment, @rgknox ! I went through the code and it looks great, I just had a couple of minor clarification questions and suggestions.

[JessicaNeedham]

Thanks Ryan this looks great! My simulations with this branch merged also show very tiny divergence between integrated fluxes and stocks.

[JessicaNeedham]

I think we discussed this but not sure where we landed on having this turned on or not. Could it be an option so a user could opt in to having their run crash if states and integrated fluxes start to diverge?

[rgknox]

I added an issue here: #1264

[rgknox]

@mpaiao and @JessicaNeedham I addressed your comments. @mpaiao I changed the name of that variable from leaf_* to surf_fine_* to be more general and include seed. @JessicaNeedham I added back in those history variables and also created an issue about graceful failure.

[rgknox]

All tests b4b except for the specific output variables that were changed in this PR with intent.
/glade/derecho/scratch/rgknox/tests_1017-183514de