Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add an optional cohort block to science experiments #170

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add an optional cohort block to science experiments #170

wants to merge 1 commit into from

Commits on Jan 3, 2022

  1. Add an optional cohort block to science experiments 

    Many experiments operate on data with a very long tail, and the most
frequent part of the distribution can wash out notable results in
sub-groups.  For example, experiment results derived from the data of
very large customers often look quite different than the much more
common results from the small data.  Even the use of percentile metrics
can't overcome these effects since often the relevant percentiles are
very high (above 99-percentile).

This adds an optional block to Science::Experiment which should return a
"cohort" when called.  The cohort is passed the result of the experiment
so it can determine the cohort from the context data, whether the result
is a mismatch or any of the observation data.

The determined cohort value is available as `Scientist::Result#cohort`
and is intended to be used by the user-defined publication mechanism.
    Carl Brasic committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    1174575 View commit details
    Browse the repository at this point in the history