Progressive tests

[nmilan]

Hello there, we have tried to set up progressive tests and it seems it's not possible? So what we need to do it's to start two experiments at the same time with the same alternative.
For example:
One experiment is: landing_page_cvr(alternatives: a,b),
Second experiment is: landing_page_cta(alternatives: a,b)
We should start both in the same time and it should be same alternative(for example a for both)

What we tried so far:

1. We use Vanity.playground.experiments[:experiment_name].chooses(alternative) and test start with chosen alternative. But when we have to finish that test like this Vanity.track!(:experiment_name) that didn't work.
2. We try to start the second test and after that, we try to update VanityParticipants.update_columns(seen: alternative_index)
3. Also, we try to add a variable in an experiment for alternatives.

We think the best solution will be if we can somehow pass alternative as a variable(ex: Vanity.ab_test(:experiment_name, :alternative)) on the starting experiment. Do you have any solutions for this?

[phillbaker]

Hi @nmilan and the others who +1'ed this - thanks for opening an issue. I'm not totally sure I follow the example, but do any of the following seem related?

• http://vanity.labnotes.org/experimental.html#multi-series
• Multi metric support for ab tests (using redis) #318
• Multivariate testing #147