Document passing a custom lambda for testset filtering

[willow-ahrens]

There are several competing interfaces to filtering tests, it would be nice to be able to just use a lambda to filter tests based on name and tags, with whatever strategy the user feels is most helpful. i.e. what if we don't want to filter based on all the tags, but rather with any of the tags? There's many variations of this, and defining a filter lambda seems feasible.

[nickrobinson251]

This actually already works! It hasn't yet been documented as a stable feature yet (which i think turned out to be a decent choice since otherwise something like #170 would have been a breaking change), but i think we probably should commit to it as an API sooner rather than later.

Anyway, what currently works is passing to runtests, as the first argument, a function that takes a single argument that has two properties name and tags, and which returns a bool indicating if the testitem with that name and tags should be run e.g.

runtests() do ti
  contains(ti.name, "foo") && in(ti.tags, :bar)
end

Does that meet your needs / fit what you had in mind?

[willow-ahrens]

yes, that's the same as TestItems.jl and it's what I had in mind.

[willow-ahrens]

Thanks for the prompt reply, and for a great package!

[nickrobinson251]

Great! I'll keep this issue open to remind us to document this (i'm just coming back from a block of leave and need to remind myself if there was something else we were wanting to do before committing to the existing API)