Skip to content

Commit

Permalink
Make the change backwards-compatible
Browse files Browse the repository at this point in the history
This supports both the old version (single value) and new version (hash-based).

I've added back the tests from the old version to check that the code stills works
when we provide a single value.

I've also adapted the README a bit to explain both versions.
  • Loading branch information
elenatanasoiu committed Dec 13, 2024
1 parent cb642e8 commit b2311bc
Show file tree
Hide file tree
Showing 3 changed files with 53 additions and 6 deletions.
24 changes: 21 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,15 +121,15 @@ class MyWidget
e.try { UserService.slug_from_login login } # returns String instance or ArgumentError

compare_error_message_and_class = -> (control, candidate) do
control.class == candidate.class &&
control.class == candidate.class &&
control.message == candidate.message
end

compare_argument_errors = -> (control, candidate) do
control.class == ArgumentError &&
candidate.class == ArgumentError &&
control.message.start_with?("Input has invalid characters") &&
candidate.message.start_with?("Invalid characters in input")
candidate.message.start_with?("Invalid characters in input")
end

e.compare_errors do |control, candidate|
Expand Down Expand Up @@ -550,7 +550,25 @@ end

#### Providing fake timing data

If you're writing tests that depend on specific timing values, you can provide canned durations using the `fabricate_durations_for_testing_purposes` method, and Scientist will report these in `Scientist::Observation#duration` and `Scientist::Observation#cpu_time` instead of the actual execution times.
If you're writing tests that depend on specific timing values, you can provide canned durations using the `fabricate_durations_for_testing_purposes` method. This can be done using either the old version (single value) or the new version (hash-based) to include both duration and cpu_time.

##### Old version (Single Value)

In the old version, you can provide a single value for the duration:

```ruby
science "absolutely-nothing-suspicious-happening-here" do |e|
e.use { ... } # "control"
e.try { ... } # "candidate"
e.fabricate_durations_for_testing_purposes( "control" => 1.0, "candidate" => 0.5 )
end
```

`fabricate_durations_for_testing_purposes` takes a Hash of duration values, keyed by behavior names. (By default, Scientist uses `"control"` and `"candidate"`, but if you override these as shown in [Trying more than one thing](#trying-more-than-one-thing) or [No control, just candidates](#no-control-just-candidates), use matching names here.) If a name is not provided, the actual execution time will be reported instead.

##### New version (Hash-based)

Scientist will report these in `Scientist::Observation#duration` and `Scientist::Observation#cpu_time` instead of the actual execution times.

```ruby
science "absolutely-nothing-suspicious-happening-here" do |e|
Expand Down
5 changes: 4 additions & 1 deletion lib/scientist/observation.rb
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,12 @@ def initialize(name, experiment, fabricated_duration: nil, &block)
@exception = e
end

if fabricated_duration
if fabricated_duration.is_a?(Hash)
@duration = fabricated_duration["duration"]
@cpu_time = fabricated_duration["cpu_time"]
elsif fabricated_duration
@duration = fabricated_duration
@cpu_time = 0.0 # setting a default value
else
end_wall_time, end_cpu_time = capture_times
@duration = end_wall_time - start_wall_time
Expand Down
30 changes: 28 additions & 2 deletions test/scientist/experiment_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -671,7 +671,20 @@ def @ex.enabled?
end

describe "testing hooks for extending code" do
it "allows a user to provide fabricated durations for testing purposes" do
it "allows a user to provide fabricated durations for testing purposes (old version)" do
@ex.use { true }
@ex.try { true }
@ex.fabricate_durations_for_testing_purposes( "control" => 0.5, "candidate" => 1.0 )

@ex.run

cont = @ex.published_result.control
cand = @ex.published_result.candidates.first
assert_in_delta 0.5, cont.duration, 0.01
assert_in_delta 1.0, cand.duration, 0.01
end

it "allows a user to provide fabricated durations for testing purposes (new version)" do
@ex.use { true }
@ex.try { true }
@ex.fabricate_durations_for_testing_purposes({
Expand All @@ -692,7 +705,20 @@ def @ex.enabled?
assert_equal 0.9, cand.cpu_time
end

it "returns actual durations if fabricated ones are omitted for some blocks" do
it "returns actual durations if fabricated ones are omitted for some blocks (old version)" do
@ex.use { true }
@ex.try { sleep 0.1; true }
@ex.fabricate_durations_for_testing_purposes( "control" => 0.5 )

@ex.run

cont = @ex.published_result.control
cand = @ex.published_result.candidates.first
assert_in_delta 0.5, cont.duration, 0.01
assert_in_delta 0.1, cand.duration, 0.01
end

it "returns actual durations if fabricated ones are omitted for some blocks (new version)" do
@ex.use { true }
@ex.try do
start_time = Time.now
Expand Down

0 comments on commit b2311bc

Please sign in to comment.