Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IWF-232: Add disabling sticky cache option #473

Merged
merged 12 commits into from
Nov 4, 2024

Conversation

lwolczynski
Copy link
Contributor

@lwolczynski lwolczynski commented Oct 30, 2024

Description

Checklist

  • Code compiles correctly
  • Tests for the changes have been added
  • All tests passing
  • This PR change is backwards-compatible
  • This PR CONTAINS a (planned) breaking change (it is not backwards compatible)

Related Issue

Closes #466

@lwolczynski lwolczynski marked this pull request as ready for review October 31, 2024 20:15
@lwolczynski lwolczynski marked this pull request as draft October 31, 2024 20:15
@lwolczynski lwolczynski force-pushed the jira/lwolczynski/IWF-232 branch 3 times, most recently from 6ad8c6b to 1f655b8 Compare October 31, 2024 21:19
@lwolczynski lwolczynski marked this pull request as ready for review October 31, 2024 22:12
@@ -19,6 +19,10 @@ type InterpreterWorker struct {
tasklist string
}

type StartOptions struct {
DisableStickyCache bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know we have a specific use case for this start option to disable sticky cache, but I'm wondering if we should add a warning comment about the performance impacts of doing this for other reasons. (My understanding is that this would have a not insignificant impact on performance in a non-test/real-world use case. Is that right?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right. I'll add a comment pointing it out

@@ -37,12 +41,17 @@ func (iw *InterpreterWorker) Close() {
iw.worker.Stop()
}

func (iw *InterpreterWorker) Start() {
func (iw *InterpreterWorker) Start(startOptions StartOptions) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we keep the original Start function here and have a second StartWithOptions so that the launchTemporalService in iwf.go does not need to change? The Start function could just set the default StartOptions and call StartWithOptions - I think that's cleaner overall but in particular keeps launchTemporalService clean

Unless we think we have a use case where launchTemporalService would actually be calling start with different options based on runtime configuration values??

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense to me. Definitely, we can keep Start as it was and add StartWithOptions. I like the idea, thanks for this suggestion!

iw.StartWithOptions(options)
}

func (iw *InterpreterWorker) StartWithOptions(startOptions StartOptions) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To even better address @mixydubs concerns, I feel like it's even better to have this StartWithStickyCacheDisabledForTest() so that no one would use this in production.

StartWithOptions is also a little confusing with the workerOptions that we are passing through config config.Config in NewInterpreterWorker. If we really need to pass options, they can be passed from there for consistency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -13,12 +13,19 @@ jobs:
# Give the default GITHUB_TOKEN write permission to commit and push the
# added or changed files to the repository.
contents: write
strategy:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we wanted have one action to run with stickyCache, and another run without, right? (two actions run in parallel so that it won't slow down)
So I expect we need new yml files like:

  • ci-temporal-integ-test-disable-sticky.yml
  • ci-cadence-integ-test-disable-sticky.yml

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. I thought we would only do tests with no sticky cache, but it makes sense to keep both scenarios 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Makefile Outdated Show resolved Hide resolved
Makefile Show resolved Hide resolved
$Q go test -v ./integ -search=false -temporal=false -dependencyWaitSeconds=180 -disableStickyCache=true

ci-cadence-integ-test-disable-sticky:
$Q go test -v ./integ -run "(?i)^Test[${startsWith}]" -search=false -temporal=false -dependencyWaitSeconds=180 -disableStickyCache=true

ci-temporal-integ-test:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you test does -disableStickyCache=true work? I thought it has to be -disableStickyCache because it's a bool flag

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does work, but I can simplify it to disableStickyCache

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@lwolczynski lwolczynski merged commit 7fe9eb9 into main Nov 4, 2024
10 checks passed
@lwolczynski lwolczynski deleted the jira/lwolczynski/IWF-232 branch November 4, 2024 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Run integ tests with sticky cache disabled to ensure determinism
3 participants