You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Commit message(s) and PR title follows the format [fix|feat|ci|chore|doc]: TICKET-ID: Short description of change made ex. fix: DEV-XXXX: Removed inconsistent code usage causing intermittent errors
Tests for the changes have been added/updated (for bug fixes/features)
Docs have been added/updated (for bug fixes/features)
Best efforts were made to ensure docs/code are concise and coherent (checked for spelling/grammatical errors, commented out code, debug logs etc.)
Self-reviewed and ran all changes on a local instance (for bug fixes/features)
Change has impacts in these area(s)
(check all that apply)
Product design
Backend (Database)
Backend (API)
Frontend
Describe the reason for change
There was an issue where the inner_id was duplicated when importing from the API. We identified the problem because a client was importing with two requests almost at the same time, causing a race condition when calculating the inner_id.
What does this fix?
Potential duplicate inner ids due to a race condition when importing from the API.
What is the new behavior?
Acquire lock on the project before calculating inner id. I tried to acquire the lock on the Tasks queryset, but when there weren't any tasks yet it wouldn't work, because it locks on the returned rows (empty the first time).
Does this change affect performance?
If there are multiple async imports for a same project at once like this case, they will eventually be committed sequentially. This is needed to ensure inner_id consistency.
Does this PR introduce a breaking change?
(check only one)
Yes, and covered entirely by feature flag(s)
Yes, and covered partially by feature flag(s)
No
Not sure (briefly explain the situation below)
What level of testing was included in the change?
(check all that apply)
e2e
integration
unit
For testing I created a small script to replicate this behavior. Running two separate worker processes is also needed, so they can consume each import job concurrently.
importjsonfromlabel_studio_sdk.clientimportLabelStudio# Define the URL where Label Studio is accessible and the API key for your user accountLABEL_STUDIO_URL=''# API key is available at the Account & Settings > Access Tokens page in Label Studio UIAPI_KEY=''# Connect to the Label Studio API and check the connectionls=LabelStudio(base_url=LABEL_STUDIO_URL, api_key=API_KEY)
project_id=7withopen('data/5400.json', 'r') asf:
data_5400=json.load(f)
withopen('data/5500.json', 'r') asf:
data_5500=json.load(f)
ls.projects.import_tasks(id=project_id, request=data_5400)
ls.projects.import_tasks(id=project_id, request=data_5500)
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR fulfills these requirements
[fix|feat|ci|chore|doc]: TICKET-ID: Short description of change madeex.fix: DEV-XXXX: Removed inconsistent code usage causing intermittent errorsChange has impacts in these area(s)
(check all that apply)
Describe the reason for change
There was an issue where the inner_id was duplicated when importing from the API. We identified the problem because a client was importing with two requests almost at the same time, causing a race condition when calculating the inner_id.
What does this fix?
Potential duplicate inner ids due to a race condition when importing from the API.
What is the new behavior?
Acquire lock on the project before calculating inner id. I tried to acquire the lock on the Tasks queryset, but when there weren't any tasks yet it wouldn't work, because it locks on the returned rows (empty the first time).
Does this change affect performance?
If there are multiple async imports for a same project at once like this case, they will eventually be committed sequentially. This is needed to ensure inner_id consistency.
Does this PR introduce a breaking change?
(check only one)
What level of testing was included in the change?
(check all that apply)
For testing I created a small script to replicate this behavior. Running two separate worker processes is also needed, so they can consume each import job concurrently.
Which logical domain(s) does this change affect?
Imports