Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ingestion of large DK or SK records #181

Open
spacesnottabs opened this issue Aug 28, 2023 · 5 comments
Open

Add ingestion of large DK or SK records #181

spacesnottabs opened this issue Aug 28, 2023 · 5 comments

Comments

@spacesnottabs
Copy link
Collaborator

Currently DK and SK source records larger than 1Mb are not being processed and instead stored in a separate S3 folder with a link provided in the output stream.

These need to also be imported.
@spacesnottabs spacesnottabs converted this from a draft issue Aug 28, 2023
@StephenAbbott StephenAbbott changed the title Register v2: Add ingestion of large DK or SK records Add ingestion of large DK or SK records Sep 1, 2023
@StephenAbbott StephenAbbott mentioned this issue Sep 1, 2023
Open
@tiredpixel
Copy link
Contributor

Blocks #164 QA.

@spacesnottabs spacesnottabs moved this from Todo (highest priority at top) to In Progress in Open Ownership Register and BODS pipelines Sep 17, 2023
@spacesnottabs spacesnottabs moved this from In Progress to In Testing in Open Ownership Register and BODS pipelines Sep 17, 2023
@tiredpixel
Copy link
Contributor

@spacesnottabs, @StephenAbbott, what is the testing status of this ticket, please?

@StephenAbbott
Copy link
Member

@spacesnottabs Are you happy that the large DK and SK records have been reinserted now?

@StephenAbbott
Copy link
Member

Outstanding work to do here...

Large DK Records.pdf

@tiredpixel
Copy link
Contributor

I'm not sure whether this should be in testing, @StephenAbbott, if there's still work to do. But maybe this relates to other tickets (or other tickets will be created)?

@StephenAbbott StephenAbbott moved this from In Testing to In Progress in Open Ownership Register and BODS pipelines Nov 3, 2023
@tiredpixel tiredpixel removed their assignment Jun 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
Open Ownership Register and BODS pipelines
Status: Todo
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tiredpixel @StephenAbbott @spacesnottabs