Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow multi data file granules #103

Open
lisakaser opened this issue Dec 4, 2024 · 7 comments
Open

Allow multi data file granules #103

lisakaser opened this issue Dec 4, 2024 · 7 comments
Assignees
Milestone

Comments

@lisakaser
Copy link

Question: What needs to be in the UMM-G size? (Amy)

Example data set?

We could check how this is currently handled in existing UMM-G

Type in CNM message might have to change

@lisakaser lisakaser added this to the Dec-Jan-Feb milestone Dec 4, 2024
@lisakaser lisakaser added the question Further information is requested label Dec 4, 2024
@lisakaser
Copy link
Author

Specific questions for @afitzgerrell blocking this ticket: What is the file size that needs to go into the UMM-G for multi science file granules?

@afitzgerrell
Copy link
Contributor

afitzgerrell commented Jan 9, 2025

The file size reported in UMM-G (SizeInBytes) would be the sum of all the data files comprising the multi-file granule (i.e., everything packaged in a granule that is not the external, file-level umm-g metadata file, browse image, or ancillary files like QA and/or PH files). I don't recall ever receiving QA and PH files with a data producer-delivered data set). There's an example of this here in the UMM-G documentation, but I've attached a visual here in case helpful.

To state this another way as: the umm-g SizeInBytes should be the sum of all the files in a product where the files are defined as "type": "data" in the CNM.Image

@lisakaser lisakaser added high priority and removed question Further information is requested labels Feb 10, 2025
@lisakaser
Copy link
Author

SIPSmetgen has a common file name and what file types are allowed as addition to a multifile granule.

@lisakaser
Copy link
Author

In this story only deal with additional science data; A separate ticket should take care of ancillary files.

@lisakaser lisakaser changed the title Allow multi file granules Allow multi data file granules Feb 10, 2025
@juliacollins
Copy link
Contributor

juliacollins commented Mar 5, 2025

The requirements of this issue should be satisfied by issue-61. Currently metgenc can handle multiple data files and multiple browse file, but it does this by assuming anything not a browse file is a data file. The logic will need some refinement when we tackle ancillary file support. Once issue-61 is ready for Ops testing I'll move this issue into testing as well.

@juliacollins
Copy link
Contributor

juliacollins commented Mar 6, 2025

Update! The definition of "common file name" was clarified during our Slack conversation on March 5, and it is not what's implemented by issue-61! The remaining work for this story is to refine the algorithm/regex used to collect up all files related to a single granule.

@lisakaser
Copy link
Author

Work started. 2SP left from original 3SP issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants