List and prioritise next schema updates

[ESapenaVentura]

Description

As a DCP Metadata Team member, I want to list the next possible schema updates so we can discuss and prioritise the next iteration of PRs.

Library preparation/technologies support

• Add permeabilisation time and permeabilisation time unit for Visium Permeabilisation time and permeabilisation time unit in imaging_preparation_protocol #1471
• Support 10x V3 feature barcodes Add field for 10x v3 feature barcodes #1125
• Improve the way we group sets of fastq files Add a field for flow cell id  #983
• Remove the requirement for barcode information if a tag-based method is selected. E.g. if an alternative technology like Microwell-Seq is used, and we are unable to get the barcode information (which read, barcode length, barcode offset), we still want to be able to submit the dataset with the expression matrix data
• Add new fields to better support nuclei sequencing Add Lysis Method to Library Preparation Schema #1419 Add Nuclei Resuspension Reagent to Library Preparation Schema #1420 Add Nuclei Resuspension Cycles to Library Preparation Schema #1421 Add Nuclei Purification Method to Library Preparation Schema #1422 Add Nuclei Concentration to Library Preparation Schema #1423
• Allow for an array of methods in the library_preparation_protocol.method field Change protocol_core.method to an array #1330
• Allow for multiple barcode length/offset to support Microwell-Seq Update allowed format for barcode length and offset #1403
• Read3/Read4 added to sequence_file.read_index Change sequence_file.read_index enum to include Read3/Read4. #1401: Support library preparation technologies
• Schema update: update library preparation protocol barcode fields #1407
• Record Genome version used to generate analysis file #1508

Improve biomaterial metadata

• Expand disease ontology to include phenotype: Expand disease ontology to include phenotype #1460
• Gender identity: New field: Gender Identity #1409
• Donor organism genus to be required Change donor_organism.genus_species to be required #999
• Add tissue bank ID and accession to biomaterial_core Add Tissue Bank ID field to biomaterial schema #1231
• Add preservation_storage_module to cell_suspension Add preservation_storage module to cell_suspension #1213
• Add timecourse module to specimen Add timecourse module to specimen #1506
• Improve organoid metadata Review schema needs to support organoid projects #1334
• Ontologize cell_viability_method Ontologize cell_viability_method in cell_suspension.json #1201
• Add optional field crown_rump_length to donor organism Add crown_rump_length property to donor_organism schema #1198
• Move to human module and ontologize cause_of_death field Change cause_of_death guidelines or description in death.json and ontologize #1188
• Change disease to diseases and make it an array in cell_line align biomaterial type disease properties by updating cell_line.json #1162
• Remove growth_conditions from cell_suspension Remove growth_conditions from cell_suspension schema. #884
• Adding drug use field Adding drug use to the medical history schema #849
• Disallow unit without value or vice versa for organism age (I think this is extensible to other value/unit fields in the schema) Disallow unit without value or vice versa for organism age #1395
• Ontologise preservation_storage module method fields Ontologise preservation_storage methods #1463

Add enum values

• Add methanol fixation to preservation_storage.method Add methanol fixation as preservation method to preservation_storage module #1203

Improve project metadata

• Make funders.organization an array Make Funding Organization an array field #1132
• Make project_role an array Make project_role ontology field an array #1059

Patch schema fixes

• Update specimen_from_organism regex pattern Update specimen_from_organism collection_time pattern regex #1393
• Update donor_organism keyword example to examples for age field donor_organism.organism_age example doesn't match pattern #1320
• Known disease to Disease status Known disease to Disease status in friendly name #1124
• Change project.funders user-friendly name to "Funders" Change user-friendly name to "Funders" in project.json #1103
• Update comment in provenance.document_id Update comment in provenance.document_id field #1044
• Gestational age typo Correct typo on gestational age description #1194
• Wrong apostrophe in matrix module Wrong "Apostrophe" used in matrix.json enum values #1410

Supporting imaging data

• Consistency between probe and channel fields Make probe and channel schemas consistent with IDs #1328
• Support imaging data Support for imaging data #974
  • Ensure metadata is in place for Spatial Transcriptomics #758
• Update imaging targets.json Update imaging_targets.json to include a field for SAM file in place of probe seq #587

Review schema

• Review all schema regexes Review and fix all schema regexes #1318
  • Donor organism age has a broken regex donor_organism.organism_age has a broken regex  #1301
  • Allow a range of BMI values for this field: donor_organism.human_specific.body_mass_index E.g. 23-28
  • Add regex to avoid whitespaces in ID fields Add regex to all schema to prevent white space in id fields #1248
  • Update regex patterns to be consistent Update RegEx patterns to be consistent #1193
  • Update date-time regexes to capture partial dates according to RFC3339 Restrict time-date pattern catching for partial dates #1184
• Review and improve descriptions and examples across schema Review and improve description and example text across json schemas #1187
  • Remove obsolete example from schema Remove obsolete term from strain_ontology example text #1186
• Standardize use of booleans Standardize use of boolean fields. #976
• Review all ontology schemas to make sure examples/restrictions are still up to date TICKET NEEDED
• Add accessions module to make possible general accession input New module: accessions.json #1408

Technical/system updates

• Remove schema_major_version and schema_minor_version: Review and fix all schema regexes #1318
• Replace reference_bundle with reference_files in analysis process Replace reference_bundle with reference_files in analysis_process #1290
• Split memory into memory and memory_unit split memory into memory + memory_unit #1169
• lowercase HDBR_accession field Change HDBR_accesssion to all lowercase to adhere to Style Guide. #1047
• Remove checksum field from file_core Remove checksum field from file_core metadata. #883
• File format should be required Disallow empty string for file_core.format #1459

Other updates

• Update enrichment_protocol.markers to be an array Update enrichment_protocol schema #1094

Needs clarification/discussion

spatial technologies updates:

• Updates to imaging protocol schema #1405
• New schema update to specimen_from_organism schema #1406
• Addition of a new "treatment_protocol" schema: Addition of a new protocol schema: treatment_protocol #1428

Other

• Include 'Information entity' branch as potential ontology terms for file_content_description Include 'Information entity' branch as potential ontology terms for file_content_description #1450
• Modelling genome ref files as analysis files Consider modelling genome reference files in analysis_protocol #1288
• Analysis protocol method should point to protocol method ontology protocol schemas should point to protocol ontology module #1164
• Assemble organ list (New system field?) Assemble list of allowed Organ ontologies #1163
• Allow for multiple machine names in sequencing_protocol Change property sequencing_protocol.local_machine_name #1126
• Remove file_description from supplementary file Remove file_description from supplementary_file.json #1001
• Add specimen weight and size Add fields to capture specimen size and weight #984
• Ontologize file_core.format field Ontologize file_core.format field. #812
• Capture cell line name Add a field to capture cell line name #756
• Id fields scope Type and scope of most id fields is not defined in the schema #733
• FACS module? Add a FACS module  #619

Acceptance Criteria

• A list is reflected on this ticket
• The final list has been reviewed and grouped by functional purpose
• For each functional group, a ticket is created with a timeline (or, at the very least, an order) assigned to it.

[rays22]

The list of possible updates above looks sensible to me. I would suggest that we pick those updates first that block data flow as top priority. The current order of the list might already reflect that, but I am not sure.