Metadata management for the National Microbiome Data Collaborative
The purpose of this repository is to manage metadata for the National Microbiome Data Collaborative (NMDC). The NMDC is a multi-organizational effort to enable integrated microbiome data across diverse areas in medicine, agriculture, bioenergy, and the environment. This integrated platform facilitates comprehensive discovery of and access to multidisciplinary microbiome data in order to unlock new possibilities with microbiome data science.
Tasks managed by the repository are:
- Generating the schema
- Deploying the documentation
- Integrating metadata from multiple environmental data repositories
The NMDC schema is used during the translation process to specify how metadata elements are related.
Documentation for the NMDC schema can be browsed here:
Entities in the schema are annotated with characteristics. When possible, we use standard terminologies and ontologies to define these characteristics. These standards include:
We are actively involved in updating the MIxS standards (mixs-ng) and creating an RDF version of MIxS (mixs-rdf).
At present, we ingest metadata from the Joint Genome Institute (JGI) and the Environmental Molecular Sciences Lab (EMSL).
The NMDC schema and translation process will be modified as more metadata sources become available.
We use Jupyter notebooks to integrate the metadata sources. This allows us to iterate quickly in a transparent and interactive manner as new metadata sources become available.
Development of more comprehensive ETL pipeline will progress as the metadata sources and schema become more concrete.