This package has some simple, minimal preprocessing of helio-data to make it machine learning ready.
Some demos showcasing how we can download some data, do some preprocessing regimes and integrate with ML datasets.
Downloading Data. In this notebook, we have a demonstration of how we can download SDO data. See notebook for details.
Preprocessing Data. In this notebook, we have a demonstration of how we can preprocess the data using a sequence of transformations. We showcase a series of tested transformations which have had success for ML applications (e.g. ITI) See notebook for details.
Preprocessing Configurations.
In this notebook, we demonstrate how we can create configurations for these transformations.
In particular, we demonstrate how Hydra-Zen can be used to help facilitate readable transformations.
See notebook for details.
Numpy DataLoader. We demonstrate how we can use data to create a simple dataloader using numpy files downloaded. See notebook for details.
RasterVision.
We demonstrate how we can use a more complex and advanced dataloader regime from rastervision.
In particular, we showcase how we can create independently sampled time series images in addition to time series images.
See notebook for details.
We can install it directly through pip
pip install git+https://github.com/spaceml-org/helio_tools
pip install gsutilWe also use poetry for the development environment.
git clone https://github.com/spaceml-org/helio_tools
cd helio_tools
conda create -n helio_tools python=3.11 poetry
conda activate helio_tools
poetry installWe provide a test dataset for the notebooks containing data from SDO/AIA, EUI/FSI, EUI/HRI and PROBA2/SWAP which can be downloaded with gsutil
gsutil cp -r gs://iti-dataset/ [local_path]Software
- InstrumentToInstrument - Instrument-to-Instrument Translation.
Glossary