There are two partitions on NAS05 dedicated to astro ML datasets astrodata01 and astrodata02.
Different datasets used for AstroML.
- goes/goes_ts_2010-2020.parquet.zip - GOES timeseries downloaded with
sdo-cliin Parquet format. Refer to this notebook for an example. This dataset can be used as GOES cache for the SDO ML v2 data loader. - sdo_benchmark - Downloaded version of SDOBenchmark.
- sdo_raw - ??
- sdomlv1_full - Full version of the SDO ML v1 dataset. Code example SDO ML v1 data loader
- sdomlv1_mini - Substantially temporally downsized version of the SDO Dataset of A Machine-learning Data Set Prepared from the NASA Solar Dynamics Observatory Mission.
- sdomlv2_full - Full version of the SDO ML v2 dataset. SDO ML v2 data loader
- sdomlv2_small - Small version of the SDO ML v2 dataset. SDO ML v2 data loader
- sharp/derived_sharp_dataset - A derived SHARP dataset used for investigating handcrafted features and automatic features extracted with a VAE. SHARP Flare Forecasting
- swan_sf Downloaded version of multivariate time series (MVTS) data extracted from HMI Active Region Patch (SHARP) series.
/mnt/nas05/astrodata01/astroml_data
├── README.md
├── curated_image_parameter_dataset
├── goes
│ └── goes_ts_2010-2020.parquet.zip
├── sdo_benchmark
├── sdo_raw
│ ├── aiadata
│ └── hmidata
├── sdomlv1_full
│ ├── stanford_machine_learning_dataset_for_sdo
│ └── stanford_machine_learning_dataset_for_sdo_extracted
├── sdomlv1_mini
│ ├── 2011.tar
│ ├── 2012.tar
│ ├── 2013.tar
│ ├── 2014.tar
│ ├── 2015.tar
│ ├── 2016.tar
│ ├── 2017.tar
│ ├── 2018.tar
│ └── EVEPlusMeta.tar
├── sdomlv2_full
│ ├── sdomlv2.zarr
│ ├── sdomlv2_eve.zarr
│ └── sdomlv2_hmi.zarr
├── sdomlv2_small
│ ├── sdomlv2_hmi_small.zarr
│ └── sdomlv2_small.zarr
├── sharp
│ ├── derived_sharp_dataset
└── swan_sf
IRIS datasets
- iris - Dataset with selected representative observations for different observation classes.
/mnt/nas05/astrodata02
├── iris
│ ├── 2013
│ ├── 2014
│ ├── 2015
│ ├── 2016
│ ├── 2017
│ ├── 2018
│ ├── find_empty_obs
│ ├── find_stray_zips
│ ├── precompute_valid_steps
│ ├── set_permissions
│ └── uncompress_all
└── only-for-iris-data