Skip to content

Commit 3f20314

Browse files
committedMar 28, 2024·
add some summary info
1 parent fa8ef00 commit 3f20314

File tree

1 file changed

+27
-3
lines changed

1 file changed

+27
-3
lines changed
 

‎sections/reproducibility-containers.qmd

+27-3
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,35 @@ title: "Reproducibility and Containers"
44

55
## Learning Objectives
66

7-
- Think about dependency management, reproducibility, and software
8-
- Become familiar with containers as a tool to improve computational reproducibility
7+
- Think about dependency management, reproducibility and software management
98
- Discuss how the techniques from this class can improve reproducibility
109

10+
1111
- Slides: [Accelerating synthesis science through reproducible science](../images/2022-09-repro-sci.pdf)
1212

13+
## Summary
14+
15+
In this course we reviewed many tools and techniques for how to make research more reproducible and scalable. These tools focus around three main areas: the environment, the data, and the code.
16+
17+
### Reproducible Environments
18+
19+
- Virtual environments with `venv` and `virtualenvwrapper`
20+
- Python dependencies with `requirements.txt`
21+
- Containers with Docker
22+
23+
### Accessible Data
24+
25+
- Publishing with the Arctic Data Center
26+
- Formats for large datasets: NetCDF and Zarr
27+
28+
### Scalable Python
29+
30+
- Parallel with `concurrent.futures`, `parsl`, `dask`
31+
- N-dimensional data access with `xarray`
32+
- Geospatial analysis with `geopandas` and `rasterio`
33+
- Software design and python packages
34+
35+
1336
## Software collapse
1437

1538
::: {layout-ncol="2"}
@@ -44,11 +67,12 @@ This approach combines the evolving approach to using a `Dockerfile` to precisel
4467

4568
:::
4669

70+
71+
4772
## Discussion
4873

4974
To wrap up this week, let's kick off a discussion with a couple of key questions.
5075

5176
- As we learn new tools for scalable and reproducible computing, what can we as software creators do to improve robustness and ease maintenance of our packages?
5277
- Given the fragility of software ecosystems, are you worried about investing a lot of time in learning and building code for proprietary cloud systems? Can we compel vendors to keep their systems open?
53-
- How much can technological solutions such as containers truly address the issues around depencies, maintainability, and sustainability of scientific software?
5478
- What relative fraction of the research budget should funders invest in software infrastructure, data infrastructure, and research outcomes?

0 commit comments

Comments
 (0)
Please sign in to comment.