Skip to content

Commit

Permalink
Add in new CI Dockerfile, remove old Dockerfile, entrypoint, etc.
Browse files Browse the repository at this point in the history
Update README a bit.
  • Loading branch information
danscales committed Jul 11, 2024
1 parent d9a4841 commit c980c66
Show file tree
Hide file tree
Showing 4 changed files with 12 additions and 147 deletions.
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,9 +233,14 @@ The following options are supported:

* [`build.sbt`](build.sbt): Scala Build Tool build configuration file
* [`.sbtopts`](.sbtopts): Command line options for SBT, including JVM parameters
* [`project`](project): Additional configuration for SBT project, plugins, utility, versions
* [`project`](project): Additional configuration for SBT project, plugins, utility,
versions, including metals and bloop
* [`src/main/scala`](src/main/scala): Application and utility code
* [`src/test/scala`](src/test/scala): Unit test files
* [`sbt`][sbt): Script used in github CI to download sbt and all the scala
dependencies during CI tests (`.github/workflow/ci.yaml`)
* [`ci/Dockerfile`](ci/Dockerfile): Dockerfile used to build the docker image for
CI tests

## Spark Job Commands

Expand Down
22 changes: 6 additions & 16 deletions Dockerfile → ci/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,32 +1,22 @@
FROM gcr.io/datamechanics/spark:platform-3.1.1-hadoop-3.2.0-java-11-scala-2.12-python-3.8-dm12
FROM openjdk:8

ARG VERSION

ENV SCALA_VERSION=2.12
ENV SCALA_VERSION=2.12.12
ENV PYTHON_VERSION=3.8
ENV GDAL_VERSION=3.1.2
ENV GDAL_VERSION=3.8.3

USER root

# Install GDAL using Conda
RUN wget https://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh
RUN sh Miniconda-latest-Linux-x86_64.sh -b -p /usr/local/miniconda
RUN bash Miniconda-latest-Linux-x86_64.sh -b -p /usr/local/miniconda

ENV PATH=/usr/local/miniconda/bin:$PATH

RUN conda config --add channels conda-forge
RUN conda config --add channels conda-forge && conda config --set ssl_verify false
RUN /usr/local/miniconda/bin/conda install python=${PYTHON_VERSION} -y
RUN pip install tqdm six
RUN /usr/local/miniconda/bin/conda install -c anaconda hdf5 -y
RUN /usr/local/miniconda/bin/conda install -c conda-forge libnetcdf gdal=${GDAL_VERSION} -y

ENV LD_LIBRARY_PATH=/usr/local/miniconda/lib/:/usr/local/lib:/usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native:/docker/usr/lib/hadoop/lib/native:/docker/usr/lib/hadoop-lzo/lib/native:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

# Copy Tree cover loss FAT JAR into docker containter
COPY target/scala-${SCALA_VERSION}/treecoverloss-assembly-${VERSION}.jar treecoverloss-assembly.jar
# Overwrite entrypoint
COPY entrypoint.sh /opt/entrypoint.sh
RUN chmod 755 /opt/entrypoint.sh

# Change back to inital user
USER 185
ENV PROJ_LIB=/usr/local/miniconda/share/proj
15 changes: 0 additions & 15 deletions docker-compose.sbt.yml

This file was deleted.

115 changes: 0 additions & 115 deletions entrypoint.sh

This file was deleted.

0 comments on commit c980c66

Please sign in to comment.