Skip to content

Commit 6d2f889

Browse files
graebmgraebm
authored
Docker images: fix raspbian-bullseye, install python3-venv (#286)
**Issue:** awslabs/aws-crt-python#573 needs python3's `venv` installed on all docker images, due to [PEP 668](https://peps.python.org/pep-0668/). While trying to update our `raspbian-bullseye` image, I discovered it's been broken for months. We'd been [hacking around this](#256) by not even trying to build it anymore, copy/pasting an old working version instead. **Description of changes:** - Install `python3-venv` in Docker images that didn't already have it - Fix `raspbian-bullseye` image. - The old image was getting 404 from some repository while running `apt-get update` - Use different base image: https://github.com/dtcooper/raspberrypi-os-docker. This was the only easily available `bullseye` image I could find. The implementation looks simple, and it's working. - Remove extra stuff from this docker image, like building additional versions of python. I don't know what these were used for (@xiazhvera do you know?). None of our other docker images go through this trouble to build extra python versions. This is already the slowest docker image, due to being armv7, so removing extra work and complication seems extra good.
1 parent 7ee8e7f commit 6d2f889

File tree

4 files changed

+7
-150
lines changed

4 files changed

+7
-150
lines changed

‎.github/docker-images/raspbian-bullseye/Dockerfile

+4-86
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,8 @@
1-
FROM raspbian/stretch as raspbian-bullseye
2-
3-
###############################################################################
4-
# Upgrade to bullseye from stretch
5-
###############################################################################
6-
7-
RUN apt-get update \
8-
&& apt-get upgrade -y \
9-
&& apt-get full-upgrade -y \
10-
&& apt-get autoremove --purge -y \
11-
&& apt-get clean -y \
12-
# Switch to bullseye repository.
13-
&& sed -i 's/stretch/bullseye/g' /etc/apt/sources.list \
14-
# Update all packages.
15-
&& apt-get update \
16-
&& apt-get upgrade -y \
17-
&& apt-get full-upgrade -y \
18-
&& apt-get autoremove --purge -y \
19-
&& apt-get clean -y
20-
1+
FROM --platform=linux/arm/v7 dtcooper/raspberrypi-os:bullseye
212

223
###############################################################################
234
# Install prereqs
245
###############################################################################
25-
# Setup nodejs policy for npm
26-
RUN apt-cache policy nodejs
276
RUN apt-get update -qq \
287
&& apt-get -y install \
298
git \
@@ -36,85 +15,24 @@ RUN apt-get update -qq \
3615
python3.9 \
3716
python3-dev \
3817
python3-pip \
18+
python3-venv \
3919
build-essential \
4020
# For PPAs
4121
software-properties-common \
4222
apt-transport-https \
4323
ca-certificates \
4424
&& apt-get clean
4525

46-
47-
# Override python with python3
48-
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10
49-
50-
# Install libs for pythons
51-
RUN sudo apt-get install build-essential checkinstall -y
52-
RUN sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev -y
53-
5426
###############################################################################
5527
# Python/AWS CLI
5628
###############################################################################
57-
RUN python -m pip install --upgrade setuptools virtualenv \
58-
&& python -m pip install --upgrade awscli \
29+
RUN python3 -m pip install --upgrade setuptools virtualenv \
30+
&& python3 -m pip install --upgrade awscli \
5931
&& ln -s `find /opt -name aws` /usr/local/bin/aws \
6032
&& which aws \
6133
&& aws --version
6234

6335

64-
65-
###############################################################################
66-
# Install Python3.6 for Deployment
67-
###############################################################################
68-
WORKDIR /tmp
69-
RUN wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tgz \
70-
&& tar zxf Python-3.6.0.tgz \
71-
&& cd Python-3.6.0 \
72-
&& ./configure \
73-
&& make -j 4 \
74-
&& make altinstall \
75-
&& python3.6 -m pip install --no-input wheel \
76-
&& cd ..
77-
78-
79-
###############################################################################
80-
# Install Python3.7 for Deployment
81-
###############################################################################
82-
RUN wget https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tgz \
83-
&& tar zxf Python-3.7.0.tgz \
84-
&& cd Python-3.7.0 \
85-
&& ./configure \
86-
&& make -j 4 \
87-
&& make altinstall \
88-
&& python3.7 -m pip install --no-input wheel \
89-
&& cd ..
90-
91-
###############################################################################
92-
# Install Python3.8 for Deployment
93-
###############################################################################
94-
RUN wget https://www.python.org/ftp/python/3.8.0/Python-3.8.0.tgz \
95-
&& tar zxf Python-3.8.0.tgz \
96-
&& cd Python-3.8.0 \
97-
&& ./configure \
98-
&& make -j 4 \
99-
&& make altinstall \
100-
&& python3.8 -m pip install --no-input wheel \
101-
&& cd ..
102-
103-
104-
###############################################################################
105-
# Install Python3.10 for Deployment
106-
###############################################################################
107-
RUN wget https://www.python.org/ftp/python/3.10.0/Python-3.10.0.tgz \
108-
&& tar zxf Python-3.10.0.tgz \
109-
&& cd Python-3.10.0 \
110-
&& ./configure \
111-
&& make -j 4 \
112-
&& make altinstall \
113-
&& python3.10 -m pip install --no-input wheel \
114-
&& cd ..
115-
116-
117-
11836
###############################################################################
11937
# Install entrypoint
12038
###############################################################################

‎.github/docker-images/ubuntu-18-x64/Dockerfile

+1
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ RUN apt-get update -qq \
1414
# Ubuntu18's default python3 is python3.6, which is EOL.
1515
# So install python3.8 (latest version on this distro, circa Oct 2022)
1616
python3.8-dev \
17+
python3.8-venv \
1718
# This installs pip for all python versions on the system
1819
# (there is no "python3.8-pip")
1920
python3-pip \

‎.github/workflows/create-channel.yml

+1-32
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@ jobs:
7979
- rhel8-x64
8080
- opensuse-leap
8181
- fedora-34-x64
82+
- raspbian-bullseye
8283
- alpine-3.16-x64
8384
- alpine-3.16-x86
8485
- alpine-3.16-arm64
@@ -124,35 +125,3 @@ jobs:
124125
run: |
125126
export IMAGE_TAG=${{ steps.tag.outputs.release_tag }}
126127
docker push ${{ secrets.AWS_ECR_REPO }}/aws-crt-${{ matrix.variant }}:$IMAGE_TAG
127-
128-
# The job will directly pull the image from the latest, and push to the new tag.
129-
# This job is only for a temporary fix. The cheated image would not get updated.
130-
passthrough-images:
131-
name: ${{ matrix.variant }}
132-
runs-on: ubuntu-latest
133-
strategy:
134-
fail-fast: false
135-
matrix:
136-
variant:
137-
- raspbian-bullseye
138-
139-
steps:
140-
- name: Checkout Sources
141-
uses: actions/checkout@v4
142-
143-
- name: Get release tag
144-
uses: ./.github/actions/release-tag
145-
id: tag
146-
147-
- name: Login to docker repo
148-
run: aws ecr get-login-password --region us-east-1 | docker login ${{ secrets.AWS_ECR_REPO }} -u AWS --password-stdin
149-
150-
- name: Pull latest image and push
151-
run: |
152-
if ! aws --region us-east-1 ecr describe-repositories --repository-names aws-crt-${{ matrix.variant }} > /dev/null 2>&1; then \
153-
exit 1
154-
fi
155-
export IMAGE_TAG=${{ steps.tag.outputs.release_tag }}
156-
docker pull ${{ secrets.AWS_ECR_REPO }}/aws-crt-${{ matrix.variant }}:latest
157-
docker tag ${{ secrets.AWS_ECR_REPO }}/aws-crt-${{ matrix.variant }}:latest ${{ secrets.AWS_ECR_REPO }}/aws-crt-${{ matrix.variant }}:$IMAGE_TAG
158-
docker push ${{ secrets.AWS_ECR_REPO }}/aws-crt-${{ matrix.variant }}:$IMAGE_TAG

‎.github/workflows/create-release.yml

+1-32
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@ jobs:
7272
- rhel8-x64
7373
- opensuse-leap
7474
- fedora-34-x64
75+
- raspbian-bullseye
7576
- alpine-3.16-x64
7677
- alpine-3.16-x86
7778
- alpine-3.16-arm64
@@ -129,38 +130,6 @@ jobs:
129130
| aws s3 cp - s3://${{env.AWS_S3_BUCKET}}/releases/$IMAGE_TAG/aws-crt-${{ matrix.variant }}.tar.gz
130131
aws s3 cp s3://${{env.AWS_S3_BUCKET}}/releases/$IMAGE_TAG/aws-crt-${{ matrix.variant }}.tar.gz s3://${{env.AWS_S3_BUCKET}}/LATEST/aws-crt-${{ matrix.variant }}.tar.gz
131132
132-
# The job will directly pull the image from the latest, and push to the new tag.
133-
# This job is only for a temporary fix. The image would not get updated.
134-
passthrough-images:
135-
name: ${{ matrix.variant }}
136-
runs-on: ubuntu-latest
137-
strategy:
138-
fail-fast: false
139-
matrix:
140-
variant:
141-
- raspbian-bullseye
142-
143-
steps:
144-
- name: Checkout Sources
145-
uses: actions/checkout@v4
146-
147-
- name: Get release tag
148-
uses: ./.github/actions/release-tag
149-
id: tag
150-
151-
- name: Login to docker repo
152-
run: aws ecr get-login-password --region us-east-1 | docker login ${{ secrets.AWS_ECR_REPO }} -u AWS --password-stdin
153-
154-
- name: Pull latest image and push
155-
run: |
156-
if ! aws --region us-east-1 ecr describe-repositories --repository-names aws-crt-${{ matrix.variant }} > /dev/null 2>&1; then \
157-
exit 1
158-
fi
159-
export IMAGE_TAG=${{ steps.tag.outputs.release_tag }}
160-
docker pull ${{ secrets.AWS_ECR_REPO }}/aws-crt-${{ matrix.variant }}:latest
161-
docker tag ${{ secrets.AWS_ECR_REPO }}/aws-crt-${{ matrix.variant }}:latest ${{ secrets.AWS_ECR_REPO }}/aws-crt-${{ matrix.variant }}:$IMAGE_TAG
162-
docker push ${{ secrets.AWS_ECR_REPO }}/aws-crt-${{ matrix.variant }}:$IMAGE_TAG
163-
164133
upload-ci-script:
165134
name: Upload container ci script
166135
runs-on: ubuntu-latest

0 commit comments

Comments
 (0)