Allow reposync to download from Amazon Linux repos #481
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The yum repository layout for Amazon Linux repositories (AL1, AL2, AL2023, and likely future versions) have an interesting layout. All Amazon Linux releases have a mirrorlist that points to the repository. This repository is not in a fixed location but instead under a GUID. It allows content syncing (and staged) before the atomic (and fast) operation of writing a new mirrorlist making the content visible. GUID-based repository wasn't an issue for Amazon Linux 1 repositories as the GUID repos were a complete copy of the repository. But as the "updates" repo grew, the time it took to release package updates increased. Starting with Amazon Linux 2, instead of having each GUID repo have a full copy of the repository, the repodata contains relative paths over to a central blobstore. Thus the only data pushed to release a package update are the added packages and a new copy of the repo metadata. However, as of caf28c4, reposync does not want to write files outside the destination directory. It broke the ability to reposync the Amazon Linux 2 style yum repositories. This patch updates the package download path per regular expression (r"^(?:../)+blobstore/[a-fA-F0-9]{64}/"). The regular expression substitution removes the blobstore-GUID path. It does keep the downloading file within a sub-directory structure if present. = changelog = msg: Allow reposync to download from Amazon Linux repos type: enhancement resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1898089
|
Doesn't this still have the same problem that #457 had where we wind up having a CVE reintroduced by allowing path traversal outside of the parent directory? |
|
The change adds the repo data inside the parent directory. Can you share an example of outside path traversal? |
|
Hmm, actually I think I misread what this does. I assume I could observe what it does by mirroring AL2023 with it? |
|
These are the testing I did Testing different locations of blobstore and the corresponding result of the change - regex substitution AL2022 was used for testing, same for AL2023 Errors during downloading metadata for repository '_dnf_local': https://bugzilla.redhat.com/show_bug.cgi?id=1950585 |
|
For AL2023
|
|
Hi @Conan-Kudo , |
|
@Conan-Kudo , @m-blaha any updates? |
|
Apologies for the delay. This pull request (PR) introduces a fix that is specific to a particular repository layout. However, we aim to adopt a more general approach. Could you please check whether the solution at #441 would be helpful? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The yum repository layout for Amazon Linux repositories (AL1, AL2,
AL2023, and likely future versions) have an interesting layout.
All Amazon Linux releases have a mirrorlist that points to the
repository. This repository is not in a fixed location but instead under
a GUID. It allows content syncing (and staged) before the atomic (and
fast) operation of writing a new mirrorlist making the content visible.
GUID-based repository wasn't an issue for Amazon Linux 1
repositories as the GUID repos were a complete copy of the repository.
But as the "updates" repo grew, the time it took to release package
updates increased.
Starting with Amazon Linux 2, instead of having each GUID repo have a
full copy of the repository, the repodata contains relative paths over
to a central blobstore. Thus the only data pushed to release a package
update are the added packages and a new copy of the repo metadata.
However, as of caf28c4, reposync does not want to write files outside
the destination directory. It broke the ability to reposync the Amazon
Linux 2 style yum repositories.
This patch updates the package download path per regular expression
(r"^(?:../)+blobstore/[a-fA-F0-9]{64}/"). The regular expression
substitution removes the blobstore-GUID path. It does keep the
downloading file within a sub-directory structure if present.