From e4efbeb02347e7a464b4809378fa851880427751 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 10 Aug 2024 08:16:49 -0400 Subject: [PATCH 01/45] DRAFT: Swiftly proxies Add a design proposal for the new swiftly proxy system --- DESIGN.md | 41 ++++++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index dd0319aa..2aa69926 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -60,7 +60,7 @@ A simple setup for managing the toolchains could look like this: The toolchains (i.e. the contents of a given Swift download tarball) would be contained in the toolchains directory, each named according to the major/minor/patch version. `config.json` would contain any required metadata (e.g. the latest Swift version, which toolchain is selected, etc.). If pulling in Foundation to use `JSONEncoder`/`JSONDecoder` (or some other JSON tool) would be a problem, we could also use something simpler. -The `~/.local/bin` directory would include symlinks pointing to the `bin` directory of the "active" toolchain, if any. +The `~/.local/bin` directory would include symlinks pointing to swiftly itself. When these binaries are run swiftly proxies them to the requested toolchain, or a default. This is all very similar to how rustup does things, but I figure there's no need to reinvent the wheel here. @@ -78,7 +78,7 @@ The contents of `~/Library/Application Support/swiftly` would look like this: – env ``` -Instead of downloading tarballs containing the toolchains and storing them directly in `~/.local/share/swiftly/toolchains`, we instead install Swift toolchains to `~/Library/Developer/Toolchains` via the `.pkg` files provided for download at swift.org. To select a toolchain for use, we update the symlinks at `~/Library/Application Support/swiftly/bin` to point to the desired toolchain in `~/Library/Developer/Toolchains`. In the env file, we’ll contain a line that looks like `export PATH="$HOME/Library/Application Support/swiftly:$PATH"`, so the version of swift being used will automatically always be from the active toolchain. `config.json` will contain version information about the selected toolchain as well as its actual location on disk. +Instead of downloading tarballs containing the toolchains and storing them directly in `~/.local/share/swiftly/toolchains`, we instead install Swift toolchains to `~/Library/Developer/Toolchains` via the `.pkg` files provided for download at swift.org. In the env file, we’ll add a line that looks like `export PATH="$HOME/Library/Application Support/swiftly:$PATH"`, so that swiftly can proxy toolchain commands to the requested toolchain, or default. `config.json` will contain version information about the selected toolchain as well as its actual location on disk. This scheme works for ensuring the version of Swift used on the command line can be controlled, but it doesn’t affect the active toolchain used by Xcode, which uses its own mechanisms for that. Xcode, if it is installed, can find the toolchains installed by swiftly. @@ -178,7 +178,7 @@ To list all the versions of swift installed on your system #### use -“Using” a toolchain sets it as the active toolchain, meaning it will be the one found via $PATH and invoked via `swift` commands executed in the shell. Only a single toolchain can be used at a given time. Using a toolchain doesn’t uninstall anything; it only updates symlinks so that the requested toolchain can be found by the shell. +“Using” a toolchain sets it as the default toolchain, meaning it will be the default one that is used when running toolchain commands from the shell. Only a single toolchain can be the default at a given time and location. Using a toolchain doesn’t uninstall anything; it only updates the configuration. To use the toolchain associated with the most up-to-date Swift version, the “latest” version can be specified: @@ -208,6 +208,10 @@ To use the latest installed main snapshot, leave off the date: `swiftly use main-snapshot` +The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version. + +Note: The `.swift-version` file mechanisms can be overridden using the `--global-default` flag so that your swiftly installation's default toolchain can be set explicitly. + #### update Update replaces a given toolchain with a later version of that toolchain. For a stable release, this means updating to a later patch version. For snapshots, this means updating to the most recently available snapshot. @@ -266,6 +270,18 @@ This command checks to see if there are new versions of `swiftly` itself and upg `swiftly self-update` +### Proxy command invocations to a requested toolchain + +Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during installation that point to the swiftly binary itself for each of the common toolchain commands, such as swift, swiftc, clang, etc. This mechanism will allows swiftly to proxy those command invocations to a requested toolchain. A toolchain can be requested in these ways in order of precedence: + +* Special toolchain selectors among the regular tool command-line arguments (e.g. `swift build +5.10.1`) with the special '+' prefix +* The presence of a .swift-version file in the current working directory, or ancestor directory, with the required toolchain version +* The swiftly default (in-use) toolchain set in the config.json by `swiftly install` or `swiftly use` commands + +In the first two cases, if there is no matching toolchain installed, swiftly will attempt to automatically install the requested toolchain and use it if the installation succeeeds. + +Note: If swiftly automatically installs a toolchain during proxying and that toolchain requires post installation steps then the proxy will abort with those post installation instructions. + ## Detailed Design Swiftly itself will be a SPM project consisting of several executable products, one per supported platform, and all of these will share the core module that handles argument parsing, printing help information, and dispatching commands. Each platform’s executable will be built to statically link the stdlib so that they can be run without having installed Swift first. @@ -457,15 +473,7 @@ https://download.swift.org/swift-5.5.1-release/ubuntu1604/swift-5.5.1-RELEASE/sw $ tar -xf --directory ~/.local/share/swiftly/toolchains ``` -It also updates `config.json` to include this toolchain as the latest for the provided version. If installing a new patch release toolchain, the now-outdated one can be deleted (e.g. `5.5.0` can be deleted when `5.5.1` is installed). - -Finally, the use implementation executes the following to update the link: - -``` -$ ln -s ~/.local/share/swiftly/toolchains//usr/bin/swift ~/.local/bin/swift -``` - -It also updates `config.json` to include this version as the currently selected one. +It also updates `config.json` to include this toolchain as the latest for the provided version. If installing a new patch release toolchain, the now-outdated one can be deleted (e.g. `5.5.0` can be deleted when `5.5.1` is installed). The `config.json` is updated to include this version as the currently selected (default) one. ### Implementation Sketch - macOS @@ -481,18 +489,13 @@ https://download.swift.org/swift--RELEASE/xcode/swift--RELEASE `config.json` is then updated to include this toolchain as the latest for the provided version. -Finally, the use implementation executes the following to update the link: - -``` -$ ln -s ~/Library/Developer/Toolchains/ ~/.swiftly/active-toolchain -``` - -It also updates `config.json` to include this version as the currently selected one. +It also updates `config.json` to include this version as the currently selected (default) one. ### `config.json` Schema ``` { + "version": "", "platform": { "namePretty": , "fullName": , From 3b5c4044bb6f8d4c867f86ede5acc76b5d448a91 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 10 Aug 2024 08:22:13 -0400 Subject: [PATCH 02/45] Add a swiftly install workflow where the version comes from the .swift-version file --- DESIGN.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/DESIGN.md b/DESIGN.md index 2aa69926..c0b4e595 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -138,6 +138,14 @@ Installing a specific snapshot from a swift version development branch `swiftly install 5.5-snapshot-2022-1-28` +##### Installing the version from the `.swift-version` file + +A package could have a swift version file that specifies the recommended toolchain version. A swiftly install with no version will search for a version file and install that version. + +`swiftly install` + +If no swift version file can be found then the installation fails indicating that it couldn't fine the file. + #### uninstall Uninstalling versions of Swift should be in a similar form to install. Uninstalling a toolchain that is currently “in use” (see the “use” command section below) will cause swiftly to use the latest Swift release toolchain that is installed. If none are, the latest snapshot will be used. If no snapshots are installed either, then a message will be printed indicating that all Swift versions are uninstalled. From 956256ffe9bec0f684a3613c134debc8b93c53d5 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 20 Aug 2024 12:15:47 -0400 Subject: [PATCH 03/45] update design to make auto-installation an error instead --- DESIGN.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index c0b4e595..843ad2e4 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -286,9 +286,7 @@ Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during instal * The presence of a .swift-version file in the current working directory, or ancestor directory, with the required toolchain version * The swiftly default (in-use) toolchain set in the config.json by `swiftly install` or `swiftly use` commands -In the first two cases, if there is no matching toolchain installed, swiftly will attempt to automatically install the requested toolchain and use it if the installation succeeeds. - -Note: If swiftly automatically installs a toolchain during proxying and that toolchain requires post installation steps then the proxy will abort with those post installation instructions. +If swiftly cannot find an installed toolchain that matches the request then it fails with an error and instructions how to use `swiftly install` to fulfill the request. ## Detailed Design From 375df29f044160bf1cd04a93befb0046c3743eab Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 20 Aug 2024 15:15:27 -0400 Subject: [PATCH 04/45] provide a mechanism to find the currently in-use toolchain physical location clarify the boundaries of the swiftly toolchain abstraction and elaborate on how to work around them --- DESIGN.md | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 843ad2e4..2ff4bf7b 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -4,21 +4,28 @@ This document contains the high level design of swiftly. Not all features have b ## Index +- [Swiftly's purpose](#swiftlys-purpose) +- [Installation of swiftly](#installation-of-switly) - [Linux](#linux) - - [Installation of swiftly](#installation-of-swiftly) - [Installation of a Swift toolchain](#installation-of-a-swift-toolchain) - [macOS](#macos) - - [Installation of swiftly](#installation-of-swiftly-1) - [Installation of a Swift toolchain](#installation-of-a-swift-toolchain-1) - [Interface](#interface) - [Toolchain names and versions](#toolchain-names-and-versions) - [Commands](#commands) + - [Toolchain selection](#toolchain-selection) - [Detailed design](#detailed-design) - [Implementation sketch - Core](#implementation-sketch---core) - [Implementation sketch - Ubuntu 20.04](#implementation-sketch---ubuntu-2004) - [Implementation sketch - macOS](#implementation-sketch---macos) - [`config.json` schema](#configjson-schema) +## Swiftly's purpose + +Swiftly helps you to easily install different Swift toolchains locally on your account. It also provides a single path where you can run the tools in the currently selected toolchain. Toolchain selection is [configurable](#toolchain-selection) using different mechanisms. + +Note that swiftly is *not* a virtual toolchain in itself since there are cases where it cannot behave as a self-contained Swift toolchain. For example, there can be external dependencies on specific files, such as headers or libraries, far too many and variable between toolchain versions to be managed by swiftly. Also, for long-lived processes, there is no way to gracefully restart them without help from the client. + ## Installation of swiftly The installation of swiftly is divided into two phases: delivery and initialization. Delivery of the swiftly binary can be accomplished using different methods: @@ -278,15 +285,25 @@ This command checks to see if there are new versions of `swiftly` itself and upg `swiftly self-update` -### Proxy command invocations to a requested toolchain +### Toolchain selection -Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during installation that point to the swiftly binary itself for each of the common toolchain commands, such as swift, swiftc, clang, etc. This mechanism will allows swiftly to proxy those command invocations to a requested toolchain. A toolchain can be requested in these ways in order of precedence: +Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during installation that point to the swiftly binary itself for each of the common toolchain commands, such as swift, swiftc, clang, etc. This mechanism will allows swiftly to proxy those command invocations to a selected toolchain at the time of invocation. A toolchain can be selected in these ways in order of precedence: * Special toolchain selectors among the regular tool command-line arguments (e.g. `swift build +5.10.1`) with the special '+' prefix * The presence of a .swift-version file in the current working directory, or ancestor directory, with the required toolchain version -* The swiftly default (in-use) toolchain set in the config.json by `swiftly install` or `swiftly use` commands +* The swiftly default (in-use) toolchain set in the swftly config.json by `swiftly install` or `swiftly use` commands + +If swiftly cannot find an installed toolchain that matches the selection then it fails with an error and instructions how to use `swiftly install` to satisfy the selection next time. + +#### Resolve selected toolchain + +For cases where the physical toolchain must be located, such as references specific header files, or shared libraries that are not proxied by swiftly there is a method to resolve the currently selected toolchain to its physical location using `swiftly use`. + +``` +swiftly use --location +``` -If swiftly cannot find an installed toolchain that matches the request then it fails with an error and instructions how to use `swiftly install` to fulfill the request. +This command will provide the full path to the directory where the selected toolchain is installed to standard output if such a toolchain exists. An external tool can directly navigate to the resources that it requires. For external tools that manage long-lived processes from the toolchain, such as the language server, and lldb, this command can be used in a poll to detect cases where the processes should be restarted. ## Detailed Design From ed68a9e15d9ae2a1ea1de643c59c851871dfbe2e Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 21 Aug 2024 08:17:00 -0400 Subject: [PATCH 05/45] add more details about the selector prefix, and methods to escape --- DESIGN.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/DESIGN.md b/DESIGN.md index 2ff4bf7b..2c419d5f 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -295,6 +295,8 @@ Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during instal If swiftly cannot find an installed toolchain that matches the selection then it fails with an error and instructions how to use `swiftly install` to satisfy the selection next time. +A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++++' is ignored entirely, but any additional arguments are sent directly to the tool without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. + #### Resolve selected toolchain For cases where the physical toolchain must be located, such as references specific header files, or shared libraries that are not proxied by swiftly there is a method to resolve the currently selected toolchain to its physical location using `swiftly use`. From c745f4cd33f6fd9ea4252a35e47edd320d17ec86 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 29 Aug 2024 13:36:51 -0400 Subject: [PATCH 06/45] Restructure the PR to move the selector syntax from the proxies to a new swiftly run command --- DESIGN.md | 42 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 6 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 2c419d5f..e3aa5efb 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -24,7 +24,7 @@ This document contains the high level design of swiftly. Not all features have b Swiftly helps you to easily install different Swift toolchains locally on your account. It also provides a single path where you can run the tools in the currently selected toolchain. Toolchain selection is [configurable](#toolchain-selection) using different mechanisms. -Note that swiftly is *not* a virtual toolchain in itself since there are cases where it cannot behave as a self-contained Swift toolchain. For example, there can be external dependencies on specific files, such as headers or libraries, far too many and variable between toolchain versions to be managed by swiftly. Also, for long-lived processes, there is no way to gracefully restart them without help from the client. +Note that swiftly is *not* a virtual toolchain in itself since there are cases where it cannot behave as a self-contained Swift toolchain. For example, there can be external dependencies on specific files, such as headers or libraries. There are far too many files that change between toolchain versions to be managed by swiftly. Also, for long-lived processes, there is no way to gracefully restart them without help from the client. ## Installation of swiftly @@ -67,7 +67,7 @@ A simple setup for managing the toolchains could look like this: The toolchains (i.e. the contents of a given Swift download tarball) would be contained in the toolchains directory, each named according to the major/minor/patch version. `config.json` would contain any required metadata (e.g. the latest Swift version, which toolchain is selected, etc.). If pulling in Foundation to use `JSONEncoder`/`JSONDecoder` (or some other JSON tool) would be a problem, we could also use something simpler. -The `~/.local/bin` directory would include symlinks pointing to swiftly itself. When these binaries are run swiftly proxies them to the requested toolchain, or a default. +The `~/.local/bin` directory would include symlinks pointing to swiftly itself. When the proxies binaries are executed swiftly proxies them to the requested toolchain, or the default. This is all very similar to how rustup does things, but I figure there's no need to reinvent the wheel here. @@ -289,24 +289,54 @@ This command checks to see if there are new versions of `swiftly` itself and upg Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during installation that point to the swiftly binary itself for each of the common toolchain commands, such as swift, swiftc, clang, etc. This mechanism will allows swiftly to proxy those command invocations to a selected toolchain at the time of invocation. A toolchain can be selected in these ways in order of precedence: -* Special toolchain selectors among the regular tool command-line arguments (e.g. `swift build +5.10.1`) with the special '+' prefix * The presence of a .swift-version file in the current working directory, or ancestor directory, with the required toolchain version * The swiftly default (in-use) toolchain set in the swftly config.json by `swiftly install` or `swiftly use` commands If swiftly cannot find an installed toolchain that matches the selection then it fails with an error and instructions how to use `swiftly install` to satisfy the selection next time. -A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++++' is ignored entirely, but any additional arguments are sent directly to the tool without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. - #### Resolve selected toolchain For cases where the physical toolchain must be located, such as references specific header files, or shared libraries that are not proxied by swiftly there is a method to resolve the currently selected toolchain to its physical location using `swiftly use`. ``` -swiftly use --location +swiftly use --print-location ``` This command will provide the full path to the directory where the selected toolchain is installed to standard output if such a toolchain exists. An external tool can directly navigate to the resources that it requires. For external tools that manage long-lived processes from the toolchain, such as the language server, and lldb, this command can be used in a poll to detect cases where the processes should be restarted. +#### Run with a selected toolchain + +There are cases where you might want to run an arbitrary command using a selected toolchain. An example could be that you want to build something with CMake. + +``` +# CMake +swiftly run cmake -G ninja +swiftly run ninja build + +# Autoconf +swiftly run ./configure +swiftly run make +``` + +Swiftly adjusts certain environment variables, such as prefixing the PATH to the selected toolchain directory, and setting the CC and CXX variables to the locations of clang and clang++ in those toolchains so that the build tools use them. If you want to explicitly specify a toolchain for the command you can do that with a selector notation like this: + +``` +swiftly run swift build +5.10.1 # Runs swift build with the 5.10.1 toolchain +``` + +A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++++' is ignored entirely, but any additional arguments are sent directly to the command without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. + +If the selected toolchain is not installed then swiftly will exit with a message indicating that you need to run `swiftly install x.y.z` to install it. However, if you enter a special `+install` token then swiftly will automatically download and install the toolchain if it isn't already present. + +``` +# Download and install the latest main snapshot toolchain and run 'swift build' to build the package with it. +swiftly run swift build +main-snapshot +install + +# Generate makefiles with the latest released Swift toolchain, download and install it if necessary +swiftly run +latest +install cmake -G "Unix Makefile" +swiftly run +latest make +``` + ## Detailed Design Swiftly itself will be a SPM project consisting of several executable products, one per supported platform, and all of these will share the core module that handles argument parsing, printing help information, and dispatching commands. Each platform’s executable will be built to statically link the stdlib so that they can be run without having installed Swift first. From 560236d788eeaa9f0ea7207970caa43f7093e8c8 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 4 Sep 2024 15:40:46 -0400 Subject: [PATCH 07/45] Implement proxy mechanism with dynamic toolchain selection Change the nature of the swiftly symlinks so that they point to the swiftly executable at install time. These do not change when new toolchains are used. Toolchain selection happens each time when the proxies are run. The proxies are created for a well-known set of toolchain binaries that are constant for a wide variety of toolchain versions and platforms. Add support for .swift-version files for toolchain selection. Update the use command so that it can point out which toolchain is in use based on context, such as swift version files that are located in the current working directory or above. The fallback selection comes from the global default configuration's 'inUse' setting. When querying for what's in use the global default is shown with the "(default)" tag. If the in-use toolchain is selected by a swift-version file the path to that file is displayed. Provide a print location flag to the use subcommand that can print the file path of the toolchain that is in use in the current context. When using a new toolchain, depending on whether a swift version is selecting the current one, update the swift version file with the selected toolchain version. If no swift version file can be located, attempt to create a new one at the top of the git worktree. If there is no git worktree, then fallback to updating the global default in the configuration. Provide a global default flag for the use subcommand so that only the global default in-use toolchain is considered and not any of the swift version files. --- Sources/LinuxPlatform/Linux.swift | 86 +------------ Sources/MacOSPlatform/MacOS.swift | 96 +------------- Sources/Swiftly/Init.swift | 60 ++++++++- Sources/Swiftly/Install.swift | 7 +- Sources/Swiftly/Proxy.swift | 64 ++++++++++ Sources/Swiftly/Swiftly.swift | 1 - Sources/Swiftly/Uninstall.swift | 3 +- Sources/Swiftly/Use.swift | 166 +++++++++++++++++++++--- Sources/SwiftlyCore/Platform.swift | 29 +++-- Tests/SwiftlyTests/E2ETests.swift | 12 -- Tests/SwiftlyTests/PlatformTests.swift | 41 ------ Tests/SwiftlyTests/SwiftlyTests.swift | 20 ++- Tests/SwiftlyTests/UseTests.swift | 167 ++++++++++--------------- 13 files changed, 370 insertions(+), 382 deletions(-) create mode 100644 Sources/Swiftly/Proxy.swift diff --git a/Sources/LinuxPlatform/Linux.swift b/Sources/LinuxPlatform/Linux.swift index 7a96a5ec..d9a2e025 100644 --- a/Sources/LinuxPlatform/Linux.swift +++ b/Sources/LinuxPlatform/Linux.swift @@ -295,86 +295,6 @@ public struct Linux: Platform { try FileManager.default.removeItem(at: toolchainDir) } - public func use(_ toolchain: ToolchainVersion, currentToolchain: ToolchainVersion?) throws -> Bool { - let toolchainBinURL = self.swiftlyToolchainsDir - .appendingPathComponent(toolchain.name, isDirectory: true) - .appendingPathComponent("usr", isDirectory: true) - .appendingPathComponent("bin", isDirectory: true) - - if !FileManager.default.fileExists(atPath: toolchainBinURL.path) { - return false - } - - // Delete existing symlinks from previously in-use toolchain. - if let currentToolchain { - try self.unUse(currentToolchain: currentToolchain) - } - - // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. - let swiftlyBinDirContents = try FileManager.default.contentsOfDirectory(atPath: self.swiftlyBinDir.path) - let toolchainBinDirContents = try FileManager.default.contentsOfDirectory(atPath: toolchainBinURL.path) - let willBeOverwritten = Set(toolchainBinDirContents).intersection(swiftlyBinDirContents) - if !willBeOverwritten.isEmpty { - SwiftlyCore.print("The following existing executables will be overwritten:") - - for executable in willBeOverwritten { - SwiftlyCore.print(" \(self.swiftlyBinDir.appendingPathComponent(executable).path)") - } - - let proceed = SwiftlyCore.readLine(prompt: "Proceed? (y/n)") ?? "n" - - guard proceed == "y" else { - SwiftlyCore.print("Aborting use") - return false - } - } - - for executable in toolchainBinDirContents { - let linkURL = self.swiftlyBinDir.appendingPathComponent(executable) - let executableURL = toolchainBinURL.appendingPathComponent(executable) - - // Deletion confirmed with user above. - try linkURL.deleteIfExists() - - try FileManager.default.createSymbolicLink( - atPath: linkURL.path, - withDestinationPath: executableURL.path - ) - } - - return true - } - - public func unUse(currentToolchain: ToolchainVersion) throws { - let currentToolchainBinURL = self.swiftlyToolchainsDir - .appendingPathComponent(currentToolchain.name, isDirectory: true) - .appendingPathComponent("usr", isDirectory: true) - .appendingPathComponent("bin", isDirectory: true) - - for existingExecutable in try FileManager.default.contentsOfDirectory(atPath: currentToolchainBinURL.path) { - guard existingExecutable != "swiftly" else { - continue - } - - let url = self.swiftlyBinDir.appendingPathComponent(existingExecutable) - let vals = try url.resourceValues(forKeys: [.isSymbolicLinkKey]) - - guard let islink = vals.isSymbolicLink, islink else { - throw Error(message: "Found executable not managed by swiftly in SWIFTLY_BIN_DIR: \(url.path)") - } - let symlinkDest = url.resolvingSymlinksInPath() - guard symlinkDest.deletingLastPathComponent() == currentToolchainBinURL else { - throw Error(message: "Found symlink that points to non-swiftly managed executable: \(symlinkDest.path)") - } - - try self.swiftlyBinDir.appendingPathComponent(existingExecutable).deleteIfExists() - } - } - - public func listAvailableSnapshots(version _: String?) async -> [Snapshot] { - [] - } - public func getExecutableName() -> String { #if arch(x86_64) let arch = "x86_64" @@ -387,8 +307,6 @@ public struct Linux: Platform { return "swiftly-\(arch)-unknown-linux-gnu" } - public func currentToolchain() throws -> ToolchainVersion? { nil } - public func getTempFilePath() -> URL { FileManager.default.temporaryDirectory.appendingPathComponent("swiftly-\(UUID())") } @@ -605,5 +523,9 @@ public struct Linux: Platform { return "/bin/bash" } + public func findToolchainLocation(_ toolchain: ToolchainVersion) -> URL { + self.swiftlyToolchainsDir.appendingPathComponent("\(toolchain.name)") + } + public static let currentPlatform: any Platform = Linux() } diff --git a/Sources/MacOSPlatform/MacOS.swift b/Sources/MacOSPlatform/MacOS.swift index a7c0dd2f..218bbdec 100644 --- a/Sources/MacOSPlatform/MacOS.swift +++ b/Sources/MacOSPlatform/MacOS.swift @@ -101,102 +101,10 @@ public struct MacOS: Platform { try? runProgram("pkgutil", "--volume", homedir, "--forget", pkgInfo.CFBundleIdentifier) } - public func use(_ toolchain: ToolchainVersion, currentToolchain: ToolchainVersion?) throws -> Bool { - let toolchainBinURL = self.swiftlyToolchainsDir - .appendingPathComponent(toolchain.identifier + ".xctoolchain", isDirectory: true) - .appendingPathComponent("usr", isDirectory: true) - .appendingPathComponent("bin", isDirectory: true) - - if !FileManager.default.fileExists(atPath: toolchainBinURL.path) { - return false - } - - // Delete existing symlinks from previously in-use toolchain. - if let currentToolchain { - try self.unUse(currentToolchain: currentToolchain) - } - - // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. - let swiftlyBinDirContents = try FileManager.default.contentsOfDirectory(atPath: self.swiftlyBinDir.path) - let toolchainBinDirContents = try FileManager.default.contentsOfDirectory(atPath: toolchainBinURL.path) - let willBeOverwritten = Set(toolchainBinDirContents).intersection(swiftlyBinDirContents) - if !willBeOverwritten.isEmpty { - SwiftlyCore.print("The following existing executables will be overwritten:") - - for executable in willBeOverwritten { - SwiftlyCore.print(" \(self.swiftlyBinDir.appendingPathComponent(executable).path)") - } - - let proceed = SwiftlyCore.readLine(prompt: "Proceed? (y/n)") ?? "n" - - guard proceed == "y" else { - SwiftlyCore.print("Aborting use") - return false - } - } - - for executable in toolchainBinDirContents { - let linkURL = self.swiftlyBinDir.appendingPathComponent(executable) - let executableURL = toolchainBinURL.appendingPathComponent(executable) - - // Deletion confirmed with user above. - try linkURL.deleteIfExists() - - try FileManager.default.createSymbolicLink( - atPath: linkURL.path, - withDestinationPath: executableURL.path - ) - } - - SwiftlyCore.print(""" - NOTE: On macOS it is possible that the shell will pick up the system Swift on the path - instead of the one that swiftly has installed for you. You can run the 'hash -r' - command to update the shell with the latest PATHs. - - hash -r - - """ - ) - - return true - } - - public func unUse(currentToolchain: ToolchainVersion) throws { - let currentToolchainBinURL = self.swiftlyToolchainsDir - .appendingPathComponent(currentToolchain.identifier + ".xctoolchain", isDirectory: true) - .appendingPathComponent("usr", isDirectory: true) - .appendingPathComponent("bin", isDirectory: true) - - for existingExecutable in try FileManager.default.contentsOfDirectory(atPath: currentToolchainBinURL.path) { - guard existingExecutable != "swiftly" else { - continue - } - - let url = self.swiftlyBinDir.appendingPathComponent(existingExecutable) - let vals = try url.resourceValues(forKeys: [URLResourceKey.isSymbolicLinkKey]) - - guard let islink = vals.isSymbolicLink, islink else { - throw Error(message: "Found executable not managed by swiftly in SWIFTLY_BIN_DIR: \(url.path)") - } - let symlinkDest = url.resolvingSymlinksInPath() - guard symlinkDest.deletingLastPathComponent() == currentToolchainBinURL else { - throw Error(message: "Found symlink that points to non-swiftly managed executable: \(symlinkDest.path)") - } - - try self.swiftlyBinDir.appendingPathComponent(existingExecutable).deleteIfExists() - } - } - - public func listAvailableSnapshots(version _: String?) async -> [Snapshot] { - [] - } - public func getExecutableName() -> String { "swiftly-macos-osx" } - public func currentToolchain() throws -> ToolchainVersion? { nil } - public func getTempFilePath() -> URL { FileManager.default.temporaryDirectory.appendingPathComponent("swiftly-\(UUID()).pkg") } @@ -226,5 +134,9 @@ public struct MacOS: Platform { return "/bin/zsh" } + public func findToolchainLocation(_ toolchain: ToolchainVersion) -> URL { + self.swiftlyToolchainsDir.appendingPathComponent("\(toolchain.identifier).xctoolchain") + } + public static let currentPlatform: any Platform = MacOS() } diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index 5d86e95e..8b89438d 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -54,6 +54,24 @@ internal struct Init: SwiftlyCommand { } } + // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. + let swiftlyBinDir = Swiftly.currentPlatform.swiftlyBinDir + let swiftlyBinDirContents = try FileManager.default.contentsOfDirectory(atPath: swiftlyBinDir.path) + let willBeOverwritten = Set(proxyList + ["swiftly"]).intersection(swiftlyBinDirContents) + if !willBeOverwritten.isEmpty && !overwrite { + SwiftlyCore.print("The following existing executables will be overwritten:") + + for executable in willBeOverwritten { + SwiftlyCore.print(" \(swiftlyBinDir.appendingPathComponent(executable).path)") + } + + let proceed = SwiftlyCore.readLine(prompt: "Proceed? [y/N]") ?? "n" + + guard proceed == "y" else { + throw Error(message: "Swiftly installation has been cancelled") + } + } + let shell = if let s = ProcessInfo.processInfo.environment["SHELL"] { s } else { @@ -113,11 +131,7 @@ internal struct Init: SwiftlyCommand { SwiftlyCore.print("Moving swiftly into the installation directory...") if swiftlyBin.fileExists() { - if !overwrite { - throw Error(message: "Swiftly binary already exists. You can try again with `--overwrite` to replace it.") - } else { - try FileManager.default.removeItem(at: swiftlyBin) - } + try FileManager.default.removeItem(at: swiftlyBin) } do { @@ -128,6 +142,30 @@ internal struct Init: SwiftlyCommand { } } + // Don't create the proxies in the tests + if !cmd.path.hasSuffix("xctest") { + SwiftlyCore.print("Setting up toolchain proxies...") + + let proxyTo = if systemManaged { + cmd.path + } else { + swiftlyBin.path + } + + for p in proxyList { + let proxy = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(p) + + if proxy.fileExists() { + try FileManager.default.removeItem(at: proxy) + } + + try FileManager.default.createSymbolicLink( + atPath: proxy.path, + withDestinationPath: proxyTo + ) + } + } + if overwrite || !FileManager.default.fileExists(atPath: envFile.path) { SwiftlyCore.print("Creating shell environment file for the user...") var env = "" @@ -201,7 +239,19 @@ internal struct Init: SwiftlyCommand { SwiftlyCore.print(""" To begin using installed swiftly from your current shell, first run the following command: \(sourceLine) + + """) + +#if os(macOS) + SwiftlyCore.print(""" + NOTE: On macOS it is possible that the shell will pick up the system Swift on the path + instead of the one that swiftly has installed for you. You can run the 'hash -r' + command to update the shell with the latest PATHs. + + hash -r + """) +#endif } } } diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 63f82c1d..a618b5b3 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -77,7 +77,7 @@ struct Install: SwiftlyCommand { let selector = try ToolchainSelector(parsing: self.version) SwiftlyCore.httpClient.githubToken = self.token - let toolchainVersion = try await self.resolve(selector: selector) + let toolchainVersion = try await Self.resolve(selector: selector) var config = try Config.load() let postInstallScript = try await Self.execute( version: toolchainVersion, @@ -218,7 +218,8 @@ struct Install: SwiftlyCommand { // If this is the first installed toolchain, mark it as in-use regardless of whether the // --use argument was provided. if useInstalledToolchain || config.inUse == nil { - try await Use.execute(version, &config) + // TODO: consider adding the global default option to this commands flags + try await Use.execute(version, false, &config) } SwiftlyCore.print("\(version) installed successfully!") @@ -227,7 +228,7 @@ struct Install: SwiftlyCommand { /// Utilize the GitHub API along with the provided selector to select a toolchain for install. /// TODO: update this to use an official swift.org API - func resolve(selector: ToolchainSelector) async throws -> ToolchainVersion { + static func resolve(selector: ToolchainSelector) async throws -> ToolchainVersion { switch selector { case .latest: SwiftlyCore.print("Fetching the latest stable Swift release...") diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift new file mode 100644 index 00000000..8d9c9c60 --- /dev/null +++ b/Sources/Swiftly/Proxy.swift @@ -0,0 +1,64 @@ +import Foundation +import SwiftlyCore + +// This is the allowed list of executables that we will proxy +let proxyList = [ + "clang", + "lldb", + "lldb-dap", + "lldb-server", + "clang++", + "sourcekit-lsp", + "clangd", + "swift", + "docc", + "swiftc", + "lld", + "llvm-ar", + "plutil", + "repl_swift", + "wasm-ld", +] + +@main +public enum Proxy { + static func main() async throws { + do { + let zero = CommandLine.arguments[0] + guard let binName = zero.components(separatedBy: "/").last else { + fatalError("Could not determine the binary name for proxying") + } + + guard proxyList.contains(binName) else { + // Treat this as a swiftly invocation + await Swiftly.main() + return + } + + let config = try Config.load() + let toolchain: ToolchainVersion + + if let (selectedToolchain, versionFile, selector) = try swiftToolchainSelection(config: config) { + guard let selectedToolchain = selectedToolchain else { + if let versionFile = versionFile { + throw if let selector = selector { + Error(message: "No installed swift toolchain matches the version \(selector) from \(versionFile). You can try installing one with `swiftly install \(selector)`.") + } else { + Error(message: "Swift version file is malformed and cannot be used to select a swift toolchain: \(versionFile)") + } + } + fatalError("error in toolchain selection logic") + } + + toolchain = selectedToolchain + } else { + throw Error(message: "No swift toolchain could be determined either from a .swift-version file, or the default. You can try using `swiftly use ` to set it.") + } + + try await Swiftly.currentPlatform.proxy(toolchain, binName, Array(CommandLine.arguments[1...])) + } catch { + SwiftlyCore.print("\(error)") + exit(1) + } + } +} diff --git a/Sources/Swiftly/Swiftly.swift b/Sources/Swiftly/Swiftly.swift index ce141b37..caa473ed 100644 --- a/Sources/Swiftly/Swiftly.swift +++ b/Sources/Swiftly/Swiftly.swift @@ -14,7 +14,6 @@ public struct GlobalOptions: ParsableArguments { public init() {} } -@main public struct Swiftly: SwiftlyCommand { public static var configuration = CommandConfiguration( abstract: "A utility for installing and managing Swift toolchains.", diff --git a/Sources/Swiftly/Uninstall.swift b/Sources/Swiftly/Uninstall.swift index a763cd4f..1bfb8ff2 100644 --- a/Sources/Swiftly/Uninstall.swift +++ b/Sources/Swiftly/Uninstall.swift @@ -100,10 +100,9 @@ struct Uninstall: SwiftlyCommand { ?? config.listInstalledToolchains(selector: .latest).filter({ !toolchains.contains($0) }).max() ?? config.installedToolchains.filter({ !toolchains.contains($0) }).max() { - try await Use.execute(toUse, &config) + try await Use.execute(toUse, true, &config) } else { // If there are no more toolchains installed, just unuse the currently active toolchain. - try Swiftly.currentPlatform.unUse(currentToolchain: toolchain) config.inUse = nil try config.save() } diff --git a/Sources/Swiftly/Use.swift b/Sources/Swiftly/Use.swift index b1cc9238..740ca0ab 100644 --- a/Sources/Swiftly/Use.swift +++ b/Sources/Swiftly/Use.swift @@ -1,4 +1,5 @@ import ArgumentParser +import Foundation import SwiftlyCore internal struct Use: SwiftlyCommand { @@ -6,6 +7,12 @@ internal struct Use: SwiftlyCommand { abstract: "Set the active toolchain. If no toolchain is provided, print the currently in-use toolchain, if any." ) + @Flag(name: .shortAndLong, help: "Print the location of the in-use toolchain. This is valid only when there is no toolchain argument.") + var printLocation: Bool = false + + @Flag(name: .shortAndLong, help: "Use the global default, ignoring any .swift-version files.") + var globalDefault: Bool = false + @Argument(help: ArgumentHelp( "The toolchain to use.", discussion: """ @@ -46,13 +53,44 @@ internal struct Use: SwiftlyCommand { try validateSwiftly() var config = try Config.load() + // This is the bare use command where we print the selected toolchain version (or the path to it) guard let toolchain = self.toolchain else { - if let inUse = config.inUse { - SwiftlyCore.print("\(inUse) (in use)") + let selected = try swiftToolchainSelection(config: config, globalDefault: self.globalDefault) + + guard let selected = selected else { + // No toolchain selected, so we just output nothing + return + } + + let (selectedVersion, versionFile, selector) = selected + + if let versionFile = versionFile, selector == nil { + throw Error(message: "Swift version file is malformed and cannot be used to select a swift toolchain: \(versionFile)") + } + + guard let selectedVersion = selectedVersion else { + fatalError("error in toolchain selection logic") + } + + if self.printLocation { + // Print the toolchain location and exit + SwiftlyCore.print("\(Swiftly.currentPlatform.findToolchainLocation(selectedVersion).path)") + return + } + + if let versionFile = versionFile { + SwiftlyCore.print("\(selectedVersion) (\(versionFile.path))") + } else { + SwiftlyCore.print("\(selectedVersion) (default)") } + return } + guard !self.printLocation else { + throw Error(message: "The print location flag cannot be used with a toolchain version.") + } + let selector = try ToolchainSelector(parsing: toolchain) guard let toolchain = config.listInstalledToolchains(selector: selector).max() else { @@ -60,29 +98,125 @@ internal struct Use: SwiftlyCommand { return } - try await Self.execute(toolchain, &config) + try await Self.execute(toolchain, self.globalDefault, &config) } - /// Use a toolchain. This method modifies and saves the input config. - internal static func execute(_ toolchain: ToolchainVersion, _ config: inout Config) async throws { - let previousToolchain = config.inUse + /// Use a toolchain. This method can modify and save the input config. + internal static func execute(_ toolchain: ToolchainVersion, _ globalDefault: Bool, _ config: inout Config) async throws { + let previousToolchain = try swiftToolchainSelection(config: config, globalDefault: globalDefault) - guard toolchain != previousToolchain else { - SwiftlyCore.print("\(toolchain) is already in use") - return + if let (selectedVersion, _, _) = previousToolchain { + guard selectedVersion != toolchain else { + SwiftlyCore.print("\(toolchain) is already in use") + return + } } - guard try Swiftly.currentPlatform.use(toolchain, currentToolchain: previousToolchain) else { - return + if let (_, versionFile, _) = previousToolchain, let versionFile = versionFile { + try toolchain.name.write(to: versionFile, atomically: true, encoding: .utf8) + } else if let newVersionFile = findNewVersionFile(), !globalDefault { + try toolchain.name.write(to: newVersionFile, atomically: true, encoding: .utf8) + } else { + config.inUse = toolchain + try config.save() } - config.inUse = toolchain - try config.save() - var message = "Set the active toolchain to \(toolchain)" - if let previousToolchain { - message += " (was \(previousToolchain))" + var message = "Set the used toolchain to \(toolchain)" + if let (selectedVersion, _, _) = previousToolchain, + let selectedVersion = selectedVersion + { + message += " (was \(selectedVersion.name))" } SwiftlyCore.print(message) } + + internal static func findNewVersionFile() -> URL? { + var cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) + + while cwd.path != "" && cwd.path != "/" { + guard FileManager.default.fileExists(atPath: cwd.path) else { + break + } + + let gitDir = cwd.appendingPathComponent(".git") + + if FileManager.default.fileExists(atPath: gitDir.path) { + return cwd.appendingPathComponent(".swift-version") + } + + cwd = cwd.deletingLastPathComponent() + } + + return nil + } +} + +/// Returns the currently selected swift toolchain with optional details. +/// +/// Selection of a toolchain can be accomplished in a number of ways. There is the +/// the configuration's global default 'inUse' setting. This is the fallback selector +/// if there are no other selections. In this case the returned tuple will contain +/// only the selected toolchain version. None of the other values are provided. +/// +/// A toolchain can also be selected from a `.swift-version` file in the current +/// working directory, or an ancestor directory. The nearest version file is +/// returned as a URL if it is present, even if the file is malformed or it +/// doesn't select any of the installed toolchains. A well-formed version file +/// will additionally return the toolchain selector that it represents. Finally, +/// if that selector selects one of the installed toolchains then all three +/// values are returned. +/// +/// Note: if no swift version files are found at all then the return will be nil +/// +public func swiftToolchainSelection(config: Config, globalDefault: Bool = false) throws -> (ToolchainVersion?, URL?, ToolchainSelector?)? { + if !globalDefault { + var cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) + + while cwd.path != "" && cwd.path != "/" { + guard FileManager.default.fileExists(atPath: cwd.path) else { + break + } + + let svFile = cwd.appendingPathComponent(".swift-version") + + if FileManager.default.fileExists(atPath: svFile.path) { + let contents = try? String(contentsOf: svFile, encoding: .utf8) + + guard let contents = contents else { + return (nil, svFile, nil) + } + + guard !contents.isEmpty else { + return (nil, svFile, nil) + } + + let selectorString = contents.replacingOccurrences(of: "\n", with: "").replacingOccurrences(of: "\r", with: "") + let selector: ToolchainSelector? + do { + selector = try ToolchainSelector(parsing: selectorString) + } catch { + return (nil, svFile, nil) + } + + guard let selector = selector else { + return (nil, svFile, nil) + } + + guard let selectedToolchain = config.listInstalledToolchains(selector: selector).max() else { + return (nil, svFile, selector) + } + + return (selectedToolchain, svFile, selector) + } + + cwd = cwd.deletingLastPathComponent() + } + } + + if let inUse = config.inUse { + return (inUse, nil, nil) + } + + return nil } diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index 2eb0f833..6c8b1fb0 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -54,18 +54,6 @@ public protocol Platform { /// If this version is in use, the next latest version will be used afterwards. func uninstall(_ version: ToolchainVersion) throws - /// Select the toolchain associated with the given version. - /// Returns whether the selection was successful. - func use(_ version: ToolchainVersion, currentToolchain: ToolchainVersion?) throws -> Bool - - /// Clear the current active toolchain. - func unUse(currentToolchain: ToolchainVersion) throws - - /// Get a list of snapshot builds for the platform. If a version is specified, only - /// return snapshots associated with the version. - /// This will likely have a default implementation. - func listAvailableSnapshots(version: String?) async -> [Snapshot] - /// Get the name of the swiftly release binary. func getExecutableName() -> String @@ -98,6 +86,9 @@ public protocol Platform { /// Get the user's current login shell func getShell() async throws -> String + + /// Find the location where the toolchain should be installed. + func findToolchainLocation(_ toolchain: ToolchainVersion) -> URL } extension Platform { @@ -126,7 +117,16 @@ extension Platform { } #if os(macOS) || os(Linux) + public func proxy(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws { + let cmd = self.findToolchainLocation(toolchain).appendingPathComponent("usr/bin/\(command)") + try self.runProgram([cmd.path] + arguments) + } + public func runProgram(_ args: String..., quiet: Bool = false) throws { + try self.runProgram([String](args), quiet: quiet) + } + + public func runProgram(_ args: [String], quiet: Bool = false) throws { let process = Process() process.executableURL = URL(fileURLWithPath: "/usr/bin/env") process.arguments = args @@ -150,6 +150,10 @@ extension Platform { } public func runProgramOutput(_ program: String, _ args: String...) async throws -> String? { + try await self.runProgramOutput(program, [String](args)) + } + + public func runProgramOutput(_ program: String, _ args: [String]) async throws -> String? { let process = Process() process.executableURL = URL(fileURLWithPath: "/usr/bin/env") process.arguments = [program] + args @@ -202,6 +206,7 @@ extension Platform { return true } + #endif } diff --git a/Tests/SwiftlyTests/E2ETests.swift b/Tests/SwiftlyTests/E2ETests.swift index 88cbe0b3..29ac97a0 100644 --- a/Tests/SwiftlyTests/E2ETests.swift +++ b/Tests/SwiftlyTests/E2ETests.swift @@ -61,18 +61,6 @@ final class E2ETests: SwiftlyTests { XCTAssertTrue(release >= ToolchainVersion.StableRelease(major: 5, minor: 8, patch: 0)) try await validateInstalledToolchains([installedToolchain], description: "install latest") - - if let envScript = envScript { - let whichCmd = if shell.hasSuffix("bash") { - "type -P swift" - } else { - "which swift" - } - - // Check that within a new shell, the swift version succeeds and is the version we expect - let whichSwift = (try? await Swiftly.currentPlatform.runProgramOutput(shell, "-c", ". \(envScript.path) ; \(whichCmd)")) ?? "" - XCTAssertTrue(whichSwift.hasPrefix(Swiftly.currentPlatform.swiftlyBinDir.path)) - } } } } diff --git a/Tests/SwiftlyTests/PlatformTests.swift b/Tests/SwiftlyTests/PlatformTests.swift index d9f07e61..01e5841c 100644 --- a/Tests/SwiftlyTests/PlatformTests.swift +++ b/Tests/SwiftlyTests/PlatformTests.swift @@ -73,45 +73,4 @@ final class PlatformTests: SwiftlyTests { XCTAssertEqual(0, toolchains.count) } } - - func testUse() async throws { - try await self.rollbackLocalChanges { - // GIVEN: toolchains have been downloaded, and installed - var (mockedToolchainFile, version) = try await self.mockToolchainDownload(version: "5.8.0") - try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version) - (mockedToolchainFile, version) = try await self.mockToolchainDownload(version: "5.6.3") - try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version) - // WHEN: one of the toolchains is used - var result = try Swiftly.currentPlatform.use(ToolchainVersion(parsing: "5.8.0"), currentToolchain: nil) - // THEN: there are symbolic links for the toolchain binaries in the bin dir that point to the toolchain - XCTAssertTrue(result) - var swiftLinkTarget = try? FileManager.default.destinationOfSymbolicLink(atPath: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swift").path) - guard let target = swiftLinkTarget else { - throw Error(message: "swift symlink was not found") - } - XCTAssertTrue(target.contains("5.8")) - - // GIVEN: toolchains have been downloaded, installed, and a toolchain is in use - // WHEN: another toolchain is used - result = try Swiftly.currentPlatform.use(ToolchainVersion(parsing: "5.6.3"), currentToolchain: ToolchainVersion(parsing: "5.8.0")) - // THEN: there are symbolic links for the toolchain binaries in the bin dir that point to the toolchain - XCTAssertTrue(result) - swiftLinkTarget = try? FileManager.default.destinationOfSymbolicLink(atPath: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swift").path) - guard let target2 = swiftLinkTarget else { - throw Error(message: "swift symlink was not found") - } - XCTAssertTrue(target2.contains("5.6.3")) - - // GIVEN: toolchains have been downloaded, installed, and a toolchain is in use - // WHEN: a toolchain is used that has not been installed - result = try Swiftly.currentPlatform.use(ToolchainVersion(parsing: "5.2.1"), currentToolchain: ToolchainVersion(parsing: "5.6.3")) - // THEN: the symbolic links remain the same - XCTAssertFalse(result) - swiftLinkTarget = try? FileManager.default.destinationOfSymbolicLink(atPath: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swift").path) - guard let target3 = swiftLinkTarget else { - throw Error(message: "swift symlink was not found") - } - XCTAssertTrue(target3.contains("5.6.3")) - } - } } diff --git a/Tests/SwiftlyTests/SwiftlyTests.swift b/Tests/SwiftlyTests/SwiftlyTests.swift index 3fed966a..0b4e9656 100644 --- a/Tests/SwiftlyTests/SwiftlyTests.swift +++ b/Tests/SwiftlyTests/SwiftlyTests.swift @@ -140,7 +140,7 @@ class SwiftlyTests: XCTestCase { } class func getTestHomePath(name: String) -> URL { - URL(fileURLWithPath: FileManager.default.currentDirectoryPath).appendingPathComponent(name, isDirectory: true) + FileManager.default.temporaryDirectory.appendingPathComponent("swiftly-tests-\(name)-\(UUID())") } /// Create a fresh swiftly home directory, populate it with a base config, and run the provided closure. @@ -168,6 +168,13 @@ class SwiftlyTests: XCTestCase { let config = try await self.baseTestConfig() try config.save() + let cwd = FileManager.default.currentDirectoryPath + defer { + FileManager.default.changeCurrentDirectoryPath(cwd) + } + + FileManager.default.changeCurrentDirectoryPath(testHome.path) + try await f() } @@ -292,17 +299,6 @@ class SwiftlyTests: XCTestCase { func validateInUse(expected: ToolchainVersion?) async throws { let config = try Config.load() XCTAssertEqual(config.inUse, expected) - - let executable = SwiftExecutable(path: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swift")) - - XCTAssertEqual(executable.exists(), expected != nil) - - guard let expected else { - return - } - - let inUseVersion = try await executable.version() - XCTAssertEqual(inUseVersion, expected) } /// Validate that all of the provided toolchains have been installed. diff --git a/Tests/SwiftlyTests/UseTests.swift b/Tests/SwiftlyTests/UseTests.swift index 724e2cce..604af79a 100644 --- a/Tests/SwiftlyTests/UseTests.swift +++ b/Tests/SwiftlyTests/UseTests.swift @@ -9,15 +9,10 @@ final class UseTests: SwiftlyTests { /// Execute a `use` command with the provided argument. Then validate that the configuration is updated properly and /// the in-use swift executable prints the the provided expectedVersion. func useAndValidate(argument: String, expectedVersion: ToolchainVersion) async throws { - var use = try self.parseCommand(Use.self, ["use", argument]) + var use = try self.parseCommand(Use.self, ["use", "-g", argument]) try await use.run() XCTAssertEqual(try Config.load().inUse, expectedVersion) - - let toolchainVersion = try self.getMockedToolchainVersion( - at: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swift") - ) - XCTAssertEqual(toolchainVersion, expectedVersion) } /// Tests that the `use` command can switch between installed stable release toolchains. @@ -171,13 +166,13 @@ final class UseTests: SwiftlyTests { /// Tests that the `use` command gracefully exits when executed before any toolchains have been installed. func testUseNoInstalledToolchains() async throws { try await self.withMockedHome(homeName: Self.homeName, toolchains: []) { - var use = try self.parseCommand(Use.self, ["use", "latest"]) + var use = try self.parseCommand(Use.self, ["use", "-g", "latest"]) try await use.run() var config = try Config.load() XCTAssertEqual(config.inUse, nil) - use = try self.parseCommand(Use.self, ["use", "5.6.0"]) + use = try self.parseCommand(Use.self, ["use", "-g", "5.6.0"]) try await use.run() config = try Config.load() @@ -213,117 +208,81 @@ final class UseTests: SwiftlyTests { } } - /// Tests that the `use` command symlinks all of the executables provided in a toolchain and removes any existing - /// symlinks from the previously active toolchain. - func testOldSymlinksRemoved() async throws { - try await self.withMockedHome(homeName: Self.homeName, toolchains: Self.allToolchains) { - let spec = [ - ToolchainVersion(major: 1, minor: 2, patch: 3): ["a", "b"], - ToolchainVersion(major: 2, minor: 3, patch: 4): ["b", "c", "d"], - ToolchainVersion(major: 3, minor: 4, patch: 5): ["a", "c", "d", "e"], - ] - - for (toolchain, files) in spec { - try await self.installMockedToolchain(toolchain: toolchain, executables: files) - } - - // Add an unrelated executable to the binary directory. - let existingFileName = "existing" - let existingExecutableURL = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(existingFileName) - let data = Data("hello world\n".utf8) - try data.write(to: existingExecutableURL) - - for (toolchain, files) in spec { - var use = try self.parseCommand(Use.self, ["use", toolchain.name]) + /// Tests that running a use command without an argument prints the currently in-use toolchain. + func testPrintInUse() async throws { + let toolchains = [ + Self.newStable, + Self.newMainSnapshot, + Self.newReleaseSnapshot, + ] + try await self.withMockedHome(homeName: Self.homeName, toolchains: Set(toolchains)) { + for toolchain in toolchains { + var use = try self.parseCommand(Use.self, ["use", "-g", toolchain.name]) try await use.run() - // Verify that only the symlinks for the active toolchain remain. - let symlinks = try FileManager.default.contentsOfDirectory( - atPath: Swiftly.currentPlatform.swiftlyBinDir.path - ) - XCTAssertEqual(symlinks.sorted(), (files + [existingFileName]).sorted()) - - // Verify that any all the symlinks point to the right toolchain. - for file in files { - guard file != existingFileName else { - continue - } - let observedVersion = try self.getMockedToolchainVersion( - at: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(file) - ) - XCTAssertEqual(observedVersion, toolchain) - } - } - } - } - - /// Tests that any executables that already exist in SWIFTLY_BIN_DIR. - func testExistingExecutablesNotOverwritten() async throws { - try await self.withMockedHome(homeName: Self.homeName, toolchains: []) { - let existingExecutables = ["a", "b", "c"] - let existingText = "existing" - for fileName in existingExecutables { - let existingExecutableURL = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(fileName) - let data = Data(existingText.utf8) - try data.write(to: existingExecutableURL) - } - - let toolchain = ToolchainVersion(major: 7, minor: 2, patch: 3) - try await self.installMockedToolchain( - toolchain: toolchain, - executables: ["a", "b", "c", "d", "e"] - ) - - var use = try self.parseCommand(Use.self, ["use", toolchain.name]) - let nOutput = try await use.runWithMockedIO(input: ["n"]) + var useEmpty = try self.parseCommand(Use.self, ["use", "-g"]) + var output = try await useEmpty.runWithMockedIO() - for exec in existingExecutables { - // Ensure we were prompted for each existing executable. - XCTAssert(nOutput.contains(where: { $0.contains(exec) })) - - // Ensure files were not overwritten. - let existingExecutableURL = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(exec) - let contents = try String(contentsOf: existingExecutableURL, encoding: .utf8) - XCTAssertEqual(contents, existingText) - } - - let nConfig = try Config.load() - XCTAssertEqual(nConfig.inUse, nil) - - let yOutput = try await use.runWithMockedIO(input: ["y"]) + XCTAssert(output.contains(where: { $0.contains(String(describing: toolchain)) })) - // Ensure we were prompted for each existing executable. - for exec in existingExecutables { - XCTAssert(yOutput.contains(where: { $0.contains(exec) })) + useEmpty = try self.parseCommand(Use.self, ["use", "-g", "--print-location"]) + output = try await useEmpty.runWithMockedIO() - // Ensure files were overwritten. - let existingExecutableURL = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(exec) - let contents = try String(contentsOf: existingExecutableURL, encoding: .utf8) - XCTAssertNotEqual(contents, existingText) + XCTAssert(output.contains(where: { $0.contains(Swiftly.currentPlatform.findToolchainLocation(toolchain).path) })) } - - let yConfig = try Config.load() - XCTAssertEqual(yConfig.inUse, toolchain) } } - /// Tests that running a use command without an argument prints the currently in-use toolchain. - func testPrintInUse() async throws { + /// Tests in-use toolchain selected by the .swift-version file. + func testSwiftVersionFile() async throws { let toolchains = [ Self.newStable, Self.newMainSnapshot, Self.newReleaseSnapshot, ] try await self.withMockedHome(homeName: Self.homeName, toolchains: Set(toolchains)) { - for toolchain in toolchains { - var use = try self.parseCommand(Use.self, ["use", toolchain.name]) - try await use.run() - - var useEmpty = try self.parseCommand(Use.self, ["use"]) - let output = try await useEmpty.runWithMockedIO() - - XCTAssert(output.contains(where: { $0.contains(String(describing: toolchain)) })) - } + let versionFile = URL(fileURLWithPath: FileManager.default.currentDirectoryPath).appendingPathComponent(".swift-version") + + // GIVEN: a directory with a swift version file that selects a particular toolchain + try Self.newStable.name.write(to: versionFile, atomically: true, encoding: .utf8) + // WHEN: checking which toolchain is selected with the use command + var useCmd = try self.parseCommand(Use.self, ["use"]) + var output = try await useCmd.runWithMockedIO() + // THEN: the output shows this toolchain is in use with this working directory + XCTAssert(output.contains(where: { $0.contains(Self.newStable.name) })) + + // GIVEN: a directory with a swift version file that selects a particular toolchain + // WHEN: using another toolchain version + useCmd = try self.parseCommand(Use.self, ["use", Self.newMainSnapshot.name]) + output = try await useCmd.runWithMockedIO() + // THEN: the swift version file is updated to this toolchain version + var versionFileContents = try String(contentsOf: versionFile, encoding: .utf8) + XCTAssertEqual(Self.newMainSnapshot.name, versionFileContents) + // THEN: the use command reports this toolchain to be in use + XCTAssert(output.contains(where: { $0.contains(Self.newMainSnapshot.name) })) + + // GIVEN: a directory with no swift version file at the top of a git repository + try FileManager.default.removeItem(atPath: versionFile.path) + let gitDir = URL(fileURLWithPath: FileManager.default.currentDirectoryPath).appendingPathComponent(".git") + try FileManager.default.createDirectory(atPath: gitDir.path, withIntermediateDirectories: false) + // WHEN: using a toolchain version + useCmd = try self.parseCommand(Use.self, ["use", Self.newReleaseSnapshot.name]) + try await useCmd.run() + // THEN: a swift version file is created + XCTAssert(FileManager.default.fileExists(atPath: versionFile.path)) + // THEN: the version file contains the specified version + versionFileContents = try String(contentsOf: versionFile, encoding: .utf8) + XCTAssertEqual(Self.newReleaseSnapshot.name, versionFileContents) + + // GIVEN: a directory with a swift version file at the top of a git repository + try "1.2.3".write(to: versionFile, atomically: true, encoding: .utf8) + // WHEN: using with a toolchain selector that can select more than one version, but matches one of the installed toolchains + let broadSelector = ToolchainSelector.stable(major: Self.newStable.asStableRelease!.major, minor: nil, patch: nil) + useCmd = try self.parseCommand(Use.self, ["use", broadSelector.description]) + try await useCmd.run() + // THEN: the swift version file is set to the specific toolchain version that was installed including major, minor, and patch + versionFileContents = try String(contentsOf: versionFile, encoding: .utf8) + XCTAssertEqual(Self.newStable.name, versionFileContents) } } } From ce0d27a7892df4d251ed0dcb9e726421817baac9 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 5 Sep 2024 15:09:42 -0400 Subject: [PATCH 08/45] Rewrite the select toolchain function with a type for the selection result and centralized error messages --- Sources/Swiftly/Proxy.swift | 23 ++++------ Sources/Swiftly/Use.swift | 90 +++++++++++++++++++------------------ 2 files changed, 54 insertions(+), 59 deletions(-) diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 8d9c9c60..45b3dcea 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -36,23 +36,16 @@ public enum Proxy { } let config = try Config.load() - let toolchain: ToolchainVersion - if let (selectedToolchain, versionFile, selector) = try swiftToolchainSelection(config: config) { - guard let selectedToolchain = selectedToolchain else { - if let versionFile = versionFile { - throw if let selector = selector { - Error(message: "No installed swift toolchain matches the version \(selector) from \(versionFile). You can try installing one with `swiftly install \(selector)`.") - } else { - Error(message: "Swift version file is malformed and cannot be used to select a swift toolchain: \(versionFile)") - } - } - fatalError("error in toolchain selection logic") - } + let (toolchain, result) = selectToolchain(config: config) - toolchain = selectedToolchain - } else { - throw Error(message: "No swift toolchain could be determined either from a .swift-version file, or the default. You can try using `swiftly use ` to set it.") + // Abort on any errors relating to swift version files + if case let .swiftVersionFile(_, error) = result, let error = error { + throw error + } + + guard let toolchain = toolchain else { + throw Error(message: "No swift toolchain could be selected from either from a .swift-version file, or the default. You can try using `swiftly install ` to install one.") } try await Swiftly.currentPlatform.proxy(toolchain, binName, Array(CommandLine.arguments[1...])) diff --git a/Sources/Swiftly/Use.swift b/Sources/Swiftly/Use.swift index 740ca0ab..ced9e4c9 100644 --- a/Sources/Swiftly/Use.swift +++ b/Sources/Swiftly/Use.swift @@ -55,21 +55,16 @@ internal struct Use: SwiftlyCommand { // This is the bare use command where we print the selected toolchain version (or the path to it) guard let toolchain = self.toolchain else { - let selected = try swiftToolchainSelection(config: config, globalDefault: self.globalDefault) + let (selectedVersion, result) = selectToolchain(config: config, globalDefault: self.globalDefault) - guard let selected = selected else { - // No toolchain selected, so we just output nothing - return - } - - let (selectedVersion, versionFile, selector) = selected - - if let versionFile = versionFile, selector == nil { - throw Error(message: "Swift version file is malformed and cannot be used to select a swift toolchain: \(versionFile)") + // Abort on any errors with the swift version files + if case let .swiftVersionFile(_, error) = result, let error = error { + throw error } guard let selectedVersion = selectedVersion else { - fatalError("error in toolchain selection logic") + // Return with nothing if there's no toolchain that is selected + return } if self.printLocation { @@ -78,12 +73,17 @@ internal struct Use: SwiftlyCommand { return } - if let versionFile = versionFile { - SwiftlyCore.print("\(selectedVersion) (\(versionFile.path))") - } else { - SwiftlyCore.print("\(selectedVersion) (default)") + var message = "\(selectedVersion)" + + switch result { + case let .swiftVersionFile(versionFile, _): + message += " (\(versionFile.path))" + case .globalDefault: + message += " (default)" } + SwiftlyCore.print(message) + return } @@ -103,16 +103,17 @@ internal struct Use: SwiftlyCommand { /// Use a toolchain. This method can modify and save the input config. internal static func execute(_ toolchain: ToolchainVersion, _ globalDefault: Bool, _ config: inout Config) async throws { - let previousToolchain = try swiftToolchainSelection(config: config, globalDefault: globalDefault) + let (selectedVersion, result) = selectToolchain(config: config, globalDefault: globalDefault) - if let (selectedVersion, _, _) = previousToolchain { + if let selectedVersion = selectedVersion { guard selectedVersion != toolchain else { SwiftlyCore.print("\(toolchain) is already in use") return } } - if let (_, versionFile, _) = previousToolchain, let versionFile = versionFile { + if case let .swiftVersionFile(versionFile, _) = result { + // We don't care in this case if there were any problems with the swift version files, just overwrite it with the new value try toolchain.name.write(to: versionFile, atomically: true, encoding: .utf8) } else if let newVersionFile = findNewVersionFile(), !globalDefault { try toolchain.name.write(to: newVersionFile, atomically: true, encoding: .utf8) @@ -122,9 +123,7 @@ internal struct Use: SwiftlyCommand { } var message = "Set the used toolchain to \(toolchain)" - if let (selectedVersion, _, _) = previousToolchain, - let selectedVersion = selectedVersion - { + if let selectedVersion = selectedVersion { message += " (was \(selectedVersion.name))" } @@ -152,24 +151,31 @@ internal struct Use: SwiftlyCommand { } } -/// Returns the currently selected swift toolchain with optional details. +public enum ToolchainSelectionResult { + case globalDefault + case swiftVersionFile(URL, Error?) +} + +/// Returns the currently selected swift toolchain, if any, with details of the selection. +/// +/// The first portion of the returned tuple is the version that was selected, which +/// can be nil if none can be selected. /// /// Selection of a toolchain can be accomplished in a number of ways. There is the /// the configuration's global default 'inUse' setting. This is the fallback selector -/// if there are no other selections. In this case the returned tuple will contain -/// only the selected toolchain version. None of the other values are provided. +/// if there are no other selections. The returned tuple will contain the default toolchain +/// version and the result will be .default. /// /// A toolchain can also be selected from a `.swift-version` file in the current -/// working directory, or an ancestor directory. The nearest version file is -/// returned as a URL if it is present, even if the file is malformed or it -/// doesn't select any of the installed toolchains. A well-formed version file -/// will additionally return the toolchain selector that it represents. Finally, -/// if that selector selects one of the installed toolchains then all three -/// values are returned. +/// working directory, or an ancestor directory. If it successfully selects a toolchain +/// then the result will be .swiftVersionFile with the URL of the file that made the +/// selection and the first item of the tuple is the selected toolchain version. /// -/// Note: if no swift version files are found at all then the return will be nil -/// -public func swiftToolchainSelection(config: Config, globalDefault: Bool = false) throws -> (ToolchainVersion?, URL?, ToolchainSelector?)? { +/// However, there are cases where the swift version file fails to select a toolchain. +/// If such a case happens then the toolchain version in the tuple will be nil, but the +/// result will be .swiftVersionFile and a detailed error about the problem. This error +/// can be thrown by the client, or ignored. +public func selectToolchain(config: Config, globalDefault: Bool = false) -> (ToolchainVersion?, ToolchainSelectionResult) { if !globalDefault { var cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) @@ -184,11 +190,11 @@ public func swiftToolchainSelection(config: Config, globalDefault: Bool = false) let contents = try? String(contentsOf: svFile, encoding: .utf8) guard let contents = contents else { - return (nil, svFile, nil) + return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file could not be read: \(svFile)"))) } guard !contents.isEmpty else { - return (nil, svFile, nil) + return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is empty: \(svFile)"))) } let selectorString = contents.replacingOccurrences(of: "\n", with: "").replacingOccurrences(of: "\r", with: "") @@ -196,27 +202,23 @@ public func swiftToolchainSelection(config: Config, globalDefault: Bool = false) do { selector = try ToolchainSelector(parsing: selectorString) } catch { - return (nil, svFile, nil) + return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is malformed: \(svFile) \(error)"))) } guard let selector = selector else { - return (nil, svFile, nil) + return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is malformed: \(svFile)"))) } guard let selectedToolchain = config.listInstalledToolchains(selector: selector).max() else { - return (nil, svFile, selector) + return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file didn't select any of the installed toolchains. You can install one with `swiftly install \(selector.description)`."))) } - return (selectedToolchain, svFile, selector) + return (selectedToolchain, .swiftVersionFile(svFile, nil)) } cwd = cwd.deletingLastPathComponent() } } - if let inUse = config.inUse { - return (inUse, nil, nil) - } - - return nil + return (config.inUse, .globalDefault) } From 66dd45985ffc88adf153ae0bf841ffd4ac62e90c Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 5 Sep 2024 15:36:59 -0400 Subject: [PATCH 09/45] Update the documentation --- .../SwiftlyDocs.docc/swiftly-cli-reference.md | 20 ++++++++--- .../GenerateDocsReference.swift | 33 +++++++++++-------- 2 files changed, 34 insertions(+), 19 deletions(-) diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 74af4da5..9ba8b475 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -23,7 +23,7 @@ swiftly [--version] [--help] Install a new toolchain. ``` -swiftly install [--use] [--token=] [--verify] [--post-install-file=] [--version] [--help] +swiftly install [--use] [--token=] [--verify|no-verify] [--post-install-file=] [--version] [--help] ``` **version:** @@ -67,7 +67,7 @@ This is useful to avoid GitHub's low rate limits. If an installation fails with an "unauthorized" status code, it likely means the rate limit has been hit. -**--verify:** +**--verify|no-verify:** *Verify the toolchain's PGP signature before proceeding with installation.* @@ -142,9 +142,19 @@ Note that listing available snapshots before 6.0 is unsupported. Set the active toolchain. If no toolchain is provided, print the currently in-use toolchain, if any. ``` -swiftly use [] [--version] [--help] +swiftly use [--print-location] [--global-default] [] [--version] [--help] ``` +**--print-location:** + +*Print the location of the in-use toolchain. This is valid only when there is no toolchain argument.* + + +**--global-default:** + +*Use the global default, ignoring any .swift-version files.* + + **toolchain:** *The toolchain to use.* @@ -293,7 +303,7 @@ The installed snapshots for a given devlopment branch can be listed by specifyin Update an installed toolchain to a newer version. ``` -swiftly update [] [--assume-yes] [--verify] [--post-install-file=] [--version] [--help] +swiftly update [] [--assume-yes] [--verify|no-verify] [--post-install-file=] [--version] [--help] ``` **toolchain:** @@ -339,7 +349,7 @@ A specific snapshot toolchain can be updated by including the date: *Disable confirmation prompts by assuming 'yes'* -**--verify:** +**--verify|no-verify:** *Verify the toolchain's PGP signature before proceeding with installation.* diff --git a/Tools/generate-docs-reference/GenerateDocsReference.swift b/Tools/generate-docs-reference/GenerateDocsReference.swift index 77cc2cde..539a3ab4 100644 --- a/Tools/generate-docs-reference/GenerateDocsReference.swift +++ b/Tools/generate-docs-reference/GenerateDocsReference.swift @@ -142,11 +142,14 @@ extension ArgumentInfoV0 { return "" } - let name: String - if let preferred = self.preferredName { - name = preferred.name + let names: [String] + + if let myNames = self.names { + names = myNames.filter { $0.kind == .long }.map(\.name) + } else if let preferred = self.preferredName { + names = [preferred.name] } else if let value = self.valueName { - name = value + names = [value] } else { return "" } @@ -157,11 +160,11 @@ extension ArgumentInfoV0 { switch self.kind { case .positional: - "<\(name)>" + "<\(names.joined(separator: "|"))>" case .option: - "--\(name)=<\(self.valueName ?? "")>" + "--\(names.joined(separator: "|"))=<\(self.valueName ?? "")>" case .flag: - "--\(name)" + "--\(names.joined(separator: "|"))" } if self.isRepeating { @@ -176,11 +179,13 @@ extension ArgumentInfoV0 { } public func identity() -> String { - let name: String - if let preferred = self.preferredName { - name = preferred.name + let names: [String] + if let myNames = self.names { + names = myNames.filter { $0.kind == .long }.map(\.name) + } else if let preferred = self.preferredName { + names = [preferred.name] } else if let value = self.valueName { - name = value + names = [value] } else { return "" } @@ -191,11 +196,11 @@ extension ArgumentInfoV0 { switch self.kind { case .positional: - "\(name)" + "\(names.joined(separator: "|"))" case .option: - "--\(name)=\\<\(self.valueName ?? "")\\>" + "--\(names.joined(separator: "|"))=\\<\(self.valueName ?? "")\\>" case .flag: - "--\(name)" + "--\(names.joined(separator: "|"))" } return inner From 2df7357799b9140324c2714bc44588cfe9a49964 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Fri, 6 Sep 2024 15:56:04 -0400 Subject: [PATCH 10/45] Create a swiftly run command Provide a run command that allows arbitrary commands to be run in the context of the selected toolchain. Provide a one-off selection mechanism with a special syntax on the run command. --- .../SwiftlyDocs.docc/swiftly-cli-reference.md | 58 +++++++ Sources/Swiftly/Install.swift | 4 +- Sources/Swiftly/Proxy.swift | 6 +- Sources/Swiftly/Run.swift | 146 ++++++++++++++++++ Sources/Swiftly/Swiftly.swift | 1 + Sources/Swiftly/Use.swift | 21 ++- Sources/SwiftlyCore/Platform.swift | 71 ++++++++- 7 files changed, 293 insertions(+), 14 deletions(-) create mode 100644 Sources/Swiftly/Run.swift diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 9ba8b475..106202f3 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -434,3 +434,61 @@ swiftly self-update [--version] [--help] +## run + +Run a command while proxying to the selected toolchain commands. + +``` +swiftly run ... [--version] [--help] +``` + +**command:** + +*Run a command while proxying to the selected toolchain commands.* + + +Run a command with a selected toolchain, so that all toolchain commands are become the default added to the system path and other common environment variables. + +You can run one of the usual toolchain commands directly: + + $ swiftly run swift build + +Or you can run another program (or script) that runs one or more toolchain commands: + + $ swiftly run make # Builds targets using clang/swiftc + $ swiftly run ./build-things.sh # Script invokes 'swift build' to create certain product binaries + +Toolchain selection is determined by swift version files `.swift-version`, with a default global as the fallback. See the `swiftly use` command for more details. + +You can also override the selection mechanisms temporarily for the duration of the command using a special syntax. An argument prefixed with a '+' will be treated as the selector. + + $ swiftly run swift build +latest + $ swiftly run swift build +5.10.1 + +The first command builds the swift package with the latest toolchain and the second selects the 5.10.1 toolchain. Note that if these aren't installed then run will fail with an error message. You can pre-install the toolchain using `swiftly install ` to ensure success. There is also a `+install` argument that will automatically download and install the toolchain if necessary. + + $ swiftly run swift build +latest +install + +If the command that you are running needs the arguments with the '+' prefixes then you can escape it by doubling the '++'. + + $ swiftly run ./myscript.sh ++abcde + +The script will receive the argument as '+abcde'. If there are multiple arguments with the '+' prefix that should be escaped you can disable the selection using a '++++' argument, which turns off any selector argument processing for subsequent arguments. This is anologous to the '--' that turns off flag and option processing for subsequent arguments in many argument parsers. + + $ swiftly run ./myscript.sh ++++ +abcde +xyz + +The script will receive the argument '+abcde' followed by '+xyz'. + + +**--version:** + +*Show the version.* + + +**--help:** + +*Show help information.* + + + + diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 4b097997..57b0806f 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -101,7 +101,7 @@ struct Install: SwiftlyCommand { } } - internal static func execute( + public static func execute( version: ToolchainVersion, _ config: inout Config, useInstalledToolchain: Bool, @@ -227,7 +227,7 @@ struct Install: SwiftlyCommand { /// Utilize the GitHub API along with the provided selector to select a toolchain for install. /// TODO: update this to use an official swift.org API - static func resolve(config: Config, selector: ToolchainSelector) async throws -> ToolchainVersion { + public static func resolve(config: Config, selector: ToolchainSelector) async throws -> ToolchainVersion { switch selector { case .latest: SwiftlyCore.print("Fetching the latest stable Swift release...") diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 45b3dcea..3deab60e 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -35,9 +35,9 @@ public enum Proxy { return } - let config = try Config.load() + var config = try Config.load() - let (toolchain, result) = selectToolchain(config: config) + let (toolchain, result) = try await selectToolchain(config: &config) // Abort on any errors relating to swift version files if case let .swiftVersionFile(_, error) = result, let error = error { @@ -49,6 +49,8 @@ public enum Proxy { } try await Swiftly.currentPlatform.proxy(toolchain, binName, Array(CommandLine.arguments[1...])) + } catch let terminated as RunProgramError { + exit(terminated.exitCode) } catch { SwiftlyCore.print("\(error)") exit(1) diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift new file mode 100644 index 00000000..8aaeac1e --- /dev/null +++ b/Sources/Swiftly/Run.swift @@ -0,0 +1,146 @@ +import ArgumentParser +import Foundation +import SwiftlyCore + +internal struct Run: SwiftlyCommand { + public static var configuration = CommandConfiguration( + abstract: "Run a command while proxying to the selected toolchain commands." + ) + + @Argument(parsing: .captureForPassthrough, help: ArgumentHelp( + "Run a command while proxying to the selected toolchain commands.", + discussion: """ + + Run a command with a selected toolchain, so that all toolchain commands \ + are become the default added to the system path and other common environment \ + variables. + + You can run one of the usual toolchain commands directly: + + $ swiftly run swift build + + Or you can run another program (or script) that runs one or more toolchain commands: + + $ swiftly run make # Builds targets using clang/swiftc + $ swiftly run ./build-things.sh # Script invokes 'swift build' to create certain product binaries + + Toolchain selection is determined by swift version files `.swift-version`, with a default global \ + as the fallback. See the `swiftly use` command for more details. + + You can also override the selection mechanisms temporarily for the duration of the command using \ + a special syntax. An argument prefixed with a '+' will be treated as the selector. + + $ swiftly run swift build +latest + $ swiftly run swift build +5.10.1 + + The first command builds the swift package with the latest toolchain and the second selects the \ + 5.10.1 toolchain. Note that if these aren't installed then run will fail with an error message. \ + You can pre-install the toolchain using `swiftly install ` to ensure success. There is \ + also a `+install` argument that will automatically download and install the toolchain if necessary. + + $ swiftly run swift build +latest +install + + If the command that you are running needs the arguments with the '+' prefixes then you can escape \ + it by doubling the '++'. + + $ swiftly run ./myscript.sh ++abcde + + The script will receive the argument as '+abcde'. If there are multiple arguments with the '+' prefix \ + that should be escaped you can disable the selection using a '++++' argument, which turns off any \ + selector argument processing for subsequent arguments. This is anologous to the '--' that turns off \ + flag and option processing for subsequent arguments in many argument parsers. + + $ swiftly run ./myscript.sh ++++ +abcde +xyz + + The script will receive the argument '+abcde' followed by '+xyz'. + """ + )) + var command: [String] + + internal mutating func run() async throws { + try validateSwiftly() + + guard self.command.count > 0 else { + throw Error(message: "Provide at least one command to run") + } + + var escapedCommand: [String] = [] + var selector: ToolchainSelector? + var install = false + var disableEscaping = false + for c in self.command { + if !disableEscaping && c == "++++" { + disableEscaping = true + continue + } + + if !disableEscaping && c.hasPrefix("++") { + escapedCommand.append("+\(String(c.dropFirst(2)))") + continue + } + + if !disableEscaping && c == "+install" { + install = true + continue + } + + if !disableEscaping && c.hasPrefix("+") { + selector = try ToolchainSelector(parsing: String(c.dropFirst())) + continue + } + + escapedCommand.append(c) + } + + var config = try Config.load() + + let toolchain: ToolchainVersion? + + if let selector = selector { + if install { + let version = try await Install.resolve(config: config, selector: selector) + let postInstallScript = try await Install.execute(version: version, &config, useInstalledToolchain: false, verifySignature: true) + if let postInstallScript = postInstallScript { + throw Error(message: """ + + There are some system dependencies that should be installed before using this toolchain. + You can run the following script as the system administrator (e.g. root) to prepare + your system: + + \(postInstallScript) + """) + } + + toolchain = version + } else { + let matchedToolchain = config.listInstalledToolchains(selector: selector).max() + guard let matchedToolchain = matchedToolchain else { + throw Error(message: "The selected toolchain \(selector.description) didn't matched any of the installed toolchains. You can install it by adding '+install' to your command, or `swiftly install \(selector.description)`") + } + + toolchain = matchedToolchain + } + } else { + let (version, result) = try await selectToolchain(config: &config, install: install) + + // Abort on any errors relating to swift version files + if case let .swiftVersionFile(_, error) = result, let error = error { + throw error + } + + toolchain = version + } + + guard let toolchain = toolchain else { + throw Error(message: "No swift toolchain could be selected from either from a .swift-version file, or the default. You can try using `swiftly install ` to install one.") + } + + do { + try await Swiftly.currentPlatform.proxy(toolchain, escapedCommand[0], [String](escapedCommand[1...])) + } catch let terminated as RunProgramError { + Foundation.exit(terminated.exitCode) + } catch { + throw error + } + } +} diff --git a/Sources/Swiftly/Swiftly.swift b/Sources/Swiftly/Swiftly.swift index 7d9789e0..1652c9a2 100644 --- a/Sources/Swiftly/Swiftly.swift +++ b/Sources/Swiftly/Swiftly.swift @@ -29,6 +29,7 @@ public struct Swiftly: SwiftlyCommand { Update.self, Init.self, SelfUpdate.self, + Run.self, ] ) diff --git a/Sources/Swiftly/Use.swift b/Sources/Swiftly/Use.swift index ced9e4c9..38066c65 100644 --- a/Sources/Swiftly/Use.swift +++ b/Sources/Swiftly/Use.swift @@ -55,7 +55,7 @@ internal struct Use: SwiftlyCommand { // This is the bare use command where we print the selected toolchain version (or the path to it) guard let toolchain = self.toolchain else { - let (selectedVersion, result) = selectToolchain(config: config, globalDefault: self.globalDefault) + let (selectedVersion, result) = try await selectToolchain(config: &config, globalDefault: self.globalDefault) // Abort on any errors with the swift version files if case let .swiftVersionFile(_, error) = result, let error = error { @@ -103,7 +103,7 @@ internal struct Use: SwiftlyCommand { /// Use a toolchain. This method can modify and save the input config. internal static func execute(_ toolchain: ToolchainVersion, _ globalDefault: Bool, _ config: inout Config) async throws { - let (selectedVersion, result) = selectToolchain(config: config, globalDefault: globalDefault) + let (selectedVersion, result) = try await selectToolchain(config: &config, globalDefault: globalDefault) if let selectedVersion = selectedVersion { guard selectedVersion != toolchain else { @@ -175,7 +175,7 @@ public enum ToolchainSelectionResult { /// If such a case happens then the toolchain version in the tuple will be nil, but the /// result will be .swiftVersionFile and a detailed error about the problem. This error /// can be thrown by the client, or ignored. -public func selectToolchain(config: Config, globalDefault: Bool = false) -> (ToolchainVersion?, ToolchainSelectionResult) { +public func selectToolchain(config: inout Config, globalDefault: Bool = false, install: Bool = false) async throws -> (ToolchainVersion?, ToolchainSelectionResult) { if !globalDefault { var cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) @@ -209,6 +209,21 @@ public func selectToolchain(config: Config, globalDefault: Bool = false) -> (Too return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is malformed: \(svFile)"))) } + if install { + let version = try await Install.resolve(config: config, selector: selector) + let postInstallScript = try await Install.execute(version: version, &config, useInstalledToolchain: false, verifySignature: true) + if let postInstallScript = postInstallScript { + throw Error(message: """ + + There are some system dependencies that should be installed before using this toolchain. + You can run the following script as the system administrator (e.g. root) to prepare + your system: + + \(postInstallScript) + """) + } + } + guard let selectedToolchain = config.listInstalledToolchains(selector: selector).max() else { return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file didn't select any of the installed toolchains. You can install one with `swiftly install \(selector.description)`."))) } diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index b7c2e99e..1ca5fe7e 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -29,6 +29,11 @@ public struct PlatformDefinition: Codable, Equatable { public static let amazonlinux2 = PlatformDefinition(name: "amazonlinux2", nameFull: "amazonlinux2", namePretty: "Amazon Linux 2") } +public struct RunProgramError: Swift.Error { + public let exitCode: Int32 + public let program: String +} + public protocol Platform { /// The platform-specific location on disk where applications are /// supposed to store their custom data. @@ -124,20 +129,62 @@ extension Platform { } #if os(macOS) || os(Linux) + /// Proxy the invocation of the provided command to the chosen toolchain. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// public func proxy(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws { - let cmd = self.findToolchainLocation(toolchain).appendingPathComponent("usr/bin/\(command)") - try self.runProgram([cmd.path] + arguments) + // The toolchain goes to the beginning of the path, and the SWIFTLY_BIN_DIR is removed from it + let tcPath = self.findToolchainLocation(toolchain).appendingPathComponent("usr/bin") + var newEnv = ProcessInfo.processInfo.environment + + // Prevent circularities with a memento environment variable + guard newEnv["SWIFTLY_PROXY_IN_PROGRESS"] == nil else { + throw Error(message: "Circular swiftly proxy invocation") + } + newEnv["SWIFTLY_PROXY_IN_PROGRESS"] = "1" + + var newPath = newEnv["PATH"] ?? "" + if !newPath.hasPrefix(tcPath.path + ":") { + newPath = ([tcPath.path] + newPath.split(separator: ":").map { String($0) }.filter { $0 != swiftlyBinDir.path }).joined(separator: ":") + } + newEnv["PATH"] = newPath + + // Remove traces of swiftly environment variables + newEnv.removeValue(forKey: "SWIFTLY_BIN_DIR") + newEnv.removeValue(forKey: "SWIFTLY_HOME_DIR") + + // Add certain common environment variables that can be used to proxy to the toolchain + newEnv["CC"] = tcPath.appendingPathComponent("clang").path + newEnv["CXX"] = tcPath.appendingPathComponent("clang++").path + + try self.runProgram([command] + arguments, env: newEnv) } - public func runProgram(_ args: String..., quiet: Bool = false) throws { - try self.runProgram([String](args), quiet: quiet) + /// Run a program. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// + public func runProgram(_ args: String..., quiet: Bool = false, env: [String: String]? = nil) throws { + try self.runProgram([String](args), quiet: quiet, env: env) } - public func runProgram(_ args: [String], quiet: Bool = false) throws { + /// Run a program. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// + public func runProgram(_ args: [String], quiet: Bool = false, env: [String: String]? = nil) throws { let process = Process() process.executableURL = URL(fileURLWithPath: "/usr/bin/env") process.arguments = args + if let env = env { + process.environment = env + } + if quiet { process.standardOutput = nil process.standardError = nil @@ -152,14 +199,24 @@ extension Platform { process.waitUntilExit() guard process.terminationStatus == 0 else { - throw Error(message: "\(args.first!) exited with non-zero status: \(process.terminationStatus)") + throw RunProgramError(exitCode: process.terminationStatus, program: args.first!) } } + /// Run a program and capture its output. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// public func runProgramOutput(_ program: String, _ args: String...) async throws -> String? { try await self.runProgramOutput(program, [String](args)) } + /// Run a program and capture its output. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// public func runProgramOutput(_ program: String, _ args: [String]) async throws -> String? { let process = Process() process.executableURL = URL(fileURLWithPath: "/usr/bin/env") @@ -181,7 +238,7 @@ extension Platform { process.waitUntilExit() guard process.terminationStatus == 0 else { - throw Error(message: "\(args.first!) exited with non-zero status: \(process.terminationStatus)") + throw RunProgramError(exitCode: process.terminationStatus, program: args.first!) } if let outData = outData { From 8976bba8a60476524b30bc4b09e5a3a910e3c396 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 7 Sep 2024 07:14:29 -0400 Subject: [PATCH 11/45] Fix empty command case with a single ++++ --- Sources/Swiftly/Install.swift | 2 +- Sources/Swiftly/Run.swift | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 57b0806f..a15a931e 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -108,7 +108,7 @@ struct Install: SwiftlyCommand { verifySignature: Bool ) async throws -> String? { guard !config.installedToolchains.contains(version) else { - SwiftlyCore.print("\(version) is already installed, exiting.") + SwiftlyCore.print("\(version) is already installed.") return nil } diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift index 8aaeac1e..7b31d6b0 100644 --- a/Sources/Swiftly/Run.swift +++ b/Sources/Swiftly/Run.swift @@ -60,10 +60,6 @@ internal struct Run: SwiftlyCommand { internal mutating func run() async throws { try validateSwiftly() - guard self.command.count > 0 else { - throw Error(message: "Provide at least one command to run") - } - var escapedCommand: [String] = [] var selector: ToolchainSelector? var install = false @@ -92,6 +88,10 @@ internal struct Run: SwiftlyCommand { escapedCommand.append(c) } + guard escapedCommand.count > 0 else { + throw Error(message: "Provide at least one command to run") + } + var config = try Config.load() let toolchain: ToolchainVersion? From 3caab6679634299a841221a6129f6ee39b743785 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Mon, 9 Sep 2024 11:14:05 -0400 Subject: [PATCH 12/45] Write run command and proxy tests --- Sources/Swiftly/Run.swift | 87 +++++++++++++++---------- Sources/SwiftlyCore/Platform.swift | 45 ++++++++----- Tests/SwiftlyTests/RunTests.swift | 101 +++++++++++++++++++++++++++++ 3 files changed, 181 insertions(+), 52 deletions(-) create mode 100644 Tests/SwiftlyTests/RunTests.swift diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift index 7b31d6b0..12df7634 100644 --- a/Sources/Swiftly/Run.swift +++ b/Sources/Swiftly/Run.swift @@ -46,11 +46,11 @@ internal struct Run: SwiftlyCommand { $ swiftly run ./myscript.sh ++abcde The script will receive the argument as '+abcde'. If there are multiple arguments with the '+' prefix \ - that should be escaped you can disable the selection using a '++++' argument, which turns off any \ + that should be escaped you can disable the selection using a '++' argument, which turns off any \ selector argument processing for subsequent arguments. This is anologous to the '--' that turns off \ flag and option processing for subsequent arguments in many argument parsers. - $ swiftly run ./myscript.sh ++++ +abcde +xyz + $ swiftly run ./myscript.sh ++ +abcde +xyz The script will receive the argument '+abcde' followed by '+xyz'. """ @@ -60,40 +60,10 @@ internal struct Run: SwiftlyCommand { internal mutating func run() async throws { try validateSwiftly() - var escapedCommand: [String] = [] - var selector: ToolchainSelector? - var install = false - var disableEscaping = false - for c in self.command { - if !disableEscaping && c == "++++" { - disableEscaping = true - continue - } - - if !disableEscaping && c.hasPrefix("++") { - escapedCommand.append("+\(String(c.dropFirst(2)))") - continue - } - - if !disableEscaping && c == "+install" { - install = true - continue - } - - if !disableEscaping && c.hasPrefix("+") { - selector = try ToolchainSelector(parsing: String(c.dropFirst())) - continue - } - - escapedCommand.append(c) - } - - guard escapedCommand.count > 0 else { - throw Error(message: "Provide at least one command to run") - } - var config = try Config.load() + let (command, selector, install) = try extractProxyArguments(command: self.command) + let toolchain: ToolchainVersion? if let selector = selector { @@ -115,7 +85,7 @@ internal struct Run: SwiftlyCommand { } else { let matchedToolchain = config.listInstalledToolchains(selector: selector).max() guard let matchedToolchain = matchedToolchain else { - throw Error(message: "The selected toolchain \(selector.description) didn't matched any of the installed toolchains. You can install it by adding '+install' to your command, or `swiftly install \(selector.description)`") + throw Error(message: "The selected toolchain \(selector.description) didn't match any of the installed toolchains. You can install it by adding '+install' to your command, or `swiftly install \(selector.description)`") } toolchain = matchedToolchain @@ -136,7 +106,16 @@ internal struct Run: SwiftlyCommand { } do { - try await Swiftly.currentPlatform.proxy(toolchain, escapedCommand[0], [String](escapedCommand[1...])) + if let outputHandler = SwiftlyCore.outputHandler { + if let output = try await Swiftly.currentPlatform.proxyOutput(toolchain, command[0], [String](command[1...])) { + for line in output.split(separator: "\n") { + outputHandler.handleOutputLine(String(line)) + } + } + return + } + + try await Swiftly.currentPlatform.proxy(toolchain, command[0], [String](command[1...])) } catch let terminated as RunProgramError { Foundation.exit(terminated.exitCode) } catch { @@ -144,3 +123,39 @@ internal struct Run: SwiftlyCommand { } } } + +public func extractProxyArguments(command: [String]) throws -> (command: [String], selector: ToolchainSelector?, install: Bool) { + var args: (command: [String], selector: ToolchainSelector?, install: Bool) = (command: [], nil, false) + + var disableEscaping = false + + for c in command { + if !disableEscaping && c == "++" { + disableEscaping = true + continue + } + + if !disableEscaping && c.hasPrefix("++") { + args.command.append("+\(String(c.dropFirst(2)))") + continue + } + + if !disableEscaping && c == "+install" { + args.install = true + continue + } + + if !disableEscaping && c.hasPrefix("+") { + args.selector = try ToolchainSelector(parsing: String(c.dropFirst())) + continue + } + + args.command.append(c) + } + + guard args.command.count > 0 else { + throw Error(message: "Provide at least one command to run.") + } + + return args +} diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index 1ca5fe7e..9042a8c4 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -129,13 +129,7 @@ extension Platform { } #if os(macOS) || os(Linux) - /// Proxy the invocation of the provided command to the chosen toolchain. - /// - /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with - /// the exit code and program information. - /// - public func proxy(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws { - // The toolchain goes to the beginning of the path, and the SWIFTLY_BIN_DIR is removed from it + internal func proxyEnv(_ toolchain: ToolchainVersion) throws -> [String: String] { let tcPath = self.findToolchainLocation(toolchain).appendingPathComponent("usr/bin") var newEnv = ProcessInfo.processInfo.environment @@ -145,21 +139,36 @@ extension Platform { } newEnv["SWIFTLY_PROXY_IN_PROGRESS"] = "1" + // The toolchain goes to the beginning of the PATH var newPath = newEnv["PATH"] ?? "" if !newPath.hasPrefix(tcPath.path + ":") { - newPath = ([tcPath.path] + newPath.split(separator: ":").map { String($0) }.filter { $0 != swiftlyBinDir.path }).joined(separator: ":") + newPath = ([tcPath.path] + newPath.split(separator: ":").map { String($0) }).joined(separator: ":") } newEnv["PATH"] = newPath - // Remove traces of swiftly environment variables - newEnv.removeValue(forKey: "SWIFTLY_BIN_DIR") - newEnv.removeValue(forKey: "SWIFTLY_HOME_DIR") - // Add certain common environment variables that can be used to proxy to the toolchain newEnv["CC"] = tcPath.appendingPathComponent("clang").path newEnv["CXX"] = tcPath.appendingPathComponent("clang++").path - try self.runProgram([command] + arguments, env: newEnv) + return newEnv + } + + /// Proxy the invocation of the provided command to the chosen toolchain. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// + public func proxy(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws { + try self.runProgram([command] + arguments, env: self.proxyEnv(toolchain)) + } + + /// Proxy the invocation of the provided command to the chosen toolchain and capture the output. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// + public func proxyOutput(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws -> String? { + try await self.runProgramOutput(command, arguments, env: self.proxyEnv(toolchain)) } /// Run a program. @@ -208,8 +217,8 @@ extension Platform { /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with /// the exit code and program information. /// - public func runProgramOutput(_ program: String, _ args: String...) async throws -> String? { - try await self.runProgramOutput(program, [String](args)) + public func runProgramOutput(_ program: String, _ args: String..., env: [String: String]? = nil) async throws -> String? { + try await self.runProgramOutput(program, [String](args), env: env) } /// Run a program and capture its output. @@ -217,11 +226,15 @@ extension Platform { /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with /// the exit code and program information. /// - public func runProgramOutput(_ program: String, _ args: [String]) async throws -> String? { + public func runProgramOutput(_ program: String, _ args: [String], env: [String: String]? = nil) async throws -> String? { let process = Process() process.executableURL = URL(fileURLWithPath: "/usr/bin/env") process.arguments = [program] + args + if let env = env { + process.environment = env + } + let outPipe = Pipe() process.standardInput = FileHandle.nullDevice process.standardError = FileHandle.nullDevice diff --git a/Tests/SwiftlyTests/RunTests.swift b/Tests/SwiftlyTests/RunTests.swift new file mode 100644 index 00000000..ce462700 --- /dev/null +++ b/Tests/SwiftlyTests/RunTests.swift @@ -0,0 +1,101 @@ +import Foundation +@testable import Swiftly +@testable import SwiftlyCore +import XCTest + +final class RunTests: SwiftlyTests { + static let homeName = "runTests" + + /// Tests that the `run` command can switch between installed toolchains. + func testRunSelection() async throws { + try await self.withMockedHome(homeName: Self.homeName, toolchains: Self.allToolchains) { + // GIVEN: a set of installed toolchains + // WHEN: invoking the run command with a selector argument for that toolchain + var run = try self.parseCommand(Run.self, ["run", "swift", "--version", "+\(Self.newStable.name)"]) + var output = try await run.runWithMockedIO() + // THEN: the output confirms that it ran with the selected toolchain + XCTAssert(output.contains(Self.newStable.name)) + + // GIVEN: a set of installed toolchains and one is selected with a .swift-version file + let versionFile = URL(fileURLWithPath: FileManager.default.currentDirectoryPath).appendingPathComponent(".swift-version") + try Self.oldStable.name.write(to: versionFile, atomically: true, encoding: .utf8) + // WHEN: invoking the run command without any selector arguments for toolchains + run = try self.parseCommand(Run.self, ["run", "swift", "--version"]) + output = try await run.runWithMockedIO() + // THEN: the output confirms that it ran with the selected toolchain + XCTAssert(output.contains(Self.oldStable.name)) + + // GIVEN: a set of installed toolchains + // WHEN: invoking the run command with a selector argument for a toolchain that isn't installed + run = try self.parseCommand(Run.self, ["run", "swift", "+1.2.3", "--version"]) + do { + try await run.run() + XCTAssert(false) + } catch let e as Error { + XCTAssert(e.message.contains("didn't match any of the installed toolchains")) + } + // THEN: an error is shown because there is no matching toolchain that is installed + } + } + + /// Tests the `run` command verifying that the environment is as expected + func testRunEnvironment() async throws { + try await self.withMockedHome(homeName: Self.homeName, toolchains: Self.allToolchains) { + // The toolchains directory should be the fist entry on the path + var run = try self.parseCommand(Run.self, ["run", try await Swiftly.currentPlatform.getShell(), "-c", "echo $PATH"]) + var output = try await run.runWithMockedIO() + XCTAssert(output[0].contains(Swiftly.currentPlatform.swiftlyToolchainsDir.path)) + + // The CC and CXX variables should be set to clang/clang++ in the toolchains + run = try self.parseCommand(Run.self, ["run", try await Swiftly.currentPlatform.getShell(), "-c", "echo $CC; echo $CXX"]) + output = try await run.runWithMockedIO() + XCTAssert(output[0].hasPrefix(Swiftly.currentPlatform.swiftlyToolchainsDir.path)) + XCTAssert(output[0].hasSuffix("clang")) + XCTAssert(output[1].hasPrefix(Swiftly.currentPlatform.swiftlyToolchainsDir.path)) + XCTAssert(output[1].hasSuffix("clang++")) + } + } + + /// Tests the extraction of proxy arguments from the run command arguments. + func testExtractProxyArguments() throws { + var (command, selector, install) = try extractProxyArguments(command: ["swift", "build"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(nil, selector) + + (command, selector, install) = try extractProxyArguments(command: ["swift", "+1.2.3", "build"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(try! ToolchainSelector(parsing: "1.2.3"), selector) + + (command, selector, install) = try extractProxyArguments(command: ["swift", "build", "+latest"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(try! ToolchainSelector(parsing: "latest"), selector) + + (command, selector, install) = try extractProxyArguments(command: ["+5.6", "swift", "build"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(try! ToolchainSelector(parsing: "5.6"), selector) + + (command, selector, install) = try extractProxyArguments(command: ["swift", "++1.2.3", "build"]) + XCTAssertEqual(["swift", "+1.2.3", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(nil, selector) + + (command, selector, install) = try extractProxyArguments(command: ["swift", "++", "+1.2.3", "build"]) + XCTAssertEqual(["swift", "+1.2.3", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(nil, selector) + + do { + let _ = try extractProxyArguments(command: ["+1.2.3"]) + XCTAssert(false) + } catch {} + + do { + let _ = try extractProxyArguments(command: []) + XCTAssert(false) + } catch {} + } +} From 31f43272df012809020c4d0544139834613d65c7 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Mon, 9 Sep 2024 11:19:52 -0400 Subject: [PATCH 13/45] Regenerate the cli reference documentation --- Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 106202f3..9edcdc83 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -473,9 +473,9 @@ If the command that you are running needs the arguments with the '+' prefixes th $ swiftly run ./myscript.sh ++abcde -The script will receive the argument as '+abcde'. If there are multiple arguments with the '+' prefix that should be escaped you can disable the selection using a '++++' argument, which turns off any selector argument processing for subsequent arguments. This is anologous to the '--' that turns off flag and option processing for subsequent arguments in many argument parsers. +The script will receive the argument as '+abcde'. If there are multiple arguments with the '+' prefix that should be escaped you can disable the selection using a '++' argument, which turns off any selector argument processing for subsequent arguments. This is anologous to the '--' that turns off flag and option processing for subsequent arguments in many argument parsers. - $ swiftly run ./myscript.sh ++++ +abcde +xyz + $ swiftly run ./myscript.sh ++ +abcde +xyz The script will receive the argument '+abcde' followed by '+xyz'. From 16caf80730f9bbf2d73b352eef111816d2a7444c Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Mon, 9 Sep 2024 11:54:11 -0400 Subject: [PATCH 14/45] Fix design document discrepancies and add install proxy argument tests --- DESIGN.md | 2 +- Tests/SwiftlyTests/RunTests.swift | 10 ++++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/DESIGN.md b/DESIGN.md index e3aa5efb..e6a814c9 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -324,7 +324,7 @@ Swiftly adjusts certain environment variables, such as prefixing the PATH to the swiftly run swift build +5.10.1 # Runs swift build with the 5.10.1 toolchain ``` -A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++++' is ignored entirely, but any additional arguments are sent directly to the command without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. +A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++' is ignored entirely, and any additional arguments are sent directly to the command without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. If the selected toolchain is not installed then swiftly will exit with a message indicating that you need to run `swiftly install x.y.z` to install it. However, if you enter a special `+install` token then swiftly will automatically download and install the toolchain if it isn't already present. diff --git a/Tests/SwiftlyTests/RunTests.swift b/Tests/SwiftlyTests/RunTests.swift index ce462700..7b92c38b 100644 --- a/Tests/SwiftlyTests/RunTests.swift +++ b/Tests/SwiftlyTests/RunTests.swift @@ -97,5 +97,15 @@ final class RunTests: SwiftlyTests { let _ = try extractProxyArguments(command: []) XCTAssert(false) } catch {} + + (command, selector, install) = try extractProxyArguments(command: ["swift", "+1.2.3", "+install", "build"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(true, install) + XCTAssertEqual(try! ToolchainSelector(parsing: "1.2.3"), selector) + + (command, selector, install) = try extractProxyArguments(command: ["swift", "+install", "build"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(true, install) + XCTAssertEqual(nil, selector) } } From b2aa16570bd1acf60583d612b049ed775fb7544c Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 10 Sep 2024 06:51:55 -0400 Subject: [PATCH 15/45] Update the list command to decorate default, and in-use toolchains --- Sources/Swiftly/List.swift | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/Sources/Swiftly/List.swift b/Sources/Swiftly/List.swift index 285c7933..1757532e 100644 --- a/Sources/Swiftly/List.swift +++ b/Sources/Swiftly/List.swift @@ -39,16 +39,19 @@ struct List: SwiftlyCommand { try ToolchainSelector(parsing: input) } - let config = try Config.load() + var config = try Config.load() let toolchains = config.listInstalledToolchains(selector: selector).sorted { $0 > $1 } - let activeToolchain = config.inUse + let (inUse, _) = try await selectToolchain(config: &config) let printToolchain = { (toolchain: ToolchainVersion) in var message = "\(toolchain)" - if toolchain == activeToolchain { + if let inUse = inUse, toolchain == inUse { message += " (in use)" } + if toolchain == config.inUse { + message += " (default)" + } SwiftlyCore.print(message) } From 7504dd3247a85a69664289fbd4f9fdf01f2187a3 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 10 Sep 2024 07:01:39 -0400 Subject: [PATCH 16/45] Update list tests to check for in use and default labels --- Tests/SwiftlyTests/ListTests.swift | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Tests/SwiftlyTests/ListTests.swift b/Tests/SwiftlyTests/ListTests.swift index ae2c1d37..6724ab97 100644 --- a/Tests/SwiftlyTests/ListTests.swift +++ b/Tests/SwiftlyTests/ListTests.swift @@ -128,7 +128,7 @@ final class ListTests: SwiftlyTests { let output = try await list.runWithMockedIO() let inUse = output.filter { $0.contains("in use") } - XCTAssertEqual(inUse, ["\(toolchain) (in use)"]) + XCTAssertEqual(inUse, ["\(toolchain) (in use) (default)"]) } try await self.runListTest { From 0e7b6616285379961eda6975ed7d02aa570ddb9b Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 11 Sep 2024 13:58:31 -0400 Subject: [PATCH 17/45] Make the version argument optional in the install subcommand With no arguments the install subcommand will install the currently selected toolchain from the `.swift-version` files. --- .../SwiftlyDocs.docc/swiftly-cli-reference.md | 6 +++- Sources/Swiftly/Install.swift | 28 +++++++++++++++++-- Sources/Swiftly/Proxy.swift | 2 +- Sources/Swiftly/Run.swift | 2 +- Sources/Swiftly/Use.swift | 20 ++++++------- 5 files changed, 43 insertions(+), 15 deletions(-) diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 9edcdc83..887435e1 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -23,7 +23,7 @@ swiftly [--version] [--help] Install a new toolchain. ``` -swiftly install [--use] [--token=] [--verify|no-verify] [--post-install-file=] [--version] [--help] +swiftly install [] [--use] [--token=] [--verify|no-verify] [--post-install-file=] [--version] [--help] ``` **version:** @@ -53,6 +53,10 @@ Likewise, the latest snapshot associated with a given development branch can be $ swiftly install 5.7-snapshot $ swiftly install main-snapshot + Install whatever toolchain is current selected, such as a .swift-version file: + + $ swiftly install + **--use:** diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index a15a931e..b9aa1fba 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -39,9 +39,13 @@ struct Install: SwiftlyCommand { $ swiftly install 5.7-snapshot $ swiftly install main-snapshot + + Install whatever toolchain is current selected, such as a .swift-version file: + + $ swiftly install """ )) - var version: String + var version: String? @Flag(name: .shortAndLong, help: "Mark the newly installed toolchain as in-use.") var use: Bool = false @@ -74,8 +78,28 @@ struct Install: SwiftlyCommand { mutating func run() async throws { try validateSwiftly() - let selector = try ToolchainSelector(parsing: self.version) var config = try Config.load() + + var selector: ToolchainSelector + + if let version = self.version { + selector = try ToolchainSelector(parsing: version) + } else { + if case let (_, result) = try await selectToolchain(config: &config), + case let .swiftVersionFile(_, sel, error) = result + { + if let sel = sel { + selector = sel + } else if let error = error { + throw error + } else { + throw Error(message: "Internal error selecting toolchain to install.") + } + } else { + throw Error(message: "There is no toolchain selected from a .swift-version file to install.") + } + } + SwiftlyCore.httpClient.githubToken = self.token let toolchainVersion = try await Self.resolve(config: config, selector: selector) let postInstallScript = try await Self.execute( diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 3deab60e..3289a8ab 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -40,7 +40,7 @@ public enum Proxy { let (toolchain, result) = try await selectToolchain(config: &config) // Abort on any errors relating to swift version files - if case let .swiftVersionFile(_, error) = result, let error = error { + if case let .swiftVersionFile(_, _, error) = result, let error = error { throw error } diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift index 12df7634..f165b39b 100644 --- a/Sources/Swiftly/Run.swift +++ b/Sources/Swiftly/Run.swift @@ -94,7 +94,7 @@ internal struct Run: SwiftlyCommand { let (version, result) = try await selectToolchain(config: &config, install: install) // Abort on any errors relating to swift version files - if case let .swiftVersionFile(_, error) = result, let error = error { + if case let .swiftVersionFile(_, _, error) = result, let error = error { throw error } diff --git a/Sources/Swiftly/Use.swift b/Sources/Swiftly/Use.swift index 38066c65..d77aea57 100644 --- a/Sources/Swiftly/Use.swift +++ b/Sources/Swiftly/Use.swift @@ -58,7 +58,7 @@ internal struct Use: SwiftlyCommand { let (selectedVersion, result) = try await selectToolchain(config: &config, globalDefault: self.globalDefault) // Abort on any errors with the swift version files - if case let .swiftVersionFile(_, error) = result, let error = error { + if case let .swiftVersionFile(_, _, error) = result, let error = error { throw error } @@ -76,7 +76,7 @@ internal struct Use: SwiftlyCommand { var message = "\(selectedVersion)" switch result { - case let .swiftVersionFile(versionFile, _): + case let .swiftVersionFile(versionFile, _, _): message += " (\(versionFile.path))" case .globalDefault: message += " (default)" @@ -112,7 +112,7 @@ internal struct Use: SwiftlyCommand { } } - if case let .swiftVersionFile(versionFile, _) = result { + if case let .swiftVersionFile(versionFile, _, _) = result { // We don't care in this case if there were any problems with the swift version files, just overwrite it with the new value try toolchain.name.write(to: versionFile, atomically: true, encoding: .utf8) } else if let newVersionFile = findNewVersionFile(), !globalDefault { @@ -153,7 +153,7 @@ internal struct Use: SwiftlyCommand { public enum ToolchainSelectionResult { case globalDefault - case swiftVersionFile(URL, Error?) + case swiftVersionFile(URL, ToolchainSelector?, Error?) } /// Returns the currently selected swift toolchain, if any, with details of the selection. @@ -190,11 +190,11 @@ public func selectToolchain(config: inout Config, globalDefault: Bool = false, i let contents = try? String(contentsOf: svFile, encoding: .utf8) guard let contents = contents else { - return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file could not be read: \(svFile)"))) + return (nil, .swiftVersionFile(svFile, nil, Error(message: "The swift version file could not be read: \(svFile)"))) } guard !contents.isEmpty else { - return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is empty: \(svFile)"))) + return (nil, .swiftVersionFile(svFile, nil, Error(message: "The swift version file is empty: \(svFile)"))) } let selectorString = contents.replacingOccurrences(of: "\n", with: "").replacingOccurrences(of: "\r", with: "") @@ -202,11 +202,11 @@ public func selectToolchain(config: inout Config, globalDefault: Bool = false, i do { selector = try ToolchainSelector(parsing: selectorString) } catch { - return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is malformed: \(svFile) \(error)"))) + return (nil, .swiftVersionFile(svFile, nil, Error(message: "The swift version file is malformed: \(svFile) \(error)"))) } guard let selector = selector else { - return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is malformed: \(svFile)"))) + return (nil, .swiftVersionFile(svFile, nil, Error(message: "The swift version file is malformed: \(svFile)"))) } if install { @@ -225,10 +225,10 @@ public func selectToolchain(config: inout Config, globalDefault: Bool = false, i } guard let selectedToolchain = config.listInstalledToolchains(selector: selector).max() else { - return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file didn't select any of the installed toolchains. You can install one with `swiftly install \(selector.description)`."))) + return (nil, .swiftVersionFile(svFile, selector, Error(message: "The swift version file didn't select any of the installed toolchains. You can install one with `swiftly install \(selector.description)`."))) } - return (selectedToolchain, .swiftVersionFile(svFile, nil)) + return (selectedToolchain, .swiftVersionFile(svFile, selector, nil)) } cwd = cwd.deletingLastPathComponent() From 43d620a45388439ca0890679d50c35fe382dccfc Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 12 Sep 2024 15:50:28 -0400 Subject: [PATCH 18/45] Fix case of empty bin directory when checking for overwrite --- Sources/Swiftly/Init.swift | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index 8b89438d..a4b7547f 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -56,7 +56,7 @@ internal struct Init: SwiftlyCommand { // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. let swiftlyBinDir = Swiftly.currentPlatform.swiftlyBinDir - let swiftlyBinDirContents = try FileManager.default.contentsOfDirectory(atPath: swiftlyBinDir.path) + let swiftlyBinDirContents = (try? FileManager.default.contentsOfDirectory(atPath: swiftlyBinDir.path)) ?? [String]() let willBeOverwritten = Set(proxyList + ["swiftly"]).intersection(swiftlyBinDirContents) if !willBeOverwritten.isEmpty && !overwrite { SwiftlyCore.print("The following existing executables will be overwritten:") From 2e3c59dd8385ba00dee69f580e58ce3005c9c71a Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Fri, 20 Sep 2024 10:48:38 -0400 Subject: [PATCH 19/45] Remove +install selector option from swift run in favour of regular `swiftly install`. Guard automatic creation of .swift-version file from `swiftly use` around a prompt overridable using an `--assume-yes`. Minor cleanup --- DESIGN.md | 10 ++--- .../SwiftlyDocs.docc/swiftly-cli-reference.md | 13 +++--- Sources/Swiftly/Install.swift | 2 +- Sources/Swiftly/Run.swift | 45 +++++-------------- Sources/Swiftly/Uninstall.swift | 2 +- Sources/Swiftly/Use.swift | 38 +++++++--------- Sources/SwiftlyCore/Platform.swift | 2 +- Tests/SwiftlyTests/RunTests.swift | 24 ++++------ 8 files changed, 51 insertions(+), 85 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index e6a814c9..97921380 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -326,14 +326,14 @@ swiftly run swift build +5.10.1 # Runs swift build with the 5.10.1 toolchain A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++' is ignored entirely, and any additional arguments are sent directly to the command without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. -If the selected toolchain is not installed then swiftly will exit with a message indicating that you need to run `swiftly install x.y.z` to install it. However, if you enter a special `+install` token then swiftly will automatically download and install the toolchain if it isn't already present. +If the selected toolchain is not installed then swiftly will exit with a message indicating that you need to run `swiftly install x.y.z` to install it. ``` -# Download and install the latest main snapshot toolchain and run 'swift build' to build the package with it. -swiftly run swift build +main-snapshot +install +# Use the latest main snapshot toolchain and run 'swift build' to build the package with it. +swiftly run swift build +main-snapshot -# Generate makefiles with the latest released Swift toolchain, download and install it if necessary -swiftly run +latest +install cmake -G "Unix Makefile" +# Generate makefiles with the latest released Swift toolchain +swiftly run +latest cmake -G "Unix Makefile" swiftly run +latest make ``` diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 887435e1..7db3879d 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -143,10 +143,10 @@ Note that listing available snapshots before 6.0 is unsupported. ## use -Set the active toolchain. If no toolchain is provided, print the currently in-use toolchain, if any. +Set the in-use toolchain. If no toolchain is provided, print the currently in-use toolchain, if any. ``` -swiftly use [--print-location] [--global-default] [] [--version] [--help] +swiftly use [--print-location] [--global-default] [--assume-yes] [] [--version] [--help] ``` **--print-location:** @@ -159,6 +159,11 @@ swiftly use [--print-location] [--global-default] [] [--version] [--h *Use the global default, ignoring any .swift-version files.* +**--assume-yes:** + +*Disable confirmation prompts by assuming 'yes'* + + **toolchain:** *The toolchain to use.* @@ -469,9 +474,7 @@ You can also override the selection mechanisms temporarily for the duration of t $ swiftly run swift build +latest $ swiftly run swift build +5.10.1 -The first command builds the swift package with the latest toolchain and the second selects the 5.10.1 toolchain. Note that if these aren't installed then run will fail with an error message. You can pre-install the toolchain using `swiftly install ` to ensure success. There is also a `+install` argument that will automatically download and install the toolchain if necessary. - - $ swiftly run swift build +latest +install +The first command builds the swift package with the latest toolchain and the second selects the 5.10.1 toolchain. Note that if these aren't installed then run will fail with an error message. You can pre-install the toolchain using `swiftly install ` to ensure success. If the command that you are running needs the arguments with the '+' prefixes then you can escape it by doubling the '++'. diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index b9aa1fba..cfbb1a87 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -242,7 +242,7 @@ struct Install: SwiftlyCommand { // --use argument was provided. if useInstalledToolchain || config.inUse == nil { // TODO: consider adding the global default option to this commands flags - try await Use.execute(version, false, &config) + try await Use.execute(version, globalDefault: false, &config) } SwiftlyCore.print("\(version) installed successfully!") diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift index f165b39b..4f4f67fe 100644 --- a/Sources/Swiftly/Run.swift +++ b/Sources/Swiftly/Run.swift @@ -35,10 +35,7 @@ internal struct Run: SwiftlyCommand { The first command builds the swift package with the latest toolchain and the second selects the \ 5.10.1 toolchain. Note that if these aren't installed then run will fail with an error message. \ - You can pre-install the toolchain using `swiftly install ` to ensure success. There is \ - also a `+install` argument that will automatically download and install the toolchain if necessary. - - $ swiftly run swift build +latest +install + You can pre-install the toolchain using `swiftly install ` to ensure success. If the command that you are running needs the arguments with the '+' prefixes then you can escape \ it by doubling the '++'. @@ -62,36 +59,19 @@ internal struct Run: SwiftlyCommand { var config = try Config.load() - let (command, selector, install) = try extractProxyArguments(command: self.command) + let (command, selector) = try extractProxyArguments(command: self.command) let toolchain: ToolchainVersion? if let selector = selector { - if install { - let version = try await Install.resolve(config: config, selector: selector) - let postInstallScript = try await Install.execute(version: version, &config, useInstalledToolchain: false, verifySignature: true) - if let postInstallScript = postInstallScript { - throw Error(message: """ - - There are some system dependencies that should be installed before using this toolchain. - You can run the following script as the system administrator (e.g. root) to prepare - your system: - - \(postInstallScript) - """) - } - - toolchain = version - } else { - let matchedToolchain = config.listInstalledToolchains(selector: selector).max() - guard let matchedToolchain = matchedToolchain else { - throw Error(message: "The selected toolchain \(selector.description) didn't match any of the installed toolchains. You can install it by adding '+install' to your command, or `swiftly install \(selector.description)`") - } - - toolchain = matchedToolchain + let matchedToolchain = config.listInstalledToolchains(selector: selector).max() + guard let matchedToolchain = matchedToolchain else { + throw Error(message: "The selected toolchain \(selector.description) didn't match any of the installed toolchains. You can install it with `swiftly install \(selector.description)`") } + + toolchain = matchedToolchain } else { - let (version, result) = try await selectToolchain(config: &config, install: install) + let (version, result) = try await selectToolchain(config: &config) // Abort on any errors relating to swift version files if case let .swiftVersionFile(_, _, error) = result, let error = error { @@ -124,8 +104,8 @@ internal struct Run: SwiftlyCommand { } } -public func extractProxyArguments(command: [String]) throws -> (command: [String], selector: ToolchainSelector?, install: Bool) { - var args: (command: [String], selector: ToolchainSelector?, install: Bool) = (command: [], nil, false) +public func extractProxyArguments(command: [String]) throws -> (command: [String], selector: ToolchainSelector?) { + var args: (command: [String], selector: ToolchainSelector?) = (command: [], nil) var disableEscaping = false @@ -140,11 +120,6 @@ public func extractProxyArguments(command: [String]) throws -> (command: [String continue } - if !disableEscaping && c == "+install" { - args.install = true - continue - } - if !disableEscaping && c.hasPrefix("+") { args.selector = try ToolchainSelector(parsing: String(c.dropFirst())) continue diff --git a/Sources/Swiftly/Uninstall.swift b/Sources/Swiftly/Uninstall.swift index 1bfb8ff2..b330d7bb 100644 --- a/Sources/Swiftly/Uninstall.swift +++ b/Sources/Swiftly/Uninstall.swift @@ -100,7 +100,7 @@ struct Uninstall: SwiftlyCommand { ?? config.listInstalledToolchains(selector: .latest).filter({ !toolchains.contains($0) }).max() ?? config.installedToolchains.filter({ !toolchains.contains($0) }).max() { - try await Use.execute(toUse, true, &config) + try await Use.execute(toUse, globalDefault: true, &config) } else { // If there are no more toolchains installed, just unuse the currently active toolchain. config.inUse = nil diff --git a/Sources/Swiftly/Use.swift b/Sources/Swiftly/Use.swift index d77aea57..95e6663a 100644 --- a/Sources/Swiftly/Use.swift +++ b/Sources/Swiftly/Use.swift @@ -4,7 +4,7 @@ import SwiftlyCore internal struct Use: SwiftlyCommand { public static var configuration = CommandConfiguration( - abstract: "Set the active toolchain. If no toolchain is provided, print the currently in-use toolchain, if any." + abstract: "Set the in-use toolchain. If no toolchain is provided, print the currently in-use toolchain, if any." ) @Flag(name: .shortAndLong, help: "Print the location of the in-use toolchain. This is valid only when there is no toolchain argument.") @@ -13,6 +13,8 @@ internal struct Use: SwiftlyCommand { @Flag(name: .shortAndLong, help: "Use the global default, ignoring any .swift-version files.") var globalDefault: Bool = false + @OptionGroup var root: GlobalOptions + @Argument(help: ArgumentHelp( "The toolchain to use.", discussion: """ @@ -98,11 +100,11 @@ internal struct Use: SwiftlyCommand { return } - try await Self.execute(toolchain, self.globalDefault, &config) + try await Self.execute(toolchain, globalDefault: self.globalDefault, assumeYes: self.root.assumeYes, &config) } /// Use a toolchain. This method can modify and save the input config. - internal static func execute(_ toolchain: ToolchainVersion, _ globalDefault: Bool, _ config: inout Config) async throws { + internal static func execute(_ toolchain: ToolchainVersion, globalDefault: Bool, assumeYes: Bool = true, _ config: inout Config) async throws { let (selectedVersion, result) = try await selectToolchain(config: &config, globalDefault: globalDefault) if let selectedVersion = selectedVersion { @@ -116,13 +118,22 @@ internal struct Use: SwiftlyCommand { // We don't care in this case if there were any problems with the swift version files, just overwrite it with the new value try toolchain.name.write(to: versionFile, atomically: true, encoding: .utf8) } else if let newVersionFile = findNewVersionFile(), !globalDefault { + if !assumeYes { + SwiftlyCore.print("A new file `\(newVersionFile)` will be created to set the new in-use toolchain for this project. Alternatively, you can set your default globally with the `--global-default` flag. Proceed with creating this file?") + + guard SwiftlyCore.promptForConfirmation(defaultBehavior: true) else { + SwiftlyCore.print("Aborting setting in-use toolchain") + return + } + } + try toolchain.name.write(to: newVersionFile, atomically: true, encoding: .utf8) } else { config.inUse = toolchain try config.save() } - var message = "Set the used toolchain to \(toolchain)" + var message = "Set the in-use toolchain to \(toolchain)" if let selectedVersion = selectedVersion { message += " (was \(selectedVersion.name))" } @@ -175,7 +186,7 @@ public enum ToolchainSelectionResult { /// If such a case happens then the toolchain version in the tuple will be nil, but the /// result will be .swiftVersionFile and a detailed error about the problem. This error /// can be thrown by the client, or ignored. -public func selectToolchain(config: inout Config, globalDefault: Bool = false, install: Bool = false) async throws -> (ToolchainVersion?, ToolchainSelectionResult) { +public func selectToolchain(config: inout Config, globalDefault: Bool = false) async throws -> (ToolchainVersion?, ToolchainSelectionResult) { if !globalDefault { var cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) @@ -209,23 +220,8 @@ public func selectToolchain(config: inout Config, globalDefault: Bool = false, i return (nil, .swiftVersionFile(svFile, nil, Error(message: "The swift version file is malformed: \(svFile)"))) } - if install { - let version = try await Install.resolve(config: config, selector: selector) - let postInstallScript = try await Install.execute(version: version, &config, useInstalledToolchain: false, verifySignature: true) - if let postInstallScript = postInstallScript { - throw Error(message: """ - - There are some system dependencies that should be installed before using this toolchain. - You can run the following script as the system administrator (e.g. root) to prepare - your system: - - \(postInstallScript) - """) - } - } - guard let selectedToolchain = config.listInstalledToolchains(selector: selector).max() else { - return (nil, .swiftVersionFile(svFile, selector, Error(message: "The swift version file didn't select any of the installed toolchains. You can install one with `swiftly install \(selector.description)`."))) + return (nil, .swiftVersionFile(svFile, selector, Error(message: "The swift version file `\(svFile.path)` uses toolchain version \(selector), but it doesn't match any of the installed toolchains. You can install the toolchain with `swiftly install`."))) } return (selectedToolchain, .swiftVersionFile(svFile, selector, nil)) diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index 9042a8c4..d0f7a39f 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -142,7 +142,7 @@ extension Platform { // The toolchain goes to the beginning of the PATH var newPath = newEnv["PATH"] ?? "" if !newPath.hasPrefix(tcPath.path + ":") { - newPath = ([tcPath.path] + newPath.split(separator: ":").map { String($0) }).joined(separator: ":") + newPath = "\(tcPath.path):\(newPath)" } newEnv["PATH"] = newPath diff --git a/Tests/SwiftlyTests/RunTests.swift b/Tests/SwiftlyTests/RunTests.swift index 7b92c38b..846509c1 100644 --- a/Tests/SwiftlyTests/RunTests.swift +++ b/Tests/SwiftlyTests/RunTests.swift @@ -58,34 +58,28 @@ final class RunTests: SwiftlyTests { /// Tests the extraction of proxy arguments from the run command arguments. func testExtractProxyArguments() throws { - var (command, selector, install) = try extractProxyArguments(command: ["swift", "build"]) + var (command, selector) = try extractProxyArguments(command: ["swift", "build"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(nil, selector) - (command, selector, install) = try extractProxyArguments(command: ["swift", "+1.2.3", "build"]) + (command, selector) = try extractProxyArguments(command: ["swift", "+1.2.3", "build"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(try! ToolchainSelector(parsing: "1.2.3"), selector) - (command, selector, install) = try extractProxyArguments(command: ["swift", "build", "+latest"]) + (command, selector) = try extractProxyArguments(command: ["swift", "build", "+latest"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(try! ToolchainSelector(parsing: "latest"), selector) - (command, selector, install) = try extractProxyArguments(command: ["+5.6", "swift", "build"]) + (command, selector) = try extractProxyArguments(command: ["+5.6", "swift", "build"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(try! ToolchainSelector(parsing: "5.6"), selector) - (command, selector, install) = try extractProxyArguments(command: ["swift", "++1.2.3", "build"]) + (command, selector) = try extractProxyArguments(command: ["swift", "++1.2.3", "build"]) XCTAssertEqual(["swift", "+1.2.3", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(nil, selector) - (command, selector, install) = try extractProxyArguments(command: ["swift", "++", "+1.2.3", "build"]) + (command, selector) = try extractProxyArguments(command: ["swift", "++", "+1.2.3", "build"]) XCTAssertEqual(["swift", "+1.2.3", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(nil, selector) do { @@ -98,14 +92,12 @@ final class RunTests: SwiftlyTests { XCTAssert(false) } catch {} - (command, selector, install) = try extractProxyArguments(command: ["swift", "+1.2.3", "+install", "build"]) + (command, selector) = try extractProxyArguments(command: ["swift", "+1.2.3", "build"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(true, install) XCTAssertEqual(try! ToolchainSelector(parsing: "1.2.3"), selector) - (command, selector, install) = try extractProxyArguments(command: ["swift", "+install", "build"]) + (command, selector) = try extractProxyArguments(command: ["swift", "build"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(true, install) XCTAssertEqual(nil, selector) } } From 9dd640c5c8835de04b1085cbd46f172453748e67 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 17 Sep 2024 14:26:53 -0400 Subject: [PATCH 20/45] Import GPG keys on every install to get new signing keys from swift.org Fix all of the swift.org urls so that they use www.swift.org to avoid redirection --- Sources/LinuxPlatform/Linux.swift | 29 +++--------------------- Tests/SwiftlyTests/HTTPClientTests.swift | 4 ++-- 2 files changed, 5 insertions(+), 28 deletions(-) diff --git a/Sources/LinuxPlatform/Linux.swift b/Sources/LinuxPlatform/Linux.swift index 072c2a7e..e20eec93 100644 --- a/Sources/LinuxPlatform/Linux.swift +++ b/Sources/LinuxPlatform/Linux.swift @@ -193,44 +193,21 @@ public struct Linux: Platform { throw Error(message: msg) } - let foundKeys = (try? self.runProgram( - "gpg", - "--list-keys", - "swift-infrastructure@forums.swift.org", - "swift-infrastructure@swift.org", - quiet: true - )) != nil - if !foundKeys { - // Import the swift keys if they aren't here already + // Import the latest swift keys, but only once per session, which will help with the performance in tests + if !swiftGPGKeysRefreshed { let tmpFile = self.getTempFilePath() FileManager.default.createFile(atPath: tmpFile.path, contents: nil, attributes: [.posixPermissions: 0o600]) defer { try? FileManager.default.removeItem(at: tmpFile) } - guard let url = URL(string: "https://swift.org/keys/all-keys.asc") else { + guard let url = URL(string: "https://www.swift.org/keys/all-keys.asc") else { throw Error(message: "malformed URL to the swift gpg keys") } try await httpClient.downloadFile(url: url, to: tmpFile) try self.runProgram("gpg", "--import", tmpFile.path, quiet: true) - } - // We only need to refresh the keys once per session, which will help with performance in tests - if !swiftGPGKeysRefreshed { - SwiftlyCore.print("Refreshing Swift PGP keys...") - do { - try self.runProgram( - "gpg", - "--quiet", - "--keyserver", - "hkp://keyserver.ubuntu.com", - "--refresh-keys", - "Swift" - ) - } catch { - throw Error(message: "Failed to refresh PGP keys: \(error)") - } swiftGPGKeysRefreshed = true } } diff --git a/Tests/SwiftlyTests/HTTPClientTests.swift b/Tests/SwiftlyTests/HTTPClientTests.swift index e72155ae..1d666d68 100644 --- a/Tests/SwiftlyTests/HTTPClientTests.swift +++ b/Tests/SwiftlyTests/HTTPClientTests.swift @@ -7,7 +7,7 @@ final class HTTPClientTests: SwiftlyTests { // GIVEN: we have a swiftly http client // WHEN: we make get request for a particular type of JSON var releases: [SwiftOrgRelease] = try await SwiftlyCore.httpClient.getFromJSON( - url: "https://swift.org/api/v1/install/releases.json", + url: "https://www.swift.org/api/v1/install/releases.json", type: [SwiftOrgRelease].self, headers: [:] ) @@ -19,7 +19,7 @@ final class HTTPClientTests: SwiftlyTests { var exceptionThrown = false do { releases = try await SwiftlyCore.httpClient.getFromJSON( - url: "https://swift.org/api/v1/install/releases-invalid.json", + url: "https://www.swift.org/api/v1/install/releases-invalid.json", type: [SwiftOrgRelease].self, headers: [:] ) From 5e615ea70634166e7abd7919475821c6dd4be0eb Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Fri, 18 Oct 2024 08:23:30 -0400 Subject: [PATCH 21/45] Make recommended documentation changes. Fix symlink target selection for swiftly when it is system managed --- DESIGN.md | 6 +++--- Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md | 4 ++-- Sources/Swiftly/Init.swift | 8 ++++---- Sources/Swiftly/Install.swift | 2 +- Sources/Swiftly/Run.swift | 5 ++--- Sources/SwiftlyCore/Platform.swift | 6 +++--- 6 files changed, 15 insertions(+), 16 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 97921380..1c76c4d4 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -147,11 +147,11 @@ Installing a specific snapshot from a swift version development branch ##### Installing the version from the `.swift-version` file -A package could have a swift version file that specifies the recommended toolchain version. A swiftly install with no version will search for a version file and install that version. +A package could have a ".swift-version" file that specifies the recommended toolchain version. A swiftly install with no version will search for a version file and install that version. `swiftly install` -If no swift version file can be found then the installation fails indicating that it couldn't fine the file. +If no ".swift-version" file can be found then the installation fails indicating that it couldn't fine the file. #### uninstall @@ -223,7 +223,7 @@ To use the latest installed main snapshot, leave off the date: `swiftly use main-snapshot` -The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version. +The use subcommand also supports `.swift-version` files. If a ".swift-version" file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version. Note: The `.swift-version` file mechanisms can be overridden using the `--global-default` flag so that your swiftly installation's default toolchain can be set explicitly. diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 7db3879d..dbe97f63 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -53,7 +53,7 @@ Likewise, the latest snapshot associated with a given development branch can be $ swiftly install 5.7-snapshot $ swiftly install main-snapshot - Install whatever toolchain is current selected, such as a .swift-version file: + Install whatever toolchain is currently selected, such as the the one in the .swift-version file: $ swiftly install @@ -456,7 +456,7 @@ swiftly run ... [--version] [--help] *Run a command while proxying to the selected toolchain commands.* -Run a command with a selected toolchain, so that all toolchain commands are become the default added to the system path and other common environment variables. +Run a command with a selected toolchain. The toolchain commands become the default in the system path. You can run one of the usual toolchain commands directly: diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index a4b7547f..9a354022 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -124,10 +124,10 @@ internal struct Init: SwiftlyCommand { let swiftlyBin = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swiftly", isDirectory: false) let cmd = URL(fileURLWithPath: CommandLine.arguments[0]) - let systemManaged = try Swiftly.currentPlatform.isSystemManagedBinary(cmd.path) + let systemManagedSwiftlyBin = try Swiftly.currentPlatform.systemManagedBinary(CommandLine.arguments[0]) // Don't move the binary if it's already in the right place, this is being invoked inside an xctest, or it is a system managed binary - if cmd != swiftlyBin && !cmd.path.hasSuffix("xctest") && !systemManaged { + if cmd != swiftlyBin && !cmd.path.hasSuffix("xctest") && systemManagedSwiftlyBin == nil { SwiftlyCore.print("Moving swiftly into the installation directory...") if swiftlyBin.fileExists() { @@ -146,8 +146,8 @@ internal struct Init: SwiftlyCommand { if !cmd.path.hasSuffix("xctest") { SwiftlyCore.print("Setting up toolchain proxies...") - let proxyTo = if systemManaged { - cmd.path + let proxyTo = if let systemManagedSwiftlyBin = systemManagedSwiftlyBin { + systemManagedSwiftlyBin } else { swiftlyBin.path } diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index cfbb1a87..2e565aa1 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -40,7 +40,7 @@ struct Install: SwiftlyCommand { $ swiftly install 5.7-snapshot $ swiftly install main-snapshot - Install whatever toolchain is current selected, such as a .swift-version file: + Install whatever toolchain is currently selected, such as the the one in the .swift-version file: $ swiftly install """ diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift index 4f4f67fe..c60c5da1 100644 --- a/Sources/Swiftly/Run.swift +++ b/Sources/Swiftly/Run.swift @@ -11,9 +11,8 @@ internal struct Run: SwiftlyCommand { "Run a command while proxying to the selected toolchain commands.", discussion: """ - Run a command with a selected toolchain, so that all toolchain commands \ - are become the default added to the system path and other common environment \ - variables. + Run a command with a selected toolchain. The toolchain commands \ + become the default in the system path. You can run one of the usual toolchain commands directly: diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index d0f7a39f..8e995387 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -261,7 +261,7 @@ extension Platform { } } - public func isSystemManagedBinary(_ cmd: String) throws -> Bool { + public func systemManagedBinary(_ cmd: String) throws -> String? { let userHome = FileManager.default.homeDirectoryForCurrentUser let binLocs = [cmd] + ProcessInfo.processInfo.environment["PATH"]!.components(separatedBy: ":").map { $0 + "/" + cmd } var bin: String? @@ -278,10 +278,10 @@ extension Platform { // If the binary is in the user's home directory, or is not in system locations ("/usr", "/opt", "/bin") // then it is expected to be outside of a system package location and we manage the binary ourselves. if bin.hasPrefix(userHome.path + "/") || (!bin.hasPrefix("/usr") && !bin.hasPrefix("/opt") && !bin.hasPrefix("/bin")) { - return false + return nil } - return true + return bin } #endif From 6d6050e3e0335a7d0e5250053a0280e9fbe2ab8b Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 19 Oct 2024 07:58:41 -0400 Subject: [PATCH 22/45] Provide a better error message on swiftly install with no version Print the error in a better way --- Sources/Swiftly/Install.swift | 2 +- Sources/Swiftly/Proxy.swift | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 2e565aa1..3d14f8ce 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -96,7 +96,7 @@ struct Install: SwiftlyCommand { throw Error(message: "Internal error selecting toolchain to install.") } } else { - throw Error(message: "There is no toolchain selected from a .swift-version file to install.") + throw Error(message: "Swiftly couldn't determine the toolchain version to install. Please set a version like this and try again: `swiftly install latest`") } } diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 3289a8ab..b91bdd4f 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -51,6 +51,9 @@ public enum Proxy { try await Swiftly.currentPlatform.proxy(toolchain, binName, Array(CommandLine.arguments[1...])) } catch let terminated as RunProgramError { exit(terminated.exitCode) + } catch let error as Error { + SwiftlyCore.print(error.message) + exit(1) } catch { SwiftlyCore.print("\(error)") exit(1) From 883e6389899a74dcfa3a7cb2116b0114e1eaf08f Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 19 Oct 2024 15:27:24 -0400 Subject: [PATCH 23/45] Add new Linux platforms: Ubuntu 2404,2310, Debian 12, Fedora 39 Add support for these new platforms to swiftly, the autodetection, the tests, and docker support. Remove the swiftly installer portion of the code. --- Sources/LinuxPlatform/Linux.swift | 134 +++-- Sources/SwiftlyCore/Platform.swift | 5 + Tests/SwiftlyTests/HTTPClientTests.swift | 21 +- docker/docker-compose.2310.yaml | 28 + docker/docker-compose.2404.yaml | 28 + docker/docker-compose.debian12.yaml | 24 + docker/docker-compose.fedora39.yaml | 28 + docker/docker-compose.yaml | 2 +- docker/install-test-debian12.dockerfile | 12 + docker/install-test-fedora39.dockerfile | 5 + docker/test-debian12.dockerfile | 18 + docker/test-fedora39.dockerfile | 16 + install/run-tests.sh | 48 -- install/swiftly-install.sh | 716 ----------------------- install/test-util.sh | 88 --- install/tests/cancel-install.sh | 19 - install/tests/custom-home-install.sh | 44 -- install/tests/custom-install.sh | 70 --- install/tests/custom-tilde-install.sh | 45 -- install/tests/default-install.sh | 159 ----- install/tests/disable-prompt.sh | 45 -- install/tests/no-install-dependencies.sh | 45 -- install/tests/overwrite.sh | 89 --- install/tests/platform-option.sh | 32 - install/tests/update-bash-profile.sh | 47 -- install/tests/update-fish-config.sh | 37 -- install/tests/update-zprofile.sh | 28 - 27 files changed, 272 insertions(+), 1561 deletions(-) create mode 100644 docker/docker-compose.2310.yaml create mode 100644 docker/docker-compose.2404.yaml create mode 100644 docker/docker-compose.debian12.yaml create mode 100644 docker/docker-compose.fedora39.yaml create mode 100644 docker/install-test-debian12.dockerfile create mode 100644 docker/install-test-fedora39.dockerfile create mode 100644 docker/test-debian12.dockerfile create mode 100644 docker/test-fedora39.dockerfile delete mode 100755 install/run-tests.sh delete mode 100755 install/swiftly-install.sh delete mode 100644 install/test-util.sh delete mode 100755 install/tests/cancel-install.sh delete mode 100755 install/tests/custom-home-install.sh delete mode 100755 install/tests/custom-install.sh delete mode 100755 install/tests/custom-tilde-install.sh delete mode 100755 install/tests/default-install.sh delete mode 100755 install/tests/disable-prompt.sh delete mode 100755 install/tests/no-install-dependencies.sh delete mode 100755 install/tests/overwrite.sh delete mode 100755 install/tests/platform-option.sh delete mode 100755 install/tests/update-bash-profile.sh delete mode 100755 install/tests/update-fish-config.sh delete mode 100755 install/tests/update-zprofile.sh diff --git a/Sources/LinuxPlatform/Linux.swift b/Sources/LinuxPlatform/Linux.swift index 6deb26c2..0f3b1e8d 100644 --- a/Sources/LinuxPlatform/Linux.swift +++ b/Sources/LinuxPlatform/Linux.swift @@ -125,6 +125,26 @@ public struct Linux: Platform { "tzdata", "zlib1g-dev", ] + case "ubuntu2404": + [ + "binutils", + "git", + "unzip", + "gnupg2", + "libc6-dev", + "libcurl4-openssl-dev", + "libedit2", + "libgcc-13-dev", + "libpython3-dev", + "libsqlite3-0", + "libstdc++-13-dev", + "libxml2-dev", + "libncurses-dev", + "libz3-dev", + "pkg-config", + "tzdata", + "zlib1g-dev", + ] case "amazonlinux2": [ "binutils", @@ -158,6 +178,36 @@ public struct Linux: Platform { "unzip", "zip", ] + case "fedora39": + [ + "binutils", + "gcc", + "git", + "unzip", + "libcurl-devel", + "libedit-devel", + "libicu-devel", + "sqlite-devel", + "libuuid-devel", + "libxml2-devel", + "python3-devel", + ] + case "debian12": + [ + "binutils-gold", + "libicu-dev", + "libcurl4-openssl-dev", + "libedit-dev", + "libsqlite3-dev", + "libncurses-dev", + "libpython3-dev", + "libxml2-dev", + "pkg-config", + "uuid-dev", + "tzdata", + "git", + "gcc", + ] default: [] } @@ -169,10 +219,18 @@ public struct Linux: Platform { "apt" case "ubuntu2204": "apt" + case "ubuntu2310": + "apt" + case "ubuntu2404": + "apt" case "amazonlinux2": "yum" case "ubi9": "yum" + case "fedora39": + "yum" + case "debian12": + "apt" default: nil } @@ -196,7 +254,7 @@ public struct Linux: Platform { // Import the latest swift keys, but only once per session, which will help with the performance in tests if !swiftGPGKeysRefreshed { let tmpFile = self.getTempFilePath() - FileManager.default.createFile(atPath: tmpFile.path, contents: nil, attributes: [.posixPermissions: 0o600]) + let _ = FileManager.default.createFile(atPath: tmpFile.path, contents: nil, attributes: [.posixPermissions: 0o600]) defer { try? FileManager.default.removeItem(at: tmpFile) } @@ -390,7 +448,7 @@ public struct Linux: Platform { public func verifySignature(httpClient: SwiftlyHTTPClient, archiveDownloadURL: URL, archive: URL) async throws { SwiftlyCore.print("Downloading toolchain signature...") let sigFile = self.getTempFilePath() - FileManager.default.createFile(atPath: sigFile.path, contents: nil) + let _ = FileManager.default.createFile(atPath: sigFile.path, contents: nil) defer { try? FileManager.default.removeItem(at: sigFile) } @@ -415,33 +473,38 @@ public struct Linux: Platform { print("This platform could not be detected, but a toolchain for one of the supported platforms may work on it.") } + let linuxPlatforms = [ + PlatformDefinition.ubuntu2404, + PlatformDefinition.ubuntu2310, + PlatformDefinition.ubuntu2204, + PlatformDefinition.ubuntu2004, + PlatformDefinition.ubuntu1804, + PlatformDefinition.fedora39, + PlatformDefinition.rhel9, + PlatformDefinition.amazonlinux2, + PlatformDefinition.debian12, + ] + + let selections = linuxPlatforms.enumerated().map { "\($0)) \($1.namePretty)" }.joined(separator: "\n") + print(""" Please select the platform to use for toolchain downloads: 0) Cancel - 1) Ubuntu 22.04 - 2) Ubuntu 20.04 - 3) Ubuntu 18.04 - 4) RHEL 9 - 5) Amazon Linux 2 + \(selections) """) let choice = SwiftlyCore.readLine(prompt: "> ") ?? "0" - switch choice { - case "1": - return PlatformDefinition.ubuntu2204 - case "2": - return PlatformDefinition.ubuntu2004 - case "3": - return PlatformDefinition.ubuntu1804 - case "4": - return PlatformDefinition.rhel9 - case "5": - return PlatformDefinition.amazonlinux2 - default: + guard let choiceNum = Int(choice) else { + fatalError("Installation canceled") + } + + guard choiceNum >= 0 && choiceNum <= linuxPlatforms.count else { fatalError("Installation canceled") } + + return linuxPlatforms[choiceNum] } public func detectPlatform(disableConfirmation: Bool, platform: String?) async throws -> PlatformDefinition { @@ -508,22 +571,19 @@ public struct Linux: Platform { var id: String? var idlike: String? var versionID: String? - var ubuntuCodeName: String? for info in releaseInfo.split(separator: "\n").map(String.init) { if info.hasPrefix("ID=") { id = String(info.dropFirst("ID=".count)).replacingOccurrences(of: "\"", with: "") } else if info.hasPrefix("ID_LIKE=") { idlike = String(info.dropFirst("ID_LIKE=".count)).replacingOccurrences(of: "\"", with: "") } else if info.hasPrefix("VERSION_ID=") { - versionID = String(info.dropFirst("VERSION_ID=".count)).replacingOccurrences(of: "\"", with: "") - } else if info.hasPrefix("UBUNTU_CODENAME=") { - ubuntuCodeName = String(info.dropFirst("UBUNTU_CODENAME=".count)).replacingOccurrences(of: "\"", with: "") + versionID = String(info.dropFirst("VERSION_ID=".count)).replacingOccurrences(of: "\"", with: "").replacingOccurrences(of: ".", with: "") } else if info.hasPrefix("PRETTY_NAME=") { platformPretty = String(info.dropFirst("PRETTY_NAME=".count)).replacingOccurrences(of: "\"", with: "") } } - guard let id = id, let idlike = idlike else { + guard let id = id, let idlike = idlike, let versionID else { let message = "Unable to find release information from file \(releaseFile)" if disableConfirmation { throw Error(message: message) @@ -534,7 +594,7 @@ public struct Linux: Platform { } if (id + idlike).contains("amzn") { - guard let versionID = versionID, versionID == "2" else { + guard versionID == "2" else { let message = "Unsupported version of Amazon Linux" if disableConfirmation { throw Error(message: message) @@ -544,25 +604,9 @@ public struct Linux: Platform { return self.manualSelectPlatform(platformPretty) } - return PlatformDefinition(name: "amazonlinux2", nameFull: "amazonlinux2", namePretty: "Amazon Linux 2") - } else if (id + idlike).contains("ubuntu") { - if ubuntuCodeName == "jammy" { - return PlatformDefinition(name: "ubuntu2204", nameFull: "ubuntu22.04", namePretty: "Ubuntu 22.04") - } else if ubuntuCodeName == "focal" { - return PlatformDefinition(name: "ubuntu2004", nameFull: "ubuntu20.04", namePretty: "Ubuntu 20.04") - } else if ubuntuCodeName == "bionic" { - return PlatformDefinition(name: "ubuntu1804", nameFull: "ubuntu18.04", namePretty: "Ubuntu 18.04") - } else { - let message = "Unsupported version of Ubuntu Linux" - if disableConfirmation { - throw Error(message: message) - } else { - print(message) - } - return self.manualSelectPlatform(platformPretty) - } + return PlatformDefinition.amazonlinux2 } else if (id + idlike).contains("rhel") { - guard let versionID = versionID, versionID.hasPrefix("9") else { + guard versionID.hasPrefix("9") else { let message = "Unsupported version of RHEL" if disableConfirmation { throw Error(message: message) @@ -572,7 +616,9 @@ public struct Linux: Platform { return self.manualSelectPlatform(platformPretty) } - return PlatformDefinition(name: "ubi9", nameFull: "ubi9", namePretty: "RHEL 9") + return PlatformDefinition.rhel9 + } else if let pd = [PlatformDefinition.ubuntu1804, .ubuntu2004, .ubuntu2204, .ubuntu2310, .ubuntu2404, .debian12, .fedora39].first(where: { $0.name == id + versionID }) { + return pd } let message = "Unsupported Linux platform" diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index b4a64459..3a981bc0 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -22,11 +22,16 @@ public struct PlatformDefinition: Codable, Equatable { } public static let macOS = PlatformDefinition(name: "xcode", nameFull: "osx", namePretty: "macOS") + + public static let ubuntu2404 = PlatformDefinition(name: "ubuntu2404", nameFull: "ubuntu24.04", namePretty: "Ubuntu 24.04") + public static let ubuntu2310 = PlatformDefinition(name: "ubuntu2310", nameFull: "ubuntu23.10", namePretty: "Ubuntu 23.10") public static let ubuntu2204 = PlatformDefinition(name: "ubuntu2204", nameFull: "ubuntu22.04", namePretty: "Ubuntu 22.04") public static let ubuntu2004 = PlatformDefinition(name: "ubuntu2004", nameFull: "ubuntu20.04", namePretty: "Ubuntu 20.04") public static let ubuntu1804 = PlatformDefinition(name: "ubuntu1804", nameFull: "ubuntu18.04", namePretty: "Ubuntu 18.04") public static let rhel9 = PlatformDefinition(name: "ubi9", nameFull: "ubi9", namePretty: "RHEL 9") + public static let fedora39 = PlatformDefinition(name: "fedora39", nameFull: "fedora39", namePretty: "Fedora Linux 39") public static let amazonlinux2 = PlatformDefinition(name: "amazonlinux2", nameFull: "amazonlinux2", namePretty: "Amazon Linux 2") + public static let debian12 = PlatformDefinition(name: "debian12", nameFull: "debian12", namePretty: "Debian GNU/Linux 12") } public protocol Platform { diff --git a/Tests/SwiftlyTests/HTTPClientTests.swift b/Tests/SwiftlyTests/HTTPClientTests.swift index 1d666d68..938c0644 100644 --- a/Tests/SwiftlyTests/HTTPClientTests.swift +++ b/Tests/SwiftlyTests/HTTPClientTests.swift @@ -48,11 +48,22 @@ final class HTTPClientTests: SwiftlyTests { func testGetMetdataFromSwiftOrg() async throws { let supportedPlatforms = [ PlatformDefinition.macOS, + PlatformDefinition.ubuntu2404, + PlatformDefinition.ubuntu2310, PlatformDefinition.ubuntu2204, PlatformDefinition.ubuntu2004, // PlatformDefinition.ubuntu1804, // There are no releases for Ubuntu 18.04 in the branches being tested below PlatformDefinition.rhel9, + PlatformDefinition.fedora39, PlatformDefinition.amazonlinux2, + PlatformDefinition.debian12, + ] + + let newPlatforms = [ + PlatformDefinition.ubuntu2404, + PlatformDefinition.ubuntu2310, + PlatformDefinition.fedora39, + PlatformDefinition.debian12, ] let branches = [ @@ -66,15 +77,17 @@ final class HTTPClientTests: SwiftlyTests { // GIVEN: we have a swiftly http client with swift.org metadata capability // WHEN: we ask for the first five releases of a supported platform in a supported arch let releases = try await SwiftlyCore.httpClient.getReleaseToolchains(platform: platform, arch: arch, limit: 5) - // THEN: we get five releases - XCTAssertEqual(5, releases.count) + // THEN: we get at least 1 release + XCTAssertTrue(1 <= releases.count) + + if newPlatforms.contains(platform) { continue } // Newer distros don't have main snapshots yet for branch in branches { // GIVEN: we have a swiftly http client with swift.org metadata capability // WHEN: we ask for the first five snapshots on a branch for a supported platform and arch let snapshots = try await SwiftlyCore.httpClient.getSnapshotToolchains(platform: platform, arch: arch, branch: branch, limit: 5) - // THEN: we get five snapshots - XCTAssertEqual(5, snapshots.count) + // THEN: we get at least 3 releases + XCTAssertTrue(3 <= snapshots.count) } } } diff --git a/docker/docker-compose.2310.yaml b/docker/docker-compose.2310.yaml new file mode 100644 index 00000000..80d6673d --- /dev/null +++ b/docker/docker-compose.2310.yaml @@ -0,0 +1,28 @@ +version: "3" + +services: + + test-setup: + image: swiftly:23.10-test + build: + args: + base_image: "swift:5.10-mantic" + + test: + image: swiftly:23.10-test + environment: + - SWIFTLY_PLATFORM_NAME=ubuntu2310 + - SWIFTLY_PLATFORM_NAME_FULL=ubuntu23.10 + - SWIFTLY_PLATFORM_NAME_PRETTY="Ubuntu 23.10" + + install-test-setup: + image: swiftly:23.10-install-test + build: + args: + base_image: "ubuntu:mantic" + + install-test: + image: swiftly:23.10-install-test + + shell: + image: swiftly:23.10-test diff --git a/docker/docker-compose.2404.yaml b/docker/docker-compose.2404.yaml new file mode 100644 index 00000000..a5318c70 --- /dev/null +++ b/docker/docker-compose.2404.yaml @@ -0,0 +1,28 @@ +version: "3" + +services: + + test-setup: + image: swiftly:24.04-test + build: + args: + base_image: "swift:5.10-noble" + + test: + image: swiftly:24.04-test + environment: + - SWIFTLY_PLATFORM_NAME=ubuntu2404 + - SWIFTLY_PLATFORM_NAME_FULL=ubuntu24.04 + - SWIFTLY_PLATFORM_NAME_PRETTY="Ubuntu 24.04" + + install-test-setup: + image: swiftly:24.04-install-test + build: + args: + base_image: "ubuntu:noble" + + install-test: + image: swiftly:24.04-install-test + + shell: + image: swiftly:24.04-test diff --git a/docker/docker-compose.debian12.yaml b/docker/docker-compose.debian12.yaml new file mode 100644 index 00000000..0adbe3e6 --- /dev/null +++ b/docker/docker-compose.debian12.yaml @@ -0,0 +1,24 @@ +version: "3" + +services: + + test-setup: + image: swiftly:debian12-test + build: + args: + base_image: "swift:5.10-debian12" + + test: + image: swiftly:debian12-test + + install-test-setup: + image: swiftly:debian12-install-test + build: + args: + base_image: "debian:12" + + install-test: + image: swiftly:debian12-install-test + + shell: + image: swiftly:debian12-test diff --git a/docker/docker-compose.fedora39.yaml b/docker/docker-compose.fedora39.yaml new file mode 100644 index 00000000..db3ef162 --- /dev/null +++ b/docker/docker-compose.fedora39.yaml @@ -0,0 +1,28 @@ +version: "3" + +services: + + test-setup: + image: swiftly:fedora39-test + build: + context: .. + dockerfile: docker/test-fedora39.dockerfile + args: + base_image: "swift:5.10-fedora39" + + test: + image: swiftly:fedora39-test + + install-test-setup: + image: swiftly:fedora39-install-test + build: + context: .. + dockerfile: docker/install-test-fedora39.dockerfile + args: + base_image: "fedora:39" + + install-test: + image: swiftly:fedora39-install-test + + shell: + image: swiftly:fedora39-test diff --git a/docker/docker-compose.yaml b/docker/docker-compose.yaml index 238e6ec8..68089649 100644 --- a/docker/docker-compose.yaml +++ b/docker/docker-compose.yaml @@ -53,7 +53,7 @@ services: install-test: <<: *install-test-common - command: /usr/bin/env bash -xcl "./run-tests.sh" + command: /usr/bin/env bash -xcl "echo ./run-tests.sh" # TODO remove the install tests # util diff --git a/docker/install-test-debian12.dockerfile b/docker/install-test-debian12.dockerfile new file mode 100644 index 00000000..d5680d9b --- /dev/null +++ b/docker/install-test-debian12.dockerfile @@ -0,0 +1,12 @@ +ARG base_image=debian:12 +FROM $base_image + +# set as UTF-8 +RUN apt-get update && apt-get install -y locales locales-all sqlite3 +ENV LC_ALL en_US.UTF-8 +ENV LANG en_US.UTF-8 +ENV LANGUAGE en_US.UTF-8 + +# dependencies +RUN apt-get update --fix-missing && apt-get install -y curl gpg +RUN echo 'export PATH="$HOME/.local/bin:$PATH"' >> $HOME/.profile diff --git a/docker/install-test-fedora39.dockerfile b/docker/install-test-fedora39.dockerfile new file mode 100644 index 00000000..88048fa0 --- /dev/null +++ b/docker/install-test-fedora39.dockerfile @@ -0,0 +1,5 @@ +ARG base_image=fedora:39 +FROM $base_image + +RUN yum install -y curl util-linux gpg +RUN echo 'export PATH="$HOME/.local/bin:$PATH"' >> $HOME/.profile diff --git a/docker/test-debian12.dockerfile b/docker/test-debian12.dockerfile new file mode 100644 index 00000000..da0d7a0b --- /dev/null +++ b/docker/test-debian12.dockerfile @@ -0,0 +1,18 @@ +ARG swift_version=5.10 +ARG base_image=swift:debian12 +FROM $base_image +# needed to do again after FROM due to docker limitation +ARG swift_version + +# set as UTF-8 +RUN apt-get update && apt-get install -y locales locales-all +ENV LC_ALL en_US.UTF-8 +ENV LANG en_US.UTF-8 +ENV LANGUAGE en_US.UTF-8 + +# dependencies +RUN apt-get update --fix-missing && apt-get install -y curl build-essential gpg +COPY ./scripts/install-libarchive.sh / +RUN /install-libarchive.sh + +RUN curl -L https://swift.org/keys/all-keys.asc | gpg --import diff --git a/docker/test-fedora39.dockerfile b/docker/test-fedora39.dockerfile new file mode 100644 index 00000000..c61cbd9e --- /dev/null +++ b/docker/test-fedora39.dockerfile @@ -0,0 +1,16 @@ +ARG base_image=swift:5.10-fedora39 +FROM $base_image +# needed to do again after FROM due to docker limitation +ARG swift_version + +# dependencies +RUN yum install -y \ + curl \ + gcc \ + gcc-c++ \ + make \ + gpg +COPY ./scripts/install-libarchive.sh / +RUN /install-libarchive.sh + +RUN curl -L https://swift.org/keys/all-keys.asc | gpg --import diff --git a/install/run-tests.sh b/install/run-tests.sh deleted file mode 100755 index f8eea39c..00000000 --- a/install/run-tests.sh +++ /dev/null @@ -1,48 +0,0 @@ -#!/usr/bin/env bash - -# Script used to execute all the swiftly-install tests found in install/tests. -# This should be run from the install directory of the repository. -# WARNING: these tests make changes to the local filesystem and are intended to be run in a containerized environment. - -source ./test-util.sh - -line_print () { - echo "==============================================" -} - -if ! has_command "curl"; then - echo "curl must be installed in order to run the tests" - exit 1 -fi - -tests_failed=0 -tests_passed=0 -failed_tests=() - -for t in tests/*.sh; do - test_name=$(basename "$t") - line_print - echo "Running test $test_name" - echo "" - if bash "$t"; then - ((tests_passed++)) - else - ((tests_failed++)) - failed_tests+=("$test_name") - fi -done - -line_print - -if [[ "$tests_failed" -gt 0 ]]; then - echo "" - echo "$tests_failed test(s) FAILED, $tests_passed test(s) PASSED" - echo "Failed tests:" - for failed_test in "${failed_tests[@]}"; do - echo "- $failed_test" - done - exit 1 -else - echo "All tests PASSED" - exit 0 -fi diff --git a/install/swiftly-install.sh b/install/swiftly-install.sh deleted file mode 100755 index 780475cc..00000000 --- a/install/swiftly-install.sh +++ /dev/null @@ -1,716 +0,0 @@ -#!/usr/bin/env bash - -# swiftly-install -# Script used to install and configure swiftly. -# -# This script will download the latest released swiftly executable and install it -# to $SWIFTLY_BIN_DIR, or ~/.local/bin (Linux) and ~/Library/Application Support/swiftly/bin (macOS) -# if that variable isn't specified. -# -# This script will also create a directory at $SWIFTLY_HOME_DIR, or -# $XDG_DATA_HOME/swiftly if that variable isn't specified. If XDG_DATA_HOME is also unset, -# ~/.local/share/swiftly (Linux) or ~/Library/Application Support/swift (macOS) will be used as a default -# instead. swiftly will use this directory to store platform information, downloaded toolchains, and other -# state required to manage the toolchains. -# -# After installation, the script will create $SWIFTLY_HOME_DIR/env.{sh,fish}, which can be sourced -# to properly set up the environment variables required to run swiftly. Unless --no-modify-profile -# was specified, the script will also update ~/.profile, ~/.bash_profile, ~/.bash_login, ~/.zprofile, -# $XDG_CONFIG_HOME/fish/conf.d/swiftly.fish, or ~/.config/fish/conf.d/swiftly.fish depending on -# the value of $SHELL and the existence of the files, to source the env.{sh,fish} file on login. -# This will ensure that future logins will automatically configure SWIFTLY_HOME_DIR, SWIFTLY_BIN_DIR, -# and PATH. -# -# Unless the --disable-confirmation flag is set, this script will allow the runner to -# configure either of those two directory paths. -# -# Unless the --no-install-system-deps flag is set, this script will attempt to install Swift's -# system dependencies using the system package manager. -# -# curl and getopt (from the util-linux package) are required to run this script. - -has_command () { - command -v "$1" > /dev/null -} - -read_input_with_default () { - echo -n "> " - # The installer script is usually run by "curl ... | bash", which means that - # stdin is not a tty but the script content itself. In that case, "read" builtin - # command receives EOF immediately. To avoid that, we use /dev/tty as stdin explicitly. - # SWIFTLY_READ_FROM_STDIN is used for testing interactive input - if [[ -t 0 ]] || [[ ${SWIFTLY_READ_FROM_STDIN+set} == "set" ]]; then - read READ_INPUT_RETURN - else - read READ_INPUT_RETURN < /dev/tty - fi - - if [ -z "$READ_INPUT_RETURN" ]; then - READ_INPUT_RETURN="$1" - fi -} - -yn_prompt () { - if [[ "$1" == "true" ]]; then - echo "(Y/n)" - else - echo "(y/N)" - fi -} - -# Read a y/n input. -# First argument is the default value (must be "true" or "false"). -# -# Sets READ_INPUT_RETURN to "true" for an input of "y" or "Y", "false" for an input -# of "n" or "N", or the default value for a blank input -# -# For all other inputs, a message is printed and the user is prompted again. -read_yn_input () { - while [[ true ]]; do - read_input_with_default "$1" - - case "$READ_INPUT_RETURN" in - "y" | "Y") - READ_INPUT_RETURN="true" - return - ;; - - "n" | "N") - READ_INPUT_RETURN="false" - return - ;; - - "$1") - return - ;; - - *) - echo "Please input either \"y\" or \"n\", or press ENTER to use the default." - ;; - esac - done -} - -# Replaces the actual path to $HOME at the beginning of the provided string argument with -# the string "$HOME". This is used when printing to stdout. -# e.g. "home/user/.local/bin" => "$HOME/.local/bin" -replace_home_path () { - if [[ "$1" =~ ^"$HOME"(/|$) ]]; then - echo "\$HOME${1#$HOME}" - else - echo "$1" - fi -} - -# Replaces the string "$HOME" or "~" in the argument with the actual value of $HOME. -# e.g. "$HOME/.local/bin" => "/home/user/.local/bin" -# e.g. "~/.local/bin" => "/home/user/.local/bin" -expand_home_path () { - echo "${1/#@(~|\$HOME)/$HOME}" -} - -# Prints the provided argument using the terminal's bold text effect. -bold () { - echo "$(tput bold)$1$(tput sgr0)" -} - -# Fetch the list of required system dependencies from the apple/swift-docker -# repository and attempt to install them using the system's package manager. -# -# $docker_platform_name, $docker_platform_version, and $package manager need -# to be set before calling this function. -install_system_deps () { - if [[ "$(id --user)" != "0" ]] && ! has_command sudo ; then - echo "Warning: sudo not installed and current user is not root, skipping system dependency installation." - return - elif ! has_command "$package_manager" ; then - echo "Warning: package manager \"$package_manager\" not found, skipping system dependency installation." - return - fi - - dockerfile_url="https://raw.githubusercontent.com/apple/swift-docker/main/nightly-main/$docker_platform_name/$docker_platform_version/Dockerfile" - dockerfile="$(curl --silent --retry 3 --location --fail $dockerfile_url)" - if [[ "$?" -ne 0 ]]; then - echo "Error enumerating system dependencies, skipping installation of system dependencies." - fi - - # Find the line number of the RUN command associated with installing system dependencies. - beg_line_num=$(printf "$dockerfile" | grep -n --max-count=1 "$package_manager.*install" | cut -d ":" -f1) - - # Starting from there, find the first line that starts with an & or doesn't end in a backslash. - relative_end_line_num=$(printf "$dockerfile" | - tail --lines=+"$((beg_line_num + 1))" | - grep -n --max-count=1 --invert-match '[[:space:]]*[^&].*\\$' | cut -d ":" -f1) - end_line_num=$((beg_line_num + relative_end_line_num)) - - # Read the lines between those two, deleting any spaces and backslashes. - readarray -t package_list < <(printf "$dockerfile" | sed -n "$((beg_line_num + 1)),${end_line_num}p" | sed -r 's/[\ ]//g') - - # If the installation command from the Dockerfile included some cleanup as part of a second command, drop that. - if [[ "${package_list[-1]}" =~ ^\&\& ]]; then - unset 'package_list[-1]' - fi - - # Always install gpg, since swiftly itself needs it for signature verification. - package_list+=("gpg") - - install_args=(--quiet -y) - - # Disable errexit since failing to install system dependencies is not swiftly installation-fatal. - set +o errexit - if [[ "$(id --user)" == "0" ]]; then - "$package_manager" install "${install_args[@]}" "${package_list[@]}" - else - sudo "$package_manager" install "${install_args[@]}" "${package_list[@]}" - fi - if [[ "$?" -ne 0 ]]; then - echo "System dependency installation failed." - if [[ "$package_manager" == "apt-get" ]]; then - echo "You may need to run apt-get update before installing system dependencies." - fi - fi - set -o errexit -} - -set_platform_ubuntu () { - docker_platform_name="ubuntu" - package_manager="apt-get" - export DEBIAN_FRONTEND=noninteractive - - PLATFORM_NAME="ubuntu$1$2" - PLATFORM_NAME_FULL="ubuntu$1.$2" - docker_platform_version="$1.$2" - - if [[ -z "$PLATFORM_NAME_PRETTY" ]]; then - PLATFORM_NAME_PRETTY="Ubuntu $1.$2" - fi -} - -set_platform_amazonlinux () { - PLATFORM_NAME="amazonlinux$1" - PLATFORM_NAME_FULL="amazonlinux$1" - docker_platform_name="amazonlinux" - docker_platform_version="$1" - package_manager="yum" - - if [[ -z "$PLATFORM_NAME_PRETTY" ]]; then - PLATFORM_NAME_PRETTY="Amazon Linux $1" - fi -} - -set_platform_rhel () { - PLATFORM_NAME="ubi$1" - PLATFORM_NAME_FULL="ubi$1" - docker_platform_name="rhel-ubi" - docker_platform_version="$1" - package_manager="yum" - - if [[ -z "$PLATFORM_NAME_PRETTY" ]]; then - PLATFORM_NAME_PRETTY="RHEL 9" - fi -} - -detect_platform () { - if [[ -f "/etc/os-release" ]]; then - OS_RELEASE="/etc/os-release" - elif [[ -f "/usr/lib/os-release" ]]; then - OS_RELEASE="/usr/lib/os-release" - else - manually_select_platform - fi - - source "$OS_RELEASE" - PLATFORM_NAME_PRETTY="$PRETTY_NAME" - - case "$ID$ID_LIKE" in - *"amzn"*) - if [[ "$VERSION_ID" != "2" ]]; then - manually_select_platform - else - set_platform_amazonlinux "2" - fi - ;; - - *"ubuntu"*) - case "$UBUNTU_CODENAME" in - "jammy") - set_platform_ubuntu "22" "04" - ;; - - "focal") - set_platform_ubuntu "20" "04" - ;; - - "bionic") - set_platform_ubuntu "18" "04" - ;; - - *) - manually_select_platform - ;; - esac - ;; - - *"rhel"*) - if [[ "$VERSION_ID" != 9* ]]; then - manually_select_platform - else - set_platform_rhel "9" - fi - ;; - - *) - manually_select_platform - ;; - esac -} - -manually_select_platform () { - if [[ "$DISABLE_CONFIRMATION" == "true" ]]; then - echo "Error: Unsupported platform: $PRETTY_NAME" - exit 1 - fi - echo "$PLATFORM_NAME_PRETTY is not an officially supported platform, but the toolchains for another platform may still work on it." - echo "" - echo "Please select the platform to use for toolchain downloads:" - - echo "0) Cancel" - echo "1) Ubuntu 22.04" - echo "2) Ubuntu 20.04" - echo "3) Ubuntu 18.04" - echo "4) RHEL 9" - echo "5) Amazon Linux 2" - - read_input_with_default "0" - case "$READ_INPUT_RETURN" in - "1" | "1)") - set_platform_ubuntu "22" "04" - ;; - - "2" | "2)") - set_platform_ubuntu "20" "04" - ;; - - "3" | "3)") - set_platform_ubuntu "18" "04" - ;; - - "4" | "4)") - set_platform_rhel "9" - ;; - - "5" | "5)") - set_platform_amazonlinux "2" - ;; - - *) - echo "Cancelling installation." - exit 0 - ;; - esac -} - -verify_getopt_install () { - if ! has_command "getopt" ; then - return 1 - fi - - getopt --test - # getopt --test exiting with status code 4 implies getopt from util-linux is being used, which we need. - [[ "$?" -eq 4 ]] - return "$?" -} - -SWIFTLY_INSTALL_VERSION="0.4.0" - -MODIFY_PROFILE="true" -SWIFTLY_INSTALL_SYSTEM_DEPS="true" - -if ! has_command "curl" ; then - echo "Error: curl must be installed to download swiftly" - exit 1 -fi - -IS_MACOS="false" -if [ "$(uname -s)" == "Darwin" ] ; then - IS_MACOS="true" -fi - -if [ "$IS_MACOS" == "false" ]; then - if ! verify_getopt_install ; then - echo "Error: getopt must be installed from the util-linux package to run swiftly-install" - exit 1 - fi -fi - -set -o errexit -shopt -s extglob - -if [ "$IS_MACOS" == "true" ]; then - args=$(getopt ynohvp: $*) -else - args=$(getopt --options ynohvp: --longoptions disable-confirmation,no-modify-profile,no-install-system-deps,help,version,platform:,overwrite --name swiftly-install -- "${@}") -fi - -eval "set -- ${args}" - -while [ true ]; do - case "$1" in - "--help" | "-h") - cat < Specifies which platform's toolchains swiftly will download. If - unspecified, the platform will be automatically detected. Available - options are "ubuntu22.04", "ubuntu20.04", "ubuntu18.04", "rhel9", and - "amazonlinux2". (LINUX ONLY) - -o, --overwrite Overwrite the existing swiftly installation found at the configured - SWIFTLY_HOME, if any. If this option is unspecified and an existing - installation is found, the swiftly executable will be updated, but - the rest of the installation will not be modified. - -h, --help Prints help information. - -v, --version Prints version information. - -NOTES: - macOS only works with the short options (e.g. -v and not --version). - -EOF - exit 0 - ;; - - "--disable-confirmation" | "-y") - DISABLE_CONFIRMATION="true" - shift - ;; - - "--no-modify-profile" | "-n") - MODIFY_PROFILE="false" - shift - ;; - - "--no-install-system-deps") - SWIFTLY_INSTALL_SYSTEM_DEPS="false" - shift - ;; - - "--no-import-pgp-keys") - swiftly_import_pgp_keys="false" - shift - ;; - - "--version" | "-v") - echo "$SWIFTLY_INSTALL_VERSION" - exit 0 - ;; - - "--platform" | "-p") - case "$2" in - "ubuntu22.04") - set_platform_ubuntu "22" "04" - ;; - - "ubuntu20.04") - set_platform_ubuntu "20" "04" - ;; - - "ubuntu18.04") - set_platform_ubuntu "18" "04" - ;; - - "amazonlinux2") - set_platform_amazonlinux "2" - ;; - - "rhel9") - set_platform_rhel "9" - ;; - - *) - echo "Error: unrecognized platform $2" - exit 1 - ;; - esac - shift 2 - ;; - - "--overwrite" | "-o") - overwrite_existing_intallation="true" - shift - ;; - - --) - shift - break - ;; - *) - echo "Error: unrecognized option \"$1\"" - if [ "$IS_MACOS" == "true" ]; then - echo "Note that on macOS you must use the short options (e.g. -v, not --version)." - fi - exit 1 - ;; - esac -done - -if [ "$IS_MACOS" == "false" ]; then - if [[ -z "$PLATFORM_NAME" ]]; then - detect_platform - fi -fi - -RAW_ARCH="$(uname -m)" -case "$RAW_ARCH" in - "x86_64") - ARCH="x86_64" - PLATFORM_ARCH="null" - ;; - - "aarch64" | "arm64") - ARCH="aarch64" - PLATFORM_ARCH='"aarch64"' - ;; - - *) - echo "Error: Unsupported CPU architecture: $RAW_ARCH" - exit 1 - ;; -esac - -if [ "$IS_MACOS" == "false" ]; then - JSON_OUT=$(cat < "$HOME_DIR/config.json" - fi - - # Verify the downloaded executable works. The script will exit if this fails due to errexit. - SWIFTLY_HOME_DIR="$HOME_DIR" SWIFTLY_BIN_DIR="$BIN_DIR" "$BIN_DIR/swiftly" --version > /dev/null - - echo "$ENV_OUT" > "$HOME_DIR/$ENV_FILE" - - if [[ "$MODIFY_PROFILE" == "true" ]]; then - # Only append the line if it isn't in .profile already. - if [[ ! -f "$PROFILE_FILE" ]] || [[ ! "$(cat $PROFILE_FILE)" =~ "$SOURCE_LINE" ]]; then - echo "$SOURCE_LINE" >> "$PROFILE_FILE" - fi - fi -fi - -if [ "$IS_MACOS" == "false" ]; then - if [[ "$SWIFTLY_INSTALL_SYSTEM_DEPS" != "false" ]]; then - echo "" - echo "Installing Swift's system dependencies via $package_manager (note: this may require root access)..." - install_system_deps - fi -fi - -if [ "$IS_MACOS" == "false" ]; then - if [[ "$swiftly_import_pgp_keys" != "false" ]]; then - if has_command gpg ; then - echo "" - echo "Importing Swift's PGP keys..." - curl --silent --retry 3 --location --fail https://swift.org/keys/all-keys.asc | gpg --import - - else - echo "gpg not installed, skipping import of Swift's PGP keys." - fi - fi -fi - -echo "" -echo "swiftly has been successfully installed!" -echo "" - -if ! has_command "swiftly" || [[ "$HOME_DIR" != "$DEFAULT_HOME_DIR" || "$BIN_DIR" != "$DEFAULT_BIN_DIR" ]] ; then - if [[ "$MODIFY_PROFILE" == "true" ]]; then - echo "Once you log in again, swiftly should be accessible from your PATH." - fi - echo "To begin using swiftly from your current shell, first run the following command:" - echo "" - echo " $SOURCE_LINE" - echo "" - echo "Then to install the latest version of Swift, run 'swiftly install latest'" -else - echo "To install the latest version of Swift, run 'swiftly install latest'" -fi - -if has_command "swift" ; then - echo "" - echo "Warning: existing installation of Swift detected at $(command -v swift)" - echo "To ensure swiftly-installed toolchains can be found by the shell, uninstall any existing Swift installation(s)." - echo "To ensure the current shell can find swiftly-installed toolchains, you may also need to run 'hash -r'." -fi diff --git a/install/test-util.sh b/install/test-util.sh deleted file mode 100644 index f2c2ffad..00000000 --- a/install/test-util.sh +++ /dev/null @@ -1,88 +0,0 @@ -#!/usr/bin/env bash - -# Common utility functionality used in the various bash tests for swiftly-install.sh. - -export SWIFTLY_READ_FROM_STDIN=1 - -test_log () { - echo "===========================" - echo "$1" - echo "===========================" -} - -has_command () { - command -v "$1" > /dev/null -} - -test_name () { - basename "$0" -} - -test_fail () { - if [ ! -z "$1" ]; then - printf "$1\n" - fi - - if [ ! -z "$3" ]; then - printf "actual: $2\n" - printf "expected: $3\n" - fi - echo "" - echo "$(test_name) FAILED" - exit 1 -} - -test_pass () { - echo "" - echo "$(test_name) PASSED" - exit 0 -} - -get_os () { - if [[ -f "/etc/os-release" ]]; then - OS_RELEASE="/etc/os-release" - elif [[ -f "/usr/lib/os-release" ]]; then - OS_RELEASE="/usr/lib/os-release" - else - echo "Error: could not detect OS information" - exit 1 - fi - - source "$OS_RELEASE" - - case "$ID" in - "amzn") - echo "amazonlinux2" - ;; - - "ubuntu") - case "$UBUNTU_CODENAME" in - "jammy") - echo "ubuntu2204" - ;; - - "focal") - echo "ubuntu2004" - ;; - - "bionic") - echo "ubuntu1804" - ;; - - *) - echo "Unsupported Ubuntu version: $PRETTY_NAME" - exit 1 - ;; - esac - ;; - - "rhel") - echo "rhel-ubi9" - ;; - - *) - echo "Unsupported platform: $PRETTY_NAME" - exit 1 - ;; - esac -} diff --git a/install/tests/cancel-install.sh b/install/tests/cancel-install.sh deleted file mode 100755 index f8ab2d43..00000000 --- a/install/tests/cancel-install.sh +++ /dev/null @@ -1,19 +0,0 @@ -#!/usr/bin/env bash - -# Tests that installation can be cancelled. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -echo "3" | ./swiftly-install.sh - -if has_command "swiftly" ; then - test_fail "swiftly executable should not have been installed" -fi - -if [[ -d "$HOME/.local/share/swiftly" ]]; then - test_fail "SWIFTLY_HOME_DIR should not have been created" -fi - -test_pass diff --git a/install/tests/custom-home-install.sh b/install/tests/custom-home-install.sh deleted file mode 100755 index 3a0ec831..00000000 --- a/install/tests/custom-home-install.sh +++ /dev/null @@ -1,44 +0,0 @@ -#!/usr/bin/env bash - -# Tests that custom install paths that include the string "$HOME" are expanded properly. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -export CUSTOM_HOME_DIR_NAME="home-substitution-test-home" -export CUSTOM_HOME_DIR="$HOME/$CUSTOM_HOME_DIR_NAME" -export CUSTOM_BIN_DIR="$CUSTOM_HOME_DIR/bin" - -cp "$HOME/.profile" "$HOME/.profile.bak" - -cleanup () { - rm -r "$CUSTOM_HOME_DIR" - mv "$HOME/.profile.bak" "$HOME/.profile" -} -trap cleanup EXIT - -printf "2\n\$HOME/${CUSTOM_HOME_DIR_NAME}\n\$HOME/${CUSTOM_HOME_DIR_NAME}/bin\ny\nn\n1\n" | ./swiftly-install.sh - -# .profile should be updated to update PATH and SWIFTLY_HOME_DIR/SWIFTLY_BIN_DIR. -bash --login -c "swiftly --version" - -. "$CUSTOM_HOME_DIR/env.sh" - -if ! has_command "swiftly" ; then - test_fail "Can't find swiftly on the PATH" -fi - -if [[ "$SWIFTLY_HOME_DIR" != "$CUSTOM_HOME_DIR" ]]; then - test_fail "SWIFTLY_HOME_DIR ($SWIFTLY_HOME_DIR) did not equal $CUSTOM_HOME_DIR" -fi - -if [[ "$SWIFTLY_BIN_DIR" != "$CUSTOM_BIN_DIR" ]]; then - test_fail "SWIFTLY_BIN_DIR ($SWIFTLY_BIN_DIR) did not equal $CUSTOM_BIN_DIR" -fi - -if [[ ! -d "$CUSTOM_HOME_DIR/toolchains" ]]; then - test_fail "the toolchains directory was not created in SWIFTLY_HOME_DIR" -fi - -test_pass diff --git a/install/tests/custom-install.sh b/install/tests/custom-install.sh deleted file mode 100755 index 3b0b0c1f..00000000 --- a/install/tests/custom-install.sh +++ /dev/null @@ -1,70 +0,0 @@ -#!/usr/bin/env bash - -# Tests that an installation using custom paths works properly. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -export CUSTOM_HOME_DIR="$(pwd)custom-test-home" -export CUSTOM_BIN_DIR="$CUSTOM_HOME_DIR/bin" -export PATH="$CUSTOM_BIN_DIR:$PATH" - -cp "$HOME/.profile" "$HOME/.profile.bak" -cleanup () { - set +o errexit - - mv "$HOME/.profile.bak" "$HOME/.profile" - - if has_command "swiftly" ; then - swiftly uninstall -y latest > /dev/null - fi - - rm -r "$CUSTOM_HOME_DIR" -} -trap cleanup EXIT - -# Custom home dir -# Custom bin dir -# Modify login config (yes) -# Install system dependencies (no) -printf "2\n$CUSTOM_HOME_DIR\n$CUSTOM_BIN_DIR\ny\nn\n1\n" | ./swiftly-install.sh - -# .profile should be updated to update PATH and SWIFTLY_HOME_DIR/SWIFTLY_BIN_DIR. -bash --login -c "swiftly --version" - -. "$CUSTOM_HOME_DIR/env.sh" - -if ! has_command "swiftly" ; then - test_fail "Can't find swiftly on the PATH" -fi - -if [[ "$SWIFTLY_HOME_DIR" != "$CUSTOM_HOME_DIR" ]]; then - test_fail "SWIFTLY_HOME_DIR ($SWIFTLY_HOME_DIR) did not equal $CUSTOM_HOME_DIR" -fi - -if [[ "$SWIFTLY_BIN_DIR" != "$CUSTOM_BIN_DIR" ]]; then - test_fail "SWIFTLY_BIN_DIR ($SWIFTLY_BIN_DIR) did not equal $CUSTOM_BIN_DIR" -fi - -if [[ ! -d "$CUSTOM_HOME_DIR/toolchains" ]]; then - test_fail "the toolchains directory was not created in SWIFTLY_HOME_DIR" -fi - -if [[ -d "$HOME/.local/share/swiftly" ]]; then - test_fail "expected default home directory to not be created, but it was" -fi - -swiftly install 5.8.0 - -swift --version - -if [[ ! -d "$CUSTOM_HOME_DIR/toolchains/5.8.0" ]]; then - test_fail "the toolchain was not installed to the custom directory" -fi - -if [[ -d "$HOME/.local/share/swiftly" ]]; then - test_fail "expected default home directory to not be created, but it was" -fi - -test_pass diff --git a/install/tests/custom-tilde-install.sh b/install/tests/custom-tilde-install.sh deleted file mode 100755 index 5c0d1de8..00000000 --- a/install/tests/custom-tilde-install.sh +++ /dev/null @@ -1,45 +0,0 @@ -#!/usr/bin/env bash - -# Tests that custom install paths that include the "~" character are expanded properly. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -export CUSTOM_HOME_DIR_NAME="tilde-substitution-test-home" -export CUSTOM_HOME_DIR="$HOME/$CUSTOM_HOME_DIR_NAME" -export CUSTOM_BIN_DIR="$CUSTOM_HOME_DIR/bin" - -cp "$HOME/.profile" "$HOME/.profile.bak" - -cleanup () { - rm -r "$CUSTOM_HOME_DIR" - mv "$HOME/.profile.bak" "$HOME/.profile" -} -trap cleanup EXIT - -# Make sure that the "~" character is handled properly. -printf "2\n~/${CUSTOM_HOME_DIR_NAME}\n~/${CUSTOM_HOME_DIR_NAME}/bin\ny\nn\n1\n" | ./swiftly-install.sh - -# .profile should be updated to update PATH and SWIFTLY_HOME_DIR/SWIFTLY_BIN_DIR. -bash --login -c "swiftly --version" - -. "$CUSTOM_HOME_DIR/env.sh" - -if ! has_command "swiftly" ; then - test_fail "Can't find swiftly on the PATH" -fi - -if [[ "$SWIFTLY_HOME_DIR" != "$CUSTOM_HOME_DIR" ]]; then - test_fail "SWIFTLY_HOME_DIR ($SWIFTLY_HOME_DIR) did not equal $CUSTOM_HOME_DIR" -fi - -if [[ "$SWIFTLY_BIN_DIR" != "$CUSTOM_BIN_DIR" ]]; then - test_fail "SWIFTLY_BIN_DIR ($SWIFTLY_BIN_DIR) did not equal $CUSTOM_BIN_DIR" -fi - -if [[ ! -d "$CUSTOM_HOME_DIR/toolchains" ]]; then - test_fail "the toolchains directory was not created in SWIFTLY_HOME_DIR" -fi - -test_pass diff --git a/install/tests/default-install.sh b/install/tests/default-install.sh deleted file mode 100755 index df218101..00000000 --- a/install/tests/default-install.sh +++ /dev/null @@ -1,159 +0,0 @@ -#!/usr/bin/env bash - -# Tests that an unconfigured installation works properly. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -cp "$HOME/.profile" "$HOME/.profile.bak" - -cleanup () { - set +o errexit - - mv "$HOME/.profile.bak" "$HOME/.profile" - - if has_command "swiftly" ; then - swiftly uninstall -y latest > /dev/null - fi - - rm -r "$HOME/.local/share/swiftly" - rm "$HOME/.local/bin/swiftly" -} -trap cleanup EXIT - -case "$(get_os)" in - "ubuntu1804") - system_deps=(binutils - git - libc6-dev - libcurl4-openssl-dev - libedit2 - libgcc-5-dev - libpython3.6 - libstdc++-5-dev - libxml2-dev - pkg-config - tzdata - zip - zlib1g-dev) - ;; - - "ubuntu2004") - system_deps=(binutils - git - gnupg2 - libc6-dev - libcurl4-openssl-dev - libedit2 - libgcc-9-dev - libpython3.8 - libstdc++-9-dev - libxml2-dev - libz3-dev - pkg-config - tzdata - zip - zlib1g-dev) - ;; - - "ubuntu2204") - system_deps=(binutils - git - gnupg2 - libc6-dev - libcurl4-openssl-dev - libedit2 - libgcc-11-dev - libpython3-dev - libstdc++-11-dev - libxml2-dev - libz3-dev - pkg-config - tzdata - zip - zlib1g-dev) - ;; - - "amazonlinux2") - system_deps=(binutils - gcc - git - glibc-static - libcurl-devel - libedit - libicu - libxml2-devel - tar - unzip - zip - zlib-devel) - ;; - - "rhel-ubi9") - system_deps=(git - gcc-c++ - libcurl-devel - libedit-devel - libuuid-devel - libxml2-devel - ncurses-devel - python3-devel - rsync - sqlite-devel - unzip - zip) - ;; - - *) - echo "Unrecognized platform" - exit 1 - ;; -esac - -if has_command apt-get ; then - apt-get update - apt-get remove -y "${system_deps[@]}" -elif has_command yum ; then - yum remove -y "${system_deps[@]}" -fi - -printf "1\n" | ./swiftly-install.sh - -# .profile should be updated to update PATH. -bash --login -c "swiftly --version" - -. "$HOME/.local/share/swiftly/env.sh" - -if ! has_command "swiftly" ; then - test_fail "Can't find swiftly on the PATH" -fi - -if [[ ! -d "$HOME/.local/share/swiftly/toolchains" ]]; then - test_fail "the toolchains directory was not created in SWIFTLY_HOME_DIR" -fi - -echo "Verifying system dependencies were installed..." -for dep in "${system_deps[@]}"; do - if has_command dpkg ; then - if ! dpkg --status "$dep" > /dev/null ; then - test_fail "System dependency $dep was not installed properly" - fi - elif has_command rpm ; then - if ! rpm -q "$dep" > /dev/null ; then - test_fail "System dependency $dep was not installed properly" - fi - fi - echo "System dependency $dep was installed successfully" -done - - -if ! gpg --list-keys Swift ; then - test_fail "Swift PGP keys were not installed by default." -fi - -swiftly install latest - -swift --version - -test_pass diff --git a/install/tests/disable-prompt.sh b/install/tests/disable-prompt.sh deleted file mode 100755 index fbe66807..00000000 --- a/install/tests/disable-prompt.sh +++ /dev/null @@ -1,45 +0,0 @@ -#!/usr/bin/env bash - -# Tests that the --disable-confirmation argument works. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -cp "$HOME/.profile" "$HOME/.profile.bak" - -cleanup () { - mv "$HOME/.profile.bak" "$HOME/.profile" - rm -r "$HOME/.local/share/swiftly" - rm -r "$HOME/.local/bin/swiftly" -} -trap cleanup EXIT - -if has_command apt-get ; then - apt-get remove -y zlib1g-dev -elif has_command yum ; then - yum remove -y libcurl-devel -fi - -./swiftly-install.sh -y - -# .profile should be updated to update PATH. -bash --login -c "swiftly --version" - -. "$HOME/.local/share/swiftly/env.sh" - -if ! has_command "swiftly" ; then - test_fail "Can't find swiftly on the PATH" -fi - -if has_command dpkg ; then - if ! dpkg --status zlib1g-dev ; then - test_fail "System dependencies were not installed properly" - fi -elif has_command rpm ; then - if ! rpm -q libcurl-devel ; then - test_fail "System dependencies were not installed properly" - fi -fi - -test_pass diff --git a/install/tests/no-install-dependencies.sh b/install/tests/no-install-dependencies.sh deleted file mode 100755 index 352f088c..00000000 --- a/install/tests/no-install-dependencies.sh +++ /dev/null @@ -1,45 +0,0 @@ -#!/usr/bin/env bash - -# Tests passing --no-install-system-deps disables installing system dependencies. -# Also verifies that interactive customization also does cancels them. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -cleanup () { - set +o errexit - - rm -r "$HOME/.local/share/swiftly" - rm "$HOME/.local/bin/swiftly" -} -trap cleanup EXIT - -verify_dependencies_not_installed () { - if has_command dpkg ; then - if dpkg --status zlib1g-dev ; then - test_fail "System dependencies were installed when they shouldn't have been" - fi - elif has_command rpm ; then - if rpm -q libcurl-devel ; then - test_fail "System dependencies were installed when they shouldn't have been" - fi - fi -} - -if has_command apt-get ; then - apt-get remove -y zlib1g-dev -elif has_command yum ; then - yum remove -y libcurl-devel -fi - -echo "1" | ./swiftly-install.sh --no-install-system-deps - -verify_dependencies_not_installed - -# Use all defaults except "n" for system dependency installation. -printf "2\n\n\n\nn\n1\ny\n" | ./swiftly-install.sh - -verify_dependencies_not_installed - -test_pass diff --git a/install/tests/overwrite.sh b/install/tests/overwrite.sh deleted file mode 100755 index 7a935115..00000000 --- a/install/tests/overwrite.sh +++ /dev/null @@ -1,89 +0,0 @@ -#!/usr/bin/env bash - -# Tests that swiftly-install properly handles an existing installation of swiftly. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -export SWIFTLY_HOME_DIR="./overwrite-test-home" -export SWIFTLY_BIN_DIR="./overwrite-bin-dir" - -cp "$HOME/.profile" "$HOME/.profile.bak" - -cleanup () { - mv "$HOME/.profile.bak" "$HOME/.profile" - rm -r "$SWIFTLY_HOME_DIR" - rm -r "$SWIFTLY_BIN_DIR" -} -trap cleanup EXIT - -test_log "Performing initial installation" -./swiftly-install.sh -y --no-install-system-deps - -. "$SWIFTLY_HOME_DIR/env.sh" - -if ! has_command "swiftly" ; then - test_fail "Can't find swiftly on the PATH" -fi - -# Modify the home dir to be able to to tell if it is changed with subsequent installs. -DUMMY_CONFIG_CONTENTS="hello world" -PROFILE_CONTENTS="$(cat $HOME/.profile)" -echo "$DUMMY_CONFIG_CONTENTS" > "$SWIFTLY_HOME_DIR/config.json" - -toolchain_dir="$SWIFTLY_HOME_DIR/toolchains/5.7.3" -mkdir -p "$toolchain_dir/usr/bin" -dummy_executable_name="foo" -touch "$toolchain_dir/usr/bin/$dummy_executable_name" - -# Also set up a symlink as if the toolchain were in use. -ln -s -t $SWIFTLY_BIN_DIR "$toolchain_dir/usr/bin/$dummy_executable_name" - -test_log "Attempting the same installation (no --overwrite flag specified)" -./swiftly-install.sh -y --no-install-system-deps - -if ! has_command "swiftly" ; then - test_fail "Can't find swiftly on the PATH" -fi - -NEW_CONFIG_CONTENTS="$(cat $SWIFTLY_HOME_DIR/config.json)" -if [[ "$NEW_CONFIG_CONTENTS" != "$DUMMY_CONFIG_CONTENTS" ]]; then - test_fail "Expected config to remain unchanged" "$NEW_CONFIG_CONTENTS" "$DUMMY_CONFIG_CONTENTS" -fi - -if ! [ -L "$SWIFTLY_BIN_DIR/$dummy_executable_name" ]; then - test_fail "Expected symlink to still exist, but it has been deleted" -fi - -if [[ ! -d "$SWIFTLY_HOME_DIR/toolchains/5.7.3" ]]; then - test_fail "Expected installed toolchain directory to still exist, but it has been deleted" -fi - -test_log "Attempting the same installation (--overwrite flag is specified)" -./swiftly-install.sh -y --overwrite --no-install-system-deps - -if ! has_command "swiftly" ; then - test_fail "Can't find swiftly on the PATH" -fi - -NEW_CONFIG_CONTENTS="$(cat $SWIFTLY_HOME_DIR/config.json)" -if [[ "$NEW_CONFIG_CONTENTS" == "DUMMY_CONFIG_CONTENTS" ]]; then - test_fail "Expected config to be reset but it was not" -fi - -if [[ "$(cat $HOME/.profile)" != "$PROFILE_CONTENTS" ]]; then - test_fail "Expected .profile not to be updated on overwrite install" -fi - -if [[ -d "$SWIFTLY_HOME_DIR/toolchains/5.7.3" ]]; then - test_fail "Expected installed toolchain directory to have been overwritten, but it still exists" -fi - -if [ -L "$SWIFTLY_BIN_DIR/$dummy_executable_name" ]; then - test_fail "Expected symlink to have been deleted, but it still exists" -fi - -swiftly --version - -test_pass diff --git a/install/tests/platform-option.sh b/install/tests/platform-option.sh deleted file mode 100755 index fe776f93..00000000 --- a/install/tests/platform-option.sh +++ /dev/null @@ -1,32 +0,0 @@ -#!/usr/bin/env bash - -# Tests that a platform can be manually specified via the --platform option. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -cleanup () { - set +o errexit - - rm -r "$HOME/.local/share/swiftly" - rm "$HOME/.local/bin/swiftly" -} -trap cleanup EXIT - -platforms=("ubuntu22.04" "ubuntu20.04" "ubuntu18.04" "amazonlinux2" "rhel9") - -for platform in "${platforms[@]}"; do - ./swiftly-install.sh --overwrite --disable-confirmation --no-install-system-deps --platform "$platform" - cat $HOME/.local/share/swiftly/config.json - - if [[ "$platform" == "rhel9" ]]; then - platform="ubi9" - fi - - if ! grep -q "\"nameFull\": \"$platform\"" "$HOME/.local/share/swiftly/config.json" ; then - test_fail "platform option had no effect for platform \"$platform\"" - fi -done - -test_pass diff --git a/install/tests/update-bash-profile.sh b/install/tests/update-bash-profile.sh deleted file mode 100755 index ff89b805..00000000 --- a/install/tests/update-bash-profile.sh +++ /dev/null @@ -1,47 +0,0 @@ -#!/usr/bin/env bash - -# Tests that .zprofile is updated properly if the user's shell is zsh. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -cleanup () { - set +o errexit - - if [[ -f "$HOME/.bash_profile" ]]; then - rm "$HOME/.bash_profile" - fi - rm "$HOME/.bash_login" - - rm -r "$HOME/.local/share/swiftly" - rm "$HOME/.local/bin/swiftly" -} -trap cleanup EXIT - -touch "$HOME/.bash_profile" -touch "$HOME/.bash_login" -export SHELL="bash" - -echo "1" | ./swiftly-install.sh --no-install-system-deps - -if [[ ! "$(cat $HOME/.bash_profile)" =~ "swiftly/env.sh" ]]; then - test_fail "install did not update .bash_profile" -fi - -if [[ "$(cat $HOME/.bash_login)" != "" ]]; then - test_fail "install updated .bash_login when .bash_profile existed" -fi - -rm "$HOME/.bash_profile" -./swiftly-install.sh -y --overwrite --no-install-system-deps - -if [[ -f "$HOME/.bash_profile" ]]; then - test_fail "install created .bash_profile when it should not have" -fi - -if [[ ! "$(cat $HOME/.bash_login)" =~ "swiftly/env.sh" ]]; then - test_fail "install did not update .bash_login" -fi - -test_pass diff --git a/install/tests/update-fish-config.sh b/install/tests/update-fish-config.sh deleted file mode 100755 index 2f1fc525..00000000 --- a/install/tests/update-fish-config.sh +++ /dev/null @@ -1,37 +0,0 @@ -#!/usr/bin/env bash - -# Tests that swiftly.fish is updated properly if the user's shell is fish. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -cleanup () { - set +o errexit - - rm "$HOME/.config/fish/conf.d/swiftly.fish" - rm "$HOME/.xdg/config/fish/conf.d/swiftly.fish" - - rm -r "$HOME/.local/share/swiftly" - rm "$HOME/.local/bin/swiftly" -} -trap cleanup EXIT - -export SHELL="fish" - -mkdir -p "$HOME/.config/fish/conf.d" -echo "1" | ./swiftly-install.sh --no-install-system-deps - -if [[ ! "$(cat $HOME/.config/fish/conf.d/swiftly.fish)" =~ "swiftly/env.fish" ]]; then - test_fail "install did not update ~/.config/fish/conf.d/swiftly.fish" -fi - -export XDG_CONFIG_HOME="$HOME/.xdg/config" -mkdir -p "$XDG_CONFIG_HOME/fish/conf.d" -./swiftly-install.sh -y --overwrite --no-install-system-deps - -if [[ ! "$(cat $XDG_CONFIG_HOME/fish/conf.d/swiftly.fish)" =~ "swiftly/env.fish" ]]; then - test_fail "install did not update \$XDG_CONFIG_HOME/fish/conf.d/swiftly.fish" -fi - -test_pass diff --git a/install/tests/update-zprofile.sh b/install/tests/update-zprofile.sh deleted file mode 100755 index 5eb7cc1d..00000000 --- a/install/tests/update-zprofile.sh +++ /dev/null @@ -1,28 +0,0 @@ -#!/usr/bin/env bash - -# Tests that .zprofile is updated properly if the user's shell is zsh. -# WARNING: this test makes changes to the local filesystem and is intended to be run in a containerized environment. - -set -o errexit -source ./test-util.sh - -cleanup () { - set +o errexit - - rm "$HOME/.zprofile" - - rm -r "$HOME/.local/share/swiftly" - rm "$HOME/.local/bin/swiftly" -} -trap cleanup EXIT - -touch "$HOME/.zprofile" -export SHELL="zsh" - -echo "1" | ./swiftly-install.sh --no-install-system-deps - -if [[ ! "$(cat $HOME/.zprofile)" =~ "swiftly/env.sh" ]]; then - test_fail "install did not update .zprofile" -fi - -test_pass From 0cc129caad3de9d3cc679ed23bab93a1e011e6e3 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 19 Oct 2024 16:20:58 -0400 Subject: [PATCH 24/45] Add the new entries to the --platform option --- Sources/LinuxPlatform/Linux.swift | 68 ++++++++++--------------------- 1 file changed, 21 insertions(+), 47 deletions(-) diff --git a/Sources/LinuxPlatform/Linux.swift b/Sources/LinuxPlatform/Linux.swift index 0f3b1e8d..70931bd8 100644 --- a/Sources/LinuxPlatform/Linux.swift +++ b/Sources/LinuxPlatform/Linux.swift @@ -466,7 +466,7 @@ public struct Linux: Platform { } } - private func manualSelectPlatform(_ platformPretty: String?) -> PlatformDefinition { + private func manualSelectPlatform(_ platformPretty: String?) async -> PlatformDefinition { if let platformPretty = platformPretty { print("\(platformPretty) is not an officially supported platform, but the toolchains for another platform may still work on it.") } else { @@ -485,7 +485,7 @@ public struct Linux: Platform { PlatformDefinition.debian12, ] - let selections = linuxPlatforms.enumerated().map { "\($0)) \($1.namePretty)" }.joined(separator: "\n") + let selections = linuxPlatforms.enumerated().map { "\($0 + 1)) \($1.namePretty)" }.joined(separator: "\n") print(""" Please select the platform to use for toolchain downloads: @@ -494,36 +494,29 @@ public struct Linux: Platform { \(selections) """) - let choice = SwiftlyCore.readLine(prompt: "> ") ?? "0" + let choice = SwiftlyCore.readLine(prompt: "Pick one of the available selections [0-\(linuxPlatforms.count)] ") ?? "0" guard let choiceNum = Int(choice) else { fatalError("Installation canceled") } - guard choiceNum >= 0 && choiceNum <= linuxPlatforms.count else { + guard choiceNum > 0 && choiceNum <= linuxPlatforms.count else { fatalError("Installation canceled") } - return linuxPlatforms[choiceNum] + return linuxPlatforms[choiceNum - 1] } public func detectPlatform(disableConfirmation: Bool, platform: String?) async throws -> PlatformDefinition { // We've been given a hint to use - if let platform = platform { - switch platform { - case "ubuntu22.04": - return PlatformDefinition.ubuntu2204 - case "ubuntu20.04": - return PlatformDefinition.ubuntu2004 - case "ubuntu18.04": - return PlatformDefinition.ubuntu1804 - case "amazonlinux2": - return PlatformDefinition.amazonlinux2 - case "rhel9": - return PlatformDefinition.rhel9 - default: - fatalError("Unrecognized platform \(platform)") + if let platform { + let linuxPlatforms = [PlatformDefinition.ubuntu2404, .ubuntu2310, .ubuntu2204, .ubuntu2004, .ubuntu1804, .amazonlinux2, .rhel9, .fedora39, .debian12] + + guard let pd = linuxPlatforms.first(where: { $0.nameFull == platform }) else { + fatalError("Unrecognized platform \(platform). Recognized values: \(linuxPlatforms.map(\.nameFull).joined(separator: ", ")).") } + + return pd } let osReleaseFiles = ["/etc/os-release", "/usr/lib/os-release"] @@ -544,29 +537,10 @@ public struct Linux: Platform { } else { print(message) } - return self.manualSelectPlatform(platformPretty) + return await self.manualSelectPlatform(platformPretty) } - let data = FileManager.default.contents(atPath: releaseFile) - guard let data = data else { - let message = "Unable to read OS release information from file \(releaseFile)" - if disableConfirmation { - throw Error(message: message) - } else { - print(message) - } - return self.manualSelectPlatform(platformPretty) - } - - guard let releaseInfo = String(data: data, encoding: .utf8) else { - let message = "Unable to read OS release information from file \(releaseFile)" - if disableConfirmation { - throw Error(message: message) - } else { - print(message) - } - return self.manualSelectPlatform(platformPretty) - } + let releaseInfo = try String(contentsOfFile: releaseFile, encoding: .utf8) var id: String? var idlike: String? @@ -583,17 +557,17 @@ public struct Linux: Platform { } } - guard let id = id, let idlike = idlike, let versionID else { + guard let id, let versionID else { let message = "Unable to find release information from file \(releaseFile)" if disableConfirmation { throw Error(message: message) } else { print(message) } - return self.manualSelectPlatform(platformPretty) + return await self.manualSelectPlatform(platformPretty) } - if (id + idlike).contains("amzn") { + if (id + (idlike ?? "")).contains("amzn") { guard versionID == "2" else { let message = "Unsupported version of Amazon Linux" if disableConfirmation { @@ -601,11 +575,11 @@ public struct Linux: Platform { } else { print(message) } - return self.manualSelectPlatform(platformPretty) + return await self.manualSelectPlatform(platformPretty) } return PlatformDefinition.amazonlinux2 - } else if (id + idlike).contains("rhel") { + } else if (id + (idlike ?? "")).contains("rhel") { guard versionID.hasPrefix("9") else { let message = "Unsupported version of RHEL" if disableConfirmation { @@ -613,7 +587,7 @@ public struct Linux: Platform { } else { print(message) } - return self.manualSelectPlatform(platformPretty) + return await self.manualSelectPlatform(platformPretty) } return PlatformDefinition.rhel9 @@ -627,7 +601,7 @@ public struct Linux: Platform { } else { print(message) } - return self.manualSelectPlatform(platformPretty) + return await self.manualSelectPlatform(platformPretty) } public func getShell() async throws -> String { From afbd797768de728d7ed39af575ba8bec672f0e83 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 19 Oct 2024 16:24:22 -0400 Subject: [PATCH 25/45] Pull list of linux platforms out into a var. --- Sources/LinuxPlatform/Linux.swift | 36 +++++++++++++++---------------- 1 file changed, 17 insertions(+), 19 deletions(-) diff --git a/Sources/LinuxPlatform/Linux.swift b/Sources/LinuxPlatform/Linux.swift index 70931bd8..a237137d 100644 --- a/Sources/LinuxPlatform/Linux.swift +++ b/Sources/LinuxPlatform/Linux.swift @@ -7,6 +7,18 @@ var swiftGPGKeysRefreshed = false /// This implementation can be reused for any supported Linux platform. /// TODO: replace dummy implementations public struct Linux: Platform { + let linuxPlatforms = [ + PlatformDefinition.ubuntu2404, + PlatformDefinition.ubuntu2310, + PlatformDefinition.ubuntu2204, + PlatformDefinition.ubuntu2004, + PlatformDefinition.ubuntu1804, + PlatformDefinition.fedora39, + PlatformDefinition.rhel9, + PlatformDefinition.amazonlinux2, + PlatformDefinition.debian12, + ] + public init() {} public var appDataDirectory: URL { @@ -473,19 +485,7 @@ public struct Linux: Platform { print("This platform could not be detected, but a toolchain for one of the supported platforms may work on it.") } - let linuxPlatforms = [ - PlatformDefinition.ubuntu2404, - PlatformDefinition.ubuntu2310, - PlatformDefinition.ubuntu2204, - PlatformDefinition.ubuntu2004, - PlatformDefinition.ubuntu1804, - PlatformDefinition.fedora39, - PlatformDefinition.rhel9, - PlatformDefinition.amazonlinux2, - PlatformDefinition.debian12, - ] - - let selections = linuxPlatforms.enumerated().map { "\($0 + 1)) \($1.namePretty)" }.joined(separator: "\n") + let selections = self.linuxPlatforms.enumerated().map { "\($0 + 1)) \($1.namePretty)" }.joined(separator: "\n") print(""" Please select the platform to use for toolchain downloads: @@ -494,26 +494,24 @@ public struct Linux: Platform { \(selections) """) - let choice = SwiftlyCore.readLine(prompt: "Pick one of the available selections [0-\(linuxPlatforms.count)] ") ?? "0" + let choice = SwiftlyCore.readLine(prompt: "Pick one of the available selections [0-\(self.linuxPlatforms.count)] ") ?? "0" guard let choiceNum = Int(choice) else { fatalError("Installation canceled") } - guard choiceNum > 0 && choiceNum <= linuxPlatforms.count else { + guard choiceNum > 0 && choiceNum <= self.linuxPlatforms.count else { fatalError("Installation canceled") } - return linuxPlatforms[choiceNum - 1] + return self.linuxPlatforms[choiceNum - 1] } public func detectPlatform(disableConfirmation: Bool, platform: String?) async throws -> PlatformDefinition { // We've been given a hint to use if let platform { - let linuxPlatforms = [PlatformDefinition.ubuntu2404, .ubuntu2310, .ubuntu2204, .ubuntu2004, .ubuntu1804, .amazonlinux2, .rhel9, .fedora39, .debian12] - guard let pd = linuxPlatforms.first(where: { $0.nameFull == platform }) else { - fatalError("Unrecognized platform \(platform). Recognized values: \(linuxPlatforms.map(\.nameFull).joined(separator: ", ")).") + fatalError("Unrecognized platform \(platform). Recognized values: \(self.linuxPlatforms.map(\.nameFull).joined(separator: ", ")).") } return pd From d301ef37b7869fcb5e5d4a872700781a4444d0a7 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 19 Oct 2024 16:31:02 -0400 Subject: [PATCH 26/45] Remove all of the platform def environment variables to force auto-detection coverage --- docker/docker-compose.1804.yaml | 4 ---- docker/docker-compose.2004.yaml | 4 ---- docker/docker-compose.2204.yaml | 4 ---- docker/docker-compose.2310.yaml | 4 ---- docker/docker-compose.2404.yaml | 4 ---- docker/docker-compose.amazonlinux2.yaml | 4 ---- docker/docker-compose.ubi9.yaml | 4 ---- 7 files changed, 28 deletions(-) diff --git a/docker/docker-compose.1804.yaml b/docker/docker-compose.1804.yaml index da34363d..fe743194 100644 --- a/docker/docker-compose.1804.yaml +++ b/docker/docker-compose.1804.yaml @@ -10,10 +10,6 @@ services: test: image: swiftly:18.04-test - environment: - - SWIFTLY_PLATFORM_NAME=ubuntu1804 - - SWIFTLY_PLATFORM_NAME_FULL=ubuntu18.04 - - SWIFTLY_PLATFORM_NAME_PRETTY="Ubuntu 18.04" install-test-setup: image: swiftly:18.04-install-test diff --git a/docker/docker-compose.2004.yaml b/docker/docker-compose.2004.yaml index a4799ace..e1f09bb2 100644 --- a/docker/docker-compose.2004.yaml +++ b/docker/docker-compose.2004.yaml @@ -10,10 +10,6 @@ services: test: image: swiftly:20.04-test - environment: - - SWIFTLY_PLATFORM_NAME=ubuntu2004 - - SWIFTLY_PLATFORM_NAME_FULL=ubuntu20.04 - - SWIFTLY_PLATFORM_NAME_PRETTY="Ubuntu 20.04" install-test-setup: image: swiftly:20.04-install-test diff --git a/docker/docker-compose.2204.yaml b/docker/docker-compose.2204.yaml index 824721cd..542c6c36 100644 --- a/docker/docker-compose.2204.yaml +++ b/docker/docker-compose.2204.yaml @@ -10,10 +10,6 @@ services: test: image: swiftly:22.04-test - environment: - - SWIFTLY_PLATFORM_NAME=ubuntu2204 - - SWIFTLY_PLATFORM_NAME_FULL=ubuntu22.04 - - SWIFTLY_PLATFORM_NAME_PRETTY="Ubuntu 22.04" install-test-setup: image: swiftly:22.04-install-test diff --git a/docker/docker-compose.2310.yaml b/docker/docker-compose.2310.yaml index 80d6673d..9871e295 100644 --- a/docker/docker-compose.2310.yaml +++ b/docker/docker-compose.2310.yaml @@ -10,10 +10,6 @@ services: test: image: swiftly:23.10-test - environment: - - SWIFTLY_PLATFORM_NAME=ubuntu2310 - - SWIFTLY_PLATFORM_NAME_FULL=ubuntu23.10 - - SWIFTLY_PLATFORM_NAME_PRETTY="Ubuntu 23.10" install-test-setup: image: swiftly:23.10-install-test diff --git a/docker/docker-compose.2404.yaml b/docker/docker-compose.2404.yaml index a5318c70..0fb817bf 100644 --- a/docker/docker-compose.2404.yaml +++ b/docker/docker-compose.2404.yaml @@ -10,10 +10,6 @@ services: test: image: swiftly:24.04-test - environment: - - SWIFTLY_PLATFORM_NAME=ubuntu2404 - - SWIFTLY_PLATFORM_NAME_FULL=ubuntu24.04 - - SWIFTLY_PLATFORM_NAME_PRETTY="Ubuntu 24.04" install-test-setup: image: swiftly:24.04-install-test diff --git a/docker/docker-compose.amazonlinux2.yaml b/docker/docker-compose.amazonlinux2.yaml index 4874d2af..57df888b 100644 --- a/docker/docker-compose.amazonlinux2.yaml +++ b/docker/docker-compose.amazonlinux2.yaml @@ -12,10 +12,6 @@ services: test: image: swiftly:amazonlinux2-test - environment: - - SWIFTLY_PLATFORM_NAME=amazonlinux2 - - SWIFTLY_PLATFORM_NAME_FULL=amazonlinux2 - - SWIFTLY_PLATFORM_NAME_PRETTY="Amazon Linux 2" install-test-setup: image: swiftly:amazonlinux2-install-test diff --git a/docker/docker-compose.ubi9.yaml b/docker/docker-compose.ubi9.yaml index 36c85c5d..3cfedd77 100644 --- a/docker/docker-compose.ubi9.yaml +++ b/docker/docker-compose.ubi9.yaml @@ -12,10 +12,6 @@ services: test: image: swiftly:ubi9-test - environment: - - SWIFTLY_PLATFORM_NAME=ubi9 - - SWIFTLY_PLATFORM_NAME_FULL=ubi9 - - SWIFTLY_PLATFORM_NAME_PRETTY="Red Hat Enterprise Linux 9" install-test-setup: image: swiftly:ubi9-install-test From 393f9d5dd9fa49df72e864a2899585b648c313f3 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sun, 20 Oct 2024 06:50:49 -0400 Subject: [PATCH 27/45] Remove last remnants of the special platform environment variables from the tests --- Tests/SwiftlyTests/SwiftlyTests.swift | 24 ++---------------------- 1 file changed, 2 insertions(+), 22 deletions(-) diff --git a/Tests/SwiftlyTests/SwiftlyTests.swift b/Tests/SwiftlyTests/SwiftlyTests.swift index a384e37e..8e8fc346 100644 --- a/Tests/SwiftlyTests/SwiftlyTests.swift +++ b/Tests/SwiftlyTests/SwiftlyTests.swift @@ -99,25 +99,8 @@ class SwiftlyTests: XCTestCase { ] func baseTestConfig() async throws -> Config { - let getEnv = { varName in - guard let v = ProcessInfo.processInfo.environment[varName] else { - throw SwiftlyTestError(message: "environment variable \(varName) must be set in order to run tests") - } - return v - } - - let name = try? getEnv("SWIFTLY_PLATFORM_NAME") - let nameFull = try? getEnv("SWIFTLY_PLATFORM_NAME_FULL") - let namePretty = try? getEnv("SWIFTLY_PLATFORM_NAME_PRETTY") - - let pd = if let name = name, let nameFull = nameFull, let namePretty = namePretty { - PlatformDefinition(name: name, nameFull: nameFull, namePretty: namePretty) - } else { - try? await Swiftly.currentPlatform.detectPlatform(disableConfirmation: true, platform: nil) - } - - guard let pd = pd else { - throw SwiftlyTestError(message: "unable to detect platform. please set SWIFTLY_PLATFORM_NAME, SWIFTLY_PLATFORM_NAME_FULL, and SWIFTLY_PLATFORM_NAME_PRETTY to run the tests") + guard let pd = try? await Swiftly.currentPlatform.detectPlatform(disableConfirmation: true, platform: nil) else { + throw SwiftlyTestError(message: "Unable to detect the current platform.") } return Config( @@ -146,9 +129,6 @@ class SwiftlyTests: XCTestCase { /// Create a fresh swiftly home directory, populate it with a base config, and run the provided closure. /// Any swiftly commands executed in the closure will use this new home directory. /// - /// This method requires the SWIFTLY_PLATFORM_NAME, SWIFTLY_PLATFORM_NAME_FULL, and SWIFTLY_PLATFORM_NAME_PRETTY - /// environment variables to be set. - /// /// The home directory will be deleted after the provided closure has been executed. func withTestHome( name: String = "testHome", From 69897966f9c0aa0ddaed9b77590dc28bee8a9383 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sun, 20 Oct 2024 11:06:17 -0400 Subject: [PATCH 28/45] Update README, and add documentation for the new run subcommand --- Documentation/SwiftlyDocs.docc/SwiftlyDocs.md | 1 + .../SwiftlyDocs.docc/getting-started.md | 23 +++++- .../SwiftlyDocs.docc/use-toolchains.md | 79 +++++++++++++++++++ README.md | 41 ++++------ 4 files changed, 114 insertions(+), 30 deletions(-) create mode 100644 Documentation/SwiftlyDocs.docc/use-toolchains.md diff --git a/Documentation/SwiftlyDocs.docc/SwiftlyDocs.md b/Documentation/SwiftlyDocs.docc/SwiftlyDocs.md index 8af36940..bd0b4da0 100644 --- a/Documentation/SwiftlyDocs.docc/SwiftlyDocs.md +++ b/Documentation/SwiftlyDocs.docc/SwiftlyDocs.md @@ -13,6 +13,7 @@ Install and manage your Swift programming language toolchains. ### HOWTOS - +- - - - diff --git a/Documentation/SwiftlyDocs.docc/getting-started.md b/Documentation/SwiftlyDocs.docc/getting-started.md index 7d48eab1..b66d50e3 100644 --- a/Documentation/SwiftlyDocs.docc/getting-started.md +++ b/Documentation/SwiftlyDocs.docc/getting-started.md @@ -27,9 +27,11 @@ $ swift --version Swift version 5.8.1 (swift-5.8.1-RELEASE) Target: x86_64-unknown-linux-gnu + +$ swift build # Build with the latest (5.8.1) toolchain ``` -Or, you can install (and use) a swift release: +You can install (and use) another release toolchain: ``` $ swiftly install --use 5.7 @@ -38,12 +40,27 @@ $ swift --version Swift version 5.7.2 (swift-5.7.2-RELEASE) Target: x86_64-unknown-linux-gnu + +$ swift build # Build with the 5.7.2 toolchain ``` -There's also an option to install the latest snapshot release and get access to the latest features: +Quickly test your package with the latest nightly snapshot to prepare for the next release: ``` $ swiftly install main-snapshot +$ swiftly run swift test +main-snapshot # Run "swift test" with the main-snapshot toolchain +$ swift build # Continue to build with my usual toolchain ``` -> Note: This last example just installed the toolchain. You can run "swiftly use" to switch to it and other installed toolchahins when you're ready. +Uninstall this toolchain after you're finished with it: + +``` +$ swiftly uninstall main-snapshot +``` + +# See Also: + +- [Install Toolchains](install-toolchains) +- [Using Toolchains](use-toolchains) +- [Uninstall Toolchains](uninstall-toolchains) +- [Swiftly CLI Reference](swiftly-cli-reference) diff --git a/Documentation/SwiftlyDocs.docc/use-toolchains.md b/Documentation/SwiftlyDocs.docc/use-toolchains.md new file mode 100644 index 00000000..941b9a80 --- /dev/null +++ b/Documentation/SwiftlyDocs.docc/use-toolchains.md @@ -0,0 +1,79 @@ +# Use Swift Toolchains + +swiftly use and swiftly run + +Swiftly toolchains include a variety of compilers, linkers, debuggers, documentation generators, and other useful tools for working with Swift. Using a toolchain activates it so that when you run toolchain commands they are run with that version. + +When you install a toolchain you can start using it right away. If you don't have any other toolchains installed then it becomes the default. + +``` +$ swiftly install latest +$ swift --version +Swift version 6.0.1 (swift-6.0.1-RELEASE) +Target: aarch64-unknown-linux-gnu +$ swift build # Build with the current toolchain +``` + +When you have more than one toolchain installed then you can choose to use one of them with `swiftly use` for all subsequent commands like this: + +``` +$ swiftly install 5.10.1 +$ swiftly install main-snapshot +$ swiftly use 5.10.1 +$ swift build # Builds with the 5.10.1 toolchain +$ swift test # Tests with the 5.10.1 toolchain +$ swiftly use main-snapshot +$ swift build # Builds with the latest snapshot toolchain on the main branch +$ lldb # Run the debugger from the latest snapshot toolchain +``` + +If you're not certain which toolchain is in-use then use the bare `swiftly use` command to provide details: + +``` +$ swiftly use +Swift 6.0.1 (default) +``` + +You can print the exact toolchain location with the `--print-location` flag: + +``` +$ swiftly use --print-location +/Users/someuser/Library/Developer/Toolchains/swift-5.10.1-RELEASE.xctoolchain +``` + +## Sharing recommended toolchain versions + +Swiftly can create and update a special `.swift-version` file at the top of your git repository so that you can share your toolchain preference with the rest of your team: + +``` +$ cd path/to/git/repository +$ swiftly use 6.0.1 +A new file `path/to/git/repository/.swift-version` will be created to set the new in-use toolchain for this project. +Alternatively, you can set your default globally with the `--global-default` flag. Proceed with creating this file? (Y/n) Y +$ cat .swift-version +6.0.1 +``` + +When a team member uses swiftly with this git repository it can use the correct toolchain version automatically: + +``` +$ cd path/to/git/repository +$ swift --version +Swift version 6.0.1 (swift-6.0.1-RELEASE) +Target: aarch64-unknown-linux-gnu +``` + +If that team member doesn't have the toolchain installed on their system there will be a warning. They can install the selected toolchain automatically like this: + +``` +$ cd path/to/git/repository +$ swiftly install # Installs the version of the toolchain in the .swift-version file +``` + +If you want to temporarily use a toolchain version for one command you can try `swiftly run`. This will build your package with the latest snapshot toolchain: + +``` +$ swiftly run swift build +main-snapshot +``` + +> Note: The toolchain must be installed on your system before you can run with it. diff --git a/README.md b/README.md index 72edab9e..fbc9afa4 100644 --- a/README.md +++ b/README.md @@ -134,50 +134,37 @@ $ swiftly list ### Selecting a toolchain for use -“Using” a toolchain sets it as the active toolchain, meaning it will be the one found via $PATH and invoked via `swift` commands executed in the shell. +“Using” a toolchain sets it as the active toolchain, meaning it will be the one found via $PATH and invoked via `swift` commands executed in the shell. The toolchain must be installed before you can use it. -To use the toolchain associated with the most up-to-date Swift version, the “latest” version can be specified: +You can provide the same version selectors as you used with `swiftly install` to use a toolchain, including exact releacs versions "major.minor.patch", and snapshots. ``` $ swiftly use latest -``` - -To use a specific stable version of Swift already installed, specify the major/minor/patch version: - -``` $ swiftly use 5.3.1 -``` - -To use the latest installed patch version associated with a given major/minor version pair, the patch can be omitted: - -``` $ swiftly use 5.3 -``` - -To use a specific snapshot version, specify the full snapshot version name: - -``` -$ swiftly use 5.3-snapshot-YYYY-MM-DD -``` - -To use the latest installed snapshot associated with a given version, the date can be omitted: - -``` $ swiftly use 5.3-snapshot +$ swiftly use 5.3-snapshot-2022-08-16 +$ swiftly use main-snapshot +$ swiftly use main-snapshot-2024-06-18 ``` -To use a specific main snapshot, specify the full snapshot version name: +After you use a toolchain your commands at the shell will run with that toolchain: ``` -$ swiftly use main-snapshot-YYYY-MM-DD +$ swiftly use x.y.z +$ swift build # Build my package with toolchain version x.y.z +$ clang -c foo.c -o foo.o # Compile this C file using the clang compiler in toolchain version x.y.z +$ lldb # Open the debugger from toolchain version x.y.z ``` -To use the latest installed main snapshot, leave off the date: +If you want to run just one command with a particular toolchain without having to switch back to the one you used previously you can use the `swiftly run` command with the version. This command builds your current package with the latest snapshot toolchain of the current release: ``` -$ swiftly use main-snapshot +$ swiftly run swift build +main-snapshot ``` +The parameter with the "+" indicates that this is the version selector of the toolchain to use and supports the full range of selectors shown above and with the `swiftly install` command. The toolchain must be installed to run a command with that toolchain. + ### Updating a toolchain Update replaces a given toolchain with a later version of that toolchain. For a stable release, this means updating to a later patch, minor, or major version. For snapshots, this means updating to the most recently available snapshot. From 6fc24f0a3192cbe6e0b7f8c800de229e74388003 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sun, 20 Oct 2024 11:14:06 -0400 Subject: [PATCH 29/45] Remove mention of CentOS from the README.md since it is not particular relevant anymore --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 72edab9e..78dcab49 100644 --- a/README.md +++ b/README.md @@ -44,7 +44,6 @@ Target: x86_64-unknown-linux-gnu ## Platform support - Linux-based platforms listed on https://swift.org/download - - CentOS 7 will not be supported due to some dependencies of swiftly not supporting it, however. Right now, swiftly is in early stages of development and is supported on Linux and macOS. For more detailed information about swiftly's intended features and implementation, check out the [design document](DESIGN.md). From 71df3a626790efab291ece0bd6f0d0214ba50df1 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sun, 20 Oct 2024 13:58:21 -0400 Subject: [PATCH 30/45] Streamline the swiftly init process The init process just installs swiftly itself at the moment. Most users will immediately install a swift toolchain, most likely the latest available one. On Linux, there's confusing gpg messages that most users don't need to be aware. The init process will provide a summary of things that are going to happen to the user's system, including the addition of GnuPG keys on Linux, and the installation of the latest swift toolchain so that they can agree, or abort the entire process. When the process runs the user is given line-level and high level processes, not internal details. Add a verbose mode for more details, such as the messages that come from GnuPG on Linux. Add an option to the init subcommand to allow swiftly to be installed without the latest available swift toolchain so that advanced users can decide how to install a toolchain themselves after the swiftly installation. Update the documentation with the more automated workflow. --- .../SwiftlyDocs.docc/automated-install.md | 16 +++-- .../SwiftlyDocs.docc/getting-started.md | 22 ++----- .../SwiftlyDocs.docc/swiftly-cli-reference.md | 59 ++++++++++++++--- README.md | 6 +- Sources/LinuxPlatform/Linux.swift | 4 +- Sources/MacOSPlatform/MacOS.swift | 2 +- Sources/Swiftly/Init.swift | 63 ++++++++++++++----- Sources/Swiftly/Install.swift | 13 ++-- Sources/Swiftly/Proxy.swift | 6 ++ Sources/Swiftly/SelfUpdate.swift | 12 ++-- Sources/Swiftly/Swiftly.swift | 3 + Sources/Swiftly/Update.swift | 3 +- Sources/SwiftlyCore/Platform.swift | 3 +- Tests/SwiftlyTests/SelfUpdateTests.swift | 2 +- 14 files changed, 151 insertions(+), 63 deletions(-) diff --git a/Documentation/SwiftlyDocs.docc/automated-install.md b/Documentation/SwiftlyDocs.docc/automated-install.md index c698a744..ceebc2d7 100644 --- a/Documentation/SwiftlyDocs.docc/automated-install.md +++ b/Documentation/SwiftlyDocs.docc/automated-install.md @@ -4,18 +4,26 @@ Swiftly can be installed automatically in places like build/CI systems. This guide will help you to script to the installation of swiftly and toolchains so that it can be unattended. We assume that you have working understanding of your build system. The examples are based on a typical Unix environment. -First, download a swiftly binary from a trusted source, such as your artifact repository, or a well-known website for the operating system (e.g. Linux) and processor architecture (e.g. arm64, or x86_64). Here's an example using the popular curl command. +First, download the swiftly binary from swift.org for your operating system (e.g. Linux) and processor architecture (e.g. arm64, or x86_64). Here's an example using the popular curl command. ``` -curl -L > swiftly +curl -L > swiftly.tar.gz +tar zxf swiftly.tar.gz +``` + +On macOS you can download the pkg file and extract it like this from the command-line: + +``` +curl -L > swiftly.pkg +installer -pkg swiftly.pkg -target CurrentUserHomeDirectory ``` > Tip: If you are using Linux you will need the "ca-certificates" package for the root certificate authorities that will establish the trust that swiftly needs to make API requests that it needs. This package is frequently pre-installed on end-user environments, but may not be present in more minimal installations. -Once swiftly is downloaded you can run the init subcommand to finish the installation. This command will use the default initialization options and proceed without prompting. +Once swiftly is downloaded you can run the init subcommand to finish the installation. This command will print verbose outputs, assume yes for all prompts, and skip the automatic installation of the latest swift toolchain: ``` -./swiftly init --assume-yes +./swiftly init --verbose --assume-yes --skip-install # the swiftly binary is extracted to ~/local/bin/swiftly on macOS ``` Swiftly is installed, but the current shell may not yet be updated with the new environment variables, such as the PATH. The init command prints instructions on how to update the current shell environment without opening a new shell. This is an example of the output taken from Linux, but the details might be different for other OSes, username, or shell. diff --git a/Documentation/SwiftlyDocs.docc/getting-started.md b/Documentation/SwiftlyDocs.docc/getting-started.md index b66d50e3..d11b47fb 100644 --- a/Documentation/SwiftlyDocs.docc/getting-started.md +++ b/Documentation/SwiftlyDocs.docc/getting-started.md @@ -1,34 +1,20 @@ # Getting Started with Swiftly -To download swiftly and install Swift, run the following in your terminal, then follow the on-screen instructions: - -``` -curl -L https://swiftlang.github.io/swiftly/swiftly-install.sh | bash -``` - -Alternatively, you can download the swiftly binary and install itself like this: +To get started with swiftly you can download it from [swift.org](https://swift.org/download), extract the package, and install it like this: ``` swiftly init ``` -Once swiftly is installed you can use it to install the latest available swift toolchain like this: +Swiftly will install itself and download the latest available Swift toolchain. Follow the prompts for any additional steps. Once everything is done you can begin using swift. ``` -$ swiftly install latest - -Fetching the latest stable Swift release... -Installing Swift 5.8.1 -Downloaded 488.5 MiB of 488.5 MiB -Extracting toolchain... -Swift 5.8.1 installed successfully! - $ swift --version -Swift version 5.8.1 (swift-5.8.1-RELEASE) +Swift version 6.0.1 (swift-6.0.1-RELEASE) Target: x86_64-unknown-linux-gnu -$ swift build # Build with the latest (5.8.1) toolchain +$ swift build # Build your package with the latest toolchain ``` You can install (and use) another release toolchain: diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index dbe97f63..6bfd1b13 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -23,7 +23,7 @@ swiftly [--version] [--help] Install a new toolchain. ``` -swiftly install [] [--use] [--token=] [--verify|no-verify] [--post-install-file=] [--version] [--help] +swiftly install [] [--use] [--token=] [--verify|no-verify] [--post-install-file=] [--assume-yes] [--verbose] [--version] [--help] ``` **version:** @@ -84,6 +84,16 @@ If the toolchain that is installed has extra post installation steps they they w written to this file as commands that can be run after the installation. +**--assume-yes:** + +*Disable confirmation prompts by assuming 'yes'* + + +**--verbose:** + +*Enable verbose reporting from swiftly* + + **--version:** *Show the version.* @@ -146,7 +156,7 @@ Note that listing available snapshots before 6.0 is unsupported. Set the in-use toolchain. If no toolchain is provided, print the currently in-use toolchain, if any. ``` -swiftly use [--print-location] [--global-default] [--assume-yes] [] [--version] [--help] +swiftly use [--print-location] [--global-default] [--assume-yes] [--verbose] [] [--version] [--help] ``` **--print-location:** @@ -164,6 +174,11 @@ swiftly use [--print-location] [--global-default] [--assume-yes] [] [ *Disable confirmation prompts by assuming 'yes'* +**--verbose:** + +*Enable verbose reporting from swiftly* + + **toolchain:** *The toolchain to use.* @@ -213,7 +228,7 @@ Likewise, the latest snapshot associated with a given development branch can be Remove an installed toolchain. ``` -swiftly uninstall [--assume-yes] [--version] [--help] +swiftly uninstall [--assume-yes] [--verbose] [--version] [--help] ``` **toolchain:** @@ -252,6 +267,11 @@ Finally, all installed toolchains can be uninstalled by specifying 'all': *Disable confirmation prompts by assuming 'yes'* +**--verbose:** + +*Enable verbose reporting from swiftly* + + **--version:** *Show the version.* @@ -312,7 +332,7 @@ The installed snapshots for a given devlopment branch can be listed by specifyin Update an installed toolchain to a newer version. ``` -swiftly update [] [--assume-yes] [--verify|no-verify] [--post-install-file=] [--version] [--help] +swiftly update [] [--assume-yes] [--verbose] [--verify|no-verify] [--post-install-file=] [--version] [--help] ``` **toolchain:** @@ -358,6 +378,11 @@ A specific snapshot toolchain can be updated by including the date: *Disable confirmation prompts by assuming 'yes'* +**--verbose:** + +*Enable verbose reporting from swiftly* + + **--verify|no-verify:** *Verify the toolchain's PGP signature before proceeding with installation.* @@ -388,7 +413,7 @@ written to this file as commands that can be run after the installation. Perform swiftly initialization into your user account. ``` -swiftly init [--no-modify-profile] [--overwrite] [--platform=] [--assume-yes] [--version] [--help] +swiftly init [--no-modify-profile] [--overwrite] [--platform=] [--skip-install] [--assume-yes] [--verbose] [--version] [--help] ``` **--no-modify-profile:** @@ -403,7 +428,12 @@ swiftly init [--no-modify-profile] [--overwrite] [--platform=] [--assu **--platform=\:** -*Specify the current Linux platform for swiftly.* +*Specify the current Linux platform for swiftly* + + +**--skip-install:** + +*Skip installing the latest toolchain* **--assume-yes:** @@ -411,6 +441,11 @@ swiftly init [--no-modify-profile] [--overwrite] [--platform=] [--assu *Disable confirmation prompts by assuming 'yes'* +**--verbose:** + +*Enable verbose reporting from swiftly* + + **--version:** *Show the version.* @@ -428,9 +463,19 @@ swiftly init [--no-modify-profile] [--overwrite] [--platform=] [--assu Update the version of swiftly itself. ``` -swiftly self-update [--version] [--help] +swiftly self-update [--assume-yes] [--verbose] [--version] [--help] ``` +**--assume-yes:** + +*Disable confirmation prompts by assuming 'yes'* + + +**--verbose:** + +*Enable verbose reporting from swiftly* + + **--version:** *Show the version.* diff --git a/README.md b/README.md index fbc9afa4..7e4b8ef6 100644 --- a/README.md +++ b/README.md @@ -6,12 +6,8 @@ Ongoing maintenance and stewardship of this project is led by the [SSWG](https:/ ### Installation -To download swiftly and install Swift, run the following in your terminal, then follow the on-screen instructions. -``` -curl -L https://swiftlang.github.io/swiftly/swiftly-install.sh | bash -``` +Download the swiftly package from [swift.org](https://swift.org/download) and it can install itself with init: -Alternatively, you can download the swiftly binary and it can install itself: ``` swiftly init ``` diff --git a/Sources/LinuxPlatform/Linux.swift b/Sources/LinuxPlatform/Linux.swift index 4d9345c3..5c5937e1 100644 --- a/Sources/LinuxPlatform/Linux.swift +++ b/Sources/LinuxPlatform/Linux.swift @@ -305,7 +305,7 @@ public struct Linux: Platform { FileManager.default.temporaryDirectory.appendingPathComponent("swiftly-\(UUID())") } - public func verifySignature(httpClient: SwiftlyHTTPClient, archiveDownloadURL: URL, archive: URL) async throws { + public func verifySignature(httpClient: SwiftlyHTTPClient, archiveDownloadURL: URL, archive: URL, verbose: Bool) async throws { SwiftlyCore.print("Downloading toolchain signature...") let sigFile = self.getTempFilePath() FileManager.default.createFile(atPath: sigFile.path, contents: nil) @@ -320,7 +320,7 @@ public struct Linux: Platform { SwiftlyCore.print("Verifying toolchain signature...") do { - try self.runProgram("gpg", "--verify", sigFile.path, archive.path) + try self.runProgram("gpg", "--verify", sigFile.path, archive.path, quiet: !verbose) } catch { throw Error(message: "Signature verification failed: \(error).") } diff --git a/Sources/MacOSPlatform/MacOS.swift b/Sources/MacOSPlatform/MacOS.swift index 761e0666..5cb2f5b3 100644 --- a/Sources/MacOSPlatform/MacOS.swift +++ b/Sources/MacOSPlatform/MacOS.swift @@ -146,7 +146,7 @@ public struct MacOS: Platform { FileManager.default.temporaryDirectory.appendingPathComponent("swiftly-\(UUID()).pkg") } - public func verifySignature(httpClient _: SwiftlyHTTPClient, archiveDownloadURL _: URL, archive _: URL) async throws { + public func verifySignature(httpClient _: SwiftlyHTTPClient, archiveDownloadURL _: URL, archive _: URL, verbose _: Bool) async throws { // No signature verification is required on macOS since the pkg files have their own signing // mechanism and the swift.org downloadables are trusted by stock macOS installations. } diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index 9a354022..aadd1f2c 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -14,39 +14,53 @@ internal struct Init: SwiftlyCommand { installation is found, the swiftly executable will be updated, but the rest of the installation will not be modified. """) var overwrite: Bool = false - @Option(name: .long, help: "Specify the current Linux platform for swiftly.") + @Option(name: .long, help: "Specify the current Linux platform for swiftly") var platform: String? + @Flag(help: "Skip installing the latest toolchain") + var skipInstall: Bool = false @OptionGroup var root: GlobalOptions + private enum CodingKeys: String, CodingKey { + case noModifyProfile, overwrite, platform, skipInstall, root + } + public mutating func validate() throws {} internal mutating func run() async throws { - try await Self.execute(assumeYes: self.root.assumeYes, noModifyProfile: self.noModifyProfile, overwrite: self.overwrite, platform: self.platform) + try await Self.execute(assumeYes: self.root.assumeYes, noModifyProfile: self.noModifyProfile, overwrite: self.overwrite, platform: self.platform, verbose: self.root.verbose, skipInstall: self.skipInstall, verbose: self.root.verbose) } /// Initialize the installation of swiftly. - internal static func execute(assumeYes: Bool, noModifyProfile: Bool, overwrite: Bool, platform: String?) async throws { + internal static func execute(assumeYes: Bool, noModifyProfile: Bool, overwrite: Bool, platform: String?, verbose _: Bool, skipInstall: Bool, verbose: Bool) async throws { try Swiftly.currentPlatform.verifySwiftlySystemPrerequisites() - let config = try? Config.load() + var config = try? Config.load() - if let config = config, !overwrite && config.version != SwiftlyCore.version { + if let config, !overwrite && config.version != SwiftlyCore.version { // We don't support downgrades, and we don't yet support upgrades throw Error(message: "An existing swiftly installation was detected. You can try again with '--overwrite' to overwrite it.") } // Give the user the prompt and the choice to abort at this point. if !assumeYes { +#if os(Linux) + let sigMsg = " In the process of installing the new toolchain swiftly will add swift.org GnuPG keys into your keychain to verify the integrity of the downloads." +#else + let sigMsg = "" +#endif + let installMsg = if !skipInstall { + "\nOnce swiftly is installed it will install the latest available swift toolchain.\(sigMsg)\n" + } else { "" } + SwiftlyCore.print(""" Swiftly will be installed into the following locations: \(Swiftly.currentPlatform.swiftlyHomeDir.path) - Data and configuration files directory including toolchains \(Swiftly.currentPlatform.swiftlyBinDir.path) - Executables installation directory - Note that the locations can be changed with SWIFTLY_HOME and SWIFTLY_BIN environment variables and run - this again. - + These locations can be changed with SWIFTLY_HOME and SWIFTLY_BIN environment variables and run this again. + \(installMsg) """) if SwiftlyCore.readLine(prompt: "Proceed with the installation? [Y/n] ") == "n" { @@ -115,12 +129,15 @@ internal struct Init: SwiftlyCommand { // Force the configuration to be present. Generate it if it doesn't already exist or overwrite is set if overwrite || config == nil { let pd = try await Swiftly.currentPlatform.detectPlatform(disableConfirmation: assumeYes, platform: platform) - var config = Config(inUse: nil, installedToolchains: [], platform: pd) + var c = Config(inUse: nil, installedToolchains: [], platform: pd) // Stamp the current version of swiftly on this config - config.version = SwiftlyCore.version - try config.save() + c.version = SwiftlyCore.version + try c.save() + config = c } + guard var config else { throw Error(message: "Configuration could not be set") } + let swiftlyBin = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swiftly", isDirectory: false) let cmd = URL(fileURLWithPath: CommandLine.arguments[0]) @@ -233,6 +250,13 @@ internal struct Init: SwiftlyCommand { addEnvToProfile = true } + var postInstall: String? + + if !skipInstall { + let latestVersion = try await Install.resolve(config: config, selector: ToolchainSelector.latest) + postInstall = try await Install.execute(version: latestVersion, &config, useInstalledToolchain: true, verifySignature: true, verbose: verbose) + } + if addEnvToProfile { try Data(sourceLine.utf8).append(to: profileHome) @@ -245,14 +269,25 @@ internal struct Init: SwiftlyCommand { #if os(macOS) SwiftlyCore.print(""" NOTE: On macOS it is possible that the shell will pick up the system Swift on the path - instead of the one that swiftly has installed for you. You can run the 'hash -r' - command to update the shell with the latest PATHs. + instead of the one that swiftly has installed for you. You can run this command to update + the shell with the latest PATHs. - hash -r + hash -r """) #endif } + + if let postInstall { + SwiftlyCore.print(""" + There are some system dependencies that should be installed before using this toolchain. + You can run the following script as the system administrator (e.g. root) to prepare + your system: + + \(postInstall) + + """) + } } } } diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 3d14f8ce..05dbad04 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -71,8 +71,10 @@ struct Install: SwiftlyCommand { )) var postInstallFile: String? + @OptionGroup var root: GlobalOptions + private enum CodingKeys: String, CodingKey { - case version, token, use, verify, postInstallFile + case version, token, use, verify, postInstallFile, root } mutating func run() async throws { @@ -106,7 +108,8 @@ struct Install: SwiftlyCommand { version: toolchainVersion, &config, useInstalledToolchain: self.use, - verifySignature: self.verify + verifySignature: self.verify, + verbose: self.root.verbose ) if let postInstallScript = postInstallScript { @@ -129,7 +132,8 @@ struct Install: SwiftlyCommand { version: ToolchainVersion, _ config: inout Config, useInstalledToolchain: Bool, - verifySignature: Bool + verifySignature: Bool, + verbose: Bool ) async throws -> String? { guard !config.installedToolchains.contains(version) else { SwiftlyCore.print("\(version) is already installed.") @@ -228,7 +232,8 @@ struct Install: SwiftlyCommand { try await Swiftly.currentPlatform.verifySignature( httpClient: SwiftlyCore.httpClient, archiveDownloadURL: url, - archive: tmpFile + archive: tmpFile, + verbose: verbose ) } diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index b91bdd4f..82b559e5 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -31,6 +31,12 @@ public enum Proxy { guard proxyList.contains(binName) else { // Treat this as a swiftly invocation + + if CommandLine.arguments.count == 1 || CommandLine.arguments[1] != "init" { + // Check if we've been invoked outside the "init" subcommand and we're not yet configured + _ = try Config.load() + } + await Swiftly.main() return } diff --git a/Sources/Swiftly/SelfUpdate.swift b/Sources/Swiftly/SelfUpdate.swift index b8534f48..e41dea60 100644 --- a/Sources/Swiftly/SelfUpdate.swift +++ b/Sources/Swiftly/SelfUpdate.swift @@ -10,7 +10,11 @@ internal struct SelfUpdate: SwiftlyCommand { abstract: "Update the version of swiftly itself." ) - private enum CodingKeys: CodingKey {} + @OptionGroup var root: GlobalOptions + + private enum CodingKeys: String, CodingKey { + case root + } internal mutating func run() async throws { try validateSwiftly() @@ -20,10 +24,10 @@ internal struct SelfUpdate: SwiftlyCommand { throw Error(message: "Self update doesn't work when swiftly has been installed externally. Please keep it updated from the source where you installed it in the first place.") } - let _ = try await Self.execute() + let _ = try await Self.execute(verbose: self.root.verbose) } - public static func execute() async throws -> SwiftlyVersion { + public static func execute(verbose: Bool) async throws -> SwiftlyVersion { SwiftlyCore.print("Checking for swiftly updates...") let swiftlyRelease = try await SwiftlyCore.httpClient.getSwiftlyRelease() @@ -91,7 +95,7 @@ internal struct SelfUpdate: SwiftlyCommand { } animation.complete(success: true) - try await Swiftly.currentPlatform.verifySignature(httpClient: SwiftlyCore.httpClient, archiveDownloadURL: downloadURL, archive: tmpFile) + try await Swiftly.currentPlatform.verifySignature(httpClient: SwiftlyCore.httpClient, archiveDownloadURL: downloadURL, archive: tmpFile, verbose: verbose) try Swiftly.currentPlatform.extractSwiftlyAndInstall(from: tmpFile) SwiftlyCore.print("Successfully updated swiftly to \(version) (was \(SwiftlyCore.version))") diff --git a/Sources/Swiftly/Swiftly.swift b/Sources/Swiftly/Swiftly.swift index 1652c9a2..e0badf85 100644 --- a/Sources/Swiftly/Swiftly.swift +++ b/Sources/Swiftly/Swiftly.swift @@ -11,6 +11,9 @@ public struct GlobalOptions: ParsableArguments { @Flag(name: [.customShort("y"), .long], help: "Disable confirmation prompts by assuming 'yes'") var assumeYes: Bool = false + @Flag(help: "Enable verbose reporting from swiftly") + var verbose: Bool = false + public init() {} } diff --git a/Sources/Swiftly/Update.swift b/Sources/Swiftly/Update.swift index 6d958ca2..bd2e050c 100644 --- a/Sources/Swiftly/Update.swift +++ b/Sources/Swiftly/Update.swift @@ -112,7 +112,8 @@ struct Update: SwiftlyCommand { version: newToolchain, &config, useInstalledToolchain: config.inUse == parameters.oldToolchain, - verifySignature: self.verify + verifySignature: self.verify, + verbose: self.root.verbose ) try await Uninstall.execute(parameters.oldToolchain, &config) diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index 1b7ee8ed..6ec7242a 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -94,8 +94,7 @@ public protocol Platform { /// Downloads the signature file associated with the archive and verifies it matches the downloaded archive. /// Throws an error if the signature does not match. - /// On Linux, signature verification will be skipped if gpg is not installed. - func verifySignature(httpClient: SwiftlyHTTPClient, archiveDownloadURL: URL, archive: URL) async throws + func verifySignature(httpClient: SwiftlyHTTPClient, archiveDownloadURL: URL, archive: URL, verbose: Bool) async throws /// Detect the platform definition for this platform. func detectPlatform(disableConfirmation: Bool, platform: String?) async throws -> PlatformDefinition diff --git a/Tests/SwiftlyTests/SelfUpdateTests.swift b/Tests/SwiftlyTests/SelfUpdateTests.swift index 1adb49a2..628ee9b8 100644 --- a/Tests/SwiftlyTests/SelfUpdateTests.swift +++ b/Tests/SwiftlyTests/SelfUpdateTests.swift @@ -21,7 +21,7 @@ final class SelfUpdateTests: SwiftlyTests { func runSelfUpdateTest(latestVersion: SwiftlyVersion) async throws { try await self.withTestHome { try await self.withMockedSwiftlyVersion(latestSwiftlyVersion: latestVersion) { - let updatedVersion = try await SelfUpdate.execute() + let updatedVersion = try await SelfUpdate.execute(verbose: true) XCTAssertEqual(latestVersion, updatedVersion) } } From 4d833f8acc313ce2c952c1c8ac05eea90afefa9e Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 23 Oct 2024 12:02:54 -0400 Subject: [PATCH 31/45] Add more detail to the getting started guide --- .../SwiftlyDocs.docc/getting-started.md | 22 +++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/Documentation/SwiftlyDocs.docc/getting-started.md b/Documentation/SwiftlyDocs.docc/getting-started.md index d11b47fb..873621d8 100644 --- a/Documentation/SwiftlyDocs.docc/getting-started.md +++ b/Documentation/SwiftlyDocs.docc/getting-started.md @@ -1,9 +1,27 @@ # Getting Started with Swiftly -To get started with swiftly you can download it from [swift.org](https://swift.org/download), extract the package, and install it like this: +Start using swiftly and swift. + +To get started with swiftly you can download it from [swift.org](https://swift.org/download), and extract the package. + +If you are using Linux then you can verify and extract the archive like this: + +``` +curl https://www.swift.org/keys/all-keys.asc | gpg --import +gpg --verify swift-x.y.z.tar.gz.sig swift-x.y.z.tar.gz +tar zxf swift-x.y.z.tar.gz +``` + +On macOS you can either run the pkg installer from the command-line like this or just run the package by double-clicking on it (not recommended): + +``` +installer -pkg swift-x.y.z.pkg -target CurrentUserHomeDirectory +``` + +Now run swiftly init to finish the installation. ``` -swiftly init +path/to/swiftly init # macOS pkg puts swiftly in either /usr/local/bin or $HOME/usr/local/bin ``` Swiftly will install itself and download the latest available Swift toolchain. Follow the prompts for any additional steps. Once everything is done you can begin using swift. From bc83ee5fdd810831b65e9ebd17264fe26f3dda4f Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 23 Oct 2024 14:51:38 -0400 Subject: [PATCH 32/45] Remove the word system from the missing dependencies message --- Sources/Swiftly/Init.swift | 2 +- Sources/Swiftly/Install.swift | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index aadd1f2c..83cb1d42 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -280,7 +280,7 @@ internal struct Init: SwiftlyCommand { if let postInstall { SwiftlyCore.print(""" - There are some system dependencies that should be installed before using this toolchain. + There are some dependencies that should be installed before using this toolchain. You can run the following script as the system administrator (e.g. root) to prepare your system: diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 05dbad04..a0f30814 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -116,7 +116,7 @@ struct Install: SwiftlyCommand { guard let postInstallFile = self.postInstallFile else { throw Error(message: """ - There are some system dependencies that should be installed before using this toolchain. + There are some dependencies that should be installed before using this toolchain. You can run the following script as the system administrator (e.g. root) to prepare your system: From be65db1114ec9b698cf75eea7eb91d9fcda741f1 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 23 Oct 2024 15:22:30 -0400 Subject: [PATCH 33/45] Add system package dependency list for Ubuntu 23.10 --- Sources/LinuxPlatform/Linux.swift | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/Sources/LinuxPlatform/Linux.swift b/Sources/LinuxPlatform/Linux.swift index a237137d..0c131b45 100644 --- a/Sources/LinuxPlatform/Linux.swift +++ b/Sources/LinuxPlatform/Linux.swift @@ -137,6 +137,27 @@ public struct Linux: Platform { "tzdata", "zlib1g-dev", ] + case "ubuntu2310": + [ + "binutils", + "git", + "unzip", + "gnupg2", + "libc6-dev", + "libcurl4-openssl-dev", + "libedit2", + "libgcc-12-dev", + "libpython3-dev", + "libsqlite3-0", + "libstdc++-12-dev", + "libxml2-dev", + "libncurses-dev", + "libz3-dev", + "pkg-config", + "python3-lldb-13", + "tzdata", + "zlib1g-dev", + ] case "ubuntu2404": [ "binutils", From 646b5cbcc493bf8253f4e026b6eda6fca17e6aad Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 23 Oct 2024 15:35:52 -0400 Subject: [PATCH 34/45] Rework the getting started guide to have a platform selector for linux/macOS and hide platform-specific details --- .../SwiftlyDocs.docc/getting-started.md | 40 ++++++++++++------- 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/Documentation/SwiftlyDocs.docc/getting-started.md b/Documentation/SwiftlyDocs.docc/getting-started.md index 873621d8..690ab795 100644 --- a/Documentation/SwiftlyDocs.docc/getting-started.md +++ b/Documentation/SwiftlyDocs.docc/getting-started.md @@ -4,25 +4,37 @@ Start using swiftly and swift. To get started with swiftly you can download it from [swift.org](https://swift.org/download), and extract the package. -If you are using Linux then you can verify and extract the archive like this: +@TabNavigator { + @Tab("Linux") { + If you are using Linux then you can verify and extract the archive like this: -``` -curl https://www.swift.org/keys/all-keys.asc | gpg --import -gpg --verify swift-x.y.z.tar.gz.sig swift-x.y.z.tar.gz -tar zxf swift-x.y.z.tar.gz -``` + ``` + curl https://www.swift.org/keys/all-keys.asc | gpg --import + gpg --verify swift-x.y.z.tar.gz.sig swift-x.y.z.tar.gz + tar zxf swift-x.y.z.tar.gz + ``` -On macOS you can either run the pkg installer from the command-line like this or just run the package by double-clicking on it (not recommended): + Now run swiftly init to finish the installation: -``` -installer -pkg swift-x.y.z.pkg -target CurrentUserHomeDirectory -``` + ``` + ./swiftly init + ``` + } -Now run swiftly init to finish the installation. + @Tab("macOS") { + On macOS you can either run the pkg installer from the command-line like this or just run the package by double-clicking on it (not recommended): -``` -path/to/swiftly init # macOS pkg puts swiftly in either /usr/local/bin or $HOME/usr/local/bin -``` + ``` + installer -pkg swift-x.y.z.pkg -target CurrentUserHomeDirectory + ``` + + Now run swiftly init to finish the installation: + + ``` + usr/local/bin/swiftly init + ``` + } +} Swiftly will install itself and download the latest available Swift toolchain. Follow the prompts for any additional steps. Once everything is done you can begin using swift. From a08faad2b3b3b617b5c872ff8fc6898ccb926a3f Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 29 Oct 2024 07:39:18 -0400 Subject: [PATCH 35/45] Permit bare swiftly command to start the init workflow when not installed --- Documentation/SwiftlyDocs.docc/getting-started.md | 7 +++---- Sources/Swiftly/Init.swift | 4 ++-- Sources/Swiftly/Proxy.swift | 12 ++++++++++-- 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/Documentation/SwiftlyDocs.docc/getting-started.md b/Documentation/SwiftlyDocs.docc/getting-started.md index 690ab795..1e7187d8 100644 --- a/Documentation/SwiftlyDocs.docc/getting-started.md +++ b/Documentation/SwiftlyDocs.docc/getting-started.md @@ -9,9 +9,8 @@ To get started with swiftly you can download it from [swift.org](https://swift.o If you are using Linux then you can verify and extract the archive like this: ``` - curl https://www.swift.org/keys/all-keys.asc | gpg --import - gpg --verify swift-x.y.z.tar.gz.sig swift-x.y.z.tar.gz - tar zxf swift-x.y.z.tar.gz + sha256sum swiftly-x.y.z.tar.gz # Check that the hash matches what's reported on swift.org + tar zxf swiftly-x.y.z.tar.gz ``` Now run swiftly init to finish the installation: @@ -31,7 +30,7 @@ To get started with swiftly you can download it from [swift.org](https://swift.o Now run swiftly init to finish the installation: ``` - usr/local/bin/swiftly init + $HOME/usr/local/bin/swiftly init ``` } } diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index 83cb1d42..57821ccf 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -28,11 +28,11 @@ internal struct Init: SwiftlyCommand { public mutating func validate() throws {} internal mutating func run() async throws { - try await Self.execute(assumeYes: self.root.assumeYes, noModifyProfile: self.noModifyProfile, overwrite: self.overwrite, platform: self.platform, verbose: self.root.verbose, skipInstall: self.skipInstall, verbose: self.root.verbose) + try await Self.execute(assumeYes: self.root.assumeYes, noModifyProfile: self.noModifyProfile, overwrite: self.overwrite, platform: self.platform, verbose: self.root.verbose, skipInstall: self.skipInstall) } /// Initialize the installation of swiftly. - internal static func execute(assumeYes: Bool, noModifyProfile: Bool, overwrite: Bool, platform: String?, verbose _: Bool, skipInstall: Bool, verbose: Bool) async throws { + internal static func execute(assumeYes: Bool, noModifyProfile: Bool, overwrite: Bool, platform: String?, verbose: Bool, skipInstall: Bool) async throws { try Swiftly.currentPlatform.verifySwiftlySystemPrerequisites() var config = try? Config.load() diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 82b559e5..8838f08c 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -32,8 +32,16 @@ public enum Proxy { guard proxyList.contains(binName) else { // Treat this as a swiftly invocation - if CommandLine.arguments.count == 1 || CommandLine.arguments[1] != "init" { - // Check if we've been invoked outside the "init" subcommand and we're not yet configured + let config = try? Config.load() + + if config == nil && CommandLine.arguments.count == 1 { + // User ran swiftly with no extra arguments in an uninstalled environment, so we skip directly into + // an init. + try await Init.execute(assumeYes: false, noModifyProfile: false, overwrite: false, platform: nil, verbose: false, skipInstall: false) + return + } else if CommandLine.arguments[1] != "init" { + // Check if we've been invoked outside the "init" subcommand and we're not yet configured. + // This will throw if the configuration couldn't be loaded and give the user an actionable message. _ = try Config.load() } From 970943f3d7695d200e20a13815c004e6bd332bd3 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 29 Oct 2024 11:51:06 -0400 Subject: [PATCH 36/45] Fix array index problem --- Sources/Swiftly/Proxy.swift | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 8838f08c..1a50dee3 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -39,7 +39,7 @@ public enum Proxy { // an init. try await Init.execute(assumeYes: false, noModifyProfile: false, overwrite: false, platform: nil, verbose: false, skipInstall: false) return - } else if CommandLine.arguments[1] != "init" { + } else if config == nil && CommandLine.arguments.count > 1 && CommandLine.arguments[1] != "init" { // Check if we've been invoked outside the "init" subcommand and we're not yet configured. // This will throw if the configuration couldn't be loaded and give the user an actionable message. _ = try Config.load() From c43fba2ff63d8ba926ab3bfec0cabf3a592ada4f Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 29 Oct 2024 12:01:11 -0400 Subject: [PATCH 37/45] Use result to simplify proxy init check --- Sources/Swiftly/Proxy.swift | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 1a50dee3..d19068e8 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -32,17 +32,25 @@ public enum Proxy { guard proxyList.contains(binName) else { // Treat this as a swiftly invocation - let config = try? Config.load() + let configResult = Result { try Config.load() } - if config == nil && CommandLine.arguments.count == 1 { - // User ran swiftly with no extra arguments in an uninstalled environment, so we skip directly into - // an init. - try await Init.execute(assumeYes: false, noModifyProfile: false, overwrite: false, platform: nil, verbose: false, skipInstall: false) + switch configResult { + case .success: + await Swiftly.main() return - } else if config == nil && CommandLine.arguments.count > 1 && CommandLine.arguments[1] != "init" { - // Check if we've been invoked outside the "init" subcommand and we're not yet configured. - // This will throw if the configuration couldn't be loaded and give the user an actionable message. - _ = try Config.load() + case let .failure(err): + guard CommandLine.arguments.count > 0 else { fatalError("argv is not set") } + + if CommandLine.arguments.count == 1 { + // User ran swiftly with no extra arguments in an uninstalled environment, so we skip directly into + // an init. + try await Init.execute(assumeYes: false, noModifyProfile: false, overwrite: false, platform: nil, verbose: false, skipInstall: false) + return + } else if CommandLine.arguments[1] != "init" { + // Check if we've been invoked outside the "init" subcommand and we're not yet configured. + // This will throw if the configuration couldn't be loaded and give the user an actionable message. + throw err + } } await Swiftly.main() From bce8ab69434255e2f327505cddd92f983793c8fd Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 30 Oct 2024 13:33:53 -0400 Subject: [PATCH 38/45] Quiet the macOS installer messages behind the verbose flag Provide more verbose untarring messages in Linux behind the verbose flag --- Sources/LinuxPlatform/Linux.swift | 11 +++++++++-- Sources/MacOSPlatform/MacOS.swift | 11 +++++++---- Sources/Swiftly/Install.swift | 2 +- Sources/SwiftlyCore/Platform.swift | 2 +- Tests/SwiftlyTests/PlatformTests.swift | 10 +++++----- 5 files changed, 23 insertions(+), 13 deletions(-) diff --git a/Sources/LinuxPlatform/Linux.swift b/Sources/LinuxPlatform/Linux.swift index 3b15d0d8..e49ca55e 100644 --- a/Sources/LinuxPlatform/Linux.swift +++ b/Sources/LinuxPlatform/Linux.swift @@ -350,7 +350,7 @@ public struct Linux: Platform { } } - public func install(from tmpFile: URL, version: ToolchainVersion) throws { + public func install(from tmpFile: URL, version: ToolchainVersion, verbose: Bool) throws { guard tmpFile.fileExists() else { throw Error(message: "\(tmpFile) doesn't exist") } @@ -371,7 +371,14 @@ public struct Linux: Platform { let relativePath = name.drop { c in c != "/" }.dropFirst() // prepend /path/to/swiftlyHomeDir/toolchains/ to each file name - return toolchainDir.appendingPathComponent(String(relativePath)) + let destination = toolchainDir.appendingPathComponent(String(relativePath)) + + if verbose { + SwiftlyCore.print("\(destination.path)") + } + + // prepend /path/to/swiftlyHomeDir/toolchains/ to each file name + return destination } } diff --git a/Sources/MacOSPlatform/MacOS.swift b/Sources/MacOSPlatform/MacOS.swift index 5cb2f5b3..ddaaf187 100644 --- a/Sources/MacOSPlatform/MacOS.swift +++ b/Sources/MacOSPlatform/MacOS.swift @@ -49,7 +49,7 @@ public struct MacOS: Platform { nil } - public func install(from tmpFile: URL, version: ToolchainVersion) throws { + public func install(from tmpFile: URL, version: ToolchainVersion, verbose: Bool) throws { guard tmpFile.fileExists() else { throw Error(message: "\(tmpFile) doesn't exist") } @@ -60,23 +60,26 @@ public struct MacOS: Platform { if SwiftlyCore.mockedHomeDir == nil { SwiftlyCore.print("Installing package in user home directory...") - try runProgram("installer", "-pkg", tmpFile.path, "-target", "CurrentUserHomeDirectory") + try runProgram("installer", "-verbose", "-pkg", tmpFile.path, "-target", "CurrentUserHomeDirectory", quiet: !verbose) } else { // In the case of a mock for testing purposes we won't use the installer, perferring a manual process because // the installer will not install to an arbitrary path, only a volume or user home directory. + SwiftlyCore.print("Expanding pkg...") let tmpDir = self.getTempFilePath() let toolchainDir = self.swiftlyToolchainsDir.appendingPathComponent("\(version.identifier).xctoolchain", isDirectory: true) if !toolchainDir.fileExists() { try FileManager.default.createDirectory(at: toolchainDir, withIntermediateDirectories: false) } - try runProgram("pkgutil", "--expand", tmpFile.path, tmpDir.path) + try runProgram("pkgutil", "--verbose", "--expand", tmpFile.path, tmpDir.path) // There's a slight difference in the location of the special Payload file between official swift packages // and the ones that are mocked here in the test framework. var payload = tmpDir.appendingPathComponent("Payload") if !payload.fileExists() { payload = tmpDir.appendingPathComponent("\(version.identifier)-osx-package.pkg/Payload") } - try runProgram("tar", "-C", toolchainDir.path, "-xf", payload.path) + + SwiftlyCore.print("Untarring pkg Payload...") + try runProgram("tar", "-C", toolchainDir.path, "-xvf", payload.path, quiet: !verbose) } } diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index dd145ab9..aeee4167 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -227,7 +227,7 @@ struct Install: SwiftlyCommand { ) } - try Swiftly.currentPlatform.install(from: tmpFile, version: version) + try Swiftly.currentPlatform.install(from: tmpFile, version: version, verbose: verbose) config.installedToolchains.insert(version) diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index eddb3dbb..21102a04 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -65,7 +65,7 @@ public protocol Platform { /// Installs a toolchain from a file on disk pointed to by the given URL. /// After this completes, a user can “use” the toolchain. - func install(from: URL, version: ToolchainVersion) throws + func install(from: URL, version: ToolchainVersion, verbose: Bool) throws /// Extract swiftly from the provided downloaded archive and install /// ourselves from that. diff --git a/Tests/SwiftlyTests/PlatformTests.swift b/Tests/SwiftlyTests/PlatformTests.swift index 01e5841c..2eff8c25 100644 --- a/Tests/SwiftlyTests/PlatformTests.swift +++ b/Tests/SwiftlyTests/PlatformTests.swift @@ -22,7 +22,7 @@ final class PlatformTests: SwiftlyTests { // GIVEN: a toolchain has been downloaded var (mockedToolchainFile, version) = try await self.mockToolchainDownload(version: "5.7.1") // WHEN: the platform installs the toolchain - try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version) + try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version, verbose: true) // THEN: the toolchain is extracted in the toolchains directory var toolchains = try FileManager.default.contentsOfDirectory(at: Swiftly.currentPlatform.swiftlyToolchainsDir, includingPropertiesForKeys: nil) XCTAssertEqual(1, toolchains.count) @@ -30,7 +30,7 @@ final class PlatformTests: SwiftlyTests { // GIVEN: a second toolchain has been downloaded (mockedToolchainFile, version) = try await self.mockToolchainDownload(version: "5.8.0") // WHEN: the platform installs the toolchain - try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version) + try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version, verbose: true) // THEN: the toolchain is added to the toolchains directory toolchains = try FileManager.default.contentsOfDirectory(at: Swiftly.currentPlatform.swiftlyToolchainsDir, includingPropertiesForKeys: nil) XCTAssertEqual(2, toolchains.count) @@ -38,7 +38,7 @@ final class PlatformTests: SwiftlyTests { // GIVEN: an identical toolchain has been downloaded (mockedToolchainFile, version) = try await self.mockToolchainDownload(version: "5.8.0") // WHEN: the platform installs the toolchain - try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version) + try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version, verbose: true) // THEN: the toolchains directory remains the same toolchains = try FileManager.default.contentsOfDirectory(at: Swiftly.currentPlatform.swiftlyToolchainsDir, includingPropertiesForKeys: nil) XCTAssertEqual(2, toolchains.count) @@ -49,9 +49,9 @@ final class PlatformTests: SwiftlyTests { try await self.rollbackLocalChanges { // GIVEN: toolchains have been downloaded, and installed var (mockedToolchainFile, version) = try await self.mockToolchainDownload(version: "5.8.0") - try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version) + try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version, verbose: true) (mockedToolchainFile, version) = try await self.mockToolchainDownload(version: "5.6.3") - try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version) + try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version, verbose: true) // WHEN: one of the toolchains is uninstalled try Swiftly.currentPlatform.uninstall(version) // THEN: there is only one remaining toolchain installed From 35a05d0f34e5cbd8838d93002de88a2bfdd4bfed Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 21 Nov 2024 09:53:50 -0500 Subject: [PATCH 39/45] Fix merge remnants and reformat --- Sources/Swiftly/Init.swift | 2 +- docker/install-test-debian12.dockerfile | 12 ------------ docker/install-test-fedora39.dockerfile | 5 ----- docker/test-debian12.dockerfile | 18 ------------------ docker/test-fedora39.dockerfile | 16 ---------------- 5 files changed, 1 insertion(+), 52 deletions(-) delete mode 100644 docker/install-test-debian12.dockerfile delete mode 100644 docker/install-test-fedora39.dockerfile delete mode 100644 docker/test-debian12.dockerfile delete mode 100644 docker/test-fedora39.dockerfile diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index a987fa2e..c8bdc8a6 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -227,7 +227,7 @@ internal struct Init: SwiftlyCommand { } var postInstall: String? - var pathChanged: Bool = false + var pathChanged = false if !skipInstall { let latestVersion = try await Install.resolve(config: config, selector: ToolchainSelector.latest) diff --git a/docker/install-test-debian12.dockerfile b/docker/install-test-debian12.dockerfile deleted file mode 100644 index d5680d9b..00000000 --- a/docker/install-test-debian12.dockerfile +++ /dev/null @@ -1,12 +0,0 @@ -ARG base_image=debian:12 -FROM $base_image - -# set as UTF-8 -RUN apt-get update && apt-get install -y locales locales-all sqlite3 -ENV LC_ALL en_US.UTF-8 -ENV LANG en_US.UTF-8 -ENV LANGUAGE en_US.UTF-8 - -# dependencies -RUN apt-get update --fix-missing && apt-get install -y curl gpg -RUN echo 'export PATH="$HOME/.local/bin:$PATH"' >> $HOME/.profile diff --git a/docker/install-test-fedora39.dockerfile b/docker/install-test-fedora39.dockerfile deleted file mode 100644 index 88048fa0..00000000 --- a/docker/install-test-fedora39.dockerfile +++ /dev/null @@ -1,5 +0,0 @@ -ARG base_image=fedora:39 -FROM $base_image - -RUN yum install -y curl util-linux gpg -RUN echo 'export PATH="$HOME/.local/bin:$PATH"' >> $HOME/.profile diff --git a/docker/test-debian12.dockerfile b/docker/test-debian12.dockerfile deleted file mode 100644 index da0d7a0b..00000000 --- a/docker/test-debian12.dockerfile +++ /dev/null @@ -1,18 +0,0 @@ -ARG swift_version=5.10 -ARG base_image=swift:debian12 -FROM $base_image -# needed to do again after FROM due to docker limitation -ARG swift_version - -# set as UTF-8 -RUN apt-get update && apt-get install -y locales locales-all -ENV LC_ALL en_US.UTF-8 -ENV LANG en_US.UTF-8 -ENV LANGUAGE en_US.UTF-8 - -# dependencies -RUN apt-get update --fix-missing && apt-get install -y curl build-essential gpg -COPY ./scripts/install-libarchive.sh / -RUN /install-libarchive.sh - -RUN curl -L https://swift.org/keys/all-keys.asc | gpg --import diff --git a/docker/test-fedora39.dockerfile b/docker/test-fedora39.dockerfile deleted file mode 100644 index c61cbd9e..00000000 --- a/docker/test-fedora39.dockerfile +++ /dev/null @@ -1,16 +0,0 @@ -ARG base_image=swift:5.10-fedora39 -FROM $base_image -# needed to do again after FROM due to docker limitation -ARG swift_version - -# dependencies -RUN yum install -y \ - curl \ - gcc \ - gcc-c++ \ - make \ - gpg -COPY ./scripts/install-libarchive.sh / -RUN /install-libarchive.sh - -RUN curl -L https://swift.org/keys/all-keys.asc | gpg --import From d732a84271d3d29332c9250ab981a7d58df88602 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 21 Nov 2024 15:03:59 -0500 Subject: [PATCH 40/45] Fixes for the system managed detection and installation logic --- Sources/Swiftly/Init.swift | 22 +--------- Sources/Swiftly/Install.swift | 12 +----- Sources/SwiftlyCore/Platform.swift | 64 +++++++++++++++++++++++------- 3 files changed, 53 insertions(+), 45 deletions(-) diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index c8bdc8a6..571b9034 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -138,26 +138,8 @@ internal struct Init: SwiftlyCommand { guard var config else { throw Error(message: "Configuration could not be set") } - let swiftlyBin = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swiftly", isDirectory: false) - - let cmd = URL(fileURLWithPath: CommandLine.arguments[0]) - let systemManagedSwiftlyBin = try Swiftly.currentPlatform.systemManagedBinary(CommandLine.arguments[0]) - - // Don't move the binary if it's already in the right place, this is being invoked inside an xctest, or it is a system managed binary - if cmd != swiftlyBin && !cmd.path.hasSuffix("xctest") && systemManagedSwiftlyBin == nil { - SwiftlyCore.print("Moving swiftly into the installation directory...") - - if swiftlyBin.fileExists() { - try FileManager.default.removeItem(at: swiftlyBin) - } - - do { - try FileManager.default.moveItem(at: cmd, to: swiftlyBin) - } catch { - try FileManager.default.copyItem(at: cmd, to: swiftlyBin) - SwiftlyCore.print("Swiftly has been copied into the installation directory. You can remove '\(cmd.path)'. It is no longer needed.") - } - } + // Move our executable over to the correct place + let _ = try Swiftly.currentPlatform.findSwiftlyBin(installSwiftly: true) if overwrite || !FileManager.default.fileExists(atPath: envFile.path) { SwiftlyCore.print("Creating shell environment file for the user...") diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 5fb34c20..9dc10cdf 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -243,22 +243,14 @@ struct Install: SwiftlyCommand { var pathChanged = false - // Don't create the proxies in the tests - if CommandLine.arguments.count > 0 && !CommandLine.arguments[0].hasSuffix("xctest") { + // Create proxies if we have a location where we can point them + if let proxyTo = try? Swiftly.currentPlatform.findSwiftlyBin(installSwiftly: false) { // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. - let swiftlyBin = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swiftly", isDirectory: false) - let systemManagedSwiftlyBin = try Swiftly.currentPlatform.systemManagedBinary(CommandLine.arguments[0]) let swiftlyBinDir = Swiftly.currentPlatform.swiftlyBinDir let swiftlyBinDirContents = (try? FileManager.default.contentsOfDirectory(atPath: swiftlyBinDir.path)) ?? [String]() let toolchainBinDir = Swiftly.currentPlatform.findToolchainBinDir(version) let toolchainBinDirContents = try FileManager.default.contentsOfDirectory(atPath: toolchainBinDir.path) - let proxyTo = if let systemManagedSwiftlyBin = systemManagedSwiftlyBin { - systemManagedSwiftlyBin - } else { - swiftlyBin.path - } - let existingProxies = swiftlyBinDirContents.filter { bin in do { let linkTarget = try FileManager.default.destinationOfSymbolicLink(atPath: swiftlyBinDir.appendingPathComponent(bin).path) diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index 3f4b3a22..d7ef12a9 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -267,27 +267,61 @@ extension Platform { } } - public func systemManagedBinary(_ cmd: String) throws -> String? { - let userHome = FileManager.default.homeDirectoryForCurrentUser - let binLocs = [cmd] + ProcessInfo.processInfo.environment["PATH"]!.components(separatedBy: ":").map { $0 + "/" + cmd } - var bin: String? - for binLoc in binLocs { - if FileManager.default.fileExists(atPath: binLoc) { - bin = binLoc - break - } + // Find the resting place for the swiftly binary, installing ourselves there if possible. + public func findSwiftlyBin(installSwiftly: Bool) throws -> String? { + let swiftlyHomeBin = self.swiftlyBinDir.appendingPathComponent("swiftly", isDirectory: false).path + + // First, let's find out where we are. + let cmd = CommandLine.arguments[0] + let cmdAbsolute = if cmd.hasPrefix("/") { + cmd + } else { + ([FileManager.default.currentDirectoryPath] + (ProcessInfo.processInfo.environment["PATH"]?.components(separatedBy: ":") ?? [])).map { + $0 + "/" + cmd + }.filter { + FileManager.default.fileExists(atPath: $0) + }.first + } + + // We couldn't find outselves in the usual places, so if we're not going to be installing + // swiftly then we can assume that we are running from the final location. + if cmdAbsolute == nil && !installSwiftly && FileManager.default.fileExists(atPath: swiftlyHomeBin) { + return swiftlyHomeBin } - guard let bin = bin else { - throw Error(message: "Could not locate source of \(cmd) binary in either the PATH, relative, or absolute path") + + // If we are system managed then we know where swiftly should be, no installation necessary. + let userHome = FileManager.default.homeDirectoryForCurrentUser + if let cmdAbsolute, !cmdAbsolute.hasPrefix(userHome.path + "/") && (cmdAbsolute.hasPrefix("/usr/") || cmdAbsolute.hasPrefix("/opt/") || cmdAbsolute.hasPrefix("/bin/")) { + return cmdAbsolute } - // If the binary is in the user's home directory, or is not in system locations ("/usr", "/opt", "/bin") - // then it is expected to be outside of a system package location and we manage the binary ourselves. - if bin.hasPrefix(userHome.path + "/") || (!bin.hasPrefix("/usr") && !bin.hasPrefix("/opt") && !bin.hasPrefix("/bin")) { + // If we're running inside an xctest or we couldn't determine our absolute path then + // we don't install nor do we have a swiftly location for the proxies. + guard let cmdAbsolute, !cmdAbsolute.hasSuffix("xctest") else { return nil } - return bin + // We're installed and running in our bin directory, return our location + if cmdAbsolute == swiftlyHomeBin { + return swiftlyHomeBin + } + + if installSwiftly { + SwiftlyCore.print("Installing swiftly in \(swiftlyHomeBin)...") + + if FileManager.default.fileExists(atPath: swiftlyHomeBin) { + try FileManager.default.removeItem(atPath: swiftlyHomeBin) + } + + do { + try FileManager.default.moveItem(atPath: cmdAbsolute, toPath: swiftlyHomeBin) + } catch { + try FileManager.default.copyItem(atPath: cmdAbsolute, toPath: swiftlyHomeBin) + SwiftlyCore.print("Swiftly has been copied into the installation directory. You can remove '\(cmd)'. It is no longer needed.") + } + } + + return FileManager.default.fileExists(atPath: swiftlyHomeBin) ? swiftlyHomeBin : nil } public func findToolchainBinDir(_ toolchain: ToolchainVersion) -> URL { From 34e4021b951e3aa9ce17c3cc0c6f17ee09ac2aac Mon Sep 17 00:00:00 2001 From: Chris McGee <87777443+cmcgee1024@users.noreply.github.com> Date: Fri, 22 Nov 2024 07:59:12 -0500 Subject: [PATCH 41/45] Update Sources/SwiftlyCore/Platform.swift Co-authored-by: Justice Adams <107649528+justice-adams-apple@users.noreply.github.com> --- Sources/SwiftlyCore/Platform.swift | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index d7ef12a9..12cd0eff 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -283,7 +283,7 @@ extension Platform { }.first } - // We couldn't find outselves in the usual places, so if we're not going to be installing + // We couldn't find ourselves in the usual places, so if we're not going to be installing // swiftly then we can assume that we are running from the final location. if cmdAbsolute == nil && !installSwiftly && FileManager.default.fileExists(atPath: swiftlyHomeBin) { return swiftlyHomeBin From 55fc274aea1e29619fd2ff11b5f44ba7c80a967c Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Fri, 22 Nov 2024 09:54:03 -0500 Subject: [PATCH 42/45] Refactor find/install swiftly binary into two functions --- Sources/Swiftly/Init.swift | 2 +- Sources/Swiftly/Install.swift | 2 +- Sources/SwiftlyCore/Platform.swift | 83 ++++++++++++++++++++---------- 3 files changed, 59 insertions(+), 28 deletions(-) diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index 571b9034..8483ad41 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -139,7 +139,7 @@ internal struct Init: SwiftlyCommand { guard var config else { throw Error(message: "Configuration could not be set") } // Move our executable over to the correct place - let _ = try Swiftly.currentPlatform.findSwiftlyBin(installSwiftly: true) + try Swiftly.currentPlatform.installSwiftlyBin() if overwrite || !FileManager.default.fileExists(atPath: envFile.path) { SwiftlyCore.print("Creating shell environment file for the user...") diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 9dc10cdf..7de678fa 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -244,7 +244,7 @@ struct Install: SwiftlyCommand { var pathChanged = false // Create proxies if we have a location where we can point them - if let proxyTo = try? Swiftly.currentPlatform.findSwiftlyBin(installSwiftly: false) { + if let proxyTo = try? Swiftly.currentPlatform.findSwiftlyBin() { // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. let swiftlyBinDir = Swiftly.currentPlatform.swiftlyBinDir let swiftlyBinDirContents = (try? FileManager.default.contentsOfDirectory(atPath: swiftlyBinDir.path)) ?? [String]() diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index 12cd0eff..a82b4ad2 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -267,8 +267,60 @@ extension Platform { } } - // Find the resting place for the swiftly binary, installing ourselves there if possible. - public func findSwiftlyBin(installSwiftly: Bool) throws -> String? { + // Install ourselves in the final location + public func installSwiftlyBin() throws { + // First, let's find out where we are. + let cmd = CommandLine.arguments[0] + let cmdAbsolute = if cmd.hasPrefix("/") { + cmd + } else { + ([FileManager.default.currentDirectoryPath] + (ProcessInfo.processInfo.environment["PATH"]?.components(separatedBy: ":") ?? [])).map { + $0 + "/" + cmd + }.filter { + FileManager.default.fileExists(atPath: $0) + }.first + } + + // We couldn't find ourselves in the usual places. Assume that no installation is necessary + // since we were most likely invoked at SWIFTLY_BIN_DIR already. + guard let cmdAbsolute else { + return + } + + // Proceed to installation only if we're in the user home directory, or a non-system location. + let userHome = FileManager.default.homeDirectoryForCurrentUser + guard cmdAbsolute.hasPrefix(userHome.path + "/") || + (!cmdAbsolute.hasPrefix("/usr/") && !cmdAbsolute.hasPrefix("/opt/") && !cmdAbsolute.hasPrefix("/bin/")) + else { + return + } + + // Proceed only if we're not running in the context of a test. + guard !cmdAbsolute.hasSuffix("xctest") else { + return + } + + // We're already running from where we would be installing ourselves. + guard case let swiftlyHomeBin = self.swiftlyBinDir.appendingPathComponent("swiftly", isDirectory: false).path, cmdAbsolute != swiftlyHomeBin else { + return + } + + SwiftlyCore.print("Installing swiftly in \(swiftlyHomeBin)...") + + if FileManager.default.fileExists(atPath: swiftlyHomeBin) { + try FileManager.default.removeItem(atPath: swiftlyHomeBin) + } + + do { + try FileManager.default.moveItem(atPath: cmdAbsolute, toPath: swiftlyHomeBin) + } catch { + try FileManager.default.copyItem(atPath: cmdAbsolute, toPath: swiftlyHomeBin) + SwiftlyCore.print("Swiftly has been copied into the installation directory. You can remove '\(cmdAbsolute)'. It is no longer needed.") + } + } + + // Find the location where swiftly should be executed. + public func findSwiftlyBin() throws -> String? { let swiftlyHomeBin = self.swiftlyBinDir.appendingPathComponent("swiftly", isDirectory: false).path // First, let's find out where we are. @@ -285,42 +337,21 @@ extension Platform { // We couldn't find ourselves in the usual places, so if we're not going to be installing // swiftly then we can assume that we are running from the final location. - if cmdAbsolute == nil && !installSwiftly && FileManager.default.fileExists(atPath: swiftlyHomeBin) { + if cmdAbsolute == nil && FileManager.default.fileExists(atPath: swiftlyHomeBin) { return swiftlyHomeBin } - // If we are system managed then we know where swiftly should be, no installation necessary. + // If we are system managed then we know where swiftly should be. let userHome = FileManager.default.homeDirectoryForCurrentUser if let cmdAbsolute, !cmdAbsolute.hasPrefix(userHome.path + "/") && (cmdAbsolute.hasPrefix("/usr/") || cmdAbsolute.hasPrefix("/opt/") || cmdAbsolute.hasPrefix("/bin/")) { return cmdAbsolute } - // If we're running inside an xctest or we couldn't determine our absolute path then - // we don't install nor do we have a swiftly location for the proxies. + // If we're running inside an xctest then we don't have a location for this swiftly. guard let cmdAbsolute, !cmdAbsolute.hasSuffix("xctest") else { return nil } - // We're installed and running in our bin directory, return our location - if cmdAbsolute == swiftlyHomeBin { - return swiftlyHomeBin - } - - if installSwiftly { - SwiftlyCore.print("Installing swiftly in \(swiftlyHomeBin)...") - - if FileManager.default.fileExists(atPath: swiftlyHomeBin) { - try FileManager.default.removeItem(atPath: swiftlyHomeBin) - } - - do { - try FileManager.default.moveItem(atPath: cmdAbsolute, toPath: swiftlyHomeBin) - } catch { - try FileManager.default.copyItem(atPath: cmdAbsolute, toPath: swiftlyHomeBin) - SwiftlyCore.print("Swiftly has been copied into the installation directory. You can remove '\(cmd)'. It is no longer needed.") - } - } - return FileManager.default.fileExists(atPath: swiftlyHomeBin) ? swiftlyHomeBin : nil } From 942c4e40cc707e15d5cad0c3e1f5f17b9b7db8d3 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Fri, 22 Nov 2024 10:14:40 -0500 Subject: [PATCH 43/45] Re-introduce the bare swiftly invocation in an uninstalled state redirecting to swiftly init subcommand --- Sources/Swiftly/Proxy.swift | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index aca9440d..486aa986 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -11,9 +11,32 @@ public enum Proxy { } guard binName != "swiftly" else { - // Treat this as a swiftly invocation - await Swiftly.main() - return + // Treat this as a swiftly invocation, but first check that we are installed, bootstrapping + // the installation process if we aren't. + let configResult = Result { try Config.load() } + + switch configResult { + case .success: + await Swiftly.main() + return + case let .failure(err): + guard CommandLine.arguments.count > 0 else { fatalError("argv is not set") } + + if CommandLine.arguments.count == 1 { + // User ran swiftly with no extra arguments in an uninstalled environment, so we jump directly into + // an simple init. + try await Init.execute(assumeYes: false, noModifyProfile: false, overwrite: false, platform: nil, verbose: false, skipInstall: false) + return + } else if CommandLine.arguments.count >= 2 && CommandLine.arguments[1] == "init" { + // Let the user run the init command with their arguments, if any. + await Swiftly.main() + return + } else { + // We've been invoked outside the "init" subcommand and we're not yet configured. + // This will throw if the configuration couldn't be loaded and give the user an actionable message. + throw err + } + } } var config = try Config.load() From e7ab371a0047355ae714e8b609d7c2fb0c8e8fff Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 5 Dec 2024 13:00:07 -0500 Subject: [PATCH 44/45] Propagate the verbose flag to the pkgutil call when installing in non-standard macOS location --- Sources/MacOSPlatform/MacOS.swift | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Sources/MacOSPlatform/MacOS.swift b/Sources/MacOSPlatform/MacOS.swift index ddaaf187..b0ad224f 100644 --- a/Sources/MacOSPlatform/MacOS.swift +++ b/Sources/MacOSPlatform/MacOS.swift @@ -70,7 +70,7 @@ public struct MacOS: Platform { if !toolchainDir.fileExists() { try FileManager.default.createDirectory(at: toolchainDir, withIntermediateDirectories: false) } - try runProgram("pkgutil", "--verbose", "--expand", tmpFile.path, tmpDir.path) + try runProgram("pkgutil", "--verbose", "--expand", tmpFile.path, tmpDir.path, quiet: !verbose) // There's a slight difference in the location of the special Payload file between official swift packages // and the ones that are mocked here in the test framework. var payload = tmpDir.appendingPathComponent("Payload") From be095e5b4ff1d5311912181146595d591e381ccb Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 5 Dec 2024 14:26:49 -0500 Subject: [PATCH 45/45] Add proxy exclusion for asking swiftly for help or dumping help json --- Sources/Swiftly/Proxy.swift | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 486aa986..72b642c2 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -31,6 +31,10 @@ public enum Proxy { // Let the user run the init command with their arguments, if any. await Swiftly.main() return + } else if CommandLine.arguments.count == 2 && (CommandLine.arguments[1] == "--help" || CommandLine.arguments[1] == "--experimental-dump-help") { + // Allow the showing of help information + await Swiftly.main() + return } else { // We've been invoked outside the "init" subcommand and we're not yet configured. // This will throw if the configuration couldn't be loaded and give the user an actionable message.