Skip to content

Commit

Permalink
Download architecture specific laucher (#30)
Browse files Browse the repository at this point in the history
  • Loading branch information
peterdeme authored Jan 12, 2023
1 parent 48959f2 commit 3916355
Show file tree
Hide file tree
Showing 12 changed files with 164 additions and 63 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ concurrency:
cancel-in-progress: true

jobs:
build:
docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@main
Expand Down
9 changes: 6 additions & 3 deletions .spacelift/config.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
version: 1
module_version: 1.2.0
module_version: 1.3.0

tests:
- name: Set up in a separate VPC
project_root: examples/separate-vpc
- name: AMD64-based workerpool
project_root: examples/amd64

- name: ARM64-based workerpool
project_root: examples/arm64
16 changes: 13 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,16 @@ The default AMI used by this module comes from the [spacelift-worker-image](http
repository. You can find the full list of AMIs on the [releases](https://github.com/spacelift-io/spacelift-worker-image/releases)
page.

## ARM-based AMI

You can use an ARM-based AMI by setting the `ami_id` variable to an arm64 AMI, and `ec2_instance_type` to an ARM-based instance type (e.g. `t4g.micro`).

We recommend using [Spacelift AMIs](https://github.com/spacelift-io/spacelift-worker-image/releases) because they come with every required tool preinstalled.

You can find an example of ARM-based workerpool in the [examples](./examples/) directory.

>❗️ If you use [custom runner images](https://docs.spacelift.io/concepts/stack/stack-settings.html#runner-image), make sure they support ARM. The default Spacelift images do support it.
## How to generate docs

The generated documentation is between `BEGIN_TF_DOCS` and `END_TF_DOCS` comments in the `README.md` file.
Expand Down Expand Up @@ -83,15 +93,15 @@ No requirements.
| <a name="input_configuration"></a> [configuration](#input\_configuration) | User configuration. This allows you to decide how you want to pass your token<br> and private key to the environment - be that directly, or using SSM Parameter<br> Store, Vault etc. Ultimately, here you need to export SPACELIFT\_TOKEN and<br> SPACELIFT\_POOL\_PRIVATE\_KEY to the environment. | `string` | n/a | yes |
| <a name="input_disable_container_credentials"></a> [disable\_container\_credentials](#input\_disable\_container\_credentials) | If true, the run container will not be able to access the instance profile<br> credentials by talking to the EC2 metadata endpoint. This is done by setting<br> the number of hops in IMDSv2 to 1. Since the Docker container goes through an<br> extra NAT step, this still allows the launcher to talk to the endpoint, but<br> prevents the container from doing so. | `bool` | `false` | no |
| <a name="input_domain_name"></a> [domain\_name](#input\_domain\_name) | Top-level domain name to use for pulling the launcher binary | `string` | `"spacelift.io"` | no |
| <a name="input_ec2_instance_type"></a> [ec2\_instance\_type](#input\_ec2\_instance\_type) | EC2 instance type for the workers | `string` | `"t3.micro"` | no |
| <a name="input_ec2_instance_type"></a> [ec2\_instance\_type](#input\_ec2\_instance\_type) | EC2 instance type for the workers. If an arm64-based AMI is used, this must be an arm64-based instance type. | `string` | `"t3.micro"` | no |
| <a name="input_enabled_metrics"></a> [enabled\_metrics](#input\_enabled\_metrics) | List of CloudWatch metrics enabled on the ASG | `list(string)` | <pre>[<br> "GroupDesiredCapacity",<br> "GroupInServiceInstances",<br> "GroupMaxSize",<br> "GroupMinSize",<br> "GroupPendingInstances",<br> "GroupStandbyInstances",<br> "GroupTerminatingInstances",<br> "GroupTotalInstances"<br>]</pre> | no |
| <a name="input_max_size"></a> [max\_size](#input\_max\_size) | Maximum number of workers to spin up | `number` | `10` | no |
| <a name="input_min_size"></a> [min\_size](#input\_min\_size) | Minimum numbers of workers to spin up | `number` | `0` | no |
| <a name="input_security_groups"></a> [security\_groups](#input\_security\_groups) | List of security groups to use | `list(string)` | n/a | yes |
| <a name="input_volume_encryption"></a> [volume\_encryption](#input\_volume\_encryption) | Whether to encrypt the EBS volume | `bool` | `false` | no |
| <a name="input_volume_size"></a> [volume\_size](#input\_volume\_size) | Size of instance EBS volume | `number` | `40` | no |
| <a name="input_vpc_subnets"></a> [vpc\_subnets](#input\_vpc\_subnets) | List of VPC subnets to use | `list(string)` | n/a | yes |
| <a name="input_worker_pool_id"></a> [worker\_pool\_id](#input\_worker\_pool\_id) | ID of the the worker pool | `string` | n/a | yes |
| <a name="input_worker_pool_id"></a> [worker\_pool\_id](#input\_worker\_pool\_id) | ID of the the worker pool. It is used for the naming convention of the resources. | `string` | n/a | yes |

## Outputs

Expand All @@ -101,4 +111,4 @@ No requirements.
| <a name="output_instances_role_arn"></a> [instances\_role\_arn](#output\_instances\_role\_arn) | ARN of the IAM role of the EC2 instances |
| <a name="output_instances_role_name"></a> [instances\_role\_name](#output\_instances\_role\_name) | Name of the IAM role of the EC2 instances |
| <a name="output_launch_template_id"></a> [launch\_template\_id](#output\_launch\_template\_id) | ID of the launch template |
<!-- END_TF_DOCS -->
<!-- END_TF_DOCS -->
7 changes: 6 additions & 1 deletion ami.tf
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
data "aws_ami" "this" {
most_recent = true
name_regex = "^spacelift-\\d{10}$"
name_regex = "^spacelift-\\d{10}-x86_64$"
owners = ["643313122712"]

filter {
Expand All @@ -12,4 +12,9 @@ data "aws_ami" "this" {
name = "virtualization-type"
values = ["hvm"]
}

filter {
name = "architecture"
values = ["x86_64"]
}
}
18 changes: 15 additions & 3 deletions asg.tf
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,27 @@ set -e
EOF

user_data_tail = <<EOF
currentArch=$(uname -m)
if [[ "$currentArch" != "x86_64" && "$currentArch" != "aarch64" ]]; then
echo "Unsupported architecture: $currentArch" >> /var/log/spacelift/error.log
return 1
fi
baseURL="https://downloads.${var.domain_name}/spacelift-launcher"
binaryURL=$(printf "%s-%s" "$baseURL" "$currentArch")
shaSumURL=$(printf "%s-%s_%s" "$baseURL" "$currentArch" "SHA256SUMS")
shaSumSigURL=$(printf "%s-%s_%s" "$baseURL" "$currentArch" "SHA256SUMS.sig")
echo "Downloading Spacelift launcher" >> /var/log/spacelift/info.log
curl https://downloads.${var.domain_name}/spacelift-launcher --output /usr/bin/spacelift-launcher 2>>/var/log/spacelift/error.log
curl "$binaryURL" --output /usr/bin/spacelift-launcher 2>>/var/log/spacelift/error.log
echo "Importing public GPG key" >> /var/log/spacelift/info.log
curl https://keys.openpgp.org/vks/v1/by-fingerprint/175FD97AD2358EFE02832978E302FB5AA29D88F7 | gpg --import 2>>/var/log/spacelift/error.log
echo "Downloading Spacelift launcher checksum file and signature" >> /var/log/spacelift/info.log
curl https://downloads.${var.domain_name}/spacelift-launcher_SHA256SUMS --output spacelift-launcher_SHA256SUMS 2>>/var/log/spacelift/error.log
curl https://downloads.${var.domain_name}/spacelift-launcher_SHA256SUMS.sig --output spacelift-launcher_SHA256SUMS.sig 2>>/var/log/spacelift/error.log
curl "$shaSumURL" --output spacelift-launcher_SHA256SUMS 2>>/var/log/spacelift/error.log
curl "$shaSumSigURL" --output spacelift-launcher_SHA256SUMS.sig 2>>/var/log/spacelift/error.log
echo "Verifying checksum signature..." >> /var/log/spacelift/info.log
gpg --verify spacelift-launcher_SHA256SUMS.sig 1>>/var/log/spacelift/info.log 2>>/var/log/spacelift/error.log
Expand Down
3 changes: 3 additions & 0 deletions examples/amd64/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# AMD64 (x86_64) based worker pool

In this example, we are setting up an amd64-based EC2 autoscaling group inside a VPC.
46 changes: 46 additions & 0 deletions examples/amd64/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "<5.0"
}

random = { source = "hashicorp/random" }
}
}

provider "aws" {
region = "eu-west-1"
}

data "aws_vpc" "this" {
default = true
}

data "aws_security_group" "this" {
name = "default"
vpc_id = data.aws_vpc.this.id
}

data "aws_subnets" "this" {
filter {
name = "vpc-id"
values = [data.aws_vpc.this.id]
}
}

resource "random_pet" "this" {}

#### Spacelift worker pool ####

module "this" {
source = "../../"

configuration = <<-EOT
export SPACELIFT_TOKEN="<token-here>"
export SPACELIFT_POOL_PRIVATE_KEY="<private-key-here>"
EOT
security_groups = [data.aws_security_group.this.id]
vpc_subnets = data.aws_subnets.this.ids
worker_pool_id = random_pet.this.id
}
3 changes: 3 additions & 0 deletions examples/arm64/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# ARM64 pool

In this example, we'll create a worker pool that will run on ARM64 instances. We'll use the `t4g.micro` instance type, which is the smallest ARM64 instance type available on AWS.
69 changes: 69 additions & 0 deletions examples/arm64/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "<5.0"
}

random = { source = "hashicorp/random" }
}
}

provider "aws" {
region = "eu-west-1"
}

data "aws_vpc" "this" {
default = true
}

data "aws_security_group" "this" {
name = "default"
vpc_id = data.aws_vpc.this.id
}

data "aws_subnets" "this" {
filter {
name = "vpc-id"
values = [data.aws_vpc.this.id]
}
}

data "aws_ami" "this" {
most_recent = true
name_regex = "^spacelift-\\d{10}-arm64$"
owners = ["643313122712"]

filter {
name = "root-device-type"
values = ["ebs"]
}

filter {
name = "virtualization-type"
values = ["hvm"]
}

filter {
name = "architecture"
values = ["arm64"]
}
}

resource "random_pet" "this" {}

#### Spacelift worker pool ####

module "this" {
source = "../../"

configuration = <<-EOT
export SPACELIFT_TOKEN="<token-here>"
export SPACELIFT_POOL_PRIVATE_KEY="<private-key-here>"
EOT
security_groups = [data.aws_security_group.this.id]
vpc_subnets = data.aws_subnets.this.ids
worker_pool_id = random_pet.this.id
ami_id = data.aws_ami.this.id
ec2_instance_type = "t4g.micro"
}
3 changes: 0 additions & 3 deletions examples/separate-vpc/README.md

This file was deleted.

47 changes: 0 additions & 47 deletions examples/separate-vpc/main.tf

This file was deleted.

4 changes: 2 additions & 2 deletions variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ variable "domain_name" {

variable "ec2_instance_type" {
type = string
description = "EC2 instance type for the workers"
description = "EC2 instance type for the workers. If an arm64-based AMI is used, this must be an arm64-based instance type."
default = "t3.micro"
}

Expand Down Expand Up @@ -95,7 +95,7 @@ variable "vpc_subnets" {

variable "worker_pool_id" {
type = string
description = "ID of the the worker pool"
description = "ID of the the worker pool. It is used for the naming convention of the resources."
}

locals {
Expand Down

0 comments on commit 3916355

Please sign in to comment.