Document how to test cache invalidations on Fastly

We use Fastly to serve Rust releases and crates. Sometimes it is necessary to
test that the cache can be purged, for which a new how-to guide has been
written.

The guide has been added to what will hopefully turn into a more complete
"service catalog" that supports the infra-team in its work.

.markdownlint.yaml

@@ -0,0 +1,3 @@
+---
+MD013:
+  code_blocks: false

service-catalog/README.md

@@ -0,0 +1,15 @@
+# Service Catalog
+
+The service catalog lists the infrastructure, services, and apps that the
+maintains. It is a work-in-progress and not exhaustive. It is our intention to
+improve it as we go.
+
+The documentation in the service catalog is inspired by the [Diataxis]
+framework, which proposes a systematic approach to categorize into tutorials,
+how-to guides, explanations, and reference documentation.
+
+## External Services
+
+- [Fastly](./fastly/README.md)
+
+[diataxis]: https://diataxis.fr/

service-catalog/fastly/README.md

@@ -0,0 +1,11 @@
+# Fastly
+
+[Fastly] is a [Content Delivery Network] (CDN) that caches content and makes it
+available closer to the user.
+
+## How-to Guides
+
+- [How to test cache invalidations](./how-to-test-cache-invalidations.md)
+
+[content delivery network]: https://en.wikipedia.org/wiki/Content_delivery_network
+[fastly]: https://www.fastly.com/

service-catalog/fastly/how-to-test-cache-invalidations.md

@@ -0,0 +1,137 @@
+# How to Test Cache Invalidations
+
+We use [Fastly] to cache Rust releases and crates. Both are cached with a long
+time-to-live (TTL) to improve the cache hit ratio and reduce our costs for
+outbound traffic from S3. However, we need to invalidate the cache when new
+versions of Rust or crates are released. This is done through Fastly's API as
+part of the release and publish processes.
+
+Sometimes, it is necessary to test whether the cache invalidation works as
+expected. This document outlines the steps that need to be taken.
+
+## Prerequisites
+
+## Find an Artifact
+
+The first step is finding a suitable artifact for the test.
+
+1. Go to <https://rust-lang.awsapps.com/start/> and log into the AWS console.
+2. Open the legacy account (`Rust Admin - 8450`) and navigate to the S3 service.
+3. Open the `dev-static-rust-lang-org` bucket and go into the `rustup` folder.
+4. Find a file that is not frequently updated, e.g. one in the archive.
+
+For the rest of the guide, we will be working with the checksum file
+`/rustup/archive/1.27.0/x86_64-unknown-linux-gnu/rustup-init.sha256`.
+
+## Download the Artifact
+
+The second step is downloading the artifact to ensure it is in the cache. This
+can be done by fetching the file from the Fastly service:
+
+```shell
+curl -I https://fastly-dev-static.rust-lang.org/rustup/archive/1.27.0/x86_64-unknown-linux-gnu/rustup-init.sha256
+```
+
+Make sure that the `x-cache` header is a `HIT`. This indicates that the file is
+cached in Fastly's network. Check the `age` header and note down the age of the
+file.
+
+After invalidating the cache, we expct the `x-cache` header to be a `MISS` and
+the `age` header to be `0`.
+
+## Purge the Cache
+
+The third step is purging the cache. We use two different methods to purge the
+cache: [surrogate keys] for Rust releases and [URL purges] for crates. Follow
+the respective instructions below.
+
+### Purge a Surrogate Key
+
+Purging a surrogate key invalidates all objects that are tagged with the key.
+Which objects get tagged with which key is defined in the VCL configuration for
+the Fastly service, which can be found in [`terragrunt/modules/release-distribution/fastly-static.tf`]
+in [rust-lang/simpleinfra].
+
+For this example, we will be purging a rustup artifact using the `rustup` key.
+
+For convenience, we export the Fastly service ID as an environment variable.
+Change this if you are working on a different service or environment.
+
+```shell
+# dev-static.rust-lang.org
+export FASTLY_SERVICE_ID=5qaYFyyiorVua6uCZg7It0
+```
+
+We also need a Fastly authentication token. Replace `<YOUR_FASTLY_TOKEN>` with
+the actual token.
+
+```shell
+export FASTLY_AUTH_TOKEN=<YOUR_FASTLY_TOKEN>
+```
+
+With a valid authentication token from Fastly, use the following command to send
+a [purge request](https://www.fastly.com/documentation/reference/api/purging/#purge-tag)
+for the key.
+
+```shell
+curl -i -X POST "https://api.fastly.com/service/${FASTLY_SERVICE_ID}/purge/rustup" \
+  -H "Fastly-Key: ${FASTLY_AUTH_TOKEN}" \
+  -H "Accept: application/json"
+```
+
+We expect the response to have a `200 OK` status code and return an ID for the
+purge in its body:
+
+```json
+{ "status": "ok", "id": "8230126-1715084554-7370607" }
+```
+
+### Purge a URL
+
+The simplest way to purge an object on Fastly is by purging its URL.
+
+Export the URL that you want to purge as an environment variable:
+
+```shell
+export URL_TO_PURGE=fastly-dev-static.rust-lang.org/rustup/archive/1.27.0/x86_64-unknown-linux-gnu/rustup-init.sha256
+```
+
+We also need a Fastly authentication token. Replace `<YOUR_FASTLY_TOKEN>` with
+the actual token and export it as an environment variable for convenience.
+
+```shell
+export FASTLY_AUTH_TOKEN=<YOUR_FASTLY_TOKEN>
+```
+
+With a valid authentication token from Fastly, use the following command to send
+a [purge request](https://www.fastly.com/documentation/reference/api/purging/#purge-single-url)
+for the URL.
+
+```shell
+curl -i -X POST "https://api.fastly.com/purge/${URL_TO_PURGE}" \
+  -H "Fastly-Key: ${FASTLY_AUTH_TOKEN}" \
+  -H "Accept: application/json"
+```
+
+We expect the response to have a `200 OK` status code and return an ID for the
+purge in its body:
+
+```json
+{ "status": "ok", "id": "8230126-1715084554-7370607" }
+```
+
+## Verify Purge
+
+After purging the cache, we can verify that the cache has been invalidated by
+fetching the file again:
+
+```shell
+curl -I https://fastly-dev-static.rust-lang.org/rustup/archive/1.27.0/x86_64-unknown-linux-gnu/rustup-init.sha256
+```
+
+The `x-cache` header should now be a `MISS` and the `age` header should be `0`.
+
+[`terragrunt/modules/release-distribution/fastly-static.tf`]: https://github.com/rust-lang/simpleinfra/blob/master/terragrunt/modules/release-distribution/fastly-static.tf
+[rust-lang/simpleinfra]: https://github.com/rust-lang/simpleinfra
+[surrogate keys]: https://www.fastly.com/documentation/guides/concepts/edge-state/cache/purging/#surrogate-key-purge
+[url purges]: https://www.fastly.com/documentation/guides/concepts/edge-state/cache/purging/#url-purge