Skip to content

Commit

Permalink
bazel: Upgrade to 7.4.0, set max disk cache size (#30395)
Browse files Browse the repository at this point in the history
Upgrades the version of Bazel we use from `7.2.1` to `7.4.0`. Developers
won't need to do anything since we use `bazelisk`, the new version will
automatically get downloaded for them.

This also updates our `.bazelrc` to include:
```
common --experimental_disk_cache_gc_max_size=80G
```
This combined with users specifying `--disk_cache` in their home
`.bazelrc`, should bound the total size of artifacts that are kept
around for builds. Hopefully this should solve the "Low Disk Warnings"
folks frequently run into, if they use Bazel instead of Cargo.

### Motivation

Pickup the new automatic gc disk cache to bound the total size of
artifacts.

### Checklist

- [x] This PR has adequate test coverage / QA involvement has been duly
considered. ([trigger-ci for additional test/nightly
runs](https://trigger-ci.dev.materialize.com/))
- [x] This PR has an associated up-to-date [design
doc](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/README.md),
is a design doc
([template](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/00000000_template.md)),
or is sufficiently small to not require a design.
  <!-- Reference the design in the description. -->
- [x] If this PR evolves [an existing `$T ⇔ Proto$T`
mapping](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/command-and-response-binary-encoding.md)
(possibly in a backwards-incompatible way), then it is tagged with a
`T-proto` label.
- [x] If this PR will require changes to cloud orchestration or tests,
there is a companion cloud PR to account for those changes that is
tagged with the release-blocker label
([example](MaterializeInc/cloud#5021)).
<!-- Ask in #team-cloud on Slack if you need help preparing the cloud
PR. -->
- [x] If this PR includes major [user-facing behavior
changes](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/guide-changes.md#what-changes-require-a-release-note),
I have pinged the relevant PM to schedule a changelog post.
  • Loading branch information
ParkMyCar authored Nov 8, 2024
1 parent 38d50d9 commit 082e0e3
Show file tree
Hide file tree
Showing 4 changed files with 12 additions and 4 deletions.
3 changes: 3 additions & 0 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,9 @@ common --experimental_remote_merkle_tree_cache
common --experimental_remote_merkle_tree_cache_size=5000
# Don't make the user wait for uploading artifacts to complete, finish it in the background.
common --bes_upload_mode=fully_async
# Make sure any local disk cache stays within a reasonable size.
common --experimental_disk_cache_gc_max_size=80G

# Tells `xz` to use all available cores.
action_env=XZ_OPT=-T0

Expand Down
2 changes: 1 addition & 1 deletion .bazelversion
Original file line number Diff line number Diff line change
@@ -1 +1 @@
7.2.1
7.4.0
4 changes: 2 additions & 2 deletions ci/builder/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -283,8 +283,8 @@ ARG BAZEL_VERSION
# contain arm64 releases.
RUN arch_bazel=$(echo "$ARCH_GCC" | sed -e "s/aarch64/arm64/" -e "s/amd64/x86_64/") bazel_version=$(echo "$BAZEL_VERSION") \
&& curl -fsSL -o /usr/local/bin/bazel https://github.com/bazelbuild/bazel/releases/download/$bazel_version/bazel-$bazel_version-linux-$arch_bazel \
&& if [ "$arch_bazel" = arm64 ]; then echo '5a4cc979353671e438b9469b833924c2361e25a580cc278a75877aedc27c1c53 /usr/local/bin/bazel' | sha256sum --check; fi \
&& if [ "$arch_bazel" = x86_64 ]; then echo '80ccd1ecb4b88750fbe5d7622d67072fddcba9da7808f13356555e480bf67875 /usr/local/bin/bazel' | sha256sum --check; fi \
&& if [ "$arch_bazel" = arm64 ]; then echo 'fac4b954e0501c2be8b9653a550b443eb85284e568d08b102977e2bf587b09d7 /usr/local/bin/bazel' | sha256sum --check; fi \
&& if [ "$arch_bazel" = x86_64 ]; then echo '48ea0ff9d397a48add6369c261c5a4431fe6d5d5348cfb81411782fb80c388d3 /usr/local/bin/bazel' | sha256sum --check; fi \
&& chmod +x /usr/local/bin/bazel

# Hardcode some known SSH hosts, or else SSH will ask whether the host is
Expand Down
7 changes: 6 additions & 1 deletion misc/bazel/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,8 @@ Bazel has numerous [command line options](https://bazel.build/reference/command-
which can be defined in a `.bazelrc` file to create different configurations that you run Bazel
with. We have a [`.bazelrc`](../../.bazelrc) in the root of our repository that defines several
common build configurations, but it's also recommended that you create a `.bazelrc` in your home
directory (i.e. `~/.bazelrc`) to customize how you run Bazel locally.
directory (i.e. `~/.bazelrc`) to customize how you run Bazel locally. Options specified in your
home RC file will override those of the workspace RC file.

A good default to start with is:
```
Expand All @@ -104,6 +105,10 @@ common --local_resources=cpu="HOST_CPUS-1"
# Define a shared disk cache so builds from different Materialize repos can share artifacts.
build --disk_cache=~/.cache/bazel
# Optional. The workspace RC already sets a max disk cache size, but you can override that if you
# have more limited disk space.
common --experimental_disk_cache_gc_max-size=40G
```

# How Bazel Works
Expand Down

0 comments on commit 082e0e3

Please sign in to comment.