Skip to content

v1.3.0-RC1

Pre-release
Pre-release
Compare
Choose a tag to compare
@bloopoid bloopoid released this 04 Jun 16:21
v1.3.0-RC1
c382eaf
This commit was signed with the committer’s verified signature. The key has expired.
jvican Jorge
GPG key ID: B0E525E3C3F4D176
Expired
Verified
Learn about vigilant mode.

bloop v1.3.0

Bloop is a build server and CLI tool for the Scala programming language developed by the Scala Center.

Bloop v1.3.0 is the biggest release of this project to this date.

With over 520 commits since the v1.2.5 release, ~26000 additions and ~9500
deletions, it aggregates together fundamental changes in the compilation
pipeline, novel build server features, user-facing improvements and bug fixes
together.

Upgrade guide 📥

These are only the upgrade steps. If you don't have bloop installed, you must read
the installation instructions instead.

If you're on macOS, upgrade to the latest version with:

$ brew upgrade scalacenter/bloop/bloop
$ brew services restart bloop

If you're on Windows using scoop, upgrade to the latest version with:

$ scoop upgrade bloop
$ bloop ng-stop
$ // Start the server as you usually do (via systemd, manually, desktop services)

🆕 If you're on Arch Linux, upgrade to the latest version with:

$ rm -rf ~/.bloop
$ yaourt -S bloop
$ systemctl --user start bloop

Otherwise, use the generic installation method:

$ curl -L https://github.com/scalacenter/bloop/releases/download/v1.2.5/install.py | python
$ bloop ng-stop
$ // Start the server as you usually do (via systemd, manually, desktop services)

Read the complete installation instructions in our Installation page.

New compilation engine with client guarantees 💪 👪

v1.3.0 features a redesigned compilation engine built to solve tooling
challenges already present in our current developer workflow but brought to
the foreground by the formalized use of a client-server architecture. This
compilation engine sports a more efficient way of doing IO work (with no
compromises in correctness) and provides a new contract to build clients.

Bloop v1.3.0 and onwards guarantees clients that their actions will have
the smallest usage footprint possible and will not conflict with those of
other concurrent clients being served by the same server in the same build.

For example, if Metals is compiling
your project via Bloop and you spawn a bloop CLI command such as bloop test foo --watch at the same time, Bloop guarantees that:

  1. The server heavily caches compilations for the same inputs (aka compile deduplication)

    If inputs haven't changed between the requests, only the first client
    request will trigger a compilation. The compilation of the second client
    will be deduplicated based on the compilation side effects recorded by the
    build server, so only one compilation will happen.

  2. Different compilation requests in the same build can run concurrently (aka
    compile isolation)

    If inputs have changed between requests, Bloop will compile the changed
    projects concurrently, avoiding shared state and conflicts with ongoing
    compilations.

  3. The outputs produced by both requests are independent in the file system.

    The compilation products will be stored in independent target
    directories only owned by the client. This independence is essential to
    allow clients to independently run any build action without altering task
    executions. Think avoiding crashing JVMs because other processes are
    writing contents to open class file pointers!

This is the first step towards modelling these build actions as pure
functions despite their pervasive side effects. Bloop manages any internal
state as best as possible to provide the best developer experience to end
users. There will be more work in this area in the future!

Related pull requests

  1. Isolate and deduplicate compilation and Zipkin tracing support by jvican
  2. Pre-populate client classes dir if empty by jvican
  3. Deduplicate compilation if project inputs change by jvican
  4. Invalidate symbols with forked and local javac by jvican
  5. Don't create and delete empty classes dirs by jvican
  6. Cancel deduplication if ongoing compilation looks dead by jvican
  7. Use generic classes dir for the pipelined traversal by jvican
  8. Log to the user a deduplication is happening by jvican
  9. Revalidate class files compiled by dependent projects by jvican
  10. Fix not found type errors during incremental compilation by jvican
  11. Add CLI lock timeout and be more flexible when locking CLI clients
  12. Fix remaining issues around handling of compile products by jvican
  13. Catch exceptions when computing the bundle by jvican

New Zipkin tracing support 📡

Bloop v1.3.0 traces any compilation request to Zipkin if a Zipkin server is
running in the background. Here's a screenshots showing collected compilation
traces for the akka/akka project.

A Zipkin instance with build traces from the akka/akka project

And another screenshot showing the top part of a detailed compilation trace.

A Zipkin instance showing a detaild profile of a build trace from the akka/akka project

The zipkin integration is explained in the performance guide below.

In-depth compiler performance guide 🚀

The compiler performance
guide is a new
addition to our documentation that explains users how to:

  1. Get familiar with the performance characteristics of project builds
  2. Compare the performance of Bloop with that of other tools compiling Scala code
  3. Modify a complicated build to optimize build times

The guide is rich in details and written to provide pointers to those users
wanting to speed up their workflow with Bloop and optimize their current
build times further.

It is open for any community addition, so if you'd like to add something,
help us improve it!

Faster build pipelining 🏇

Build pipelining is a compilation performance technique to start the
compilation of modules in a build before their dependencies have finished
compilation.

Build pipelining was merged back in December, 2018 and made available in
bloop v1.2.1, making bloop the first Scala developer tool to implement this
technique and support it under an experimental flag.

Bloop v1.3.0 changes its internal implementation to make it faster. We're
very excited about what this entails for any Scala build in the wild. The
bloop project will publish benchmarks and a concise guide in a follow-up
post, keep tuned! 😄

Use Ammonite by default in bloop console 🎮

Contributed by @torkelrogstad.

Ammonite is a modernized Scala REPL with syntax highlighting, multi-line
editing, the ability to load maven artifacts directly in the REPL, and many
other quality-of-life improvements missing in the default Scala REPL.

Bloop v1.3.0 defaults on Ammonite every time you run bloop console on a
project. Unlike running amm in your terminal, the Ammonite integration will
automatically start an Ammonite REPL using the classpath of a project in your
build. That allows you to use all the code defined in that classpath and
experiment with its behavior.

Interested in the implementation details? Have a look at Add support for Ammonite REPL by default.

Better Metals and BSP integration 🤘 💯

BSP support in bloop v1.3.0 is vastly superior to that in v1.2.5. The new release implements a few non-implemented endpoints, fixes several non-reported issues in diagnostics and improves connection support and cancellation capabilities.

The Metals integration is superior to that in v1.2.5 thanks to this
non-exhaustive list of PRs:

  1. Guarantee cancellation upon sudden BSP client dropouts by jvican
  2. Upgrade to BSP 2.0.0-M4 by tgodzik
  3. Implement BSP buildTarget/scalaMainClasses by marek1840
  4. Implement BSP buildTarget/scalaTestClasses by marek1840
  5. Send out server BSP responses/notifications in order by jvican
  6. A few unlisted PRs that casually fix issues in BSP semantics

Add support for Scala 2.13 ▶️

Bloop v1.3.0 finally adds support for Scala
2.13.x.

Add support for JDK 9 and 11 ⏩

Contributed by @er1c, bloop v1.3.0 supports JDK 9 and JDK 11. If you run
bloop with these Java runtimes, it is recommended to enable the full parallel
GC with -XX:+UseParallelGC as compilation is faster.

For this reason, Java 8 still remains the recommended runtime for the time
being and our installers might install it by default. In the future, we might
consider upgrading to a newer Java runtime.

  1. Compile on Java9+ by er1c
  2. Add Java9+ reverse-compatible changes by er1c

New Arch Linux installer 🏄

Contributed by TheElectronWill.

Bloop has a new AUR package for Arch Linux that is updated by our release infrastructure.
You can now install bloop in Arch Linux by running pacaur -S bloop or yaourt -S bloop.

Improved file watcher responsiveness ⌚ 🔥

bloop v1.3.0 has better and snappier file watcher infrastructure, with
protection against overflowing
events and several fixes to
bugs reported after v1.2.5.
One of these bugs is a recurrent problems in macOS machines where Bloop
server's JVM would crash due to uncleaned state in the Carbon-based file
watcher.

Support for file watching in run commands has not yet been implemented. If
you want to help, drop by bloop's Gitter channel and we will give
you pointers to help implement it.

Fix use of test arguments in bloop test 📈

Running bloop test in your project or passing specific test arguments to Bloop's test runner could produce issues such as:

[E] Unexpected error during remote test execution: 'Socket closed'.
[E] Internal error when running tests: sbt.ForkMain$Run$RunAborted: java.net.SocketException: Broken pipe (Write failed)

This pull request makes sure
that we handling missing test frameworks better and that the above issues
processing test arguments are gone.

Improvements in build tools support (bloopInstall) 🛠️

Improvements in sbt

Automatic support for integration tests in sbt builds

sbt-bloop v1.3.0 will automatically export integration test
configurations to independent
build targets. If you have modified your sbt build to support integration tests
in your build, you can safely remove that logic.

Support sbt source dependencies by default

Contributed by olafurpg. Support for sbt project references was before
hidden behind an specific sbt boolean setting. Now it's enabled by
default.

Other

  1. Avoid duplication of projects in dependencies field by tkroman
  2. Fix small bug when exporting scalac and javac options in sbt by jvican

Improvements in Gradle

  1. Include Gradle's transitive dependencies in classpath by Arthurm1
  2. Ignore annotation processor in Gradle by tgodzik
  3. Don't add JAR dependencies of Gradle projects in the classpath by tgodzik
  4. Export source dependencies from Gradle by tgodzik

Thanks to these improvements, Metals will be able to support "Import Build"
in Gradle projects in its next release.

Improvements in Mill

  1. Point Mill docs to merged bloop integration upstream by Baccata
  2. Fix broken link in Mill documentation by perryao
  3. Update Mill version by robby-phd

Thanks to these improvements, Metals will be able to support "Import Build"
in mill builds in its next release.

Improvements in Maven

  1. Avoid NPE when collecting resources in Maven's Bloop plugin by artyomboyko
  2. Add source dependencies to Maven plugin
  3. Fix warning about Maven being run in interactive mode by tgodzik
  4. Fix Maven plugin in Windows by adding version of maven-invoker by tgodzik

Thanks to these improvements, Metals will be able to support "Import Build"
in Maven projects in its next release.

Miscellaneous

Here's a list of key miscellaneous changes in this release.

  1. Synchronize CLI clients to the same build by jvican
  2. Don't recompile project if server dies by jvican
  3. Add -Dbloop.computation.cores and env var to configure thread pool by jvican
  4. Display Java version and Java home in bloop about by cquiroz
  5. Add failing test case where source files are pulled in multiple times by jackkoenig
  6. Don't print warnings when autocompleting by CucumisSativus
  7. Add buildpress, a tool to automatically export from any build tool to bloop by jvican
  8. Add support for standalone repo path in CLI args by tkroman
  9. Add contributing guide to the website by jvican
  10. Improve bloop launcher's test suite by jvican
  11. Change SCOOP env variable to avoid conflict by Arthurm1
  12. Show correct default for install.py '--dest' option by BardurArantsson
  13. Use a guaranteed unique name for temporary directories by propensive
  14. Update website, enable versioning and assort fixes by jvican
  15. Update contributing guide with expected error by CucumisSativus
  16. Fix installation link in README by jrudolph
  17. Encode UUIDs with Base 64 by odisseus
  18. Fix off-one error in fish autocompletions by 2m
  19. Use two cores for Bloop's computation pool in Travis by jvican
  20. Update docs URL in Bloop error messages by AckoLF
  21. Update Homebrew server log path by regadas

Next steps 🚶

The recent work on supporting concurrent build clients, giving strong
building guarantees to clients and the bloop launcher released in v1.2.5
has crystallized in a robust tooling framework to take our build tool
integrations to the next level.

Bloop v1.4.0 will enable sbt to offload compilation of projects
automatically to Bloop, speeding up sbt project load and making the
cutting-edge performance improvements in Bloop available to sbt.
The goal is that users will be able to use bloop without changing their
current development workflow.

Such integrations for other build tools will also be possible. The bloop team
volunteers to mentor people interested in contributing such integrations for
popular build tools such as mill or Gradle. If you're interested, please get
in contact with us.

Special acknowledgements 🙏

bloop v1.3.0 is an exciting release because of the vast amount of external
contributors that we have had. Since v1.2.5, bloop has received
contributions by more than 27 OSS contributors.

Additionally, the bloop team has welcomed three new more members: Tomasz
Godzik (@tgodzik) and Marek Żarnowski (@marek1840)
from VirtusLab which have made this release a reality and Roman
Tkalenko (@tkroman) which has recently started contributing in
making Bloop better for CI workflows.

I am thrilled to see the bloop community grow and be working with such
skilled people in improving the tooling in the Scala community.

According to git shortlog -sn --no-merges v1.2.5..v1.3.0, 27 people
contributed to this release: Guillaume R, Roman Tkalenko, tgodzik, Jorge,
Eric Peters, Michał Gutowski, Marek Żarnowski, Carlos Quiroz, Artyom Boyko,
Arthur McGibbon, Tomasz Godzik, Torkel Rogstad, odisseus, AckoLF, Ólafur Páll
Geirsson, ArthurM, Bardur Arantsson, David Gregory, Enver Osmanov, Filipe
Regadas, Johannes Rudolph, Jon Pretty, Martynas Mickevičius, Mike, Olivier
Mélois, Jack Koenig and Robby.

Assets 7
Loading