bug: expecting build reproducibility for forge test, forge build and forge coverage

[gnapoli23]

Component

Forge

Have you ensured that all of these are up to date?

• Foundry
• Foundryup

What version of Foundry are you on?

forge 0.2.0 (27d008f 2024-09-07T11:27:46.212820056Z)

What command(s) is the bug in?

forge test - forge build - forge coverage

Operating System

Linux

Describe the bug

I have a Foundry project which builds, tests and creates test coverage in a docker container based on the latest version of ghcr.io/foundry-rs/foundry image.

The project is hosted in an internal repository, and I expect that when I run forge build then running forge test would not recompile the contracts, but in many cases it happens that even if I don't make any change to the contracts source code, other colleagues running forge test on their machines have a recompilation even if they are "pre-compiled" and versioned in the repo.

This does not happen always, is there a reason behind that?

More over, we noticed that when we run forge coverage the recompilation always happens, indeed we have diffs in the compiled contracts. The diffs seems to be in the formatting of the file, btw.

Is there a way to have build reproducibility for these 3 commands whatever is the machine we are running them on?
Is the compilation somehow based on the timestamp of the files?
Thanks for your support.

[zerosnacks]

Regarding forge coverage, it requires optimizations to be disabled (which is enabled by default, see #2486) so recompilation is expected here.

Does running forge test require a full recompilation or a partial recomplilation?

[gnapoli23]

Hi @zerosnacks , thanks for the info about forge coverage.

Regarding forge test usually it's a partial recompilation, not a full one.

[SocksNFlops]

This is giving weird behaviors when utilizing the creationCode of contracts in lib dependencies. In other words, it's impossible to use precomputed hashes. For instance, if you run this in forge test and forge compile, you will get different results:

import {CONTRACT} from ...
...
bytes32 initHashCode = keccak256(abi.encodePacked(type(CONTRACT).creationCode));
console.log("initHashCode");
console.logBytes(initHashCode);

If you pre-compute them and hardcode them in the contracts, forge test works fine.

However, if you run forge coverage, it will recompile them with different creationCodes than the ones used in forge test, and these won't match the precomputed ones anymore. This breaks all of the unit tests where you're attempting to use the hash to predict a create2 output.

What's even worse, is after you run forge coverage, it caches the build of the lib-dependencies, and then proceeds to break every run of forge test unless you run forge clean.

I believe this is because it's recompiling the lib-dependencies without optimizations, whereas in unit tests and deployments, they have optimizations on. If forge coverage is going to recompile everything in src/, is there a way we can at least allow our lib dependencies to be compiled with the standard optimizations? Left as is, either forge test or forge coverage will run successfully, but not both. This isn't ideal.