Freshly generated snapshot is corrupt

[angeloashmore]

Hi there!

AVA reports the the following error when using snapshots. The snapshot has been freshly generated locally and pushed to GitHub to run in an GitHub Actions workflow.

  value-group.ts › creates a mock Group field value

  Could not compare snapshot

  The snapshot file is corrupted.

  File path: /home/runner/work/prismic-mock/prismic-mock/test/snapshots/value-group.test.ts.snap

  Please run AVA again with the --update-snapshots flag to recreate it.

  › snapshotTwiceMacro (/home/runner/work/prismic-mocktest/__testutils__/snapshotTwiceMacro.ts:7:4)

This error only appears when running within GitHub Actions. The test passes locally.

Any idea what could cause a corrupted snapshot to be generated?

Relevant files and tests:

• Test: https://github.com/prismicio/prismic-mock/blob/97861bf3cf46cb9a42c6e2b5d6b4c5c60aab6cd2/test/value-group.test.ts#L8

• Macro used by the test: https://github.com/prismicio/prismic-mock/blob/97861bf3cf46cb9a42c6e2b5d6b4c5c60aab6cd2/test/__testutils__/snapshotTwiceMacro.ts#L3-L8

• Snapshot (Markdown): https://github.com/prismicio/prismic-mock/blob/97861bf3cf46cb9a42c6e2b5d6b4c5c60aab6cd2/test/snapshots/value-group.test.ts.md

• Snapshot (binary): https://github.com/prismicio/prismic-mock/blob/97861bf3cf46cb9a42c6e2b5d6b4c5c60aab6cd2/test/snapshots/value-group.test.ts.snap

• Failing test: https://github.com/prismicio/prismic-mock/runs/3305415519?check_suite_focus=true#step:7:199

AVA config

// ava.config.js

export default {
	extensions: ["ts"],
	files: ["./test/**/*.test.ts"],
	require: ["ts-eager/register"],
	verbose: true,
};

AVA version

$ npx ava --version
3.15.0

Thank you!

[novemberborn]

That's strange!

The snapshot format has changed in AVA 4 (in pre-release). Do you think you could give that a go? Perhaps the problem goes away… see https://github.com/avajs/ava/releases/tag/v4.0.0-alpha.2 and https://github.com/avajs/ava/releases/tag/v4.0.0-alpha.1.

[angeloashmore]

@novemberborn Thanks for the suggestion. I wasn't aware there was a AVA 4 pre-release.

Unfortunately, the same error occurs. This PR updates the previously linked test to AVA 4 with new snapshots: prismicio/prismic-mock#1

I can get around this by not using snapshots for those tests so it's not blocking me. I'm happy to help diagnose the issue, however. Thank you!

[angeloashmore]

In case it is relevant, here is my local system info where tests pass:

  System:
    OS: macOS 10.15.6
    CPU: (12) x64 Intel(R) Core(TM) i5-10400 CPU @ 2.90GHz
    Shell: 4.4.23 - /nix/store/n6kzbax7sjy3kha78hpfh024ghbykgxa-bash-interactive-4.4-p23/bin/bash
  Binaries:
    Node: 14.17.0 - /nix/store/jvkhrd4b513zrsbbmssg07hpkjpc9bmh-nodejs-14.17.0/bin/node
    npm: 7.15.1 - /nix/store/v8wdcakdc4k9pbykikl7vsijq4dp6z1c-node_npm-7.15.1/bin/npm

[novemberborn]

Hey @angeloashmore I haven't had a chance yet to look into this further.

[angeloashmore]

@novemberborn Not a problem! I really appreciate you following up.

[novemberborn]

@angeloashmore sorry for the wait.

I couldn't actually run the tests from prismicio/prismic-mock#1 but I copied a snapshot file into a separate project and set things up so AVA would load it.

AVA snapshots start with some readable text, followed by a version number, followed by a SHA256 checksum of the rest of the file. This checksum is corrupted.

This is how the checksum is computed and the data is prepared before file is written:

ava/lib/snapshot-manager.js

Lines 190 to 196 in 725c5d6

	const sha256sum = crypto.createHash('sha256').update(compressed).digest();
	return Buffer.concat([
	READABLE_PREFIX,
	VERSION_HEADER,
	sha256sum,
	compressed
	], READABLE_PREFIX.byteLength + VERSION_HEADER.byteLength + SHA_256_HASH_LENGTH + compressed.byteLength);

And how the checksum is compared:

ava/lib/snapshot-manager.js

Lines 221 to 223 in 725c5d6

	const sha256sum = crypto.createHash('sha256').update(compressed).digest();
	const expectedSum = buffer.slice(sha256sumOffset, compressedOffset);
	if (!sha256sum.equals(expectedSum)) {

I don't see room for false positives there, so I'm pretty confident the snapshot is corrupted. Indeed zlib.gunzipSync() throws its own data check error.

Why that is happening I do not know. My best guess is that there are concurrent writes, except that we're (attempting) to write the files atomically. And regardless that assumes you're running the same test file twice, concurrently.

Perhaps you can try and isolate the error?

[angeloashmore]

@novemberborn Thanks for taking a look, and sorry about the @prismicio/mock tests not running. I've been working with local package links and didn't realize external builds were failing.

I'll try to come up with a smaller reproduction and narrow down what triggers this to happen. Thanks for the details!

[onigoetz]

Hi, I have the exact same issue on AVA 4.0

I can delete the file, run tests to create the snapshots, re-run the tests -> it complains that the file is corrupted.

I don't have a small repro case, but everything I have is pushed in that branch : https://github.com/swissquote/crafty/tree/ava-integration-tests

Steps to see the error:

1. clone https://github.com/swissquote/crafty
2. checkout ava-integration-tests
3. yarn install
4. yarn build
5. cd packages/integration
6. yarn ava __tests__/crafty-preset-typescript-webpack.js

Running the full test suite shows more corrupt files, but this is the shortest path to see errors.

My environment:

• macOS
• Node 16.13.0

I can run more tests and experiments if you need me to, I'm not exactly sure where to start though

[jonahsnider]

I experienced a similar issue as the original commenter on v4.0.0 and it was fixed by v4.0.1

[onigoetz]

Indeed it does seem to work with 4.0.1 👍

[angeloashmore]

Thanks for the heads up @jonahsnider!