-
Notifications
You must be signed in to change notification settings - Fork 425
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
v2.2: Stops flushing the accounts hash cache mmaps (backport of #5490) #5507
base: v2.2
Are you sure you want to change the base?
Conversation
Cherry-pick of 9d662b8 has failed:
To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally |
@Mergifyio rebase |
✅ Branch has been successfully rebased |
3090420
to
0a8def9
Compare
CI is blocked until the version is bumped:
|
@Mergifyio rebase |
(cherry picked from commit 9d662b8) # Conflicts: # accounts-db/src/cache_hash_data.rs
✅ Branch has been successfully rebased |
0a8def9
to
a6ef354
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## v2.2 #5507 +/- ##
=======================================
Coverage 83.3% 83.3%
=======================================
Files 800 800
Lines 370256 370247 -9
=======================================
+ Hits 308537 308586 +49
+ Misses 61719 61661 -58 🚀 New features to boost your workflow:
|
Problem
mmap
flush
is very slow, and should be avoided whenever possible. (This also includesmsync/fsync
.)We currently flush the accounts hash cache mmaps after creating them. This is unnecessary, and contributes to OS contention.
I originally added flushing the mmaps to fix a bug with accounts hash mismatches, but this was incorrect. Ultimately #5032 correctly identified and fixed the underlying issue.
Here's the accounts hash calc process. The peaks are during full accounts hash calcs. This shows the time spent writing the files themselves, and the time spent flushing the mmaps. It takes 115 seconds to write the files, but 190 seconds to flush the mmaps❗

Summary of Changes
Stop flushing the mmaps.
Note, I intend to backport this to v2.2.
Justification to Backport
Until the lattice hash feature are activated, the accounts hash calculation will still be running. This mmap flush is a big performance hit on all nodes taking snapshots. And the mmap flush was added to fix an issue that turns out it did not fix.
See #5490 (comment) for performance results 🚀
This is an automatic backport of pull request #5490 done by [Mergify](https://mergify.com).