Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT REVIEW] HCD 1.1.0 hotfix #1611

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
CNDB-11950: cache sstable density result to speed up test (#1454)
### What is the issue
riptano/cndb#11950

### What does this PR fix and why was it fixed
We saw many test failures in `UnifiedCompactionStrategyTest` after
#1407. After investigating it a bit, it seems that the root cause to the
unit test failure is likely the cost associated with the mockito calls
to get different values.

However, without changing anything in Mockito, I was able to optimize
the `UCS::getLevels` method enough to make the test suite go from timing
out to taking 3 minutes 11 seconds when running `ant test
-Dtest.name=UnifiedCompactionStrategyTest` on the command line.

Let's see if the test passes in butler.

### Checklist before you submit for review
- [ ] Make sure there is a PR in the CNDB project updating the Converged
Cassandra version
- [ ] Use `NoSpamLogger` for log lines that may appear frequently in the
logs
- [ ] Verify test results on Butler
- [ ] Test coverage for new/modified code is > 80%
- [ ] Proper code formatting
- [ ] Proper title for each commit staring with the project-issue
number, like CNDB-1234
- [ ] Each commit has a meaningful description
- [ ] Each commit is not very long and contains related changes
- [ ] Renames, moves and reformatting are in distinct commits
michaeljmarshall authored and szymon-miezal committed Feb 12, 2025
commit adee5dbc048012fadfa8df4c2962b6c68565dc1f
Original file line number Diff line number Diff line change
@@ -28,7 +28,6 @@
import java.util.Set;
import java.util.TreeMap;
import java.util.UUID;
import java.util.concurrent.atomic.AtomicLong;
import java.util.function.BiPredicate;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
@@ -873,14 +872,20 @@ public Map<Arena, List<Level>> getLevels(Collection<? extends CompactionSSTable>
for (Arena arena : arenas)
{
List<Level> levels = new ArrayList<>(MAX_LEVELS);
arena.sstables.sort(currentShardManager::compareByDensity);

// Precompute the density, then sort.
List<SSTableWithDensity> ssTableWithDensityList = new ArrayList<>(arena.sstables.size());
for (CompactionSSTable sstable : arena.sstables)
ssTableWithDensityList.add(new SSTableWithDensity(sstable, currentShardManager.density(sstable)));
Collections.sort(ssTableWithDensityList);

double maxSize = controller.getMaxLevelDensity(0, controller.getBaseSstableSize(controller.getFanout(0)) / currentShardManager.localSpaceCoverage());
int index = 0;
Level level = new Level(controller, index, 0, maxSize);
for (CompactionSSTable candidate : arena.sstables)
for (SSTableWithDensity candidateWithDensity : ssTableWithDensityList)
{
final double size = currentShardManager.density(candidate);
final CompactionSSTable candidate = candidateWithDensity.sstable;
final double size = candidateWithDensity.density;
if (size < level.max)
{
level.add(candidate);
@@ -1445,4 +1450,25 @@ public String toString()
FBUtilities.prettyPrintMemory(spaceAvailable), rateLimitLog, remainingAdaptiveCompactions);
}
}

/**
* Utility wrapper to efficiently store the density of an SSTable with the SSTable itself.
*/
private static class SSTableWithDensity implements Comparable<SSTableWithDensity>
{
final CompactionSSTable sstable;
final double density;

SSTableWithDensity(CompactionSSTable sstable, double density)
{
this.sstable = sstable;
this.density = density;
}

@Override
public int compareTo(SSTableWithDensity o)
{
return Double.compare(density, o.density);
}
}
}