[Remote Compaction] Parse options from OPTIONS file instead of using CompactionServiceInput #13025
+104
−109
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
We've been serializing and deserializing DBOptions and CFOptions (and other CF into) as part of
CompactionServiceInput
. These are all readily available in the OPTIONS file and the remote worker can read the OPTIONS file to obtain the same information. This helps reducing the size of payload significantly.The only downside of this approach is that if
fail_if_options_file_error
isfalse
, we cannot safely guarantee that OPTIONS file is up-to-date with DB's current state.fail_if_options_file_error
option is marked for deprecation. We will considerfail_if_options_file_error=false
with remote compaction as incompatible features.This also solves the problem where we had to open the default CF with the CFOption from another CF if the remote compaction is for a non-default column family. (TODO comment in /db_impl_secondary.cc)
Test Plan
Unit Tests
Also tested with Meta's internal Offload Infra