Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add transactions #71

Open
jpsamaroo opened this issue Oct 30, 2023 · 0 comments
Open

Add transactions #71

jpsamaroo opened this issue Oct 30, 2023 · 0 comments

Comments

@jpsamaroo
Copy link
Collaborator

At the moment, our storage layer (in StorageState) is non-transactional, and so it's not possible to ensure that a certain set of operations occur without other (incompatible or performance-degrading) operations occurring in-between. This can make it impossible for the SimpleRecencyAllocator (which provides our swap-to-disk functionality) to be able to provide any guarantees that memory/disk usage actually falls within user-provided limits.

The obvious solution here is to add "transactions", where the set of operations to be performed in sequence are batched up and submitted as a single logical operation. Any transaction in progress can exclude other operations or transactions from occurring on the same piece(s) of data until the current transaction completes. Additionally, for performance, we can implement some logic to "merge" compatible operations across multiple transactions (or within the same transaction) when the end result will be identical to running the operations exclusively.

@dbedi3311

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant