feat: add StateRoot migration construct #216

seanmcgary · 2025-02-03T18:52:58Z

This PR introduces the concept of a state migration. When introducing new features, it's possible that there is existing transaction data that was not codified into a model that now needs to be. Rather than truncating the database, re-indexing blocks resulting in modifying existing StateRoots, this migration construct allows us to populate new models with existing data, leaving existing StateRoots unchanged, and codifying the changes in a sub-tree whos root is represented in the top-level StateRoot tree.

The StateMigrator specifies which migrations run at which block height for a given chain. Multiple migrations can be run for a single block and will be run in the order specified. Each migration will produce a root representing the changes made. Each of those roots are composed of the same StateRoot structure used at the very top-level, one for each block that was touched in the migration process. In the majority of cases where we're populating models with only a few events, these StateRoot trees will be rather sparse and only include the new model.

Each migration works by:

Creating a new instance of the StateManager
Querying the database for the TransactionLogs that need to be processed
Groups them by block number
In the order of blockNumber ascending, feeds each log through the StateManager as it would have if it came through in real time.

The same lifecycle of HandleLogStateChange --> CommitFinalState --> GenerateStateRoot is followed.

todo:

Update fork block heights and dates

internal/config/config.go

pkg/eigenState/stateManager/stateManager.go

pkg/eigenState/stateMigrator/stateMigrator.go

pkg/eigenState/stateMigrator/stateMigrations/202502031222_operatorSets/migration.go

0xrajath

LGTM! I think we just need to update the fork height and date for preprod.

This PR introduces the concept of a state migration. When introducing new features, it's possible that there is existing transaction data that was not codified into a model that now needs to be. Rather than truncating the database, re-indexing blocks resulting in modifying existing StateRoots, this migration construct allows us to populate new models with existing data, leaving existing StateRoots unchanged, and codifying the changes in a sub-tree whos root is represented in the top-level StateRoot tree. ![Screenshot 2025-02-04 at 8 25 47 AM](https://github.com/user-attachments/assets/149021bc-7457-49f0-b4f2-2091a4d0b46c) The StateMigrator specifies which migrations run at which block height for a given chain. Multiple migrations can be run for a single block and will be run in the order specified. Each migration will produce a root representing the changes made. Each of those roots are composed of the same StateRoot structure used at the very top-level, one for each block that was touched in the migration process. In the majority of cases where we're populating models with only a few events, these StateRoot trees will be rather sparse and only include the new model. Each migration works by: * Creating a new instance of the StateManager * Querying the database for the TransactionLogs that need to be processed * Groups them by block number * In the order of blockNumber ascending, feeds each log through the StateManager as it would have if it came through in real time. The same lifecycle of HandleLogStateChange --> CommitFinalState --> GenerateStateRoot is followed.

seanmcgary changed the title ~~feat: add stateRoot migration construct~~ feat: add StateRoot migration construct Feb 4, 2025

seanmcgary force-pushed the sm-stateMigration branch from 2d4be85 to 87a21ef Compare February 4, 2025 17:02

seanmcgary marked this pull request as ready for review February 5, 2025 13:54

seanmcgary requested a review from a team as a code owner February 5, 2025 13:54

seanmcgary requested a review from 0xrajath February 5, 2025 13:54