Accelerate reference walk when determining relevant obliterates #4
Conversation
| undefined, | ||
| false, | ||
| ); | ||
| } | ||
| return true; |
There was a problem hiding this comment.
can we do anything about this true to end the walk earlier? same thing below
| ) { | ||
| seg.localRefs?.walkReferences((ref) => { | ||
| const obliterate = ref.properties?.obliterate as ObliterateInfo | undefined; | ||
| if (obliterate?.end === ref && starts.has(obliterate)) { |
There was a problem hiding this comment.
i think we can remove the start when we find a match, and when starts are empty, we can stop looking
| false, | ||
| ); | ||
| if ( | ||
| (seg.movedSeq !== undefined && |
There was a problem hiding this comment.
this won't work once we have expansion, as the segment that has the ref won't be moved, so i'm not sure we should add this.
There was a problem hiding this comment.
an idea could be to put all the obliterate refs in a stored segment set, and do that scan on that, rather than the tree. something like that should probably be a pr of its own. maybe even two sets, one for starts, and one for ends, so we can quickly find all the starts, then fine corresponding ends.
| this.moveSeqs.includes(seg.movedSeq)) || | ||
| (seg.localMovedSeq !== undefined && this.localMoveSeqs.has(seg.localMovedSeq)) | ||
| ) { | ||
| seg.localRefs?.walkReferences( |
There was a problem hiding this comment.
i think it would be safe to only check the first and last position of the segment, rather than scan the whole thing, as we shouldn't merge segments above the min seq, so the obliterate refs should always stay on the ends of the segments
|
Closing in favor of #6 |
Make use of the moveSeqs and localMoveSeqs field on mergeTree to avoid walking all of the segments during block insert. Tests are not passing at the moment, I think #3 is needed before they will pass.