Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark all objects reachable from roots as live before doing main cyclic GC pass #126491

Open
markshannon opened this issue Nov 6, 2024 · 3 comments
Labels
3.14 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage

Comments

@markshannon
Copy link
Member

markshannon commented Nov 6, 2024

Objects can only be cyclic garbage if they are not reachable.
So, if we can cheaply identify the majority of reachable objects before performing the (relatively slow) cycle detecting pass, we can save a lot of time.

Performing a transitive closure of all objects reachable from global roots (the sys and builtins modules as well as builtin class's dicts and sublasses) plus a transitive closure of all objects reachable from the stacks can eliminate >90% of all objects relatively cheaply.

Initial experiments show a ~3% speedup, with an almost 50% speedup of the most gc-heavy benchmark.

This idea has been proposed a few times.
@nascheme has definitely suggested it before. Perhaps he can add links to any prior discussion and/or experiments?

What makes this more feasible now is that the GC can see the evaluation stack of frames, thanks to #124392, so we would now expect that the vast majority of reachable objects can be cheaply marked, thus improving the efficiency of cycle detection considerably.

Linked PRs

@markshannon markshannon added performance Performance or resource usage interpreter-core (Objects, Python, Grammar, and Parser dirs) 3.14 new features, bugs and security fixes labels Nov 6, 2024
@kumaraditya303
Copy link
Contributor

Is that benchmark public, link isn't working for me?

@markshannon
Copy link
Member Author

Fixed the link

@nascheme
Copy link
Member

nascheme commented Nov 6, 2024

FYI, I made an incremental "mark alive" version of my idea. My PR is based off of 3.13 but it could be done on top of the "incremental gc" that's in the "main" branch as well. This approach avoids a problem with the current incremental GC: if you happen to pull in an object near the root of the object graph, the GC pause could be long because all reachable objects are pull in to the cyclic GC pass.

#126511

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.14 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage
Projects
None yet
Development

No branches or pull requests

3 participants