chore(perf): Make all gems load faster #28104
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Gem.find_latest_files's performance scales with the amount of files contained across all gems of a project. This can quickly get out of hand, especially on projects with many gems, or large gems. Gem.loaded_spec is already populated by bundler, and we can iterate over only the name of the gems to detect those matching the google-cloud-* pattern. We can then guess the name of the file we expect to be present. On my machine, this makes `auto_load_files` go from ~100ms to <1ms, and requiring `bigtable` goes from ~220ms to ~110.
|
I don't understand what the CI failure is |
dazuma
requested changes
Jan 24, 2025
| # Ruby 2.0 does not have Gem.find_latest_files | ||
| Gem.find_files "google-cloud-*.rb" | ||
| Gem.loaded_specs.values.filter_map do |spec| | ||
| next unless spec.name.start_with?("google-cloud-") |
There was a problem hiding this comment.
The CI failures are rubocop warnings. To conform with this library's style, omit the method-argument parentheses on this line and the following two lines.
| Gem.find_files "google-cloud-*.rb" | ||
| Gem.loaded_specs.values.filter_map do |spec| | ||
| next unless spec.name.start_with?("google-cloud-") | ||
| path = File.join(spec.lib_dirs_glob, spec.name + ".rb") |
There was a problem hiding this comment.
My understanding is, spec.lib_dirs_glob can return glob syntax (i.e. including commas etc) in cases where there are multiple source paths. That would cause this to fail. Maybe we can iterate over spec.require_paths instead? (Note that require_paths is relative to spec.full_gem_path, and can also be nil, where we should default to "lib".)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Gem.find_latest_files's performance scales with the amount of files contained across all gems of a project. This can quickly get out of hand, especially on projects with many gems, or large gems.Gem.loaded_specis already populated by bundler, and we can iterate over only the name of the gems to detect those matching thegoogle-cloud-*pattern. We can then guess the name of the file we expect to be present.On my machine, this makes
auto_load_filesgo from ~100ms to <1ms, and requiringbigtablegoes from ~220ms to ~110ms.bundle exec rake ciin the gem subdirectory.