You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When running benchmarks in a repo with a lot of packages, if two benchmarks have the same name this github-action doesn't take into account the package name cause the comparison to fail randomly.
Hey @gaby
Thanks for reporting this! I wonder how we could deal with this. I think it will create a similar issue of backward compatibility as we faced with PR #177
We could append the package name to the name of the benchmark so it would be eg
This should create a unique name to compare
As a remedy to not break existing metrics we could treat benchmarks with only one package as special cases and not append the package name in those cases. Or add both cases: old (without suffix) and new (with suffix) so that it can be then dropped in the next release. WDYT?
@ktrz Adding both cases is probably the way to go now. Other option I can think is having a config option to specify if you want benchmarks with suffix or not.
When running benchmarks in a repo with a lot of packages, if two benchmarks have the same name this
github-action
doesn't take into account the package name cause the comparison to fail randomly.This can be seen on this run: https://github.com/gofiber/fiber/actions/runs/10873073504
Where
BenchmarkAppendMsgitem
shows up multiple times with different values even though they are in different packages.The text was updated successfully, but these errors were encountered: