PR Guidelines

PRs on Functionality

To be written

PRs on Performance Optimization

If performance optimization is the main purpose of your PR, follow the below guidelines. These are general guidelines and not strict requirements. Use your judgment or seek feedback to adapt them to each particular case.

Overall Principles

• Try to provide actual data rather than speculative discussions
• Focus on what matters the most
• Set a performance goal before jumping on implementation. Without a goal, nobody can evaluate if a mission is accomplished
• Seek feedback early and regularly as necessary.

Preliminary

• What this PR is about
  • Provide a clear scope of the PR
• Background context. Why it matters
  • Provide enough information to discuss why this problem should be addressed
  • E.g., 10% slower than the PyTorch eager mode
• SOL analysis
  • Set the expected performance as accurately and precisely as possible
  • The roofline model is a good starting point
  • An existing highly tuned implementation can also be a SOL target (e.g., CUTLASS matmul performance)
• Current status
  • Where we are
  • What are the reasons of the performance gap with SOL
• Actual measurement data to support the above analysis

Approach(es)

• Technical details of an approach
• Expected performance gains
• Limitations and concerns
  • What will not be addressed by this approach
  • It is completely fine not to solve the whole problem at once
• Discuss and compare multiple approaches if applicable

Results

• Correctness is more important than performance. Make sure all existing tests pass. Add more tests if necessary.
• Evaluate the performance of the implementation as thoroughly as possible
• Provide a concise summary of the performance results
• How close is the actual performance to the SOL performance?
  • It’s completely fine not hitting the SOL performance as it provides an upper bound.
  • Understanding why there’s still a gap is more important. If necessary, discuss if filling the remaining gap is important.
• Is there any regression?
  • In particular for heuristics development, it’s a difficult challenge to develop robust heuristics that can uniformly optimize a given fusion.
  • Having a regression may be unavoidable and may be acceptable depending on its impact and the overall performance benefit.
  • Try to provide quantitative results why this optimization should be adopted even with a regression.
• If multiple approaches are tried, do not just discard failed ones as they may still have insightful results.