Skip to content

Issues: flashinfer-ai/flashinfer

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

71 Open 132 Closed
71 Open 132 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

I can't install. Please help.
#845 opened Feb 14, 2025 by kgboyko
1
[Feature] Can BatchDecodeWithPagedKVCacheWrapper return attention scores to all tokens, not just logsumexp?
#838 opened Feb 14, 2025 by yawnzh
1
PrefillPlan tries to allocate more memory than float_workspace_size_in_bytes passed in.
#809 opened Feb 12, 2025 by rchardx
1
BatchPrefillWithPagedKVCacheSM90Run failed with error: operation not supported
#807 opened Feb 12, 2025 by sfc-gh-yewang
1
[BUG] attention/prefill.cuh(138): error: expression must have a constant value
#806 opened Feb 11, 2025 by haohaibo
4
Numerical stability issue in recent commits since 0.2.0 bug Something isn't working priority: high
#805 opened Feb 11, 2025 by rchardx
5
[Docs] TypeError: single_prefill_with_kv_cache() got an unexpected keyword argument 'use_fp16_qk_reduction'
#802 opened Feb 10, 2025 by sleepwalker2017
[Tracing Issue] Multi-head Latent Attention
#792 opened Feb 6, 2025 by yzh119
3 of 4 tasks
[Feature] Reuse JIT code path for building AOT wheel enhancement New feature or request
#791 opened Feb 6, 2025 by abcdabcd987
@abcdabcd987
6
Question about cascade inference
#789 opened Feb 5, 2025 by sleepwalker2017
11
[Tracking Issue] Performance with CUDA 12.8
#780 opened Feb 4, 2025 by yzh119
6 tasks
PyTorch 2.6 wheels
#771 opened Feb 1, 2025 by yzh119
read cached content
#750 opened Jan 24, 2025 by Mrxiangli
4
[Feature] Llama3.1 RoPE on the fly enhancement New feature or request
#746 opened Jan 21, 2025 by turboderp
@yzh119
BatchPrefillWithPagedKVCacheDispatched for sm90
#745 opened Jan 21, 2025 by tangcy98
1
[Tracking Issue] Checklist for 8-bit KV-Cache improments
#742 opened Jan 17, 2025 by yzh119
5 tasks
AttributeError: module 'flashinfer._kernels' has no attribute 'apply_rope_pos_ids_cos_sin_cache'
#741 opened Jan 17, 2025 by fergusfinn
1
[Feature] MiniMax-01 Lightning Attention
#739 opened Jan 16, 2025 by yzh119
4
Build from source fails with older versions of setuptools (<72.2.0) because of PosixPath types returned from .glob
#738 opened Jan 16, 2025 by fergusfinn
Flashinfer==0.2.0 precision error when tested on vLLM unit tests
#736 opened Jan 13, 2025 by Dr-Left
@yzh119
C++ benchmarks CMake error caused by enable_fp16 option in generate.py
#734 opened Jan 13, 2025 by rtxxxpro
[RFC]: Introducing ReproSpec for Strong Reproducibility in LLM Inference RFC Request for Comments
#733 opened Jan 11, 2025 by yzh119
1
ROPE problem in MLA decode kernel
#730 opened Jan 9, 2025 by lsw825
2
after torch compile with 0.2.0, speed is become very slow
#727 opened Jan 9, 2025 by MichoChan
6
Custom mask slows down attention.
#724 opened Jan 8, 2025 by qiyuxinlin
9
ProTip! Mix and match filters to narrow down what you’re looking for.