Skip to content

Issues: flashinfer-ai/flashinfer

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

101 Open 169 Closed
101 Open 169 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Does flashinfer support head_size = 576 for Ampere GPUs?
#1043 opened Apr 28, 2025 by ghostplant
2
flashinfer.jit: Loading JIT ops: batch_decode_with_kv_cache_dtype_q_f16_dtype_kv_f16_dtype_o_f16_dtype_idx_i32_head_dim_qk_64_head_dim_vo_64_posenc_0_use_swa_False_use_logits_cap_False
#1038 opened Apr 24, 2025 by yawnzh
1
Which one of FlashMLA is faster on Hopper?
#1037 opened Apr 24, 2025 by ghostplant
Can flashinfer's CutlassSegmentGEMMSM90Run function be used for LoRA computing on H20?
#1034 opened Apr 23, 2025 by chenhongyu2048
2
flashinfer.decode.single_decode_with_kv_cache: Floating point exception (core dumped)
#1027 opened Apr 20, 2025 by MenHimChan
[Performance Issue] FlashInfer shows no performance improvement with FP8 compared to BF16 in BatchDecodeWithPagedKVCacheWrapper with page_size=1
#1024 opened Apr 18, 2025 by cscyuge
1
[Bug] FP8 scaling factors (k_scale/v_scale) not taking effect in BatchPrefillWithPagedKVCacheWrapper
#1023 opened Apr 17, 2025 by cscyuge
@yzh119
1
Low performance of POD Attention compared to BatchPrefillWithPagedKVCache
#1022 opened Apr 17, 2025 by Edenzzzz
@yzh119
1
[Feature Request] Llama 4
#1004 opened Apr 6, 2025 by yzh119
3 tasks
1
Does FlashInfer support ViT attention in Qwen2.5_Vl?
#992 opened Apr 1, 2025 by IdeaMeshDyx
[Feature Tracking] MLA-FP8 Hopper kernel
#990 opened Mar 31, 2025 by yzh119
C++ API Stability
#988 opened Mar 30, 2025 by AgrawalAmey
2
Unclear behavior of top_k for k < 1
#979 opened Mar 28, 2025 by sharvil
2
top_k_top_p_sampling_from_logits incompatible with torch.compile + CUDAGraph
#978 opened Mar 28, 2025 by sharvil
6
struck in the JIT loading stage without any errors
#972 opened Mar 25, 2025 by ZAntonyH
4
how to use mla C++ API [C++ compilation error ]
#963 opened Mar 20, 2025 by paperHZ
1
Question about which kernel to use for non-squared mask
#959 opened Mar 19, 2025 by RobeZH
2
CUDA error: too many blocks in cooperative launch(72) when use multi GPU
#938 opened Mar 13, 2025 by Zzzer0o
2
batchsize dim for flashinfer api
#927 opened Mar 10, 2025 by arthursunbao
“BatchPrefillWithPagedKVCache failed with error an illegal memory access was encountered” when I use a cuda that is not cuda:0
#924 opened Mar 9, 2025 by wfloveiu
2
Anyway to build flashinfer with pytorch 2.7.0 and cuda 12.8
#922 opened Mar 8, 2025 by ryderhwang
@yzh119
RMSNorm failed with error code no kernel image is available for execution on the device
#920 opened Mar 7, 2025 by nanmi
2
[Bugfix] CUDAGraph Compatibility in AppendPagedKVCacheKernel for Variable-Length Inputs
#919 opened Mar 7, 2025 by SungBalance
1
ValueError: Invalid mode: forward_mode=<ForwardMode.TARGET_VERIFY: 6>
#916 opened Mar 6, 2025 by tchaton
flashinfer.prefill.single_prefill_with_kv_cache meets error when running on A100
#915 opened Mar 6, 2025 by FengzhuoZhang
ProTip! Follow long discussions with comments:>50.