Skip to content

Issues: NVIDIA/TensorRT-LLM

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

394 Open 1,825 Closed
394 Open 1,825 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Feature Request] TensorRT-LLM libs should work with older versions of GLIBC
#2795 opened Feb 19, 2025 by vihangm
[Model Requests] Support Qwen2.5-VL Architecture
#2794 opened Feb 18, 2025 by mtezgider
deepseek branch link error: undefined reference to `std::ios_base_library_init()' from libfp8_blockscale_gemm.a bug Something isn't working
#2791 opened Feb 18, 2025 by wahaha22
2 of 4 tasks
2
Incompatible buffer sizes when running Speculative decoding with draft/target models on Qwen2 bug Something isn't working
#2789 opened Feb 17, 2025 by gloritygithub11
1 of 4 tasks
When executing this command, many modules are missing, such as luguru, hydra, and llava. Is there any way to install all of these at once? bug Something isn't working
#2788 opened Feb 17, 2025 by J4S0N666
4 tasks
2
Deepseek-v3 running on 2xH100 nodes getting poor performanc bug Something isn't working
#2786 opened Feb 14, 2025 by zymy-chen
2 of 4 tasks
The performance of Qwen1.5-7B based on the trtllm-bench test was very poor bug Something isn't working
#2785 opened Feb 14, 2025 by ruru5697
3 of 4 tasks
Does anyone try TensorRT-LLM for ComfyUI
#2784 opened Feb 13, 2025 by TuanNT-ZenAI
Bug when loading an engine using LoRA through LLM API bug Something isn't working Investigating LLM API/Workflow triaged Issue has been triaged by maintainers
#2782 opened Feb 13, 2025 by pei0033
2 of 4 tasks
@nv-guomingz
2
User Buffer and Reduce Fusion overwritten to False
#2781 opened Feb 12, 2025 by jolyons123
GPU Utilization drops gradually over time using Executor API bug Something isn't working
#2778 opened Feb 12, 2025 by MahmoudAshraf97
3 of 4 tasks
1
Inconsistent Batch Index Order in Decoupled Mode with trt-llm bug Something isn't working
#2777 opened Feb 12, 2025 by Oldpan
2 of 4 tasks
8
DeepSeek-V3 fp8 tp32 failed to convert chectpoint bug Something isn't working
#2776 opened Feb 12, 2025 by MtFitzRoy
2 of 4 tasks
Processing multi concurrent request by Qwen2-VL is slow. It seems infer in queue.
#2775 opened Feb 12, 2025 by zhaocc1106
Installation broken with 0.17.0.post1 with poetry due to git / flash infer dependency. bug Something isn't working
#2774 opened Feb 12, 2025 by michaelfeil
1 of 4 tasks
Limit max GPU memory used
#2773 opened Feb 11, 2025 by bri25yu
Cannot create checkpoint for llama-3.2 (1B, 3B) bug Something isn't working
#2772 opened Feb 11, 2025 by falkbene
3 of 4 tasks
TypeError: quantize_and_export() got an unexpected keyword argument 'cp_size'
#2771 opened Feb 11, 2025 by yanduoduan
Unable to install tensorrt_llm on Amazon Linux 2.
#2769 opened Feb 9, 2025 by eduardzl
1
CUDA Illegal memory access for certain input sizes to Whisper bug Something isn't working
#2767 opened Feb 9, 2025 by MahmoudAshraf97
2 of 4 tasks
2
Are there any plans to implement DualPipe parallelism from DeepSeek
#2765 opened Feb 7, 2025 by ttim
2
OSError: bfloat16 is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
#2763 opened Feb 7, 2025 by zmtttt
2
Mixtral SmoothQuant
#2759 opened Feb 7, 2025 by shana34
tensorrtllm [0.16] protobuf input data type mismatch
#2758 opened Feb 7, 2025 by sujituk
Building from source does not work bug Something isn't working Investigating triaged Issue has been triaged by maintainers
#2757 opened Feb 6, 2025 by maximzubkov
2 of 4 tasks
9
ProTip! Type g i on any issue or pull request to go back to the issue listing page.