Search results

[Feature]: trtllm-bench latency doesn't support --report_json

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#10528 In NVIDIA/TensorRT-LLM;

· tdjackey opened on Jan 8, 2026

[Bug]: AutoDeploy: correct EP sharding based on MLP type + unit test

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Infra<NV>automated tests, build checks, github actions, system stability & efficiency.<NV>automated tests, build checks, github actions, system stability & efficiency.

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

Bug

#10526 In NVIDIA/TensorRT-LLM;

· lucaslie opened on Jan 8, 2026

[Bug][AutoDeploy]: Demollm multiprocess executor is broken

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

Status: Open.

#10520 In NVIDIA/TensorRT-LLM;

· bmarimuthu-nv opened on Jan 7, 2026

[Feature]: AutoDeploy: support HF-native fp8 dynamic quantization

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Low PrecisionLower-precision formats (INT8/INT4/FP8) for TRTLLM quantization (AWQ, GPTQ).Lower-precision formats (INT8/INT4/FP8) for TRTLLM quantization (AWQ, GPTQ).

Status: Open.

Task

#10519 In NVIDIA/TensorRT-LLM;

· lucaslie opened on Jan 7, 2026

[Feature]: AutoDeploy: support rms_norm for attention sharding heuristic

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

Task

#10518 In NVIDIA/TensorRT-LLM;

· lucaslie opened on Jan 7, 2026

[Bug]: regression due to the MOE refactor: AttributeError: 'FP8EPShardingInfo' object has no attribute 'mlp_type'

bugSomething isn't workingSomething isn't working

Model optimization<NV>Model-specific performance optimizations and tuning<NV>Model-specific performance optimizations and tuning

Status: Open.

#10513 In NVIDIA/TensorRT-LLM;

· MrGeva opened on Jan 7, 2026

[AutoDeploy]: Standardize perf experiments logging

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#10486 In NVIDIA/TensorRT-LLM;

· galagam opened on Jan 7, 2026

[Feature]: Adding support for Fuzzy Speculative Decoding (ACL Findings 2025)

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Speculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter

Status: Open.

#10481 In NVIDIA/TensorRT-LLM;

· maxholsman opened on Jan 7, 2026

[Bug][AutoDeploy]: Llama3.2 Vision models fail - exported model signature mismatch for use_cache/return_dict

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

bugSomething isn't workingSomething isn't working

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

Bug

#10464 In NVIDIA/TensorRT-LLM;

· tcherckez-nvidia opened on Jan 6, 2026

[AutoDeploy]: Support for GLM 4.7 text model

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#10462 In NVIDIA/TensorRT-LLM;

· suyoggupta opened on Jan 6, 2026

[Bug]: starcoder2-15b fails with flashinfer backend due to float32 dtype incompatibility

bugSomething isn't workingSomething isn't working

CUDA Graph

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

Bug

#10458 In NVIDIA/TensorRT-LLM;

· tcherckez-nvidia opened on Jan 6, 2026

[Bug]: Export fails for google/gemma-3-27b-it

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

bugSomething isn't workingSomething isn't working

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

Bug

#10454 In NVIDIA/TensorRT-LLM;

· tcherckez-nvidia opened on Jan 6, 2026