TensorRT Memory Leak

[frenetj]

Describe the issue

We are currently experiencing a memory leak when using the TensorRT execution provider. The leak is considerable (from 3.8 GB to 7.7 GB depending on the model being run). Running in CPU also leaks about 1 GB for some models.

To reproduce

Attached is a little test program to reproduce the issue. See README.txt for details,

problem_adsk.zip

Urgency

This issue doesn't prevent us from shipping, but since the leak is considerable, we would like to address this issue ASAP.

Platform

Linux

OS Version

Rocky Linux 8.10

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.22

ONNX Runtime API

C

Architecture

X64

Execution Provider

TensorRT

Execution Provider Library Version

CUDA 12.8 TensorRT 10.8

[frenetj]

I forgot to mention that TensorRT is also leaking about 200 MB of VRAM.

To monitor memory usage in a shell:

nvidia-smi --query-compute-apps=pid,used_memory --format=csv -l 1