[Doc]: recommended docker for visual_gen

[christopher5106]

📚 The doc issue

Hi,

In the documentation you write

Recommended docker: nvcr.io/nvidia/tensorrt-llm/release:1.2.0rc4

If you just leave like that, it's complicated to understand what to expect from that image. I saw that Python 3.12 is installed.

1. Where is Visual gen: I tried from tensorrt_llm import visual_gen without success
   Should we clone it ? Do we have to install the package ?

2. What is in /app/tensorrt_llm and in /workspace/tutorials ? I see some bin/lib/include folders in /app/tensorrt_llm, it seems you compiled something.

3. I see that pytorch is installed but some deprecations:

>>> import torch
/usr/local/lib/python3.12/dist-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]

4. Are there any SageAttention, FlashAttention installed ?

5. What is the goal of this recommendation ?

Please help

Suggest a potential alternative/fix

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

[christopher5106]

I found the doc here https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tensorrt-llm/containers/release?version=1.3.0rc5.post1 but to my point of view, it's not clear how it relates to visual_gen that I didn't find in the image.

[zhenhuaw-me]

@christopher5106 Thanks for raising this issue. The feat/visual_gen branch is a prototype branch which is not included in the RC release. You need to go through the install guide which installs visual_gen as a different package. You can reach out to @forrestl111 if any setup issue on that branch.

The "production support" in the main branch is under development with limited model and features support. You can refer to this doc: https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/models/visual-generation.md
The first weekly release that is able to do like from tensorrt_llm import VisualGen will be 1.3.0rc8 and the release is still WIP.

[zhenhuaw-me]

I am going to convert this issue to a discussion since it's not a real doc issue. Please let me know if any concerns. Thank you for trying out TensorRT-LLM VisualGen!