LoRA with TensorRT

[benihime91]

Just came acrross the TensorRT optimized community pipelines. AFAIK they do not support loading LoRAs (please correct me if i am wrong). I know one option is to directly compile the merged LoRA and base model but that doesn't suit my workflow. Is there any way i can dynamically load and merge LoRA during inference. Any helps / tips ?

[sayakpaul]

Merging LoRA is possible: check https://huggingface.co/docs/diffusers/main/en/tutorials/using_peft_for_inference