Skip to content

flux pipeline not work with DPMSolverMultistepScheduler and UniPCMultistepScheduler #11907

Open
@gameofdimension

Description

@gameofdimension

Describe the bug

FluxPipeline is not compatible with two advanced schedulers: DPMSolverMultistepScheduler and UniPCMultistepScheduler

Reproduction

python repro.py dpm or unipc, euler

import sys
import torch
from diffusers.schedulers.scheduling_dpmsolver_multistep import \
    DPMSolverMultistepScheduler
from diffusers.schedulers.scheduling_unipc_multistep import \
    UniPCMultistepScheduler

from diffusers import FluxPipeline


def main(scheduler):
    ckpt = "black-forest-labs/FLUX.1-dev"
    pipe = FluxPipeline.from_pretrained(ckpt, torch_dtype=torch.bfloat16).to("cuda")

    print("original scheduler", pipe.scheduler)

    if scheduler == 'dpm':
        pipe.scheduler = DPMSolverMultistepScheduler.from_config(
            pipe.scheduler.config, use_flow_sigmas=True, prediction_type="flow_prediction")
    elif scheduler == 'unipc':
        pipe.scheduler = UniPCMultistepScheduler.from_config(
            pipe.scheduler.config, use_flow_sigmas=True, prediction_type="flow_prediction")
    print("updated scheduler", pipe.scheduler)

    # prompt = "A cat holding a sign that says hello world"
    # prompt = "a white bird in front of a dinosaur standing by some trees"
    prompt = 'a motorcycle parked in an ornate bank lobby'
    num_inference_steps = 20
    generator = torch.Generator("cpu").manual_seed(0)
    image = pipe(
        prompt,
        height=1024,
        width=1024,
        guidance_scale=3.5,
        num_inference_steps=num_inference_steps,
        max_sequence_length=512,
        generator=generator,
    ).images[0]

    image.save(f"flux-diffusers-{num_inference_steps}.png")


if __name__ == '__main__':
    scheduler = sys.argv[1]
    main(scheduler)

Logs

Traceback (most recent call last):
  File "/root/dpm-flow-match/baseline.py", line 46, in <module>
    main(scheduler)
  File "/root/dpm-flow-match/baseline.py", line 31, in main
    image = pipe(
            ^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/diffusers/pipelines/flux/pipeline_flux.py", line 851, in __call__
    timesteps, num_inference_steps = retrieve_timesteps(
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/diffusers/pipelines/flux/pipeline_flux.py", line 133, in retrieve_timesteps
    raise ValueError(
ValueError: The current scheduler class <class 'diffusers.schedulers.scheduling_dpmsolver_multistep.DPMSolverMultistepScheduler'>'s `set_timesteps` does not support custom sigmas schedules. Please check whether you are using the correct scheduler.

System Info


- 🤗 Diffusers version: 0.34.0
- Platform: Linux-5.10.134-16.1.3.vip.an8.x86_64-x86_64-with-glibc2.39
- Running on Google Colab?: No
- Python version: 3.12.3
- PyTorch version (GPU?): 2.7.1+cu126 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Huggingface_hub version: 0.33.2
- Transformers version: 4.53.0
- Accelerate version: 1.8.1
- PEFT version: 0.16.0
- Bitsandbytes version: not installed
- Safetensors version: 0.5.3
- xFormers version: not installed
- Accelerator: NVIDIA L20, 46068 MiB
NVIDIA L20, 46068 MiB
NVIDIA L20, 46068 MiB
NVIDIA L20, 46068 MiB
NVIDIA L20, 46068 MiB
NVIDIA L20, 46068 MiB
NVIDIA L20, 46068 MiB
NVIDIA L20, 46068 MiB
- Using GPU in script?:  yes
- Using distributed or parallel set-up in script?: no

Who can help?

@yiyixuxu

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions