Skip to content

[Sana Sprint] add image-to-image pipeline #11602

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 70 commits into from
May 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
8864fe5
sana sprint img2img
linoytsaban May 15, 2025
b2740e1
fix import
linoytsaban May 15, 2025
db54f9d
fix name
linoytsaban May 15, 2025
0a4e447
fix image encoding
linoytsaban May 15, 2025
940a7a5
fix image encoding
linoytsaban May 15, 2025
e0b6c6c
fix image encoding
linoytsaban May 15, 2025
43711dd
fix image encoding
linoytsaban May 15, 2025
a70f29d
fix image encoding
linoytsaban May 15, 2025
5452431
fix image encoding
linoytsaban May 15, 2025
caa0110
try w/o strength
linoytsaban May 16, 2025
ea879c7
Merge branch 'huggingface:main' into sana
linoytsaban May 21, 2025
74b8681
Merge branch 'huggingface:main' into sana
linoytsaban May 22, 2025
2a52cd5
try scaling differently
linoytsaban May 22, 2025
b3549fb
Merge remote-tracking branch 'origin/sana' into sana
linoytsaban May 22, 2025
b247c5f
try with strength
linoytsaban May 22, 2025
c80f572
revert unnecessary changes to scheduler
linoytsaban May 22, 2025
2173054
revert unnecessary changes to scheduler
linoytsaban May 22, 2025
ac4a132
Apply style fixes
github-actions[bot] May 22, 2025
c47bb07
remove comment
linoytsaban May 22, 2025
3aead2f
add copy statements
linoytsaban May 22, 2025
9ffe0f8
Merge branch 'main' into sana
linoytsaban May 22, 2025
9d5c0b5
Merge remote-tracking branch 'origin/sana' into sana
linoytsaban May 22, 2025
3097441
add copy statements
linoytsaban May 22, 2025
7f2b21b
add to doc
linoytsaban May 22, 2025
c636c76
add to doc
linoytsaban May 22, 2025
f8b4cf9
add to doc
linoytsaban May 22, 2025
fbdaa48
add to doc
linoytsaban May 22, 2025
d330161
Apply style fixes
github-actions[bot] May 22, 2025
cfe0dec
empty commit
linoytsaban May 22, 2025
76e3482
fix copies
linoytsaban May 22, 2025
76a1cf8
fix copies
linoytsaban May 22, 2025
6f40b09
fix copies
linoytsaban May 22, 2025
b0b482a
fix copies
linoytsaban May 22, 2025
7db64a1
fix copies
linoytsaban May 22, 2025
fb99712
docs
sayakpaul May 23, 2025
f6a41db
make fix-copies.
sayakpaul May 23, 2025
042dc61
Merge branch 'main' into sana
sayakpaul May 24, 2025
75d0a77
fix doc building error.
sayakpaul May 24, 2025
0e2c037
initial commit - add img2img test
linoytsaban May 25, 2025
4dad325
initial commit - add img2img test
linoytsaban May 25, 2025
ed818f9
Merge remote-tracking branch 'origin/sana' into sana
linoytsaban May 25, 2025
4eaa7ef
fix import
linoytsaban May 25, 2025
255498b
fix imports
linoytsaban May 26, 2025
99a064e
Merge branch 'main' into sana
linoytsaban May 26, 2025
e85a201
Apply style fixes
github-actions[bot] May 26, 2025
ed560fd
empty commit
linoytsaban May 26, 2025
1bfbf8b
Merge remote-tracking branch 'origin/sana' into sana
linoytsaban May 26, 2025
b717fbd
remove
linoytsaban May 26, 2025
89551d7
Merge branch 'main' into sana
linoytsaban May 26, 2025
1658c41
empty commit
linoytsaban May 26, 2025
26234ee
Merge remote-tracking branch 'origin/sana' into sana
linoytsaban May 26, 2025
de5fad1
test vocab size
linoytsaban May 26, 2025
a9d4197
fix
linoytsaban May 26, 2025
5297450
fix prompt missing from last commits
linoytsaban May 26, 2025
479d9d2
small changes
linoytsaban May 26, 2025
0580379
fix image processing when input is tensor
linoytsaban May 26, 2025
a0803f9
fix order
linoytsaban May 26, 2025
ad68465
Apply style fixes
github-actions[bot] May 26, 2025
e2a4a93
empty commit
linoytsaban May 26, 2025
070a985
fix shape
linoytsaban May 26, 2025
8b00756
remove comment
linoytsaban May 26, 2025
10b7f27
Merge branch 'main' into sana
linoytsaban May 26, 2025
a5664ac
image processing
linoytsaban May 26, 2025
055753f
Merge remote-tracking branch 'origin/sana' into sana
linoytsaban May 26, 2025
ccab5a2
remove comment
linoytsaban May 26, 2025
422c4c8
Merge branch 'main' into sana
linoytsaban May 27, 2025
bda716c
skip vae tiling test for now
linoytsaban May 27, 2025
e30dda9
Merge branch 'main' into sana
linoytsaban May 27, 2025
d571779
Apply style fixes
github-actions[bot] May 27, 2025
ec717aa
empty commit
linoytsaban May 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions docs/source/en/api/pipelines/sana_sprint.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,12 +88,46 @@ image.save("sana.png")

Users can tweak the `max_timesteps` value for experimenting with the visual quality of the generated outputs. The default `max_timesteps` value was obtained with an inference-time search process. For more details about it, check out the paper.

## Image to Image

The [`SanaSprintImg2ImgPipeline`] is a pipeline for image-to-image generation. It takes an input image and a prompt, and generates a new image based on the input image and the prompt.

```py
import torch
from diffusers import SanaSprintImg2ImgPipeline
from diffusers.utils.loading_utils import load_image

image = load_image(
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/penguin.png"
)

pipe = SanaSprintImg2ImgPipeline.from_pretrained(
"Efficient-Large-Model/Sana_Sprint_1.6B_1024px_diffusers",
torch_dtype=torch.bfloat16)
pipe.to("cuda")

image = pipe(
prompt="a cute pink bear",
image=image,
strength=0.5,
height=832,
width=480
).images[0]
image[0].save("output.png")
```

## SanaSprintPipeline

[[autodoc]] SanaSprintPipeline
- all
- __call__

## SanaSprintImg2ImgPipeline

[[autodoc]] SanaSprintImg2ImgPipeline
- all
- __call__


## SanaPipelineOutput

Expand Down
2 changes: 2 additions & 0 deletions src/diffusers/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -441,6 +441,7 @@
"SanaControlNetPipeline",
"SanaPAGPipeline",
"SanaPipeline",
"SanaSprintImg2ImgPipeline",
"SanaSprintPipeline",
"SemanticStableDiffusionPipeline",
"ShapEImg2ImgPipeline",
Expand Down Expand Up @@ -1025,6 +1026,7 @@
SanaControlNetPipeline,
SanaPAGPipeline,
SanaPipeline,
SanaSprintImg2ImgPipeline,
SanaSprintPipeline,
SemanticStableDiffusionPipeline,
ShapEImg2ImgPipeline,
Expand Down
9 changes: 7 additions & 2 deletions src/diffusers/pipelines/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -290,7 +290,12 @@
_import_structure["paint_by_example"] = ["PaintByExamplePipeline"]
_import_structure["pia"] = ["PIAPipeline"]
_import_structure["pixart_alpha"] = ["PixArtAlphaPipeline", "PixArtSigmaPipeline"]
_import_structure["sana"] = ["SanaPipeline", "SanaSprintPipeline", "SanaControlNetPipeline"]
_import_structure["sana"] = [
"SanaPipeline",
"SanaSprintPipeline",
"SanaControlNetPipeline",
"SanaSprintImg2ImgPipeline",
]
_import_structure["semantic_stable_diffusion"] = ["SemanticStableDiffusionPipeline"]
_import_structure["shap_e"] = ["ShapEImg2ImgPipeline", "ShapEPipeline"]
_import_structure["stable_audio"] = [
Expand Down Expand Up @@ -675,7 +680,7 @@
from .paint_by_example import PaintByExamplePipeline
from .pia import PIAPipeline
from .pixart_alpha import PixArtAlphaPipeline, PixArtSigmaPipeline
from .sana import SanaControlNetPipeline, SanaPipeline, SanaSprintPipeline
from .sana import SanaControlNetPipeline, SanaPipeline, SanaSprintImg2ImgPipeline, SanaSprintPipeline
from .semantic_stable_diffusion import SemanticStableDiffusionPipeline
from .shap_e import ShapEImg2ImgPipeline, ShapEPipeline
from .stable_audio import StableAudioPipeline, StableAudioProjectionModel
Expand Down
2 changes: 2 additions & 0 deletions src/diffusers/pipelines/sana/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
_import_structure["pipeline_sana"] = ["SanaPipeline"]
_import_structure["pipeline_sana_controlnet"] = ["SanaControlNetPipeline"]
_import_structure["pipeline_sana_sprint"] = ["SanaSprintPipeline"]
_import_structure["pipeline_sana_sprint_img2img"] = ["SanaSprintImg2ImgPipeline"]

if TYPE_CHECKING or DIFFUSERS_SLOW_IMPORT:
try:
Expand All @@ -37,6 +38,7 @@
from .pipeline_sana import SanaPipeline
from .pipeline_sana_controlnet import SanaControlNetPipeline
from .pipeline_sana_sprint import SanaSprintPipeline
from .pipeline_sana_sprint_img2img import SanaSprintImg2ImgPipeline
else:
import sys

Expand Down
Loading