Welcome FLUX.3 Image Klein in Diffusers

Back to Articles

a]:hidden">

Welcome FLUX.3 Image Klein in Diffusers: The latest open image generation model from Black Forect Labs

Published October 15, 2026

Update on GitHub

Upvote 1

Aritra Roy Gosthipaty ariG23498 Follow

Sayak Paul sayakpaul Follow

Linoy Tsaban linoyts Follow

:last-child]:mb-0"> TL;DR

Contents

Getting started

What FLUX.3 can do Text-to-image

Image-to-image

Layout control

Base model and quantized variants Base model

Single-file format checkpoints

LoRA finetuning

What it costs to run Memory

Speed

The architecture From pixels to tokens

From prompt to tokens

Two stages

Inside one block

Every token gets its own timestep

Positions in four dimensions

The denoising loop

Modular Diffusers See the workflows

Encode once, generate many

Add a step inside the loop

Try it

Black Forest Labs FLUX.3 Image Klein is here, and it runs in Diffusers from day one. One model does text-to-image and multi-reference image-to-image, and it adds something new, the layout control. You write bounding boxes into the prompt to say where each part of the image should go, or which parts of a reference image to move.

Both open checkpoints are in the FLUX.3 Image Klein collection:

You can head on to the Hugging Face space and generate some images right now!

TL;DR

One model for text-to-image, multi-reference image-to-image, and bounding-box layout control.

The distilled checkpoint makes a 1024 × 1024 image in about 2 seconds on an H100.

NVFP4 and FP8 weights load with from_single_file, for smaller GPUs.

LoRA training scripts for both text-to-image and image editing.

Ships as a Modular Diffusers pipeline, so you can pop blocks out or add your own.

Contents

Getting started

What FLUX.3 can do

Base model and quantized variants

LoRA finetuning

What it costs to run

The architecture

Modular Diffusers

Try it

Getting started

Install the latest diffusers and transformers (for the Qwen3-VL text encoder):

pip install -U diffusers transformers

Load the distilled checkpoint and generate an image:

import torch from diffusers import ModularPipeline

pipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein") pipe.load_components(dtype=torch.bfloat16) pipe.to("cuda")

pipe(prompt="a red fox", output="images")[0].save("fox.png")

What FLUX.3 can do

Text-to-image

FLUX.3 responds best to a full scene description with the subject, the setting, the light, the composition, and the style. The official prompting guide from Black Forest Labs has more tips. The distilled checkpoint generates a 1024 × 1024 image in 16 steps by default. Pass height, width, or num_inference_steps to change them.

prompt = ( "A wide-angle wildlife photograph of a red fox standing alert in a snow-covered forest clearing at golden hour. " "The fox is centered in the frame, its thick orange winter coat contrasting with the fresh white snow, breath " "visible in the cold air. Snow-laden pine trees fill the softly blurred background, and low warm sunlight rakes " "across the scene from the left, casting long blue shadows." )

fox = pipe( prompt=prompt, generator=torch.Generator().manual_seed(0), output="images" )[0] fox.save("fox.png")

Text to image

Image-to-image

Pass a reference image (or more) with the image argument. Write the prompt as a caption of the image you want, with the change included.

prompt = ( "The same red fox in the same snow-covered forest clearing, now at night under a full moon: cool blue " "moonlight, deep shadows, stars visible through the pine branches, the fox's eyes catching the light." ) night = pipe( prompt=prompt, image=fox, # from the previous text to image example generator=torch.Generator().manual_seed(0), output="images", )[0] night.save("night.png")

Image to image

With several references, refer to them in the prompt as <ref_image_0>, <ref_image_1>, ... in the order they are passed to image:

red_fox = pipe(prompt="a red fox", generator=torch.Generator().manual_seed(0), output="images")[0] bench = pipe(prompt="a wooden bench", generator=torch.Generator().manual_seed(0), output="images")[0]

combined = pipe( prompt="The fox from <ref_image_0> sitting on the wooden bench from <ref_image_1> in a snowy setting.", image=[red_fox, bench], output="images", )[0]

combined.save("combined.png")

<ref_image_0>: a red fox <ref_image_1>: a wooden bench The fox on the bench

Layout control

Add a JSON list of regions after the caption. Each region has an id, a bbox, and a desc, and the caption mentions it as <id>.

Boxes are written as [y_min, x_min, y_max, x_max], y first, on a 0–1000 scale. Because the scale is normalized, the same box works at any image size.

Placing regions in a new image

layout_prompt = ( "Minimalist graphic illustration featuring a black silhouette of a person <silhouette_1> centered against a " "solid, vibrant chartreuse background <background_1>. The figure is captured in a dynamic, mid-stride running " "pose, facing toward the left side of the frame. " '[{"id": "background_1", "bbox": [0, 0, 1000, 1000], "desc": "A flat field of neon yellow-green with a subtle ' 'paper texture and fine, organic grain."}, {"id": "silhouette_1", "bbox": [150, 150, 850, 850], "desc": "A ' "black silhouette of a person in motion, made of dense stippled ink. The leading edges are solid; the trailing " 'arm and leg dissolve into coarse square pixels and scattered dots."}]' )

runner = pipe(prompt=layout_prompt, generator=torch.Generator().manual_seed(0), output="images")[0]

Each region's desc is drawn inside its bbox

Moving regions within a reference image

For edits, each region says where its content comes from (src_bbox in the reference) and where it goes (tgt_bbox in the output). A region with the same two boxes stays where it is.

This example moves the knitted knight up and to the left, and pins the backdrop, the cliff, the needle, the dragon, and the fire:

from diffusers.utils import load_image

knight_image = load_image("knight_orig.png") edit_prompt = ( "In <ref_image_0>, move the miniature grey amigurumi knight figure <knight_1> upwards and to the left along the " "yarn cliff <cliff_1>. Keep the rest of the image completely unchanged, preserving the blurred yarn backdrop " "<backdrop_1>, the large knitted dragon <dragon_1>, and the fiery orange thread <fire_1>. " '[{"id": "knight_1", "from": "ref_image_0", "src_bbox": [500, 150, 850, 350], "tgt_bbox": [194, 55, 544, 255], ' '"desc": "A miniature amigurumi knight crocheted from thick grey woolen yarn."}, ' '{"id": "backdrop_1", "from": "ref_image_0", "src_bbox": [0, 0, 1000, 1000], "tgt_bbox": [0, 0, 1000, 1000], ' '"desc": "A soft, out-of-focus backdrop of cream and indigo knitted yarn."}, ' '{"id": "cliff_1", "from": "ref_image_0", "src_bbox": [300, 0, 1000, 450], "tgt_bbox": [300, 0, 1000, 450], ' '"desc": "A steep cliff of stacked yarn skeins in mustard, terracotta, and indigo."}, ' '{"id": "needle_1", "from": "ref_image_0", "src_bbox": [420, 250, 550, 450], "tgt_bbox": [420, 250, 550, 450], ' '"desc": "An oversized polished steel tapestry needle."}, ' '{"id": "dragon_1", "from": "ref_image_0", "src_bbox": [100, 600, 950, 1000], "tgt_bbox": [100, 600, 950, 1000], ' '"desc": "A large, fluffy knitted dragon in terracotta and mustard yarn."}, ' '{"id": "fire_1", "from": "ref_image_0", "src_bbox": [350, 400, 650, 650], "tgt_bbox": [350, 400, 650, 650], ' '"desc": "Curled puffs of fiery orange angora thread suspended in mid-air."}]' )

moved = pipe(prompt=edit_prompt, image=knight_image, generator=torch.Generator().manual_seed(0), output="images")[0]

The knight moves from its src_bbox to its tgt_bbox, and the rest of the scene stays put

Base model and quantized variants

Base model

Everything above uses the distilled checkpoint. The base checkpoint runs 50 steps with classifier-free guidance, and it's the one to fine-tune (see LoRA finetuning).

pipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein-base") pipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16}) pipe.to("cuda")

pipe.guider = pipe.guider.new(guidance_scale=5.0) # optional image = pipe(prompt=prompt, output="images")[0]

Single-file format checkpoints

Black Forest Labs also ships the transformer as a single file, in bf16 and in two quantized formats: fp8r (rowwise FP8) and nvfp4. Load any of them with from_single_file, then load the other components by name so the bf16 transformer isn't downloaded too.

from diffusers import BFLQuantizationConfig, Flux3Transformer2DModel, ModularPipeline

model_id = "black-forest-labs/flux-3-image-klein-base" variant = "fp8r" # or "nvfp4"

transformer = Flux3Transformer2DModel.from_single_file( f"https://huggingface.co/{model_id}/blob/main/variants/{variant}/model.safetensors", config=model_id, subfolder="transformer", quantization_config=BFLQuantizationConfig(compute_dtype=torch.bfloat16), dtype=torch.bfloat16, )

pipe = ModularPipeline.from_pretrained(model_id) pipe.load_components(dtype=torch.bfloat16) pipe.update_components(transformer=transformer) pipe.to("cuda")

For the unquantized single file, use https://huggingface.co/{model_id}/blob/main/model.safetensors and drop quantization_config.

LoRA finetuning

FLUX.3 works for both text-to-image and image-to-image, which makes it a great base for custom LoRAs. diffusers ships DreamBooth LoRA training scripts for both tasks (you can also train with Ostris' AI Toolkit).

Here is a text-to-image example that learns a tarot-card style from the multimodalart/1920-raider-waite-tarot-public-domain dataset, triggered with trtcrd:

accelerate launch train_dreambooth_lora_flux3_klein.py
--pretrained_model_name_or_path="black-forest-labs/flux-3-image-klein-base"
--dataset_name="multimodalart/1920-raider-waite-tarot-public-domain"
--caption_column="caption"
--instance_prompt="trtcrd"
--output_dir="trained-flux3-tarot-lora"
--mixed_precision="bf16"
--resolution=1024
--use_aspect_ratio_buckets
--train_batch_size=1
--rank=16
--lora_alpha=16
--learning_rate=1e-4
--lr_scheduler="constant"
--max_train_steps=1500
--validation_prompt="a trtcrd of a cat, tarot style"
--validation_epochs=25
--seed=0
--push_to_hub

To reduce memory usage, add a few optimization flags:

--gradient_checkpointing
--cache_latents
--use_8bit_adam

--gradient_checkpointing recomputes activations in the backward pass (lower VRAM, slightly slower), --cache_latents pre-encodes the images and frees the VAE, and --use_8bit_adam keeps the optimizer states in 8-bit.

A couple of knobs are worth experimenting with. The grid below trains the same tarot style with different LoRA ranks and optimizers; the adapters are on the Hub at diffusers-modular/flux3-tarot-lora. More training features and configurations are documented in the DreamBooth FLUX.3 README.

For instruction-based editing, the image-to-image script (train_dreambooth_lora_flux3_klein_img2img.py) trains on (reference image, target image, instruction) triplets, where the reference is addressed in the prompt as <ref_image_0>.

See the DreamBooth FLUX.3 README for all options and the full image-to-image example.

What it costs to run

In this section, we will talk about the memory consumption and the speed of generation of the models.

Memory

name peak allocated (after load) GiBs peak allocated (after inference) GiBs

nvfp4 (RTX6000) 13.4 15.3

fp8r (H100) 14.35 16.27

bf16 (H100) 18.80 20.72

If you have less memory:

from diffusers import ComponentsManager, ModularPipeline

manager = ComponentsManager() manager.enable_auto_cpu_offload(device="cuda")

pipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein