Fibo 2.0

Fibo 2.0 is an 8B unified image generation and editing model developed by Bria AI. It runs in the latent space of the FLUX.2 VAE, encodes prompts and reference images with Qwen3-VL-4B, and uses a single-stream DiT with 44 blocks. Text-to-image and image editing share one set of weights.

Fibo 2.0 Dev is the full, non-distilled base model, trained with classifier-free guidance and intended for fine-tuning. Fibo 2.0 Edit Turbo is a distilled editing-only variant for 4-step, guidance-free inference.

Installation

Before performing model inference and training, please install DiffSynth-Studio first.

git clone https://github.com/modelscope/DiffSynth-Studio.git
cd DiffSynth-Studio
pip install -e .

For more information on installation, please refer to Setup Dependencies.

Quick Start

Run the following code to quickly load the briaai/Fibo-2.0-dev model and perform text-to-image inference. VRAM management is enabled, and the framework will automatically control model parameter loading based on remaining VRAM.

import json
import torch
from diffsynth.pipelines.fibo import FiboPipeline, ModelConfig

vram_config = {
    "offload_dtype": "disk",
    "offload_device": "disk",
    "onload_dtype": torch.float8_e4m3fn,
    "onload_device": "cpu",
    "preparing_dtype": torch.float8_e4m3fn,
    "preparing_device": "cuda",
    "computation_dtype": torch.bfloat16,
    "computation_device": "cuda",
}
pipe = FiboPipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(model_id="briaai/Fibo-2.0-dev", origin_file_pattern="transformer/*.safetensors", **vram_config),
        ModelConfig(model_id="briaai/Fibo-2.0-dev", origin_file_pattern="text_encoder/model.safetensors", **vram_config),
        ModelConfig(model_id="briaai/Fibo-2.0-dev", origin_file_pattern="vae/diffusion_pytorch_model.safetensors", **vram_config),
    ],
    processor_config=ModelConfig(model_id="briaai/Fibo-2.0-dev", origin_file_pattern="processor/"),
    font_config=ModelConfig(model_id="briaai/Fibo-2.0-dev", origin_file_pattern="fonts/"),
    vram_limit=torch.cuda.mem_get_info("cuda")[1] / (1024 ** 3) - 0.5,
)
caption = {
    "short_description": "An origami whale swimming through paper clouds.",
    "objects": [{"description": "a blue origami whale, folded from glossy paper", "location": "center"}],
    "background_setting": "a bright sky made of layered white paper clouds",
    "style_medium": "paper craft photograph",
}
prompt = json.dumps(caption, separators=(",", ":"), ensure_ascii=False)
image = pipe(prompt=prompt, height=1024, width=1024, seed=42, num_inference_steps=50, cfg_scale=5.0)
image.save("image.jpg")

Model Overview

Model ID	Inference	Low VRAM Inference	Full Training	Full Training Validation	LoRA Training	LoRA Training Validation
briaai/Fibo-2.0-dev	code	code	code	code	code	code
briaai/Fibo-2.0-edit-turbo	code	code	-	-	-	-

The training columns above link to the text-to-image scripts. Fibo 2.0 Dev also supports image-editing training, see Model Training.

Model Inference

The model is loaded via FiboPipeline.from_pretrained, see Loading Models for details.

from_pretrained also accepts:

processor_config: ModelConfig of the Qwen3-VL processor.
font_config: ModelConfig of the fonts used to draw text_render images. It can point to a font directory, a font file, or a list of font files.

The input parameters for FiboPipeline inference include:

prompt: A structured JSON caption, a plain description, or an edit instruction. Edit prompts may also be a JSON caption of the result with an edit_instruction field.
negative_prompt: Used when cfg_scale > 1. Defaults to the null caption Fibo 2.0 was trained with.
edit_image: The image or list of reference images to edit. Leave empty for text-to-image.
mask: For single-image editing, white marks the region to regenerate.
text_images: Images of the text in a caption’s text_render entries. When omitted, they are drawn from the caption.
fonts: Font paths used to draw characters that Pillow’s built-in font lacks.
render_text_images: Whether to draw text images from the caption, default is True.
height, width: Output size, multiples of 16. When editing and left empty, the trained size for the first image’s aspect ratio is used.
cfg_scale: Classifier-free guidance scale, default is 5.0.
num_inference_steps: Number of inference steps, default is 30.
denoising_strength: Denoising strength, range is 0~1, default is 1. Lower values start from a lower noise level.
seed, rand_device: Random seed and the device used to generate noise.
progress_bar_cmd: Progress bar, default is tqdm.tqdm. Set to lambda x:x to disable it.

If VRAM is insufficient, please enable VRAM Management. We provide recommended low VRAM configurations for each model in the example code, see the table in the “Model Overview” section above.

Image Editing

edit_image accepts one image, or a list of up to 5 images for multi-reference editing. The example below uses the distilled Fibo-2.0-edit-turbo model:

from PIL import Image
from diffsynth.pipelines.fibo import FiboPipeline, ModelConfig

pipe = FiboPipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(model_id="briaai/Fibo-2.0-edit-turbo", origin_file_pattern="transformer/*.safetensors"),
        ModelConfig(model_id="briaai/Fibo-2.0-edit-turbo", origin_file_pattern="text_encoder/model.safetensors"),
        ModelConfig(model_id="briaai/Fibo-2.0-edit-turbo", origin_file_pattern="vae/diffusion_pytorch_model.safetensors"),
    ],
    processor_config=ModelConfig(model_id="briaai/Fibo-2.0-edit-turbo", origin_file_pattern="processor/"),
    font_config=ModelConfig(model_id="briaai/Fibo-2.0-edit-turbo", origin_file_pattern="fonts/"),
)
source = Image.open("photo.jpg").convert("RGB")
image = pipe(prompt="replace the jacket with a red leather one", edit_image=source, seed=42, num_inference_steps=4, cfg_scale=1.0)
image.save("edited.jpg")

Fibo-2.0-dev uses 50 steps with cfg_scale=5.0; Fibo-2.0-edit-turbo uses 4 steps with cfg_scale=1.0.

Masked Editing

For single-image editing, pass mask together with edit_image (white marks the region to regenerate):

mask = Image.open("mask.png").convert("L")  # white: regenerate, black: keep
image = pipe(
    prompt="replace the suitcase with a vintage brown leather one",
    edit_image=source, mask=mask, seed=42, num_inference_steps=4, cfg_scale=1.0,
)

Text Rendering

Fibo renders text from a caption’s text_render entries. Each entry’s text is drawn as a small black-on-white image and shown to Qwen3-VL together with the prompt:

caption = {
    "short_description": "A young woman holding a white sign that reads \"ModelScope\".",
    "objects": [{"description": "a white sign with bold black text", "location": "center"}],
    "style_medium": "photograph",
    "text_render": [{"text": "ModelScope"}],
}
image = pipe(
    prompt=json.dumps(caption, separators=(",", ":"), ensure_ascii=False),
    height=1024, width=1024, seed=42, num_inference_steps=50, cfg_scale=5.0,
)

font_config loads the fonts shipped with the model. To render characters that those fonts lack, pass additional font paths through fonts:

image = pipe(prompt=prompt, fonts=["/path/to/a/cjk-font.ttf"], ...)

Characters not covered by any font are drawn with Pillow’s built-in font. Text rendering accuracy is model-dependent, especially for non-Latin scripts.

Model Training

Fibo 2.0 Dev is trained via examples/fibo/model_training/train.py. The script parameters include:

General Training Parameters
- Dataset Basic Configuration
  - --dataset_base_path: Root directory of the dataset.
  - --dataset_metadata_path: Metadata file path of the dataset.
  - --dataset_repeat: Number of times the dataset is repeated in each epoch.
  - --dataset_num_workers: Number of processes for each DataLoader.
  - --data_file_keys: Field names to be loaded from metadata, usually image or video file paths, separated by ,.
- Model Loading Configuration
  - --model_paths: Paths of models to be loaded. JSON format.
  - --model_id_with_origin_paths: Model IDs with original paths, e.g., "briaai/Fibo-2.0-dev:transformer/*.safetensors". Separated by commas.
  - --extra_inputs: Extra input parameters required by the model Pipeline, e.g., edit_image and mask when training the image editing model, separated by ,.
  - --fp8_models: Models loaded in FP8 format, consistent with --model_paths or --model_id_with_origin_paths format. Currently only supports models whose parameters are not updated by gradients (no gradient backpropagation, or gradients only update their LoRA).
  - --quant_options: Dynamically quantize loaded models. Semicolon-separated entries, each <model_string>:<method>[/<exclude_modules>], where <model_string> matches an entry in --model_paths/--model_id_with_origin_paths, method is a registered method (e.g. bitsandbytes_nf4), and exclude_modules optionally lists layers kept in full precision.
- Training Basic Configuration
  - --learning_rate: Learning rate.
  - --num_epochs: Number of epochs.
  - --trainable_models: Trainable models, e.g., dit, vae, text_encoder.
  - --find_unused_parameters: Whether there are unused parameters in DDP training. Some models contain redundant parameters that do not participate in gradient calculation, and this setting needs to be enabled to avoid errors in multi-GPU training.
  - --weight_decay: Weight decay size, see torch.optim.AdamW.
  - --task: Training task, default is sft. Some models support more training modes, please refer to the documentation of each specific model.
- Output Configuration
  - --output_path: Model saving path.
  - --remove_prefix_in_ckpt: Remove prefix in the state dict of the model file.
  - --save_steps: Interval of training steps to save the model. If this parameter is left blank, the model is saved once per epoch.
- LoRA Configuration
  - --lora_base_model: Which model to add LoRA to.
  - --lora_target_modules: Which layers to add LoRA to.
  - --lora_rank: Rank of LoRA.
  - --lora_checkpoint: Path of the LoRA checkpoint. If this path is provided, LoRA will be loaded from this checkpoint.
  - --preset_lora_path: Preset LoRA checkpoint path. If this path is provided, this LoRA will be loaded in the form of being merged into the base model. This parameter is used for LoRA differential training.
  - --preset_lora_model: Model that the preset LoRA is merged into, e.g., dit.
- Gradient Configuration
  - --use_gradient_checkpointing: Whether to enable gradient checkpointing.
  - --use_gradient_checkpointing_offload: Whether to offload gradient checkpointing to memory.
  - --gradient_accumulation_steps: Number of gradient accumulation steps.
- Image Width/Height Configuration (Applicable to Image Generation and Video Generation Models)
  - --height: Height of image or video. Leave height and width blank to enable dynamic resolution.
  - --width: Width of image or video. Leave height and width blank to enable dynamic resolution.
  - --max_pixels: Maximum pixel area of image or video frames. When dynamic resolution is enabled, images with resolution larger than this value will be downscaled, and images with resolution smaller than this value will remain unchanged.
Fibo 2.0 Specific Parameters
- --processor_path: Path or model_id:origin_file_pattern of the Qwen3-VL processor, leave blank to automatically download from remote.
- --font_path: Path or model_id:origin_file_pattern of the fonts used for text rendering, leave blank to automatically download from remote.
- --initialize_model_on_cpu: Whether to initialize the model on CPU.

We have built sample datasets, including a text-to-image dataset and an image-editing dataset. You can download them with the following command:

modelscope download --dataset DiffSynth-Studio/diffsynth_example_dataset --include "fibo/*" --local_dir ./data/diffsynth_example_dataset

data/diffsynth_example_dataset/fibo/Fibo-2.0-dev: text-to-image, with metadata.csv containing image and prompt.
data/diffsynth_example_dataset/fibo/Fibo-2.0-dev-Edit: image editing, with metadata.json containing image, prompt, edit_image, and an optional mask.

Fibo 2.0 Dev supports both text-to-image and image-editing training. Training reuses the inference Pipeline: the VAE and text encoder are frozen, and the DiT is trained with flow matching. Full training uses --trainable_models "dit", while LoRA training uses --lora_base_model "dit". For image editing, edit_image and mask are loaded with UnifiedDataset and injected through --extra_inputs.

Mode	Full Training	LoRA Training	Full Validation	LoRA Validation
Text-to-image	script	script	script	script
Image editing	script	script	script	script

The full training scripts use --config_file examples/fibo/model_training/full/accelerate_config.yaml; the LoRA scripts run on a single GPU by default. mask is optional per sample, and rows without mask are trained as ordinary editing samples. Fibo 2.0 Edit Turbo is not intended for fine-tuning; train on Fibo 2.0 Dev instead.

We have written recommended training scripts for each model, please refer to the tables above. For how to write model training scripts, please refer to Model Training; for more advanced training algorithms, please refer to Training Framework Detailed Explanation.

License

Fibo 2.0 is released under CC BY-NC 4.0 for non-commercial use. Commercial use requires a separate agreement with Bria AI.