Preview for SDXL ControlNet Union
SDXL ControlNet Union workflow diagram

Run this workflow on InstaSD

Get started in minutes! Run this ComfyUI workflow online - no setup required.

Description

Create stunning AI-generated images with ControlNet for Stable Diffusion, utilizing stacked ControlNet models such as Depth and Canny. This workflow ensures that generated images follow the structure and details of a reference image, perfect for concept artists, illustrators, and AI enthusiasts.


🎯 Features

  • Multi-ControlNet Stacking – Uses Depth and Canny ControlNet for enhanced structure-based image generation.
  • High-Fidelity Image Transformation – Maintains original composition while allowing creative modifications.
  • Customizable Prompts – Fine-tune prompts for precise artistic direction.
  • Stable Diffusion SDXL Support – Works with SDXL models for high-resolution outputs.

💡 Use Cases

  • Concept Art Creation – Generate fantasy landscapes or characters based on sketches.
  • Photo-to-Art Transformations – Convert real images into stylized digital paintings.
  • AI-Assisted Illustration – Enhance outlines with AI-generated details.
  • Product Mockups – Generate detailed product designs from simple sketches.

⚙️ How It Works

  1. Load the Checkpoint Model – Select a Stable Diffusion model like Juggernaut_X_RunDiffusion.
  2. Prepare Your Image – Upload a reference image to guide the generation process.
  3. Apply Preprocessors – Use CannyEdgePreprocessor and DepthAnythingPreprocessor for detailed image structure analysis.
  4. Stack ControlNets – Utilize Multi-ControlNet Stack to enhance structure adherence.
  5. Refine with Conditioning – Use positive and negative text prompts to guide the image’s style.
  6. Run the Sampler – Generate the final image with the DPM++ 2M sampler for high-quality output.
  7. Preview and Save – Review and export your AI-generated masterpiece.

Credits: pixaroma

Models

FileDestinationSource
Juggernaut_X_RunDiffusion.safetensors/ComfyUI/models/checkpointsDownload
diffusion_pytorch_model_promax.safetensors/ComfyUI/models/controlnetDownload

Nodes

CheckpointLoaderSimpleCLIPTextEncodeVAEDecodeSaveImagePreviewImageCR Apply Multi-ControlNetAIO_PreprocessorCR Multi-ControlNet StackLoadImageVAEEncodeKSampler