Preview for Style & Motion Transfer using WAN 2.1 FLUX
Style & Motion Transfer using WAN 2.1 FLUX  workflow diagram

Run this workflow on InstaSD

Get started in minutes! Run this ComfyUI workflow online - no setup required.

Description

Unlock fluid style and motion transfer using WAN 2.1 FUN and Flux, with this innovative ComfyUI workflow. Designed for expressive and dynamic video generation, it enables seamless animation-to-animation transformations while preserving nuanced body movement and artistic styling.


🎯 Features

  • Style & Motion Fusion – Combine two videos: one as style, the other as motion input.
  • Supports WAN 2.1 FUN & Flux – Leverages cutting-edge video modeling architecture for optimal consistency.
  • Fully ComfyUI Integrated – All nodes and preprocessing handled directly in ComfyUI, no external tools required.
  • Dynamic Video Composition – Frame-by-frame synthesis with control over fps, resolution, and upscaling.
  • Supports CFGZeroStar & DiT Models – Allows for advanced model patching and diffusion skipping techniques.

πŸ’‘ Use Cases

  • Stylized Animation – Create fluid animated sequences with custom art styles.
  • AI Music Videos – Transfer dance or movement from one video into a unique stylized output.
  • Video-to-Video Transfer – Ideal for merging poses, expressions, or dynamics across different video clips.
  • Motion Testing for Game Design – Rapidly prototype character movements with visual flair.

βš™οΈ How It Works

  1. Prepare Your Inputs

    • Choose a motion source video and a style source video.
    • Ensure both are processed through video_cn_preprocess nodes.
  2. Load Essential Components

    • Load WAN FUN 1.3B, WAN 2.1 VAE, and umt5_xxl_fp8 CLIP models.
    • Include CLIP Vision, SkipLayerGuidanceDiT, and RIFE VFI nodes.
  3. Apply Preprocessing & Alignment

    • Use DWPreprocessor and resolution normalization.
    • Encode visual style via CLIPVisionEncode with img_restyle.
  4. Combine Motion & Style

    • Use WanVideoTeaCacheKJ to handle video-to-video transformation.
    • Utilize ModelSamplingSD3 and CFGZeroStar to optimize generation fidelity.
  5. Postprocessing & Output

    • Interpolate with RIFE VFI for smooth playback.
    • Export via VHS_VideoCombine with frame rate control, compression, and preview.

Credits: cgpixel23

Models

FileDestinationSource
clip_vision_h.safetensors/ComfyUI/models/clip_visionDownload
wan_2.1_vae.safetensors/ComfyUI/models/vaeDownload
umt5_xxl_fp8_e4m3fn_scaled.safetensors/ComfyUI/models/clip_visionDownload
wan fun 1.3B.safetensors/ComfyUI/models/unetDownload
yolox_l.onnx/ComfyUI/custom_nodes/ComfyUI_controlnet_aux/ckpts/yzd-v/DWPoseDownload
dw-ll_ucoco_384_bs5.torchscript.pt/ComfyUI/custom_nodes/ComfyUI_controlnet_aux/ckpts/hr16/DWPose-TorchScript-BatchSize5Download
4x-ClearRealityV1.pth/ComfyUI/models/upscale_modelsDownload
clip_l.safetensors/ComfyUI/models/clipDownload
flux.1-lite-8B-alpha.safetensors/ComfyUI/models/unetDownload
depth_anything_vitl14.pth/ComfyUI/custom_nodes/ComfyUI_controlnet_aux/ckpts/LiheYoung/Depth-Anything/checkpointsDownload
flux.1-lite-8B-alpha-Q5_K_S.gguf/ComfyUI/models/unetDownload
flux1-depth-dev-lora.safetensors/ComfyUI/models/lorasDownload
flux1-canny-dev-lora.safetensors/ComfyUI/models/lorasDownload
turbo flux lora.safetensors/ComfyUI/models/lorasDownload
Studio_Ghibli_Flux.safetensors/ComfyUI/models/lorasDownload
ae.safetensors/ComfyUI/models/vaeDownload
umt5_xxl_fp8_e4m3fn_scaled.safetensors/ComfyUI/models/clipDownload
t5xxl_fp8_e4m3fn.safetensors/ComfyUI/models/clipDownload

Nodes

SetNodeGetNodeVHS_VideoCombineImageListToImageBatchRIFE VFIImageConcatMultiCLIPVisionLoaderVAELoaderUNETLoaderSkipLayerGuidanceDiTWanVideoTeaCacheKJCLIPVisionEncodeDWPreprocessorINTConstantModelSamplingSD3CFGZeroStarVAEDecodeLayerUtility: PurgeVRAMVHS_VideoInfoLoadedImageResizeKJUpscaleModelLoaderImage Comparer (rgthree)PreviewImageeasy cleanGpuUsedImageScaleByImageUpscaleWithModelImageFromBatch+GetImageSize+SimpleMath+ImageScaleStringFunction|pysssssFlorence2ModelLoaderDualCLIPLoaderFlorence2RunCLIPTextEncodeKSamplerDepthAnythingPreprocessorDownloadAndLoadFlorence2ModelRerouteImageBlendSaveImageWanFunControlToVideoFast Groups Bypasser (rgthree)NoteUnetLoaderGGUFPower Lora Loader (rgthree)FluxGuidanceInstructPixToPixConditioningLoadImageCLIPLoaderVHS_LoadVideo