comfyui-video — Генерация видео с использованием искусственного интеллекта

--- name: comfyui-video description: Automate AI video generation with ComfyUI and LTX-2.3. Supports text-to-video (T2V), image-to-video (I2V), batch scene rendering for music videos, and multi-scene workflows. Includes progress monitoring, fault recovery, and performance tuning. Use when generating AI videos with ComfyUI, creating MV scenes in batch, troubleshooting video rendering, or optimizing generation speed. --- # ComfyUI Video Generation Automate AI video generation using ComfyUI + LTX-2.3 model. Ideal for music video (MV) production, multi-scene batch rendering, and AI video content creation. ## Requirements | Item | Spec | |------|------| | GPU | ≥24GB VRAM (Turing/Ampere/Ada) | | ComfyUI | 0.17+ | | PyTorch | 2.6+cu124 | | Access | SSH tunnel forwarding port 18188 | ## Model Setup | Model | Size | Path | |-------|------|------| | LTX-2.3 dev (bf16) | 43GB | `models/checkpoints/ltx-2.3-22b-dev.safetensors` | | Gemma 3 12B | 23GB | `models/text_encoders/comfy_gemma_3_12B_it.safetensors` | | Distilled LoRA | 7.1GB | `models/loras/ltxv/ltx2/ltx-2.3-22b-distilled-lora-384.safetensors` | | Video VAE (bf16) | - | `models/vae/LTX23_video_vae_bf16.safetensors` | **Turing GPUs** (e.g., Quadro RTX 8000) do NOT support `fp8_e4m3fn`. Use bf16/fp16 models only. ## Performance Baseline ``` Per-step time: ~221s (constant, regardless of frame count!) 15 steps: ~57 min 25 steps: ~1h45m Frames: 72=3s, 121=5s, 480=20s (24fps) ``` **Key insight**: Frame count does NOT affect total time. Bottleneck is model forward pass. ## Workflow Node Reference | Node | ID | Purpose | |------|-----|---------| | LoadImage | 2004 | I2V reference input | | CLIPTextEncode (positive) | 2483 | Positive prompt | | CLIPTextEncode (negative) | 2612 | Negative prompt | | EmptyLTXVLatentVideo | 3059 | Empty latent | | LTXVScheduler | 4966 | Steps/length params | | LoraLoaderModelOnly | 4922+ | LoRA loader | | SaveVideo | 4823/4852 | Output mp4 | ## Quick Start ### Generate a Single Video (I2V) 1. Load workflow: `/workspace/ComfyUI/custom_nodes/ComfyUI-LTXVideo/example_workflows/2.3/LTX-2.3_T2V_I2V_Single_Stage_Distilled_Full.json` 2. Set params using `scripts/batch_scenes.js` 3. Click Run 4. Wait ~1 hour 5. Download from `/workspace/ComfyUI/output/` ### Batch Scene Generation Use `scripts/batch_scenes.js` for automation: ```javascript // Load script first, then configure each scene: await comfyui_batch.configureScene({ name: "scene_01", prompt: "A lonely girl running through rain at night, neon reflections", image: "unified_ref.png", steps: 15, frames: 72 }); // Click Run, repeat for next scene ``` ## Step Count Guide | Steps | Quality | Time/Scene | Use Case | |-------|---------|------------|----------| | 8 | Rough | ~30min | Quick preview | | 15 | Good | ~57min | **Recommended sweet spot** | | 25 | Best | ~1h45m | Final quality output | I2V + LoRA at 15 steps achieves ~90% of 25-step quality with 40% less time. ## Troubleshooting ### VAEDecode Validation Failed **Error**: `Exception when validating node: 'VAEDecode'` **Cause**: VAE load timing or insufficient VRAM **Fix**: Reload the entire workflow (fetch + loadGraphData), wait for models to fully load, then run. Never reload during execution. ### Browser Tab Lost **Cause**: SSH tunnel disconnected **Fix**: 1. Rebuild tunnel: `ssh -f -N -L 18188:localhost:18188 user@host -p port` 2. Navigate to ComfyUI 3. Reload workflow ### Inconsistent Characters Across Scenes **Cause**: Different reference images per scene **Fix**: Use the SAME reference image for all scenes. Extract a clear frame from an existing video if needed. The I2V input image dictates the visual baseline. ### Output Video Not Saved **Check**: `ssh -p PORT root@HOST "ls -lht /workspace/ComfyUI/output/*.mp4"` **Fix**: Check for VAEDecode errors in log, then re-run. ## Monitoring Progress ```bash # Current sampling progress ssh -p PORT root@HOST "grep 'it/s' /tmp/comfy.log | tail -1" # Completion check ssh -p PORT root@HOST "grep 'Prompt executed' /tmp/comfy.log | tail -1" # Output files ssh -p PORT root@HOST "ls -lht /workspace/ComfyUI/output/*.mp4" ``` ## Best Practices 1. **15 steps is the sweet spot** — I2V converges at 15-20 steps, 25 has diminishing returns 2. **Unified reference image** — Same input image for all scenes ensures character consistency 3. **Reload workflow every time** — Avoids VAEDecode validation failures 4. **Never reload during execution** — Current run will fail 5. **Frame selection** — 72 frames (3s) for testing, 480 frames (20s) for final output 6. **VRAM management** — Wait for each generation to complete before starting next ## T2V vs I2V Comparison | Mode | Steps | Quality | Notes | |------|-------|---------|-------| | T2V (no LoRA) | 15 | ❌ Very blurry | Not recommended | | I2V + LoRA | 25 | ✅ Excellent | Major quality improvement | | I2V + LoRA | 15 | ✅ Very good | Best time/quality ratio | **Conclusion**: I2V + LoRA is the recommended combination. ## Resources - `scripts/batch_scenes.js` — Batch scene automation - `references/workflow_nodes.md` — Full node ID mapping - `references/tips.md` — Prompt tips, VRAM management, optimization