huggingface
added training script for instruct-pix2pix-sdxl
What does this PR do? This PR adds comprehensive documentation for training InstructPix2Pix-SDXL with LoRA adapters on diffusers/sdxl-instructpix2pix-768. The new README_lora_sdxl.md provides:
Key Additions: Complete training guide for LoRA-based InstructPix2Pix SDXL fine-tuning Quick start examples with both HuggingFace datasets and custom local datasets LoRA configuration recommendations for different use cases (fast training, balanced, high quality) Advanced training options including: Multi-GPU distributed training setup Checkpoint resumption Custom VAE integration for improved stability Comprehensive inference examples with detailed parameter explanations Troubleshooting section addressing common issues (OOM errors, NaN loss, quality issues) LoRA vs full fine-tuning comparison to help users choose the right approach Documentation Improvements: Clear argument reference organized by category (Model/Data, Training, LoRA, etc.) Practical tips for memory optimization Production-ready inference code snippets Citations for InstructPix2Pix and SDXL papers This documentation complements the existing examples/instruct_pix2pix/train_instruct_pix2pix_lora_sdxl.py script and makes it significantly easier for users to:
Get started with LoRA training quickly Understand and configure LoRA hyperparameters Troubleshoot common issues Deploy trained models for inference
Who can review? @sywangyi - Training examples and documentation
@sayakpaul - Documentation review
Hi, thanks for wanting to contribute this. But I am not if we need these examples right now.
Hi, thanks for wanting to contribute this. But I am not if we need these examples right now.
Hello dear, thanks for replying. Why? What's the problem?