LoRA-deployment

This repository demonstrates how to serve multiple LoRA fine-tuned Stable Diffusions from 🧨 Diffusers library on Hugging Face Inference Endpoint. Since only few ~ MB of checkpoint is produced after finetuning with LoRA, we can switch different checkpoint for different fine-tuned Stable Diffusion in super quick, memory efficient, and disk space efficient ways.

For demonstration purpose, I have tested the following Hugging Face Model repositories which has LoRA fine-tuned checkpoint(pytorch_lora_weights.bin ):

• ethan_ai
• noto-emoji
• pokemon

Notebook

• Pilot notebook: shows how to write and test a custom handler for Hugging Face Inference Endpoint in local or Colab environments
• Inference notebook: shows how to request inference to the custom handler deployed on Hugging Face Inference Endopint
• Multi-workers inference notebook: shows how to run simultaneous requests to the custom handler deployed on Hugging Face Inference Endpoint in Colab environment

Custom Handler

• handler.py: basic handler. This custom handler is proved to work with this Hugging Face Model repo
• multiworker_handler.py: advanced handler with multiple worker(Stable Diffusion) pool. This custom handler is proved to work with this Hugging Face Model repo

Script

• inference.py: standalone Python script to send requests to the custom handler deployed on Hugging Face Inference Endpoint

Reference

• https://huggingface.co/blog/lora

This site is open source. Improve this page.