TfiyuenLau/GirlsFrontline2_SDXL_LoRA

基于SDXL模型LoRA微调实现《少前2：追放》文生图

一、Model Library

1. 微调数据集：基于SDXL模型的《少女前线2：追放》LoRA微调数据集
2. 预训练模型：stable_diffusion_xl
3. 底模：animagine-xl-3.0
4. SDXL LoRA微调训练器：kohya_ss
5. 数据集画质增强：waifu2x

二、Prompt Dict

1. 少前2追放角色

• 佩里缇亚: PKPSP
• 塞布丽娜: SPAS12
• 托洛洛: AKAlfa
• 桑朵莱希: G36
• 琼玖: QBZ191
• 维普雷: Vepr12
• 莫辛纳甘: MosinNagant
• 黛烟: QBZ95
• 克罗丽科: Kroliko
• 夏克里: XCRL
• 奇塔: MP7
• 寇尔芙: TaurusCurve
• 科谢尼娅: APS
• 纳甘: Nagant1895
• 纳美西丝: OM50
• 莉塔拉: GalilARM
• 闪电: OTs14

2. Pixiv画师风格

• おにねこ（鬼猫）: Onineko26
• 麻生: AsouAsabu
• mignon: Mignon
• migolu: Migolu

三、使用方式

1. 安装部分环境（默认已安装pytorch等必要环境）

pip install diffusers --upgrade
pip install transformers accelerate safetensors

2. 使用Hugging Face下载并使用底模（animagine-xl-3.0）和LoRA模型

import torch
import datetime
from PIL import Image
import matplotlib.pyplot as plt
from diffusers import (
  StableDiffusionXLPipeline, 
  EulerAncestralDiscreteScheduler,
  AutoencoderKL
)

# LoRA Hugging Face ID
lora_id = "TfiyuenLau/GirlsFrontline2_SDXL_LoRA"

# Load VAE component
vae = AutoencoderKL.from_pretrained(
  "madebyollin/sdxl-vae-fp16-fix", 
  torch_dtype=torch.float16
)

# Configure the pipeline
pipe = StableDiffusionXLPipeline.from_pretrained(
  "cagliostrolab/animagine-xl-3.0", 
  vae=vae,
  torch_dtype=torch.float16, 
  use_safetensors=True, 
)
pipe.load_lora_weights(lora_id)
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
pipe.to('cuda')

3. 生成图像

# Define Prompt
output = "./output.png"
prompt = "1girl, OTs14, gloves, looking at viewer, smile, food, holding, solo, closed mouth, sitting, yellow eyes, black gloves, masterpiece, best quality"
negative_prompt = "nsfw, lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, artist name"

# Generate Image
image = pipe(
  prompt, 
  negative_prompt=negative_prompt, 
  width=1024,
  height=1024,
  guidance_scale=7,
  num_inference_steps=28
).images[0]

# Save & Show
image.save(output)
image = Image.open(output)
plt.axis('off')
plt.imshow(image)
image.close()