FiftyLove

FiftyLove

3
粉丝
0
关注
326
运行
29
下载人数
1
获赞
23
星标
LoRA训练参数分享

LoRA训练参数分享

该训练参数是我实践而来最“偷懒”的训练参数,而非“最佳”训练参数,在效果尚可的前提下尽可能降低成本,以下是参数使用方式。SD15(模型训练)model_train_type = "sd-lora"pretrained_model_name_or_path = "填写SDXL/Pony大模型地址"vae = "填写模型VAE模型地址(非必要)"v2 = falsetrain_data_dir = "填写训练数据集路径"reg_data_dir = "填写验证集路径(非必要但推荐)"prior_loss_weight = 1resolution = "1024,1024"enable_bucket = truemin_bucket_reso = 256max_bucket_reso = 1024bucket_reso_steps = 64bucket_no_upscale = trueoutput_name = "填写模型名称"output_dir = "./output"save_model_as = "safetensors"save_precision = "fp16"save_every_n_epochs = 1save_state = falsemax_train_epochs = 1train_batch_size = 1gradient_checkpointing = falsenetwork_train_unet_only = falsenetwork_train_text_encoder_only = falselearning_rate = 1unet_lr = 1text_encoder_lr = 1lr_scheduler = "constant"lr_warmup_steps = 0optimizer_type = "Prodigy"network_module = "networks.lora"network_dim = 32network_alpha = 16log_with = "tensorboard"logging_dir = "./logs"caption_extension = ".txt"shuffle_caption = truekeep_tokens = 0max_token_length = 255noise_offset = 0.1seed = 1337clip_skip = 2mixed_precision = "bf16"full_fp16 = falsexformers = truesdpa = truelowram = falsecache_latents = truecache_latents_to_disk = truepersistent_data_loader_workers = trueoptimizer_args = [ "decouple=True", "weight_decay=0.01", "use_bias_correction=True", "d_coef=2.0"]SDXL-Pony(模型训练)model_train_type = "sdxl-lora"pretrained_model_name_or_path = "填写SDXL/Pony大模型地址"vae = "填写模型VAE模型地址(非必要)"train_data_dir = "填写训练数据集路径"reg_data_dir = "填写验证集路径(非必要但推荐)"prior_loss_weight = 1resolution = "1024,1024"enable_bucket = truemin_bucket_reso = 256max_bucket_reso = 1024bucket_reso_steps = 64bucket_no_upscale = trueoutput_name = "填写模型名称"output_dir = "./output"save_model_as = "safetensors"save_precision = "fp16"save_every_n_epochs = 1save_state = falsemax_train_epochs = 1train_batch_size = 1gradient_checkpointing = falsenetwork_train_unet_only = falsenetwork_train_text_encoder_only = falselearning_rate = 1unet_lr = 1text_encoder_lr = 1lr_scheduler = "constant"lr_warmup_steps = 0optimizer_type = "Prodigy"network_module = "networks.lora"network_dim = 32network_alpha = 16log_with = "tensorboard"logging_dir = "./logs"caption_extension = ".txt"shuffle_caption = truekeep_tokens = 0max_token_length = 255noise_offset = 0.1seed = 1337mixed_precision = "bf16"full_fp16 = falsexformers = truesdpa = truelowram = falsecache_latents = truecache_latents_to_disk = truepersistent_data_loader_workers = trueoptimizer_args = [ "decouple=True", "weight_decay=0.01", "use_bias_correction=True", "d_coef=2.0"]使用须知1.使用模版,把任何中文地方进行修改,跑完整一轮的学习率测试。2.点开Tensorboard面板,找到最新的训练日志,记下lr/d*lr/textencoder、lr/d*lr/unet(有时候会没有)最右侧的Value值,这个就是神童优化器计算出的最佳学习率。3.返回LoRA训练的专家模式,将Value值填写进learning_rate、unet_lr,text_encoder_lr改回默认的1e-5,lr_scheduler改回cosine_with_restarts,optimizer_type改回AdamW8bit,max_train_epochs改回10,mixed_precision改为fp16,打开full_fp16。4.开始训练即可。ps:该参数适合训练概念(人物、服装、道具等),如果需要训练画风请将network_dim修改为128、network_alpha修改为64。
1

帖子