Qwen3-14B-QLoRA-SoulChat-R1

Qwen3-14B-QLoRA-SoulChat-R1

模型介绍

本模型基于 **Qwen3-14B**，使用 **QLoRA** 方法在 SoulChat-R1 数据集上进行微调，专注于提升模型在心理咨询对话方面的能力。通过 8-bit 量化与 LoRA 低秩适配技术，在保持轻量级的同时显著提升任务表现。

##本模型也用于目前尚未完工的智能心理对话项目LumiChat

微调配置

top.booster: liger kernel

top.finetuning type: lora

top.model name: Qwen3-14B

top.quantization bit: '8'

top.quantization method: bnb

top.rope scaling: none

top.template: qwen3

train.additional target: ''

train.apollo rank: 16

train.apollo scale: 32

train.apollo target: all

train.apollo update interval: 200

train.badam mode: layer

train.badam switch interval: 50

train.badam switch mode: ascending

train.badam update ratio: 0.05

train.batch size: 3

train.compute type: bf16

train.create new adapter: false

train.cutoff len: 4096

train.dataset: SoulChat-R1

train.dataset dir: data

train.ds offload: false

train.ds stage: none

train.enable thinking: true

train.extra args: '{"optim": "adamw torch"}'

train.freeze extra modules: ''

train.freeze language model: false

train.freeze multi modal projector: true

train.freeze trainable layers: 2

train.freeze trainable modules: all

train.freeze vision tower: true

train.galore rank: 16

train.galore scale: 2

train.galore target: all

train.galore update interval: 200

train.gradient accumulation steps: 6

train.learning rate: 5e-5

train.logging steps: 5

train.lora alpha: 16

train.lora dropout: 0

train.lora rank: 8

train.lora target: ''

train.loraplus lr ratio: 0

train.lr scheduler type: cosine

train.mask history: false

train.max grad norm: '1.0'

train.max samples: '100000'

train.neat packing: false

train.neftune alpha: 0

train.num train epochs: '8.0'

train.packing: false

train.ppo score norm: false

train.ppo whiten rewards: false

train.pref beta: 0.1

train.pref ftx: 0

train.pref loss: sigmoid

train.report to: none

train.resize vocab: false

train.reward model: []

train.save steps: 100

train.train on prompt: false

train.training stage: Supervised Fine-Tuning

train.use apollo: false

train.use badam: false

train.use dora: false

train.use galore: false

train.use llama pro: false

train.use pissa: false

train.use rslora: false

train.val size: 0.07

train.warmup steps: 0

训练损失

您可以通过如下git clone命令，或者ModelScope SDK来下载模型

SDK下载

Git下载

如果您是本模型的贡献者，我们邀请您根据模型贡献文档，及时完善模型卡片内容。