Turkish-Finance-SFT-QLoRA

Model Overview

• Base model: ytu-ce-cosmos/Turkish-Gemma-9b-T1
• Model type: Causal Language Model (9B class) fine-tuned with QLoRA + LoRA

Training Dataset

• Dataset used for supervised fine-tuning (SFT): AlicanKiraz0/Turkish-Finance-SFT-Dataset
• Preprocessing highlights: prompt/chat formatting compatible with Gemma chat format; filtering applied (e.g., minimum user/assistant lengths), validation split used for evaluation.

Intended Use

• Intended for: generating explanatory answers for Turkish finance-related questions and tasks (QA, summarization, reasoning).
• Not intended for: automated financial decision-making without human review.

Training

• Fine-tuning approach: QLoRA quantization (4-bit, NF4) with LoRA adapters applied to the base model.
• LoRA config:
  • rank r=32
  • alpha=64
  • dropout=0.05
• Precision: bf16 during training where supported.
• Optimizer: paged_adamw_32bit
• Scheduler: cosine learning rate schedule with warmup (~6% warmup ratio).
• Gradient accumulation used to achieve effective batch sizes when limited by GPU memory.

Hyperparameters

• Learning rate: 2.0e-4
• Epochs: 3 (default; adjust based on validation performance)
• Per-device train batch size: 4
• Gradient accumulation steps: 4 (effective batch ≈ 16)
• Max sequence length: 4096
• Warmup ratio: 0.06

How to use (Inference works on Colab T4, with reduced output length if needed.)

import os
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

import os, torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

BASE_MODEL = "ytu-ce-cosmos/Turkish-Gemma-9b-T1"
ADAPTER = "Dbmaxwell/Turkish-Finance-SFT-QLoRA" 

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

tok = AutoTokenizer.from_pretrained(BASE_MODEL, token=token, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    token=token,
    trust_remote_code=True,
    quantization_config=bnb,
    device_map="auto",
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True,
)
model = PeftModel.from_pretrained(
    base, ADAPTER,
    device_map="auto",
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True,
)
model.eval()


q = """Bir hisse senedinde uzun süredir devam eden yatay konsolidasyon bölgesi yukarı yönlü hacimli bir kırılımla aşıldıktan sonra, fiyatın tekrar bu banda doğru sarkmasına rağmen volatilitenin belirgin biçimde düşmesi, trendin devamı açısından nasıl yorumlanmalıdır?"""
prompt = f"<bos><start_of_turn>user\n{q}<end_of_turn>\n<start_of_turn>model\n"

inp = tok(prompt, return_tensors="pt").to(next(model.parameters()).device)
with torch.inference_mode():
    out = model.generate(
        **inp,
        max_new_tokens=4096,
        do_sample=False
    )

print(tok.decode(out[0][inp["input_ids"].shape[1]:], skip_special_tokens=True)) #output in files/ex1.md

Limitations & Risks

• Domain bias: training data focused on finance, so outputs may reflect domain-specific biases.
• Hallucinations: the model can produce incorrect or unsupported assertions; do not use outputs as sole source for critical decisions.
• Safety: outputs should be validated by domain experts; include human-in-the-loop for high-stakes use.

Technical Specifications

• Hardware: NVIDIA A100 recommended; CUDA 12.x
• Software: Python 3.10+, transformers, peft, bitsandbytes (4-bit), accelerate, deepspeed