BirdieByte1024/Doctor_Behavior_LoRA-Mistral-7B-Instruct-v0.3

🦷 LoRA Fine-Tuned Model for Dental AI Assistant (Stage 1 - Tone Fine-Tuning)

This is a LoRA adapter trained on conversational medical data to teach a base language model (Mistral-7B-Instruct) how to behave like a helpful and professional medical assistant, focusing on polite tone, empathetic responses, and clarity.

This model is intended for Stage 1 tone/behavior fine-tuning in a two-phase pipeline toward building an AI dental doctor.

---

📋 Model Details

• Developed by: BirdieByte
• Finetuned from: unsloth/mistral-7b-instruct-v0.3-bnb-4bit
• LoRA Framework: PEFT
• Training Library: Unsloth
• Language(s): English
• Model Type: LoRA adapter for causal language modeling
• License: MIT

---

💡 Intended Use

Direct Use

This model adapter is used to:

• Enhance tone and medical empathy in LLMs
• Simulate a virtual assistant that responds like a professional doctor
• Prepare a model for further domain-specific fine-tuning (e.g. dental implants)

Not for Use

• Not intended for medical decision-making or diagnosis.
• Not safe to use in real-world healthcare settings without human supervision.

---

🧠 Training Details

• Training Data: Reformatted and filtered samples from antareepdey/Patient_doctor_chat
• Data Format: ChatML-style (ChatGPT-like role-based prompting)
• Training Type: LoRA adapter tuning only (base model not updated)
• Precision: 4-bit base model (bnb), LoRA adapter in FP16
• Steps: 3 epochs on ~480 examples
• Output: LoRA adapter weights and tokenizer

---

🧪 Evaluation

Qualitative testing showed:

• Improved clarity
• Friendly tone and structure
• Useful for staged evaluation against baseline Mistral model

Quantitative metrics not reported at this stage.

---

🚀 How to Use

Load with PEFT

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
model = PeftModel.from_pretrained(base, "BirdieByte1024/Mistral-doctor-behavior")
tokenizer = AutoTokenizer.from_pretrained("BirdieByte1024/Mistral-doctor-behavior")

---

🌱 Environmental Impact

• Hardware: Google Colab (T4 GPU)
• Training time: ~30 minutes
• Energy usage: Estimated minimal for LoRA
• CO2 emissions: < 0.01 kg CO2eq (estimated)

---

📚 Citation

BirdieByte (2025). LoRA Fine-Tuned Dental Tone Assistant (Stage 1). Hugging Face model repository.

---

✉️ Contact

For questions or collaborations, please contact via Hugging Face (@your-username).

• PEFT 0.14.0