---
license: gemma
language:
- en
base_model: unsloth/gemma-3-270m-it
datasets:
- kth8/user_prompt_domain_classification-500000x
pipeline_tag: text-generation
library_name: transformers
tags:
- sft
- trl
- unsloth
- gemma
- gemma3
- gemma3_text
---
![logo](https://storage.googleapis.com/gweb-developer-goog-blog-assets/images/gemma-3_2.original.png)
A fine-tune of [unsloth/gemma-3-270m-it](https://huggingface.co/unsloth/gemma-3-270m-it) on the [kth8/user_prompt_domain_classification-500000x](https://huggingface.co/datasets/kth8/user_prompt_domain_classification-500000x) dataset.

## Usage example
**System prompt**
```
You are a text classifier. Categorize the provided text into domain and sub-domain in JSON format.
```
**User prompt**
```
Categorize the domain and sub_domain for:

"In a high-pressure mantle plume environment, what are the thermodynamic constraints on carbon isotope fractionation during subduction-induced carbonate dissolution, and how might these constraints be detected in deep-mantle xenolith isotopic signatures?"
```
**Assistant response**
```
{"domain": "Science", "sub_domain": "Geochemistry"}
```
## Model Details
- Base Model: `unsloth/gemma-3-270m-it`
- Parameter Count: 275,692,160
- Precision: torch.bfloat16

## Hardware
- GPU: NVIDIA GeForce RTX 5090
- Announced: Jan 6th, 2025
- Release Date: Jan 30th, 2025
- Memory Type: GDDR7
- Bandwidth: 1.79 TB/s
- Memory Size: 32 GB
- Memory Bus: 512 bit
- CUDA cores: 21760
- Tensor cores: 680
- TDP: 575W

## Training Settings
### PEFT
- Rank: 32
- LoRA alpha: 64
- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Gradient checkpointing: unsloth

### SFT
- Epoch: 1
- Batch size: 16
- Gradient Accumulation steps: 1
- Learning rate: 0.0002
- Optimizer: adamw_torch_fused
- Learning rate scheduler: cosine

## Training stats
- Date: 2026-04-03T16:34:46.733685
- Peak VRAM usage: 27.949 GB
- Global step: 31097
- Training runtime (seconds): 4570.1661
- Average training loss: 0.09000451330583219
- Final validation loss: 0.06542336940765381

## Framework versions
- Unsloth: 2026.4.1
- TRL: 0.24.0
- Transformers: 5.5.0
- Pytorch: 2.10.0
- Datasets: 4.3.0
- Tokenizers: 0.22.2

## License
This model is released under the Gemma license. See the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and [Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy) regarding the use of Gemma-generated content.