This guide demonstrates a practical approach to fine-tuning large language models with minimal data. The author uses a Qwen3.5-9B model and a dataset of just over 1,000 high-quality examples to teach the model to generate structured test cases in JSON format. The process involves two stages: Supervised Fine-Tuning (SFT) to establish the base behavior, followed by KTO (Kahneman-Tversky Optimization) to align the model's outputs with human preferences. The entire workflow runs on a single RTX 4090D with 24GB VRAM, showcasing that advanced fine-tuning is accessible without a large cluster. Key insights include the importance of data quality over quantity, the specific hyperparameters that worked, and the evaluation metrics used to measure success. This is a valuable reference for teams looking to customize models for specific structured output tasks efficiently.
Learn how to fine-tune a 9B parameter model on a single RTX 4090D using just 1,000+ samples with a two-stage SFT + KTO approach to generate structured JSON test cases.