Fine-Tuning Mistral 7B using QLoRA with PyTorch pt. 1: The Model | ML Engineering
Hi All Today we're working with a popular and slightly bigger model than our previous example. Mistral 7B is capable of chat and light coding tasks, for older hardware it's a winner for sure. Here's a complete, runnable example of fine-tuning Mistral 7B using QLoRA with the peft , transformers , and bitsandbytes libraries. This example assumes you're working with a single GPU (eg. an A100 or similar). First install the required packages: pip install -q bitsandbytes datasets accelerate peft transformers trl View full script below, also available here : Full breakdown of the script above, block-by-block. 1. Dataset Loading dataset = load_dataset("timdettmers/openassistant-guanaco", split="train") * Loads a preprocessed instruction-following dataset (Guanco, derived from OpenAssistant). * split="train" selects the training portion * The dataset is in a conversational ...