Bored MLE

  Hi All Continuing from Part 1, this post details the Kubernetes and Observability configs of the project. The full source is available here . Let's break the code down shall we.  1. K3's Server Config ( infra/server-config.yaml )  write-kubeconfig-mode: "0644"  *      Sets file permissions for kubeconfig file (readable by all users in the group) *      0644 means owner can read/write, group and others can only read disable: - traefik - servicelb - local-storage - metrics-server *      Disables default k3s components that we'll replace with better alternatives. Components Disabled: *      _traefik: Replaced with ingress-nginx for better control *      _servicelb: Replaced with MetalLB or cloud load balancer *      _local-storage: Replaced with Longhon for dynamic provisioning *      _metrics-server: Replaced with Prometheus for better monit...

     Hi All Today we're working with a popular and slightly bigger model than our previous example. Mistral 7B is capable of chat and light coding tasks, for older hardware it's a winner for sure.  Here's a complete, runnable example of fine-tuning Mistral 7B using QLoRA with the peft , transformers , and bitsandbytes libraries. This example assumes you're working with a single GPU (eg. an A100 or similar). First install the required packages: pip install -q bitsandbytes datasets accelerate peft transformers trl View full script below, also available here :   Full breakdown of the script above, block-by-block. 1.      Dataset Loading dataset = load_dataset("timdettmers/openassistant-guanaco", split="train") *      Loads a preprocessed instruction-following dataset (Guanco, derived from OpenAssistant). *      split="train" selects the training portion *      The dataset is in a conversational ...