Bored MLE

Hi All Today we'll be loading a pre-trained model and train it on Colab. Here's a minimal example using Huggingface for transformers and accelerate for memory efficiency. View source below. Let's break down the above code. Block by block. I'm assuming you're using a notebook for this example. However in the full example in my GitHub I've included the Ops files too (Docker and k8s for GKE or AKS). I don't go into those here, I'll have a series of articles tacking MLOps in depth, this is about fine-tuning a pre-trained small LLM in Colab.  Prep: Setup Environment   !pip install -q transformers accelerate datasets peft bitsandbytes   *     transformers provides pre-trained models and training utilities *      peft enables LoRA (low-rank Adaptation), reducing memory usage *        bitsandbytes allows 8-bit precision to shrink model size *        accelerate helps manage GPU memory and multi-GPU setups ...

  Hi All Another day, another Rust ML backend application. Today we're looking at defining custom modules in Rust in the ML backend. We're going to be creating a custom linear layer in Rust using Tch-rs ( Rust PyTorch bindings ).   View full source below:   Let's break the above code down. Block by block. 1. Imports use tch::{nn, nn::Module, Tensor}; What it does:  *        tch::nn - Neural network module containing layer definitions. *      nn:Module - The trait that all neaural network modules must implement. *      Tensor - The Tensor type used throughout tch-rs   2. Struct Definition   struct CustomLayer { weight: Tensor, bias: Tensor, } What it does: *      weight - A tensor holding a layer's weights (matrix) *      bias -  A tensor holding the layer's bias (vector) This is essentially a linear layer (fully connected layer) that we're definin...

  Hi All Today we're looking at some Rust ML with  Tch-rs . Now Tch-rs is a  Rust bindings library for the C++ api of PyTorch. "The goal of the tch crate is to provide some thin wrappers around the C++ PyTorch api (a.k.a. libtorch)" - github . Let's get to it then. Let's demonstrate tch-rs's core capability: creating tensors, moving them to devices, performing operations and getting results - all in Rust while using PyTorch's backend. See introductory example below.  use tch::{Device, Tensor, Kind}; fn main() { let device = Device::cuda_if_available(); let x = Tensor::of_slice(&[1.0, 2.0, 3.0]).to_device(device); let y = Tensor::of_slice(&[4.0, 5.0, 6.0]).to_device(device); let z = x + y; println!("{:?}", z); } Breakdown: *      Imports : Device , Tensor and Kind   from Tch-rs, the core types for device management and memory operations. *        Device Selection : Device::cuda_if_available(); a...

    I'll have a full example soon on how to wrap unsafe code (malloc/free like in a C++ Cuda kernel), in safe abstractions using Rust. See snippet below: