How to Serve Big LLM over Decentralized GPUs? | Parallax + Dynamic Programming
Deep Learning with Yacine · 37:07
Parallax is Gradient's open-source engine for running large LLMs on scattered, mixed consumer GPUs and Macs instead of a data center. It plans in two stages: first it decides which GPUs hold which model layers, then f...