Resource Management (4) - Infrastructure & Tooling - Full Stack Deep Learning

New course announcement ✨

We're teaching an in-person LLM bootcamp in the SF Bay Area on November 14, 2023. Come join us if you want to see the most up-to-date materials building LLM-powered products and learn in a hands-on environment.

https://www.scale.bythebay.io/llm-workshop

Hope to see some of you there!

--------------------------------------------------------------------------------------------- How to effectively manage compute resources?
More videos at https://course.fullstackdeeplearning.com

Summary
- Running complex deep learning models poses a very practical resource management problem: how to give every team the tools they need to train their models without requiring them to operate their own infrastructure?
- The most primitive approach is to use spreadsheets that allow people to reserve what resources they need to use.
- The next approach is to utilize a SLURM Workload Manager, a free and open-source job scheduler for Linux and Unix-like kernels.
- A very standard approach these days is to use Docker alongside Kubernetes.
- Docker is a way to package up an entire dependency stack in a lighter-than-a-Virtual-Machine package.
- Kubernetes is a way to run many Docker containers on top of a cluster.
- The last option is to use open-source projects.
- Using Kubeflow allows you to run model training jobs at scale on containers with the same scalability of container orchestration that comes with Kubernetes.
- Polyaxon is a self-service multi-user system, taking care of scheduling and managing jobs in order to make the best use of available cluster resources.