Skip to content
View usamahz's full-sized avatar
13 followers · 3 following

Sponsoring

@openclaw

Achievements

Achievement: StarstruckAchievement: Pair Extraordinairex3Achievement: Pull Sharkx4Achievement: Public SponsorAchievement: QuickdrawAchievement: YOLO

Achievements

Achievement: StarstruckAchievement: Pair Extraordinairex3Achievement: Pull Sharkx4Achievement: Public SponsorAchievement: QuickdrawAchievement: YOLO

Highlights

  • Pro

Organizations

@AlphaBloke-AI

Block or report usamahz

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
usamahz/README.md
The cover of How to Make Your Model Fast: a latency budget drawn as a ruler, cut by a line at 33 milliseconds, with the 3.6 milliseconds past the line in red.

How to Make Your Model Fast

A Systems View of Efficient Machine Learning, from Silicon to Agents

Most of what is written about machine learning assumes the model is the interesting part and the machine is a detail. In practice the machine decides what you are allowed to build. This book is about the boundary where a model meets real hardware under a real budget for latency, memory, power and money: how to predict what that boundary will do to you, and what to change first when it does.

Fourteen parts, from the silicon up: rooflines, CPUs, GPUs and NPUs, kernels, compilers, quantisation, compression, vision, LLMs on small machines, robotics, profiling, serving and agents. 36 worked problems with every answer written out in full.

It's free. Start with Part 1, the roofline.


I write at usamah.me. A post to start with: Stop Latency Laundering.

Pinned Loading

  1. cpu-performance-engineering cpu-performance-engineering Public

    A reading path for CPU performance engineering, from one instruction to production inference. Primary sources only, with a runnable benchmark for every section.

    C 58 5

  2. make-your-model-fast make-your-model-fast Public

    A Systems View of Efficient Machine Learning, from Silicon to Agents. Fourteen parts, from roofline analysis and vector units up through kernels, compilers, quantisation, compression, vision, on-de…

    6 1

  3. pytorch/executorch pytorch/executorch Public

    On-device AI across mobile, embedded and edge for PyTorch

    Python 5.1k 1.2k

  4. cnn cnn Public

    A convolutional layer with a subsequent Rectified Linear Unit (ReLU) activation. This structure forms the cornerstone of many sophisticated image processing and computer vision systems.

    C 4

  5. multi-task-loss multi-task-loss Public

    A production-ready, scalable multi-task perception system for autonomous vehicles, capable of handling multiple tasks simultaneously.

    Python 3

  6. autonomous-vehicles autonomous-vehicles Public

    Autonomous Vehicles: Implementation of various computer vision techniques for autonomous vehicles.

    Python 2