tinygrad

What tinygrad is

tinygrad is a free deep learning stack for neural network training, bringing together a tensor library, compiler, JIT, and training tools. Its tensors support automatic differentiation in both forward and backward passes, while operations execute lazily when a tensor is realized. The library includes neural network classes, optimizers, and state loading and saving. TinyJit captures and replays kernels in a decorated function, and tensors can be split across multiple GPUs. Listed accelerators include OpenCL, CPU, Metal, CUDA, AMD, NV, Qualcomm, and WebGPU. tinygrad supports Python and local training, with multiple deployment targets. The project says it is used in openpilot to run a driving model on a Snapdragon 845 GPU. It is licensed under MIT, and installation instructions cover pip as well as the recommended source installation. The documentation says the API has been stable for a while, though the project is not yet version 1.0; its FAQ says it is not faster than PyTorch for most use cases. Development discussion is directed to GitHub and Discord.

Who it suits

tinygrad is aimed at people working on deep learning frameworks and neural network training, particularly Python users who want to explore its tensor, compiler, and JIT components.

Limits

tinygrad is free. Its documented caveats are that it is not yet version 1.0 and is not faster than PyTorch for most use cases.

Getting started

1. Visit the tinygrad website.
2. Choose an installation route: the project recommends source installation and also documents pip.
3. Install tinygrad in a Python environment.
4. Use the documentation to begin working with tensors and training tools.
5. Join development discussion on GitHub or Discord.

Verdict

tinygrad combines training tools with a broad set of listed accelerators and multi-GPU tensor sharding. Its pre-1.0 status and stated performance caveat are important considerations for prospective users.