kernel-builder

---

Note: Torch 2.10 builds are still based on PyTorch release candidates. Typically the ABI does not break during release candidates. If it does, you have to recompile your kernels with the final 2.10.0 release.

Join us on Discord for questions and discussions!

This repo contains a Nix package that can be used to build custom machine learning kernels for PyTorch. The kernels are built using the PyTorch C++ Frontend and can be loaded from the Hub with the kernels Python package.

This builder is a core component of the larger kernel build/distribution system.

🚀 Quick Start

We recommend using Nix to build kernels. To speed up builds, first enable the Hugging Face binary cache:

# Install cachix and configure the cache
cachix use huggingface

# Or run once without installing cachix
nix run nixpkgs#cachix -- use huggingface

Then quick start a build with:

cd examples/relu
nix run .#build-and-copy \
  --max-jobs 2 \
  --cores 8 \
  -L

Where --max-jobs specifies the number of build variant that should be built concurrently and --cores the number of CPU cores that should be used per build variant.

The compiled kernel will then be available in the local build/ directory. We also provide Docker containers for CI builds. For a quick build:

# Using the prebuilt container
cd examples/relu
docker run --rm \
  --mount type=bind,source=$(pwd),target=/kernelcode \
  -w /kernelcode ghcr.io/huggingface/kernel-builder:main build

See dockerfiles/README.md for more options, including a user-level container for CI/CD environments.

🎯 Hardware Support

Hardware	Kernels Support	Kernel-Builder Support	Kernels Validated in CI	Tier
CUDA	✓	✓	✓	1
ROCm	✓	✓	✗	2
XPU	✓	✓	✗	2
Metal	✓	✓	✗	2
Huawei NPU	✓	✗	✗	3

📚 Documentation

• Writing Hub kernels
• Building kernels with Nix
• Framework-specific notes:
  • Metal
• Building kernels with Docker (for systems without Nix)
• Local kernel development (IDE integration)
• Kernel security
• Why Nix?

Credits

The generated CMake build files are based on the vLLM build infrastructure.