Docker¶
Turing Engine publishes a CPU image and a CUDA image to the GitHub Container Registry.
| Image | Platforms | Built from |
|---|---|---|
ghcr.io/intutic/turing:<version> |
linux/amd64, linux/arm64 | CPU PyTorch |
ghcr.io/intutic/turing:<version>-cuda |
linux/amd64 | PyTorch for CUDA 12.8 |
<version> is the release tag, for example v2.0.0; latest and latest-cuda follow the newest release. The
release workflow (.github/workflows/release.yml) publishes both images when a version tag is pushed, starting with
v2.0.0.
The container runs turing serve on port 8000 as a non-root user (turing). Arguments after the image name are
passed to turing serve (default: --model smollm2).
CPU¶
docker run --rm -p 127.0.0.1:8000:8000 \
-v ~/.cache/huggingface:/home/turing/.cache/huggingface \
ghcr.io/intutic/turing:latest --model HuggingFaceTB/SmolLM2-135M
The volume reuses your HuggingFace download cache, so weights are fetched once. Without it the container downloads
the model on every start. Pass -e HF_TOKEN=... for gated models.
curl -s localhost:8000/v1/chat/completions -H 'content-type: application/json' \
-d '{"model": "smollm2", "messages": [{"role": "user", "content": "The capital of France is"}], "max_tokens": 8}'
CUDA¶
Needs an NVIDIA driver recent enough for CUDA 12.8 and the NVIDIA Container Toolkit on the host.
docker run --rm --gpus all -p 127.0.0.1:8000:8000 \
-v ~/.cache/huggingface:/home/turing/.cache/huggingface \
ghcr.io/intutic/turing:latest-cuda --model HuggingFaceTB/SmolLM2-360M-Instruct --device cuda
Every turing serve flag works after the image name, for example --kv-cache-dtype int4 or --max-batch-size 32.
Run docker run --rm ghcr.io/intutic/turing:latest --help for the list.
Building locally¶
One Dockerfile, deploy/Dockerfile, builds both images. They differ only in the PyTorch wheels: the CUDA image
installs PyTorch's CUDA 12.8 build, which bundles the CUDA runtime.
docker build -f deploy/Dockerfile -t turing:cpu .
docker build -f deploy/Dockerfile --build-arg TORCH_INDEX=https://download.pytorch.org/whl/cu128 -t turing:cuda .
The build compiles the C++ extension into a wheel in a first stage, so the final image carries no compiler.