Run a model on your own GPU with dl model

dl model sets up an open model on an Nvidia GPU you own and connects the dl coding agent to it. It checks your GPU, picks a tested config, runs vLLM in Docker, and checks that the model answers correctly before it goes live. Updates are tested next to the old version and only switch if they pass. Your prompts, code and weights never leave your hardware.

It needs dl 0.30.1 or later.

dl model up          # check the GPU, pick a model that fits, start it, test it
dl model status      # what's running, GPU memory, last test results
dl model update      # test a new version next to the old one, switch if it passes
dl model rollback    # back to the previous version
dl model bench       # measure speed with 1, 8 and 32 requests at once

#What you need

• An Nvidia GPU with enough memory for the model you pick. dl model up only lists models that fit. The model runs on one GPU, so what fits is decided per GPU, not by adding cards together.
• The Nvidia driver, Docker, and the NVIDIA Container Toolkit. dl model up checks all three and says what's missing.
• Disk space for the model weights. They're downloaded once into a shared cache and reused on updates.

This runs on the machine you type the command on: your workstation, or a GPU server you're logged in to. --server doesn't apply.

#Start a model

dl model up shows your GPU, lists the models that fit, and asks for the context length and port (8000 by default). Then it:

1. starts vLLM in Docker, listening on 127.0.0.1 only;
2. waits for the model to load (the first answer from a fresh start can take up to 2 minutes);
3. tests it: a short chat, speed, a tool call and a JSON reply;
4. points Autopilot at it, so your next dl session uses it.

Add --dry-run to dl model up or dl model update to see every Docker command without running anything.

#From dl's first run

The first time dl starts on a machine with an Nvidia GPU of 24 GB or more, it offers to run a private model on it. Say yes and it runs the same setup with the default context length and port, without asking for them. If something's missing, it tells you, and on Ubuntu and Debian it offers to install the NVIDIA Container Toolkit for you.

#Update and roll back

dl model update starts the new version next to the old one, on the next port, when the GPU has room for both. It tests the new one, re-tests the old one, compares their speed, then switches Autopilot to the new one and stops the old container. If there isn't room for both, it warns you that the model will be down while it switches, and asks first.

If the new model fails its tests, nothing switches: the old model keeps serving.

dl model rollback brings back the previous model with the same settings and points Autopilot back at it.

#What dl checks before a model goes live

A server that responds isn't the same as a model that works for a coding agent. Before it switches, dl sends real requests and checks the answers:

• a short chat;
• speed;
• a tool call, because without the right tool-call setup a model can chat but can't edit files or run commands;
• a JSON reply.

If any check fails, nothing switches. A failed check prints a curl command that sends the same request, so you can try it by hand.

#What works today

• One Nvidia GPU, one model per machine, vLLM. Tested with small models.
• Coming soon: models that need several GPUs, such as GLM 5.3 Flash and DeepSeek V4.1 Flash, several models on one machine, and SGLang.
• On the roadmap: AMD and Huawei Ascend. Ask for early access.

#Where things live

• The running model's state: ~/.local/state/directionallylocal/model.json (or under $XDG_STATE_HOME). It's separate from a server install, so dl model works on a laptop or a server.
• Model weights: the Docker volume dl-model-hf-cache.
• The containers: dl-model-1 and dl-model-2. Each update switches to the other one, and dl model status shows which is running. docker logs <container> shows vLLM's output.
• Logs: every dl model run writes one to ~/.directionallylocal/logs, and errors name the file.

#FAQs

#Does my code leave my machine?

No. Your prompts, code and weights never leave your hardware. The model listens only on 127.0.0.1, and Autopilot talks to it locally. We count anonymous setup steps (your GPU type, and whether setup worked or where it failed) to improve dl. You can turn that off with DL_NO_TELEMETRY=1 or DO_NOT_TRACK=1, and it never runs in CI. Details are in the privacy policy.

#Which GPUs work?

One Nvidia GPU with a recent driver and the NVIDIA Container Toolkit. Models that need several GPUs are coming soon; AMD and Huawei Ascend are on the roadmap.

#Can I still use the free tier or my own key?

Yes. Switch any time with /provider in a session. That moves you off the local model.

#What if the model won't start?

The first line of the error says which step failed and why. Run dl model status to see which container is running, then docker logs <container>. The most common causes are too little GPU memory for the context length you chose, and a vLLM image that doesn't support your GPU yet.