Run open models on your own GPUs with dl model

dl model sets up an open model on an Nvidia GPU you own, checks that it answers correctly, and connects the dl coding agent to it. When a new model version ships, dl model update tests it next to the old one and only switches if it passes. If you change your mind, dl model rollback brings back the previous model with the same settings.

Last week the harness was local. This week the model is too.

#Who this is for

Teams that self-host for privacy usually already have the GPUs. What costs them is everything around the hardware: picking the right serving flags, getting tool calls working, and moving to each new model version without breaking the agents that depend on it. Each of those can eat days.

dl model is for that work. It doesn't replace your hardware or your choice of model. It makes running and updating a model a command instead of a project.

If you don't have GPUs, nothing changes for you. dl still starts with free tokens after a GitHub sign-in, or your own key for any provider.

#What it does

curl -fsSL https://dl.new | sh   # install dl (0.30.1 or later)
dl model up          # check the GPU, pick a model that fits, start it, test it
dl model status      # what's running, GPU memory, last test results
dl model update      # test a new version next to the old one, switch only if it passes
dl model rollback    # bring back the previous model
dl model bench       # measure speed with 1, 8 and 32 requests at once

dl model up checks your Nvidia GPU and lists the models that fit. It starts the one you pick with vLLM in Docker, using a config we've tested: the image version, the memory limits and the flags the model needs. It listens on your machine only. Then it tests the model, and if the tests pass, it points the dl coding agent at it.

If your GPU has 24 GB or more, you don't even need to type the command: dl offers to set up a model the first time it runs.

#Why "the server responds" isn't enough

A model server can start, answer a health check, and still be useless to a coding agent. The most common reason is tool calling. A coding agent works by calling tools: read a file, run a command, edit a file. If the server isn't started with the right tool-call settings for that model, the model chats fine but never calls a tool, and the agent looks broken.

So before a model goes live, dl sends real requests and checks the answers:

• a short chat;
• speed;
• a tool call;
• a JSON reply.

If any check fails, nothing switches. Your agents keep using the model that already works. A failed check also prints a curl command that sends the same request, so you can see it for yourself.

#Updates you can undo

dl model update starts the new version next to the old one when the GPU has room for both, tests the new one, re-tests the old one, and compares their speed. Only then does it switch the agent over and stop the old one. If there isn't room for both, it tells you the model will be down while it switches, and asks first.

dl model rollback brings back the previous model with the same settings and points the agent back at it.

#What it looked like on a real GPU

We tested it on one Nvidia RTX PRO 5000 Blackwell with 48 GB, using vLLM 0.20.0 and a small 4B open model (a 7.5 GB download). dl reserved 40% of the card, about 20 GB, so two small models fit side by side during an update.

dl model bench at an 8k context (the default is 32k) measured:

vLLM's prefix cache hit rate was 50%. The update ran side by side with no downtime: dl tested both versions, compared their speeds, then switched. Rollback worked the same way.

#What works today

• Now: one Nvidia GPU, one model per machine, vLLM. Setup, tested updates and rollback. Tested with small models.
• Coming soon: models that need several GPUs, such as GLM 5.3 Flash and DeepSeek V4.1 Flash, several models on one machine, and SGLang.
• On the roadmap: AMD and Huawei Ascend.

Your prompts, code and model weights never leave your hardware. We count anonymous setup steps, like your GPU type and whether setup worked, to improve dl, and you can turn that off.

#Run it on your GPUs

Install dl, then run dl model up. The details are in the docs.

If your team runs a bigger setup, we're working with a few design partners to get them going. Tell us what you run: your company, your GPUs, and the models you use. The same form is the early-access list for AMD and Huawei Ascend.

No GPUs? Install dl from dl.new and start with free tokens.

#FAQs

#Does my data leave my machine?

No. The model runs on your GPU and listens only on your machine. The dl coding agent talks to it locally, so your prompts, code and model weights stay on your hardware. dl does count anonymous setup steps (your GPU type, and whether setup worked or where it failed). Set DL_NO_TELEMETRY=1 or DO_NOT_TRACK=1 to turn that off. The privacy policy has the details.

#Which GPUs and models work today?

One Nvidia GPU per model, with vLLM. It's tested with small models. Models that need several GPUs, like GLM 5.3 Flash and DeepSeek V4.1 Flash, are coming soon.

#What happens if an update breaks something?

If the new version fails any check, dl doesn't switch, and the old one keeps serving. If you find a problem after switching, dl model rollback brings back the previous model with the same settings.

#Do I still get free tokens?

Yes. The free tier works the same as before. dl model is an addition for teams that own GPUs, not a replacement. /provider in a session switches between them.

#Does it support AMD or Huawei Ascend?

Not yet. Both are on the roadmap. Use the design-partner form to ask for early access.