BareMetalRT runs local AI agents on NVIDIA RTX GPUs in Windows with TensorRT-LLM. Free tier; Enterprise $20/seat/month. Features, pricing and verdict.
Researched and fact-checked by Bartu Cavusoglu · Updated October 26, 2026
BareMetalRT is a Windows-native runtime for running open-weight language models and tool-calling agents on a user's own NVIDIA RTX GPU. It is made by Bare Metal AI, Inc., a member of the NVIDIA Inception program, and is positioned as a private alternative to cloud inference. The company says it sells no hosted inference, so there is no cloud for prompts to reach.
Under the hood it uses NVIDIA TensorRT-LLM rather than llama.cpp. The vendor states that NVIDIA discontinued TensorRT-LLM on Windows and that it maintains a working port with no WSL, Docker or compatibility layers. The current release shown on the site is v0.26.1.
At install time the engine compiles each model to the specific RTX card, using fused attention kernels, paged KV-cache and in-flight batching. Context windows reach up to 512K tokens on a single machine. The site's comparison table lists 4–8 concurrent users, with the number depending on the GPU's VRAM.
BareMetalRT can run as a headless background service. A command such as bmrt serve --model qwen3-4b starts an OpenAI-compatible API at localhost:1234/v1, and an Anthropic-compatible API is also offered. For remote access, the daemon dials out to a relay that forwards TLS-encrypted bytes, so no port forwarding is needed.
The agent layer includes 318 integrations built on the Model Context Protocol, 29 ready-made skills, and workflows that chain integrations into reusable, schedulable procedures. Integrations are read-only by default, with opt-in write access gated by approval prompts. A local memory feature stores facts in an editable markdown vault that works with Obsidian.
For larger models, tensor parallelism splits a model across multiple RTX GPUs over ordinary Ethernet, using a custom transport that replaces NCCL. Cards can differ but must share an RTX generation, and the vendor says it computes in full precision. The catalog covers Qwen, Llama, DeepSeek, Mistral, Gemma and Phi, and users can load Hugging Face repos.
BareMetalRT is a Windows-native runtime from Bare Metal AI, Inc. that runs open-weight models and tool-calling agents on a user's own NVIDIA RTX GPU. It uses NVIDIA TensorRT-LLM and exposes OpenAI and Anthropic-compatible APIs. The vendor says nothing leaves the user's network.
The Free plan is $0 for one person on one machine. Enterprise is $20 per seat per month, billed monthly, with an annual option advertised at 25% savings. Enterprise Private is custom-priced.
Yes. The Free plan includes the full engine, the entire model catalog, the local API, developer and personal connectors, a chat app with voice and built-in tools. The site says it needs no account and has no meter or expiry.
The vendor says Ollama and LM Studio run llama.cpp, while BareMetalRT runs NVIDIA TensorRT-LLM and compiles models to the user's RTX card. It also says BareMetalRT is the only one of the compared tools that splits a model across multiple RTX GPUs over Ethernet.
It offers 318 integrations built on the Model Context Protocol, including AWS, Azure, Datadog, Snowflake, ServiceNow, Microsoft 365, Box, Workday, Splunk, Okta, GitHub, Postgres, Gmail and Slack. Developer connectors are free, and the business catalog needs Enterprise.
The vendor says it runs entirely on the user's hardware with no hosted inference and offers a one-click air-gap with documented attestation. A HIPAA BAA and DPA are available, and SOC 2 and ISO 27001 are listed as planned or in progress.
It is built for NVIDIA RTX GPUs on Windows. Concurrent users and model size depend on the GPU's VRAM. Models too large for one card can be split across multiple RTX GPUs of the same generation over Ethernet.
It can suit small businesses that already own RTX hardware, since the free tier covers one machine and Enterprise seats are self-serve at $20 per month with no contract. Businesses without NVIDIA GPUs or Windows machines would not be able to use it.