What “private AI” actually means

Modular house · Explainer, Private AI · ~4 min read

Private AI is your models and your agents running on infrastructure you control, with your data staying inside boundaries you set. The name is about who holds the keys. It says nothing about how capable the tools are. Your data, your rules, from dirt to desktop, one partner from the hardware to the interface. Here is what that looks like in practice, and what it does not require you to give up.

The common misconception

Say “private AI” in a planning meeting and watch what people picture. Usually something small. A stripped-down model wheezing away on a spare laptop, good for a demo and not much else, the discount version of the tools everyone already uses at work. That picture is years out of date. A private workspace runs current open-weight models, several of them, on real hardware in a real facility, and you pick the one that fits the job. Privacy and capability are two separate questions. Answering one does not cost you the other.

The confusion is easy to forgive, because the line got blurry on purpose. The public tools sell a business tier with a setting that promises your data will not be used for training, and that promise is worth something. Read where it lives, though. It lives in a contract, and a contract is a thing both parties can revisit. Meanwhile your prompts still leave your building, cross the open internet, and land on servers you have never seen, in a jurisdiction you did not pick. The setting governs what the vendor says it will do with your data. Where your data goes stays the same.

Private AI closes that gap by removing the trust exercise entirely. Data cannot leak out of a boundary when the computing happens inside the boundary. There is nothing to opt out of. A setting is a promise. A boundary is a fact.

The three things that make AI private

Start with where the model runs, because everything else follows from it. When you ask a public tool a question, the question travels. It leaves your network, crosses the internet, and gets processed in a data center you cannot name, alongside everyone else’s traffic. Private AI turns that around: instead of sending your data to the model, you bring the model to your data. For our clients that means an isolated environment in a US-based, FedRAMP-certified data center, and for organizations that want it, dedicated hardware on their own premises. The question gets answered a few racks away from the data it draws on. Sometimes a few feet.

Then the data itself, which covers a lot more than chat history. The real value of a workspace shows up when you connect it to your documents. Contracts, research, case files, the institutional memory of the whole company. That happens through retrieval: the model reads the relevant pieces of your knowledge base at query time and grounds its answer in them. In a private environment, the documents, the index built from them, and the retrieval itself all stay inside the same boundary. Nothing is used to train anyone’s model. Nothing is retained by a third party, because there is no third party in the room.

And the part that gets discussed least while mattering most: who draws the boundary. Who can see which documents. Whether the system can reach the internet at all. What gets encrypted, what gets backed up, and what happens to all of it if you decide to leave. On public platforms those rules are set by the vendor and adjusted at the vendor’s discretion. In a private environment they are yours to set: access controls that separate the legal team’s vault from marketing’s, an air gap where the work demands one, and an exit that is just your data, in usable formats, walking out the door with you. That last one tells you who really owned the environment. If leaving is easy, you did.

What you keep

Everything your team actually liked about the public tools. The chat interface, the drafting and summarizing, the document questions, the code help. Those run just as well on infrastructure you control, through a clean web interface with access to multiple models, so people pick the model that fits the task instead of the one a vendor is promoting this quarter. Good UX is a software problem, and the software exists. Nobody has to learn to love a command line.

You also keep room to grow, because private is a spectrum, and you do not have to buy the far end of it on day one. Our Wildcat tier is the entry point: an isolated environment on shared infrastructure, in the same FedRAMP facility, under the same US jurisdiction as everything above it. Panther steps up to dedicated infrastructure, with file vaults, role-based access controls, and enhanced encryption for teams handling sensitive documents. Grizzly is the top: fully dedicated hardware, zero-trust architecture, an optional air gap, and the option to run it on your own premises instead of ours. The boundary tightens as you climb. What never changes is the jurisdiction. Your data stays in the United States whether it sits in our facility or in your building.

And you keep one accountable partner across the whole stack. The facility, with its redundant power and its guarded doors. The hardware in the racks. The models, kept patched and current, swapped for better ones as better ones ship. The interface your people log into every morning. When something needs attention, there is no seam between a cloud vendor, a hosting vendor, and an AI vendor for the problem to fall through. You hold the deed, and there is one number to call.

Your data, your rules.