At Google Cloud Next 2026 in Las Vegas this week, Google made a quiet but significant announcement: Gemini can now run on a single air-gapped server, fully disconnected from the internet — and from Google itself.

The product is a Dell-certified, Google-approved hardware appliance delivered through a neocloud partner called Cirrascale Cloud Services. Eight Nvidia GPUs. Confidential computing protections. The marketing hook: “pull the plug and the model vanishes.”

We’ve been watching the coverage with genuine interest. And a fair bit of déjà vu.

The Market Just Caught Up

For years, enterprise organizations in financial services, healthcare, defense, and government faced what analysts called an impossible tradeoff: access the most powerful AI models through public cloud APIs — and surrender control of your data — or settle for less capable open-source models you could host yourself.

Google’s announcement is a formal acknowledgment that this framing was always wrong. The demand for fully private AI wasn’t a niche concern. It was the only architecturally honest answer for any organization that takes data governance seriously.

Modular Technology Group has been building on that premise since before it was a keynote slide.

What Google Is Actually Selling

Let’s be precise about the offering, because the details matter.

The Cirrascale deployment requires a Google-certified hardware platform. It requires a partnership with a specific neocloud provider. It requires Google’s approval of the appliance configuration. General availability is projected for June or July 2026 — it’s in preview now.

And the selling point — that the model “vanishes when you pull the plug” — is a confidential computing feature that ties the model weights to the specific hardware. Impressive engineering. But consider what it implies: you are still dependent on Google’s certification ecosystem to acquire and maintain access to the model. The sovereignty is physical, not architectural.

The right question for any enterprise evaluating this: What is your exit strategy?

• What happens if Cirrascale changes its pricing or partnership terms?
• What happens if Google deprecates the on-premises licensing tier?
• What happens when the certified hardware goes end-of-life?

Vendor lock-in doesn’t disappear because the server is in your rack. It moves from the network layer to the hardware and licensing layer.

A Different Architectural Bet

Modular Technology Group made a different set of choices when we designed our private AI infrastructure.

Model-agnostic. We are not tied to any single model provider. Our clients run the models that fit their use case — whether that’s an open-weight model, a fine-tuned variant, or a frontier model accessed under controlled conditions. When a better model ships, you switch. No re-certification. No new appliance.

Hardware-agnostic. We operate in a FedRAMP-authorized data center on infrastructure you control. You are not locked to a specific GPU configuration or a vendor-approved hardware stack. The architecture scales with your needs, not with a product roadmap you don’t control.

Fixed, transparent pricing. No usage-based API billing. No surprise invoices at the end of the month. You know what you’re paying. That predictability is a feature, not an accident.

Available now. Not in preview. Not GA in Q3. Running, deployed, with clients in production today.

Data Sovereignty Is Architecture, Not Proximity

The broader lesson from Google’s announcement isn’t about Google. It’s about how the enterprise AI market is maturing in its understanding of what “private” actually means.

Physical proximity — a server in your building, or in a data center you can point to — is necessary but not sufficient. True data sovereignty requires architectural ownership: control over the model, the infrastructure, the data pipeline, and the exit path.

When your AI model “vanishes when you pull the plug,” ask yourself: whose plug is it, really?

At Modular Technology Group, “Your Data, Your Rules” isn’t a product announcement. It’s been the design constraint from the beginning.

If you’re evaluating private AI infrastructure — whether in response to this week’s news or because you’ve been thinking about it longer than Google has been announcing it — we’re happy to compare architectures.

Schedule a conversation →

---

Source inspiration: LinkedIn

Same AI Model, Two Hardware Tiers — And Why Context Length Is the Hidden Variable

Modular Technology Group · April 20, 2026

---

Ask any AI vendor how fast their stack runs and you’ll get a single headline number. “40 tokens per second.” “Under a second to first token.” Impressive — until you realize the benchmark prompt was 200 words long and you’re planning to feed it a 300-page document.

This week we took Qwen 3.6 35B-A3B — a state-of-the-art Mixture-of-Experts model released a few days ago — and pointed it at two very different pieces of hardware. Same model. Same questions. Same quantization tier (4-bit). Only the hardware changed.

The result isn’t just a horse race. It’s a quiet lesson in why the specs that matter for AI aren’t always the specs that get advertised.

---

Why We Ran This

At Modular, we route the same model across different infrastructure depending on the workload. A developer laptop handles quick, short-context tasks. A dedicated AI server handles long-document analysis, multi-turn agent reasoning, and anything that needs a big context window.

The question isn’t “which is faster.” A server beats a laptop. That’s boring.

The real question: at what context length does routing to the dedicated server become worth it? Without numbers, every routing decision is a guess. So we measured.

---

The Setup

Platform	Hardware	Engine	Quantization
“Forge” — developer laptop	MacBook Pro M4 Max, 64 GB unified memory	LM Studio (MLX backend)	MLX 4-bit
“Reach” — dedicated AI server	2× NVIDIA RTX 4070 Ti, 24 GB VRAM total	Ollama v0.11.10 (llama.cpp/GGUF)	Q4_K_M GGUF

We ran the model at three context sizes — 32k, 64k, and 128k tokens — and measured how long each host took to generate a 256-token response. Three trials per cell. Temperature fixed at 0.1 for near-determinism. Prompt content matched byte-for-byte. Tokenizer output cross-checked. Apples to apples.

For the 128k results, we added six total trials across two independent sessions to nail the number down.

---

Result #1: One Host Stays Usable. The Other Doesn’t.

At 32k context, both platforms deliver workable performance. The MacBook runs at 10.8 tokens/sec — slower than the dedicated server, but perfectly fine for interactive chat.

Then the context grows.

At 64k, the MacBook drops to 4.8 tokens/sec. At 128k, it collapses to 1.7 tokens/sec.

The dedicated server, meanwhile, holds its shape:

Context	Forge (MBP)	Reach (Dual GPU)	Reach advantage
32k	10.8 tok/s	26.3 tok/s	2.4× faster
64k	4.8 tok/s	19.3 tok/s	4.0× faster
128k	1.7 tok/s	9.0 tok/s	5.3× faster

Notice the pattern: the gap widens with every doubling of context. This isn’t a flat advantage — it compounds. By the time you’re at 128k, the kind of window you need for whole-document analysis or agent reasoning, the server is over five times faster than the laptop.

---

Result #2: The Honest Metric Is Wall Time

Tokens-per-second is abstract. What does this actually feel like to a human waiting for an answer?

A 256-token reply — roughly one solid paragraph — takes:

Context	Forge	Reach
32k	23 seconds	12 seconds
64k	54 seconds	15 seconds
128k	2 minutes, 32 seconds	31 seconds

That’s the difference between a tool you can hold a conversation with and a tool you fire off and check back on later.

---

Result #3: The Bottleneck Isn’t What You Think

Here’s where it gets interesting.

During the 128k runs on the dedicated server, we monitored both GPUs continuously. The VRAM was pegged — 22.2 GB of 24 GB total, 91% saturation. So the GPUs must have been pegged too, right?

Not even close.

The two GPUs, theoretically capable of hundreds of trillions of operations per second, sat at 6–7% utilization. They weren’t waiting for work. They were waiting for memory.

At long context lengths, the model has to read the entire “KV cache” — every token it’s seen so far — to generate each new token. Enormous quantities of data move between VRAM and the compute cores every few milliseconds. The memory bus becomes the choke point long before the math does.

This is the single most important finding in the entire exercise, because it reframes how to evaluate future hardware.

More FLOPS won’t fix this. When the question becomes “should we buy the next card when it drops?” — the answer starts with its memory bandwidth spec, not its TFLOPS number. That’s the opposite of what most marketing collateral emphasizes.

---

The Same Story, Live From Production Telemetry

This is real production monitoring from our own dashboard during the benchmark — not synthetic charts. Three things worth noticing:

• Both GPU panels are nearly identical. Both cards track the same 5–7% load pattern. That’s tensor parallelism working.
• The staircase in “Total Memory Used.” Each step is a single 128k trial committing its KV cache, then holding it. Three trials, three plateaus, climbing toward the 24 GB ceiling.
• Compute is flat. Memory is climbing. The shape of the real data tells the same story as the synthetic chart: this workload lives and dies by memory, not by compute.

This is the visibility that separates production AI infrastructure from “we installed it and hope it works.”

---

Result #4: Tensor Parallelism Done Right

One thing the dedicated server does exceptionally well: split the model cleanly across both GPUs.

At 128k context, the memory load is nearly identical on both cards — 11,101 MiB on GPU 0, 11,117 MiB on GPU 1. A difference of 16 MiB out of over 11,000. That’s Ollama’s tensor-parallel splitter working exactly as designed. No card is bearing extra load. No GPU is OOMing. No spillover to CPU.

Tensor parallelism isn’t automatic. It requires compatible hardware, deliberate configuration, and a runtime that actually supports it. It’s also invisible to the end user — which is exactly how it should be.

---

What This Means for How You Deploy AI

If you’re prototyping against 4k-to-16k prompts on a decent laptop, you’re fine. For a team running real AI applications against real-world documents, the math shifts quickly.

A few honest observations from this data:

• Context length matters more than model size. A 35B-parameter model can feel snappy or geological depending entirely on how much context you feed it. Marketing benchmarks rarely mention this.
• Hardware choice is a memory problem, not just a compute problem. Two mid-range GPUs with balanced VRAM can outperform much more expensive single-GPU setups for long-context work.
• Consumer hardware has real limits. M-series Macs are remarkable for the price. But physics is physics. There’s a reason production AI workloads live on dedicated servers.
• Private infrastructure isn’t only about sovereignty. It’s also about having the right hardware for the right context, predictable performance, and the ability to scale without a surprise cloud bill.

At Modular, we deploy private AI infrastructure that gets these details right — matching the model, the quantization, the hardware, and the runtime so answers come back in seconds, not minutes. Data stays private. Costs stay fixed. Performance stays predictable.

Your data, your rules. Your hardware, matched to your workload.

---

Appendix: Methodology & Caveats

Model: Qwen 3.6 35B-A3B (Mixture-of-Experts — 36B total parameters, 3B active per token)

Prompts: Synthetic filler text sized to 85% of target context, with a single consistent question appended. Byte-identical across both hosts. Tokenizer output verified to match (prompt_tokens reported identically on each side).

Trials: Three per context-size × host cell for the primary run. Six additional trials at 128k on the dedicated server across two independent sessions. Variance across all six 128k runs: under 2% (8.94–9.03 tok/s).

Completion target: 256 tokens, temperature=0.1.

Ollama configuration: Explicit num_ctx override on every request. Default caps context at 4,096 tokens — enough to silently invalidate every long-context test if you miss it.

Caveats:

• Quantization formats differ (MLX 4-bit vs Q4_K_M GGUF). Both are 4-bit but not bit-identical.
• The MacBook was running normal background workloads during the test, not dedicated. A clean bench would improve its numbers modestly but not flip the conclusion.
• Single model tested. Different architectures — dense transformers, larger MoEs, specialized coding models — will scale differently.
• The 6–7% GPU utilization figure reflects generation phase only. Prompt evaluation phase utilization was much higher, but brief.

Raw data and all benchmark scripts: Available on request. Fully reproducible.

---

Modular Technology Group builds and hosts private AI workspaces with open-source components, in a FedRAMP data center. We use what we sell.