Your information security policy says it somewhere, in writing: “Confidential information must not be transmitted to external servers.” So here is a question. That source code your engineer just pasted into an AI chat — whose server processed it?
In 2026, the technical answer to this contradiction already exists. GLM, Kimi, DeepSeek — download the open weights, run them on your own GPUs, and nothing leaves the building. Not a prompt, not a line of code, not a single byte of customer data.
And let’s be equally clear about the second point: building this requires no consulting firm and no system integrator. Worse — putting them in the middle is itself a security risk. This article explains why, step by step.
What You’re Actually Trading for the Convenience of Cloud APIs
Using a cloud LLM API means every prompt travels as an HTTP request to someone else’s server. Source code. Incident logs. Data pipelines full of customer records. Summaries of board meeting minutes. The more seriously you use AI for development, the more of your company’s core information flows outward, every single day.
“The terms of service say our data won’t be used for training” — true. But think it through. A contract does not prevent a leak. Only data that never moves is data that never leaks. Terms get revised. Vendors get acquired. And data sitting on a foreign provider’s servers is subject to that country’s jurisdiction, not yours — the US has the CLOUD Act, and every other jurisdiction has its equivalent. Whichever country’s API you call, the structure is the same.
In finance, healthcare, defense, and government, “the data left the premises” is disqualifying by itself. And even outside regulated industries, there’s a strange asymmetry: companies preach zero trust and defense in depth, then take the single most valuable thing they own — the code of the product they’re about to ship — and send it out on a promise.
The Open-Weight Trio: GLM, Kimi, DeepSeek
The biggest tectonic shift of 2025 was open-weight models catching up to the frontier. Three names lead it.
DeepSeek. The company behind the “R1 shock” of January 2025. A 671-billion-parameter Mixture-of-Experts design that activates only ~37 billion per token — enormous knowledge at a realistic inference cost, with particular strength in math, coding, and reasoning. License: MIT. No strings worth mentioning.
Kimi K2. Moonshot AI’s trillion-parameter MoE (~32 billion active), which made its name on agentic tasks — using tools and completing multi-step work autonomously. License: Modified MIT, with one clause — products exceeding 100 million monthly active users or $20 million in monthly revenue must display “Kimi K2” in their UI. For virtually every company reading this, that’s unconditional.
GLM. The flagship of Z.ai (Zhipu AI). GLM-4.6 is a 355-billion-parameter MoE with a particularly strong reputation in coding-agent workloads, under an MIT license. Its smaller sibling matters just as much: GLM-4.5-Air, at 106 billion parameters, runs quantized on workstation-class hardware. As a “start with one machine” entry point, it is currently the strongest candidate.
What the three share: coding and agentic benchmark scores that match or beat frontier closed models from a year ago — and weights you can download for free.
”But Chinese Models Are Scary” — Your Fear Is Pointed the Wrong Way
Raise this topic and the same reaction always comes: “Is it really safe to run a Chinese company’s model internally?”
Start with the technical facts. Model weights are static files in safetensors format. They are matrices of numbers, not executable code. The thing that executes is an inference engine — vLLM or SGLang, open-source software audited by the entire world — and you choose which one.
A self-hosted model has no outbound traffic. You don’t have to take that on faith: block all egress at the firewall and run it. It works. It works in a fully air-gapped environment. The vague fear that “it might be sending data somewhere” is a fear you can kill with a verifiable test.
And here is the sharp irony. “Chinese models are scary, so we’ll use a big foreign API instead” — that decision is precisely the act of shipping your data to an external server. If you are genuinely worried about where your data goes, self-hosted open weights are the only defensible answer. A hosted API is a black box nobody outside can inspect; published weights are examined by researchers worldwide. It is not close.
Data moves because you send it to an API. Download the weights, and the only thing that runs is your own GPU.
Self-Hosting Is Not a Six-Month Project. It’s a Friday Afternoon.
So what does the build actually take? Here’s the real procedure — not the SI’s estimate.
Provision a GPU server. Run pip install vllm. Then:
vllm serve zai-org/GLM-4.5-Air --tensor-parallel-size 2That’s it. An OpenAI-compatible API endpoint is now live inside your network. Coding agents like Cline, Roo Code, and aider work by changing one connection URL. Put a proxy like LiteLLM in front and it slots into almost any existing AI workflow. No requirements document. No kickoff meeting.
Think about hardware in three tiers.
Entry — $10,000–20,000. Quantized GLM-4.5-Air runs on a workstation with two 48GB-class GPUs, or one 96GB-class card. Use it to run real internal tasks for two weeks.
Full scale — a few hundred thousand dollars, or rental from a few thousand per month. Full-size DeepSeek, Kimi K2, or GLM-4.6 wants an 8×H100/H200-class node. Buying one is a capital expense — a depreciable asset. Renting dedicated GPU cloud gets you started with no upfront cost.
The comparison that matters — the consulting quote. “AI strategy engagement,” $200,000, deliverable: a slide deck. You could buy the top-tier GPU server for that and keep the change. Three years later, one of them is still running inference. The other is landfill in a shared drive.
Why Consultants and SIs Will Never Tell You This
If you’ve read this far, one question should be nagging you. If it’s this simple, why is “LLM platform construction” sold as a six-month, multi-hundred-thousand-dollar project?
The answer is structural. Their business model is billable hours times headcount, and a job that ends in one command is unsellable. So it gets inflated. A current-state assessment phase. Use-case definition workshops. Vendor selection support. PoC planning. A steering committee. Search the estimate line by line — you will not find the step that says “type vllm serve.” Even though that step is nearly the entire substance of the work.
Be careful with “let’s start with a PoC.” When the PoC ends, a “next phase proposal” arrives. When that ends, an “assessment for full-scale rollout” arrives. Production never comes — because when production comes, the invoices stop.
The subcontracting pyramid is even cruder. Of the $1,200 per day you pay, the third-tier engineer actually touching the keyboard sees perhaps $250. And what is that engineer doing? Reading the official docs and community write-ups, trying things — exactly what your own junior engineer could start doing today. The delta is pure intermediary margin.
And the most important point of all: outsourcing the construction of your “secure LLM platform” is itself a breach of the security it exists to provide. Network diagrams, authentication design, operational runbooks — the complete blueprint of the system that guards your secrets, passing through outside hands. Building the thing that keeps information inside by handing information outside. As jokes go, it’s not even a good one.
What they sell is not technology. It is the feeling that this must be difficult — and that feeling evaporates after one day of reading documentation.
Security Is Only the Entry Point — Four Things In-House Self-Hosting Changes
1. Total data sovereignty. Prompts, outputs, logs — all of it stays inside. You design the audit trail to your own requirements. The worry about training-data reuse doesn’t get mitigated; it ceases to exist as a concept. When the auditor asks where the data is processed, you point at the floor.
2. An inverted cost structure. API billing is metered — run agents seriously and the bill grows without ceiling. Self-hosting is a fixed cost. Let a coding agent refactor your codebase all night, and the only thing that grows is the power bill. Never having to watch a token meter changes the experimental velocity of in-house development at the root.
3. Compounding know-how. Model selection instincts, quantization configs, internal evaluation suites, prompt assets — all of it accumulates inside your organization. Outsource it, and that knowledge walks out with the final invoice, to be resold to your competitor on the next engagement.
4. Zero dependency. API price hikes, rate-limit changes, sudden model deprecations, terms-of-service revisions — cloud API users live at the mercy of all of them. None of it touches you. The weights are on your disk. Nobody can take them back.
In Honesty: When Self-Hosting Is the Wrong Call
An article that only escalates isn’t credible, so here are the real limits.
GPU operations are real work — monitoring, driver updates, model swaps. But size it correctly: this is “one infra person, part-time,” not “a new dedicated team,” and it is certainly not a reason to sign an operations contract with an SI.
And when data sensitivity is low and you need maximum frontier performance, a cloud frontier model is the rational choice. For most organizations the optimum is hybrid: self-host for confidential data and high-volume workloads; cloud frontier models for public-data work that needs the absolute best. The test is a single question. If this data leaked, would the business bleed? If yes, there is no argument for sending that workload outside.
What to Do on Monday
1. Split your workloads by sensitivity. What may leave the building, and what must not. The second list is your self-hosting scope.
2. Secure one GPU. For evaluation, a single workstation with 48GB-class GPUs is enough. A dedicated cloud GPU instance also works — just verify it’s single-tenant, check the region, and confirm storage encryption.
3. Stand up GLM-4.5-Air on vLLM. Half a day including the model download. One day including the troubleshooting.
4. Connect a coding agent and measure for two weeks on real tasks. Refactoring internal repos, generating tests, fixing documentation. Those two weeks will produce more data than a consultant’s six-month PoC report — more accurate, and free.
5. Write the approval request like this: “Upfront cost: less than one-tenth of the consulting quote. Data leaving the premises: zero bytes. Results: already measured.” No executive has a counter to that.
The Fork in the Road
Right now, the same phrase — “information security” — is splitting into two opposite futures.
One company says “due to information security concerns, generative AI use is postponed for now,” and schedules another review committee. Another says “due to information security requirements, we are going all-in on self-hosting,” and orders the GPUs. A year from now, the gap in development velocity between those two companies will be permanent.
The weights are published. The inference engines are open source. The documentation is free. The GPUs can be ordered today. Between you and serious AI adoption, there is no longer a consultant or an SI standing in the way.
If anything is still standing there, it is only the fear they sold you on the way out.
NeoAnalogLab provides self-hosted LLM platform construction and in-house AI enablement: model selection and evaluation, coding-agent integration, and hands-on buildout — with operations fully handed over to your team. Enablement without dependency. If you want to run AI development without a single byte leaving your network, let’s talk.