The case for bringing AI compute in-house (and on a road trip)

Alex Lisle’s desk plants are in trouble—they’re next to four AI computers that have been running hot lately.

Lisle, CTO at cybersecurity platform Reality Defender, has a quartet of DGX Spark machines, all featuring a Nvidia GB10 Grace Blackwell Superchip. Each workstation hosts a different large language model (LLM) to create what coders call an adversarial loop: models like Qwen3.8b write the code, and DeepSeek 4.1 Flash audits any flaws in the generators’ work, with the aim of improving it.

“They’re all different models and they all hate each other, so it’s effectively just a way of replacing the human to make good code for mundane tasks,” he said, which means jobs like fixing workflows and reports.

Lisle had tried the adversarial loop with cloud-based models and token-based pricing, which led to high costs as the LLMs went eight or nine rounds before concluding.

“That was costing millions of tokens,” he said. “The only reason that adversarial loops work for me is the fact that I can do this without paying for tokens.”

Lisle isn’t the only tech pro considering the local train these days, experimenting with in-house hardware to support AI workflows. But this bring-your-own-AI-device option, which requires hosting open models, has IT pros juggling the cost benefits with security concerns.

Keep it open? Tech companies like Google, Anthropic, and OpenAI offer large-scale models closed from download or modification. A paper from an MIT and Georgia Tech researcher, published in November 2025, found closed models account for 80% of model usage.

Open-source models, however, are free and provide access to:

• Training data and its source
• The source code training and running the model
• The parameters supporting decision-making, or “weights” (“open-weight” AI does not necessarily mean open-source)

Open models running on one’s own hardware eliminate unpredictable, per-token costs. AT&T’s chief data and AI officer told WSJ in August that a quarter of the company’s AI usage now comes from open models. (Other organizations are onboard, too, as IT Brew reported.)

On the road. Parker Johnston, agentic field CTO at solutions integrator Insight, recently chronicled his road trip picking up a brand-new ZGX Fury GB300 from HP’s factory. He used the Fury hardware—a station with a Nvidia Blackwell Ultra​ GPU and 748 GB of coherent memory—to run four large language models (LLMs) simultaneously and 30-plus agents; he burned 11 million tokens in 90 minutes, according to his LinkedIn post.

Johnston sees owning AI hardware as a way to help employees explore beyond the token-based constraints of cloud-based, third-party AI services.

“From 6 pm to 6 am I have agents that can run now completely overnight and give my development team a workload or a capability they never thought possible,” Johnston told IT Brew, citing tasks like vulnerability management or spotting dependency drift, which is when a system uses different versions of the same package. “All of these things—you might shy away from…because one, it takes labor time to do it, and two, it takes tokens.”

While Johnston notes the importance of putting any model behind a corporate network with access controls, he sees local models helping industries handling sensitive information.

“Maybe it’s a bank-loan application, and I can go onto this device, process the information I need, and immediately remove the files. I no longer have to worry about where they’re being kept, what access folks might have, who configured the cloud environment, what’s going on on the other side of it. It’s all for me to now wholly own across that entire landscape,” he said.

AI everywhere. Neil Ward-Dutton, research VP of agentic automation and AI technologies at tech-intelligence firm IDC, sees businesses running models on modest, local hardware options. Tools like Ollama help open-weight models run on a MacBook pro, Chrome includes an on-device LLM, and today’s PCs have specialized GPUs for on-device GenAI tasks, he mentioned.

Ward-Dutton doesn’t envision a future, however, where every employee has an expensive, chunky workstation running their AI: “Do I see a future where enterprises will have AI models running in lots of different places, including on laptops, company phones, local private clouds, on-premise data centers, and so on? Absolutely.”

Security! Executives have raised concerns about open models’ lack of centralized control over their use and modification.

While local AI diminishes some security risks—you keep your sensitive data in-house, for example—Ward-Dutton mentioned in a follow-up email to IT Brew that security pros must monitor access to system resources and protect keys and credentials

Regarding security, Lisle appreciates that open-weight models (unlike SaaS-based models) offer full access to their chain of thought.

“I’m fine-tuning them for my particular use cases, I can reason about them because I see the reasoning chains, the data never leaves my environment, and then the fixed cost. Those are all very powerful indicators of where the market is going,” he said.