Tired of expensive token costs? Try running your AI models locally?

The cost of cloud AI was creeping up, so we went local with open AI models.

For months, our team relied on cloud-based LLMs like GPT 5.0 running on Copilot to automate data cleaning. The initial setup was seamless, but the monthly bills began to spiral unpredictably. We found ourselves paying a premium every time we processed large datasets, with costs compounding as usage scaled.

The decision to pivot wasn't just financial; it was about control. We needed to eliminate recurring token costs whilst retaining full ownership of our data and workflows. By building a simple windows server with an RTX 3090 (that's 24GB of VRAM enough to run most open models comfortably) we are able to run Qwen on Ollama to replicate what we were doing with cloud LLMs. In simple terms we transformed AI from a variable cost into a fixed cost asset [W4].

Fig 1 — We cleared the table tennis table and got to work. A custom machine built from old PCs and some newer bits we got from eBay.

Why cloud APIs stopped making sense for our workflow?

The limitations of cloud solutions became apparent when we examined our operational reality. Cloud APIs offer managed infrastructure and automatic scaling, but they lack the fine-grained control required for sensitive financial data [W3]. More importantly, as output-heavy tasks like code generation or data cleaning scale, token costs compound rapidly compared to input-only queries [W6].

For instance, flagship models charge roughly £15 per million input tokens and £75 per million output tokens [W3]. When you are processing thousands of rows of financial data, these micro-charges add up to a significant operational expenditure (OPEX). We were effectively renting intelligence for tasks that could be handled by our own hardware.

Building the local engine: hardware and software choices.

We opted for a practical, cost-effective build rather than chasing the latest flagship silicon. The core of our setup was a desktop PC equipped with an RTX 3090, chosen for its affordability on the used market and sufficient VRAM to run a quantised model like Qwen [U1]. While newer cards offer higher throughput, the RTX 3090 remains a cost-effective workhorse for specific model sizes [W2].

The software stack was straightforward:

• Hardware: Used RTX 3090 (24GB VRAM) and standard office PC components.

• Inference Engine: Ollama, selected for its ease of deployment on Windows office environments [U1].

• Model: Qwen 3.6, chosen for its balance of performance and resource efficiency.

This approach bypasses the need for complex cloud API integrations, allowing us to keep the inference layer entirely within our office network [W1].

Fig 2 — The final build — the core components we went with.

How we automated Excel cleaning without sending data to the cloud.

The operational workflow is now seamless and secure. We feed Excel spreadsheets directly into the local model, which cleans and codes data according to our in-house guidelines. The most critical benefit is privacy: sensitive financial data never leaves the office network [U1].

Previously, this process had to be done manually or via cloud APIs that required uploading client data to external servers. By hosting the model locally, we gained better control over how the model was used and ensured compliance with strict data protection standards [[W3]]. The users now have easier access to AI capabilities without worrying about data leakage.

The trade-offs: upfront cost versus long-term savings.

Addressing the financial reality honestly, this shift requires a significant upfront capital expenditure (CAPEX). An £700 used RTX 3090 can replace roughly £50–£500 in monthly API costs, depending on usage volume [W4]. For us the break-even point was reached within weeks of deployment.

However, local AI becomes cheaper only after a certain usage threshold is crossed [W12]. If your business uses AI lightly, cloud APIs may still be more cost-effective. But for high-volume, repetitive tasks like data cleaning, the long-term savings are substantial. We also account for electricity and cooling costs, which are negligible compared to the monthly API fees we used to pay [W7].

What we would improve and who this is for?

The RTX 3090 is older technology, and power consumption under load is a factor we monitor. Additionally, local models may lack the 'reasoning' depth of flagship cloud models like Claude Opus for complex strategic tasks [W15]. We would improve our setup by adding more RAM to handle larger context windows in the future.

This approach suits businesses with high-volume, repetitive data tasks where privacy and cost control matter more than frontier reasoning. It is not a replacement for cloud APIs in every scenario; rather, it is a pragmatic choice for daily, privacy-sensitive operational workflows [W16]. For many small businesses, the 'smart play' is a hybrid approach: route routine work locally, reach for the cloud only when complex reasoning is needed [W15].

Taking back control of your AI infrastructure

For small businesses, owning the inference layer offers predictability and security. It transforms AI from a variable cost into a fixed asset that you control completely. While cloud APIs have their place for prototyping or heavy reasoning, local hosting is the pragmatic choice for daily operations where data privacy is paramount [W19].

As the 2026 landscape matures, local LLMs are no longer just a proof of concept; they are a viable, mature alternative that can significantly reduce operational costs and enhance data security [W10]. We have taken back control of our AI infrastructure, and it has paid for itself in full.

Sources

1. Self-Hosted LLMs vs Cloud APIs: Cost & Performance Comparison

2. Free Local AI vs Paid Cloud APIs: Real Cost Comparison

3. Local LLMs vs Cloud APIs: 2026 Total Cost of Ownership Analysis

4. Local LLMs vs Cloud APIs: The 2026 Cost Analysis

5. Local LLMs vs. Cloud APIs — The 5-Year Cost Reality

6. Is Running a Local LLM Cheaper Than Cloud API? A Developer's Cost ...

7. LLM Local vs API Cost Calculator - AgentCalc

8. Local LLM vs Cloud API Cost Comparison 2026 - presenc.ai

9. Own Server or API: LLM Cost Comparison 2026 - momentschool.com

10. Local LLM vs Cloud API Cost: Small Business Guide 2026 | Vucense