用淘汰的 Windows 10 PC 做本地大语言模型推理

Abstract

The Windows 10 end-of-life provides a material incentive to repurpose legacy PCs as on-premises large language model (LLM) inference servers. This study assesses the technical feasibility, economic viability and the environmental and social issues of converting an enterprise workstation with an NVIDIA Graphics Processing Unit (GPU) into a dedicated AI node. Using Proxmox virtualisation, Ollama, and Open WebUI to benchmark several open-source LLMs, this study shows that a repurposed workstation, fitted with a single consumer-grade NVIDIA GPU, can deliver stable inference throughput of 29–65 tokens/s for bounded, conversational coding and document-assistant workflows, while preserving full data sovereignty and eliminating recurring subscription fees. Performance analysis reveals that resource contention and GPU memory allocation are the primary determinants of throughput. Correctness, assessed against three established public benchmarks with gold answers, does not track the same ranking as throughput; combining both axes identifies gemma4:e4b as the strongest model evaluated on this hardware for both speed and correctness, while the slowest model tested was among the most accurate. A cost-benefit model shows local operation is substantially cheaper than mid-range and premium cloud alternatives, and only modestly costlier than the cheapest cloud tier on a per-prompt basis—a gap this study argues is outweighed by the digital-sovereignty benefits of local control, particularly where compliance-driven data-handling constraints apply. The study summarizes key background motivations for on-premises AI—privacy, security, regulatory risk—and estimates that this repurposed workstation's usage-phase CO₂e footprint is lower than each of the cloud alternatives compared, in addition to avoiding the embodied manufacturing emissions of new hardware. Repurposing legacy workstations with readily available open source tools offers a sustainable, cost-effective pathway for organizations to deploy practical LLM inference capabilities locally, providing a strategic alternative to cloud services for entities that prioritize privacy, regulatory compliance or long-term cost containment.

1 Introduction

Microsoft's decision to end support for Windows 10 on 14 October 2025 (Microsoft, 2025b; Kelly, 2023) affects a large installed base. Canalys estimated that roughly one fifth of PCs in use—some 240 million—would be ineligible for Windows 11 and therefore unlikely to be refurbished or resold (Caddy and Jessop, 2023). This is a proprietary industry estimate rather than a measurement: the underlying installed-base data and the basis for the one-fifth proportion are not published, and the authors have found no independent corroboration. It is widely reproduced in subsequent work (Gould et al., 2024), and is used here to indicate scale rather than as a precise quantity. As operating-system vendors phase out legacy systems, an industry-wide surge in surplus hardware is now underway. The IT management company Lansweeper found that, based on an estimated 30 million Windows devices from 60,000 organizations, 42% lacked approved CPUs and 14% lack TPM 2.0 security requirements (Othman, 2025).

Concurrently, the rise of cloud-hosted large language model (LLM) services has introduced a new set of constraints including subscription fees, utilization limits, and external data dependencies (Naughton, 2025). These dynamics converge to create a notable opportunity: repurposing otherwise idle PCs into dedicated AI inference machines. Of course, other options are also available, such as converting laptops and smaller PCs to Linux as a mature and effective alternative to Windows where appropriate.

Benchmarking the performance of LLM inference is itself an active and rapidly growing area of research, spanning both large-scale High Performance Computing (HPC) facilities and consumer-grade hardware. Industry-standard suites such as MLPerf Inference (Reddi et al., 2020) established rigorous, reproducible methodologies for comparing machine-learning hardware and software stacks, and this tradition continues at major high-performance-computing venues. Recent work presented at the International Conference for High Performance Computing, Networking, Storage and Analysis (SC) includes multi-accelerator inference benchmarking across NVIDIA, AMD, and specialized AI hardware (Chitty-Venkata et al., 2024), and predictive characterization of LLM inference-service performance across Graphics Processing Unit (GPU) configurations (Lazuka et al., 2024). Practice and Experience in Advanced Research Computing (PEARC) has likewise documented the acquisition, benchmarking, and power-consumption analysis of GPU-equipped HPC resources purchased specifically to support AI workloads (Bachstein et al., 2024). These efforts share this study's concern with rigorous, reproducible measurement, but target institutional-scale, purpose-built accelerator infrastructure rather than the repurposed, consumer-grade single-GPU hardware considered here. Section 2.5 reviews the literature closest in scope to this study's setting.

Large language models are increasingly central to business, research, and personal productivity, and most organizations reach them through third-party cloud providers—exposing sensitive data to external actors, subjecting workloads to network latency, and incurring subscription costs that matter most at high call volumes. A local AI server offers organizations an alternative that retains full control over data, eliminates subscription costs, and leverages the growing ecosystem of effective open-source models. By converting legacy Windows 10 hardware—equipped with, or easily upgraded to include, commodity GPUs—into on-premises inference engines, organizations can achieve comparable performance to basic commercial cloud offerings while maintaining strict compliance with data-protection regulations.

We acknowledge that cloud AI service and local AI inference are not directly equivalent. Cloud services are backed with large data center resources, offering services beyond the reach of local systems, using very large proprietary models with many features—especially the latest “Frontier” models. Furthermore, stale information in frozen-in-time local AI inference models may not be appropriate where the latest time-sensitive news or direct web searching is required.

Furthermore local models—especially the smallest ones for example deepseek-R1:1.5b used in this evaluation can appear very responsive while generating results with mistakes or completely misunderstanding the context, ignoring parts of the input prompt—a pattern consistent with broader findings that smaller and weaker models score substantially lower on ground-truth-verified reasoning and factuality benchmarks (Muhlgay et al., 2024; White et al., 2024). The other fully local models (e.g., gemma4, qwen3, phi4, mistral, granite) show a more varied picture under the rigorous benchmark evaluation reported in Section 4.3.3: gemma4:e4b and qwen3:14b score highest on code generation (91.5 and 90.2% pass@1), with arithmetic-reasoning scores in line with the rest of the group, while mistral-nemo:12b reaches only 62.8% on HumanEval, the weakest of the throughput-stable group, despite otherwise stable, competitive throughput. It remains imperative to verify that a specific local model is suitable and reliable for a given task, using a rigorous benchmark rather than informal impressions alone.

Linux is a viable alternative operating system for this hardware, but it is a steep incline for less technical users to manage without help, and those invested in the Windows ecosystem or tied to applications such as Photoshop or AutoCAD will find migration challenging at best. Where a Windows 11 conformant replacement is bought instead, the fate of the obsolete PC remains, and repurposing it—with Linux or as an LLM server—comes into play (Othman, 2025). Furthermore, the Proxmox container approach used in this study may be beyond the IT and DevOps skills available to smaller organizations or individuals.

This study's contribution is not a new benchmarking methodology: Section 2.5 shows that rigorous, multi-axis local-inference benchmarking of consumer-grade hardware is already well-established, including studies evaluating cost, capability, and viability against cloud alternatives on hardware comparable to that used here (Khalil et al., 2025; Knoop and Holtmann, 2026; Nguyen and Nguyen, 2025). What these studies do not address is the specific practitioner decision this study responds to: what to do, right now, with a wave of hardware rendered obsolete not by wear or a deliberate purchase decision, but by a single, dated operating-system end-of-life event. That end-of-support date creates a concrete, time-bounded decision point for the owners of those PCs: dispose of otherwise-functional hardware, migrate it to Linux, or repurpose it in place. This study's contribution is to evaluate that third option end-to-end, on hardware that underwent exactly this transition, combining technical feasibility, economic viability, and the e-waste and digital-sovereignty motivations that make the choice worth making in the first place—a combination existing benchmarking literature, focused on hardware performance in isolation, does not provide.

Two narrower findings from the correctness evaluation (Section 4.3.3) are worth noting alongside this central contribution. First, throughput and output correctness do not track each other within this model set: a model identified as reliable by speed and stability alone is not necessarily the most correct for a given task, a result specific to the models and hardware evaluated here rather than a general claim. Second, evaluating reasoning-capable local models against established public benchmarks surfaced several failure modes in scoring chat-served models with EleutherAI's lm-evaluation-harness, arising both from incompatibilities between the harness and Ollama-served chat-completion backends and from the design of the custom task written to bridge them. Each silently produces false negatives rather than raising an error, so none is detectable from the reported scores alone. All are described in Section 4.3.3, with the corrected task published in the replication package (Curington and Lano, 2026a), for other practitioners attempting the same evaluation.

2 Background

2.1 Avoiding e-waste

Repurposing and continued use of devices beyond obsolescence rather than disposal is a highly effective way to reduce both e-waste and carbon emissions, measured here in carbon dioxide equivalent (CO₂e)—all greenhouse gases expressed as the equivalent mass of CO₂.

Oxford University's life-cycle assessment of a standard office desktop and screen measured 40 W in active use, 15 W idle and 2 W after shutdown. That gives approximately 70 g CO₂e per 8-h working day from electricity alone. Over a 6-year service life the whole-life footprint is approximately 621 kg CO₂e. Around 85% of that arises from manufacture and shipping, and only 15% from electricity consumed in use (University of Oxford IT Services Department, 2022).

The same source elsewhere reports 778 kg CO₂e as an annual figure, which its own measurements do not support. The 15% use-phase share would require some 13,900 h of 40 W operation a year, against the 8,760 a year contains. Read as a 6-year total, it agrees with the 621 kg figure used here.

Reusing an existing machine avoids that embodied component entirely, which is what matters for this study. Replacing a working one incurs it again in full.

The 85/15 split should not be read as applying directly to the server evaluated here. A GPU-equipped workstation draws far more power than an office desktop—150 W idle and 410 W under load against Oxford's 40 W active—so its use-phase share is proportionally much larger. The embodied saving is unaffected: it is fixed at manufacture, whatever the machine later draws.

Estimating the e-waste impact, Othman (2025) estimates that globally, the Windows 10 end-of-life (EOL) transition threatens to generate 850,000–1,000,000 metric tons of e-waste through premature disposal of 130–155 million functional computing devices—a single corporate policy decision producing e-waste volumes exceeding the annual generation of major developed economies. Othman bases this estimate on a disposal rate projection of 60% in developed economies, 45% in developing economies, arriving at a global weighted average of 55%. With 130–155 million devices disposed of between 2025 and 2027, and an average PC weight of 6.5 kg, total e-waste approaches one million metric tons. This is a risk rather than a certainty, however: Windows 10 continues to receive Extended Security Updates until October 2027, and repurposing an aging PC—with Linux, or as a local LLM server as considered in this study—offers a non-disposal alternative to the outcome Othman projects.

2.2 Digital sovereignty

Digital sovereignty is the ability of individuals, organizations, and states to control their own data, software, and computing infrastructure, rather than depending on external providers whose jurisdiction and commercial terms lie outside their control (Floridi, 2020). Running open-weight large language models on privately-owned, on-premises hardware increases digital sovereignty along each of these dimensions: model weights and inference execution remain under the operator's own administration, no prompt or response data leaves the local network, and operation does not depend on a third-party vendor's pricing, availability, or terms of service. This concern is already shaping national policy responses to dependence on non-domestic software providers.

This dynamic is already visible in national policy: Microsoft's Windows policy has motivated national governments to take action. For instance, in France, the National Gendarmerie (equivalent to a national police force) transitioned 37,000 desktop PCs to Linux, achieving significant cost savings (Finley, 2013). Furthermore, the French State has issued a directive to all government departments to initiate a similar transition, following the National Gendarmerie's success (Vaughan-Nichols, 2026). The motivation is clearly expressed by David Amiel, France's Minister of Public Action and Accounts, who states

The State can no longer be content with acknowledging its dependence; it must break free from it. We must desensitize ourselves from American tools and reclaim control of our digital destiny. We can no longer accept that our data, our infrastructure, and our strategic decisions depend on solutions over which we control neither the rules, nor the pricing, nor the evolution, nor the risks. The transition is underway: our ministries, our operators, and our industrial partners are committing today to an unprecedented initiative to map our dependencies and strengthen our digital sovereignty. Digital sovereignty is not an option. (France, 2026)

This also overlaps with “device neutrality” concerns, highlighted by a statement from the Free Software Foundation Europe:

Manufacturers, vendors, and platforms controlling devices may perform a gatekeeper function in similar ways to a provider of an internet access connection controlling a gateway to the internet. The more important the device features and components are for end-users, the more entrenched the position of the gatekeeper can become in the digital markets. (F.S.F.E., 2026)

2.3 Privacy and chat surveillance

Beyond operating system choices, the use of cloud AI tools raises privacy concerns. AI prompt interaction and web-search summarisation tools expose far more personal information than traditional web searching does (Schneier, 2025). In the era of surveillance capitalism, these systems provide a new goldmine for cloud providers, enabling them to gather highly granular data about individuals (Schneier, 2025). Opt-out mechanisms for this kind of data harvesting are often limited or absent. For example, Meta has announced that data collected from user interactions with its AI products will be used to sell targeted advertisements—via a privacy policy update—without providing an opt-out for users worldwide, except in some regions where such an opt-out is legally required (Zeff, 2025).

Keeping AI inference local ensures that sensitive information remains within local systems, eliminating the need to transmit data to external servers. This is particularly important for healthcare and financial organizations that handle confidential data, and anyone that must comply with strict data privacy regulations. Regulations such as the European General Data Protection Regulation (GDPR) impose data handling responsibilities on businesses, whilst private home users also benefit from privacy-oriented AI environments (Kivimäki, 2025).

Operating a private, secure server on-premises—whether at home or in the office— eliminates the need to share data with external providers. Recent hardware and software advances have made it both feasible and cost-effective to host an AI server locally. Importantly, a local AI system does not query the internet to answer questions; it relies solely on its pre-trained model data and, in its standard configuration, requires no network connections.

2.4 Non-local alternatives

Some privacy concerns associated with cloud services can be addressed through intermediary layers that anonymise prompts. One such service is “Duck.AI”, which aims to minimize privacy concerns by routing user prompts through a managed Application Programming Interface (API) to one of four mainstream AI cloud models (Wolfe et al., 2024).

Another approach is Google's NotebookLM, based on Gemini AI cloud service. This service offers a privacy policy that prohibits training on or repurposing user data; its interface organizes projects into notebooks containing user-uploaded reference material (Rahman et al., 2025).

However, neither alternative fully alleviates privacy concerns, nor do they address the repurposing of PCs lacking Windows 11 compatibility or enable unbounded local use.

2.5 Benchmarking methodologies

Large language model (LLM) deployment is shifting toward localized, cost-effective inference. Evaluation of local LLMs requires balancing usage methods with hardware constraints. This section reviews related contemporary approaches to local AI benchmarking and empirical performance evaluations on consumer-grade hardware.

2.5.1 Runtime environment

Evaluating localized LLM performance necessitates structured frameworks that move beyond simplistic, single-axis metrics. Stuhlmann et al. (2025) introduced Bench360, a comprehensive framework benchmarking local inference across four core tasks—General Knowledge and Reasoning, QA, Summarisation, and Text-to-SQL—spanning multiple hardware configurations and inference engines. Their findings emphasize that because no single hardware-engine configuration dominates, evaluating local deployment requires a multi-faceted methodology. Similarly, the benchmark design in this study is expanded to cover ten prompt dimensions.

To address operational consistency, recent studies emphasize statistical repetition and prompt diversity. Investigating highly constrained environments, Nguyen and Nguyen (2025) evaluated 25 quantised open-source LLMs on single-board computers (SBCs) utilizing the Ollama (2026a) ecosystem. Crucially, their methodology mitigates run-to-run variance by executing each prompt four times per model-device pairing to calculate stable mean token generation speeds, while deploying varied prompt workloads to simulate realistic operational diversity. Adopting the same approach for this study, five repetitions and ten prompts are used to measure performance mean and variance.

Beyond raw token speed, modern benchmarking frameworks incorporate holistic hardware metrics. Renney et al. (2026) expanded edge inference evaluation by establishing a rigorous framework tracking throughput, Time-to-First-Token (TTFT), and energy consumption (measured in MJ per million tokens) under strict generation caps to ensure baseline comparability. If evaluations require greater detail, this approach is considered for future work.

Furthermore, local deployment evaluations increasingly combine both computational efficiency and model intelligence. Khalil et al. (2025) examined the viability of private LLM servers for small-to-medium businesses (SMBs) across a dual-axis framework that measures both server performance under load and localized model capability. By leveraging standardized public benchmarks, single consumer GPU and single LLM (Qwen), their approach enables direct comparisons between on-premises open weights and proprietary cloud ecosystems. This result validates the goals of this study, here expanded to cover a suite of LLMs.

2.5.2 Cost estimation

The financial and operational sovereignty advantages of self-hosted inference have driven extensive evaluation of consumer-grade hardware. Knoop and Holtmann (2026) executed a large-scale evaluation of consumer Blackwell GPUs across 79 distinct configurations, modifying quantisations, context lengths (8k–64k), and high-concurrency workloads. Their rigorous cost model demonstrates that self-hosted consumer infrastructure limits operational costs to electricity consumption ($0.001 to $0.04 per million tokens), yielding a 40–200 × cost reduction compared to budget cloud APIs and achieving hardware break-even within 4 months. This result confirms viability and cost efficiency characteristics found in this study.

To review consumer-tier capabilities within the broader architectural landscape, Li et al. (2024) provided a comprehensive hardware perspective on LLM acceleration. Their survey highlights that FP16 performance scaling is predominantly dictated by the volume and generational tier of a GPU's Tensor Cores, noting substantial performance disparities between tiers, such as an RTX 4090 desktop achieving 110 tokens/s on Llama-2-13B compared to 33 tokens/s on a laptop-grade RTX 4070. While using alternative related GPUs than this study, it confirms broadly similar results.

For the specific architectural class of mid-range NVIDIA Ampere class GPUs, empirical industry benchmarks complement academic studies. Testing consumer-grade hardware within popular deployment stacks, an empirical evaluation by DatabaseMart (2026) demonstrated that an RTX 3060 Ti with 8 GB Video random-access memory (VRAM) running Mistral (7B) via Ollama consistently achieves over 70 tokens/s while maintaining 98% GPU utilization. This demonstrates that sub-13B parameter models can achieve highly competitive, low-resource efficiency on widely accessible consumer hardware, using less VRAM than this study (utilizing 12 GB VRAM).

A review of the current literature highlights a clear consensus: local LLM benchmarking cannot rely on a single run, single prompt type, or isolated metrics. Existing frameworks successfully combine multi-run statistical averages, diverse workload dimensions, and energy efficiency—to properly characterize local performance. Correctness is a further axis this study treats as inseparable from throughput: a response generated quickly but incorrectly has no practical value, so Section 4.3 reports both throughput/stability and output correctness for the same set of models, rather than substituting one for the other. This study builds upon these insights methodologically (multi-run statistical averages, multiple prompt dimensions), while differing in aim: none of the studies reviewed above evaluate hardware specifically because it was rendered obsolete by an operating-system end-of-life transition, nor connect their performance findings back to the e-waste and digital-sovereignty motivations discussed in Sections 2.1, 2.2. This study's contribution is that connection, not a new benchmarking technique.

2.6 Trade-offs

This section examines what it means in practice to run a local AI server, assessing the key trade-offs as summarized in Table 1. An return on investment (ROI) comparison may begin with typical LLM API subscription costs (NanoGPT, 2025) vs. time and effort required to reconfigure hardware.

Table 1

Local LLM trade-offs, listing key advantages and disadvantages.

Several of these trade-offs merit further comment. Maintenance and system uptime sit opposite each other across the two columns: an on-premises operator controls the timing of hardware upgrades, OS patching, and software updates rather than being subject to a vendor's release schedule, but in exchange takes on full responsibility for keeping the system running, with no service-level agreement or built-in redundancy in the event of local hardware failure, network loss, or power outage. Similarly, the institutional-knowledge trade-off cuts both ways: building and operating this stack in-house develops transferable AI-infrastructure expertise across an organization, which a cloud subscription does not, but it requires an ongoing investment in staff training and skills maintenance that a managed cloud service is designed to remove entirely.

3 Local pathways

3.1 Dedicated server—repurposing spare computing systems

Utilizing an enterprise workstation removed from service owing to the Windows 10 end-of-life (Gould et al., 2024) issue, very little hardware modification is needed other than a memory upgrade. For this study, a typical workstation, widely available from refurbished system suppliers, is paired with a consumer-grade second-hand GPU.

3.2 Host OS options

Given a suitable desktop computer with a consumer GPU, such as an NVIDIA RTX with sufficient VRAM, it is possible to install a local AI environment directly within the host operating system (Linux, MacOS or Windows) without requiring low-level virtualisation (such as Proxmox). Proxmox virtualisation imposes the least overhead, with the entire GPU dedicated to exclusive use by the LLM.

Installing Ollama (2026a) on Linux, where care is taken to avoid concurrent applications, imposes less than 2% overhead over Proxmox; offering the significant advantage of low administrative complexity.

Resource contention is the principal drawback in operating LLMs concurrently with Windows. On a Windows 11 workstation, 0.3–0.8 GB VRAM is consumed at idle prior to loading any LLM, compared with near zero on the headless Proxmox configuration, a difference attributable to the Windows Desktop Window Manager and CUDA display context (Microsoft, 2025a).

In most cases the critical factors are the amount of dedicated GPU video memory (VRAM) and the selection of an LLM that fits within that constraint, as it must accommodate the LLM weights and context window for reasonable performance.

3.3 Multi-GPU configurations

For organizations requiring inference capacity beyond that of a single GPU, two broad architectural approaches exist: multiple GPUs within a single workstation, and distributed inference across multiple networked machines. Each carries distinct trade-offs in cost, complexity, and practical accessibility.

Single-chassis multi-GPU configurations are well-established in home-lab and small enterprise contexts, with two to four GPUs being common. The principal hardware requirement is a sufficiently large chassis and a motherboard providing multiple full-bandwidth PCIe × 16 slots with adequate physical spacing between cards. Modern inference frameworks, including Ollama and llama.cpp, are capable of distributing model layers across multiple GPUs effectively, with VRAM pooled transparently. The primary internal bottleneck is the PCIe bus itself; higher-end NVIDIA data center GPUs address this through NVLink, a proprietary high-bandwidth direct interconnect. However, NVLink is not available on consumer-grade GeForce GPUs of the type used in this study, meaning PCIe remains the sole interconnect pathway at this hardware tier.

Distributing inference across multiple separate machines is technically feasible but substantially more complex. The critical constraint is inter-node bandwidth: standard Gigabit Ethernet is wholly inadequate for the continuous high-volume tensor data transfers required during inference, and specialized high-bandwidth networking —such as InfiniBand or high-speed RDMA-capable Ethernet—becomes a prerequisite. The tooling required to orchestrate distributed inference across nodes is also considerably more specialized than single-machine deployments, placing such configurations outside the reach of most small organizations without dedicated infrastructure expertise or external specialist support. Recovering idle compute across machine fleets is a well-established discipline in high-throughput computing, using systems such as HTCondor (Thain et al., 2005) or the Open Science Grid (Pordes et al., 2007) to schedule batch and parallel jobs across clusters of shared resources, with queuing, priority, and fault-tolerance mechanisms. This addresses a different problem to the one considered here. The single-user, single-workstation deployment in this study serves interactive prompts directly, with no contention or batch scheduling to manage. Such tooling would only become relevant if inference were scaled toward the distributed, multi-machine configuration discussed above.

For the majority of organizations considering local LLM deployment, the single-chassis multi-GPU approach represents the more practical scaling path. Concentrating capital expenditure on the highest available VRAM capacity within one machine—whether one, two or four GPUs—yields the best performance-to-complexity ratio, avoids the networking constraints of distributed deployment, and remains manageable with the same toolchain described in this study.

3.4 Cloud vs. local cost analysis

The financial cost of running a local AI server may be compared with that of using a similar cloud-hosted service. Since usage patterns of each could vary considerably, a comparison is made using the specific local study configuration described in this paper. The workstation itself was already available, having been freed by the Windows 10 to Windows 11 transition, so the comparison below considers ongoing energy use rather than the cost of the machine. One component was purchased: the second-hand NVIDIA RTX 3060 was acquired for this work at approximately $282 (£210). That outlay does not alter the comparisons that follow. Against the midrange and premium cloud tiers it is recovered within roughly 3 months of the annual savings reported below, and against the cheapest tier it widens a gap that local operation already loses on energy alone. Being second-hand, it also carries no new embodied manufacturing footprint, the asymmetry discussed in Section 2.1. In particular, we consider if the energy cost of running a local AI server is more or less expensive than using a remote commercial AI cloud service.

Assumptions made about the local AI server workstation, used by a single researcher in a domestic setting are shown in Table 2. The energy cost figure is the UK domestic electricity unit rate under Ofgem's regulated price cap for Q3 2026: 26.11 pence/kWh, inclusive of VAT (Ofgem, 2026). It is converted to USD at the 2026 average GBP/USD exchange rate of 1.3443. A regulated rate is used in place of a single household bill so that the analysis reflects a nationally representative tariff rather than one property's.

Table 2

Local LLM usage and cost assumptions, used as the basis for energy and cost analysis.

Server power draw was measured directly rather than estimated, using a Lowenergie plug-in mains energy monitor at the wall socket. The instrument is rated to ±2 W below 100 W and ±2% from 100 to 3,000 W (BS 1363 / BS 5733). It recorded 150 W with the machine idle and 410 W as a sustained average under inference load. That tolerance is approximately ±8 W at the loaded figure, or about ±2% on the derived energy cost. Both are small beside the uncertainty in the usage assumptions themselves.

These are whole-system figures at the wall. They cover the dual-processor workstation, its memory and storage, and the GPU together, rather than a GPU-only figure or a sum of component ratings. The comparison in Section 2.1 is therefore like for like: Oxford's 40 W is also a whole-machine active measurement.

The average of 500 prompts per day represents a moderately heavy single-user workload, set toward the high end of plausible personal usage so as not to underestimate the local operating cost: spread evenly across the 10-h working day, it corresponds to roughly one prompt every 72 s, consistent with an AI-assisted workflow combining coding, editing, and documentation tasks as described in Section 4, rather than isolated occasional queries. The 40-s loaded-time-per-prompt estimate approximates the generation time for the assumed 1,500-token response length; this is consistent with the overall mean throughput of the six production-ready models characterized in Section 4 (approximately 43 tokens/s across the reported prompt types), which would generate a response of this length in approximately 35 s, validating 40 s as a reasonable, slightly conservative round-number estimate.

For this estimate we assume the workstation is powered only during working hours on weekdays. Counting idle power requirements plus additional load when the local AI is evaluating a prompt yields a power estimate. Assuming daily use at these activity levels, the local energy cost can be compared with cloud service costs for the same prompt-response activity. To provide a range of potential cloud costs, three models are considered—a small efficient open-source model (matching the local AI server), a midrange commercial model, and a premium top-end model, as detailed in Table 3. The cost for these is pay-as-you-go based on the number of prompts and the tokens per prompt.

Table 3

Cost metrics of representative cloud LLMs, comparing small, medium and large models with local energy spending.

Per-prompt pricing is sourced from NanoGPT's aggregated multi-provider pricing table (NanoGPT, 2025), Accessed 2025-09-26.

The local AI server energy cost for 1 year is $207 USD. If powered 24-h a day rather than working hours, the cost increases to $354 per year. In comparison, for an organization making 100,000 prompts per year, the small model's cloud cost ($48.75) is approximately four times lower than the local server's electricity footprint. However at midrange, the economic dynamic flips; the cloud API cost ($1,562) is approximately 7.6 times greater than the local energy cost, yielding over $1,350 in annual operation savings. At the premium end, the cloud cost ($12,000) becomes severely prohibitive, at approximately 58 times greater the cost of local electricity.

Although the small model remains cheaper in principle, and Gemini 3 Flash may also match or exceed its raw capability, cost and capability are not the only factors that determine which option is appropriate. This is corroborated by the providers' own published evaluations—Gemini 3 Flash reports 90.4% on GPQA Diamond and 78% on SWE-bench Verified (Google, 2026), and Claude 4 Opus reports strong results on SWE-bench Verified and Terminal-Bench (Anthropic, 2025)—though a direct, benchmark-matched comparison with this study's own GSM8K/HumanEval/TruthfulQA results (Section 4.3.3) is not possible: both providers have moved to a harder, more recent benchmark suite for these model generations, and neither publishes scores on the classic GSM8K/HumanEval/TruthfulQA battery used here. The cited figures are offered only as independent evidence that these are highly capable contemporary systems, not as a like-for-like score comparison. This holds even more strongly against flat-rate subscription products such as Claude Pro, which for a suitable usage pattern can be more capable and cheaper still than the metered comparison used here, as discussed above.

As argued in Section 2.2, local deployment increases digital sovereignty along three dimensions—who controls model execution, whether data leaves the local network, and independence from a vendor's pricing, availability, and terms of service—none of which any cloud alternative, however cheap or capable, can substitute for where privacy, regulatory compliance, or long-term cost predictability are the decisive requirements rather than raw performance-per-dollar. This argument is deliberately a risk- and values-based one rather than a quantified cost comparison: unlike the dollar and carbon-footprint figures elsewhere in this section, sovereignty's value has no natural common unit, and its practical weight will vary by reader according to their own regulatory exposure and risk tolerance. Section 2.2's examples—compliance obligations under GDPR, and national governments' own policy responses to vendor dependence—illustrate that this weight can be substantial and sometimes carries real, quantifiable stakes (compliance penalties, vendor lock-in switching costs), even though this study does not attempt to price it directly.

For use cases where these constraints do not apply, a cloud option may indeed be the more sensible choice; this study's contribution is characterizing when the local alternative becomes viable and preferable, not asserting that it always is. Note that the full functionality of the midrange and premium models cannot be replicated on low-cost local AI server hardware. Ongoing energy cost of running the workstation is negligible compared with capital expenditure or maintenance services—factors not considered in this evaluation.

The return on repurposing therefore depends on which cloud tier is the realistic alternative. At a 500-prompt daily average and a 40-s prompt execution time, local operation is substantially cheaper than the midrange tier (approximately 7.6 times) and the premium tier (approximately 58 times), saving over $1,350 a year against the former. Against the cheapest metered tier it is not: that option remains roughly four times cheaper than the local server's electricity alone, and remains cheaper even at maximum utilization. The case for repurposing rests on the midrange and premium comparisons, and on the sovereignty, privacy and carbon considerations set out elsewhere in this section, rather than on beating every cloud option on price. Which of these applies depends on usage patterns, hardware availability, and the weight given to data privacy and control.

A further scenario worth quantifying is maximum theoretical utilization: if the workstation were operated continuously at full load with zero idle time, at the 40-s loaded-time-per-prompt rate above, it would process 86, 400s/40s = 2, 160 prompts per day, or approximately 788,940 prompts over a 365.25-day year. At full load continuously (410 W, no idle contribution), this yields an annual electricity cost of approximately $1,262—compared with approximately $394 for the same prompt volume via Gemini 3 Flash (Table 3), so cloud pricing remains roughly 3.2 × cheaper even at this extreme utilization. At this scale, however, the per-prompt cost is dominated almost entirely by the electricity rate itself, since idle overhead becomes negligible: the same scenario breaks even against Gemini 3 Flash's pricing at an electricity rate of approximately $0.11/kWh, well below the UK rate used above and achievable in many lower-cost electricity regions. This scenario is not far-fetched for batch or semi-supervised learning workloads that saturate hardware continuously rather than following an interactive, working-hours usage pattern, and suggests that sustained, high-utilization local deployment can become cost-competitive with even the cheapest metered cloud tier where electricity is inexpensive.

A further consideration raised in Section 2.1 is the carbon footprint of local vs. cloud inference, distinct from cost. Applying the UK government's published grid electricity carbon-intensity factor (0.196 kg CO₂e/kWh, generation and transmission losses combined) (Department for Energy Security and Net Zero, 2025) to the local server's annual usage-phase electricity consumption at the same 100,000-prompt/year volume (589.7 kWh) gives an estimated 115 kg CO₂e/year. The equivalent figure for each cloud model was estimated using EcoLogits (Rincé and Banse, 2026), an open-source methodology that models per-request energy and emissions from model architecture, measured token throughput, and each provider's data-center location, power-usage effectiveness, and electricity mix. At this usage volume, the resulting estimates are approximately 170 kg CO₂e/year (range 113–228) for Gemini 3 Flash, 1,132 kg CO₂e/year (range 674–1,590) for Gemini 2.5 Pro, and 1,387 kg CO₂e/year (range 890–1,884) for Claude 4 Opus—meaning the local server's own usage-phase footprint is lower than even the cheapest cloud tier compared here, by a factor of approximately 1.5 ×, rising to roughly 10–12 × against the midrange and premium tiers. These figures are collected in Table 4.

Table 4

Estimated annual usage-phase carbon footprint of the local server compared with the three representative cloud models from Table 3, at the same 100,000-prompt/year volume (Table 2).

Cloud figures are computed using the EcoLogits (Rincé and Banse, 2026) emissions-estimation methodology; the local figure uses the UK government's published grid electricity carbon-intensity factor (Department for Energy Security and Net Zero, 2025).

Several caveats apply to these cloud-side estimates. None of the three providers discloses the underlying model architecture; EcoLogits therefore substitutes a generic parameter-count estimate for closed models (Claude 4 Opus and Gemini 2.5 Pro receive an identical modeled architecture, flagged by the tool itself as a low-confidence substitution), and Gemini 3 Flash has no production-model entry in EcoLogits' database at the time of writing, so the estimate instead uses its preview-release proxy entry. Data-center location, power-usage effectiveness, and water-usage effectiveness are likewise provider-level defaults rather than measurements of the specific request path. These estimates should therefore be read as order-of-magnitude comparisons rather than precise figures.

The comparison also omits embodied manufacturing footprint on both sides: Section 2.1's life-cycle finding that a typical desktop attributes roughly 85% of its lifetime emissions to manufacture and shipping (University of Oxford IT Services Department, 2022) applies in full to any new cloud-side hardware, whereas this study's local server incurs none of that footprint, since it reuses hardware that already existed—an asymmetry omitted above that would only widen the gap further in the local server's favor. Both the cloud pricing in Table 3 and the EcoLogits model-database entries used here are snapshots as of their respective access dates (2025-09-26 and 2026-07-30); cloud providers revise pricing and release new models frequently, so these comparisons should be re-verified against current figures before being relied upon for a deployment decision made substantially after this study's publication.

The cloud-cost comparisons above use metered, pay-as-you-go API pricing, which is the appropriate baseline for this study's steady, scripted prompt volume. Consumer-facing flat-rate subscriptions (e.g., monthly AI assistant plans) represent a different commercial tier: providers price these against expected average usage pooled across many subscribers, so a heavy user can consume many times their subscription cost in equivalent metered-API value before hitting the fair-use or rate limits such plans typically carry. Flat subscriptions can therefore be substantially cheaper than metered API pricing for the right usage pattern, but are bounded by those limits and are not designed for the sustained, scripted, high-volume workloads modeled in this study.

4 Evaluation methodology

This study establishes AI inference servers from scratch, using a custom benchmark suite (Curington and Lano, 2026a). The techniques and tool-chain landscape described here are evolving rapidly; the authors acknowledge that this example may be out-of-date by the time of publication. The workstation used for the evaluation was used for many years as both a windows desktop and for virtual Linux services. After wiping the Windows 10 boot disk, a second-hand consumer GPU was added. This dedicated AI server hardware is a Dell Precision T7810 tower workstation, dual Xeon 12-core CPUs (48 threads) at 2.6 GHz, 192 GB ECC RAM, 1 TB NVMe SSD storage and a single NVIDIA GeForce RTX 3060 GPU with 12 GB VRAM. There are no monitors connected to this server, as the GPU is purely used for computation. This study acknowledges this premium enterprise class workstation is more powerful than a typical obsolete consumer PC, however the server CPUs, memory and PCIe bus advantages are minimal as most LLM computation is expected to occur on the GPU.

The server is organized as illustrated in the Figure 1 diagram, Proxmox 9 with LXC virtualisation, supporting Ollama and Open WebUI LLM components are established following published guides (Srinivas et al., 2025). Proxmox is the base virtualisation control layer, supporting multiple instances of services and applications. Multiple lightweight virtual Linux containers (LXCs) are enabled within Proxmox. One LXC container is configured to host Ollama, which is responsible for maintaining the local LLM model collection and dynamically responding to input prompts—returning LLM output to the requesting process. Open WebUI (Open WebUI, 2026) serves client-side front-end clients for controlling Ollama using a second LXC container.

Figure 1

The principal resource constraint is the 12 GB VRAM, so LLMs are selected specifically to fit this level. The GPU has a maximum rated power consumption of 170 W; it was capped at 140 W throughout, so all throughput figures reported here were obtained under that cap. The cap's effect on throughput was not separately quantified, since no measurements were taken with it removed. Such a server therefore does not require extraordinary power consumption for a domestic setting. This study's results (Section 4.3.1) confirm 12 GB VRAM as sufficient for models up to approximately 14 billion parameters at four-bit quantisation. It is not a minimum: 7B-class models are reported to run well on 8 GB (DatabaseMart, 2026). VRAM scales with model size, quantisation, and context length.

The server is accessed from Microsoft Visual Studio Code for code development (Falkingham, 2025), and through a web browser to Open WebUI for chat sessions.

Model functionality is critically constrained by finite hardware resources: in practice the two key computational constraints are context window size and the physical memory allocated for model weights (Na et al., 2024). The server offers a basic but sufficient GPU, as well as adequate CPU and memory to run a range of models, although tuning, not considered in this evaluation, may have a significant effect. Section 4.1 treats both constraints in detail, including the conditions under which a model that appears to fit within GPU memory nonetheless spills onto the CPU; the effect is visible in the evaluation results below.

4.1 Context constraint

The physical VRAM capacity of the local GPU is the primary constraint on LLM operation in practice. Models whose weight size exceeds available VRAM spill over to host system RAM and CPU computation, incurring a severe performance penalty. Selecting a model whose weights fit entirely within VRAM is therefore the most critical first step in configuration.

A second, less obvious constraint is the token context window (Na et al., 2024). The key-value (KV) cache that stores the active context also resides in VRAM, and its size scales linearly with context length. Ollama defaults conservatively to a 4,096-token context window regardless of what the underlying model architecture supports. For example, Gemma4:e4b supports context lengths up to 128k tokens, yet Ollama will use only 4k unless explicitly reconfigured.

Table 5 illustrates the VRAM allocation for Gemma4:e4b at three context lengths on a 12 GB GPU. Model weights consume 9.6 GB and the Ollama runtime approximately 0.4 GB, leaving roughly 2.0 GB available for the KV cache. At the default 4k context the KV cache requires approximately 0.5 GB, leaving a comfortable 1.5 GB margin. Doubling the context to 8k raises the KV cache to approximately 1.0 GB, reducing the free margin to 1.0 GB—the minimum recommended for stable operation when attaching source files or documents as context. Increasing to 16k tokens exhausts the remaining headroom entirely, placing the system at the spill threshold. Unlike model loading, where VRAM exhaustion is immediate and predictable, context-driven spilling occurs mid-session as the context window fills, making it difficult to detect without active monitoring.

Table 5

VRAM allocation for Gemma4:e4b on a 12 GB GPU at three context lengths.

Model weights (9.6 GB) and Ollama runtime (~0.4 GB) are fixed; only the KV cache scales with context length. KV cache sizes are approximate. The 8k configuration is empirically confirmed to remain within GPU VRAM; 16k sits at the spill threshold. Spilling occurs mid-session as context fills, not at model load time.

Practitioners should therefore treat context length as a tunable parameter requiring explicit validation rather than accepting the Ollama default uncritically.

4.2 Structured 10-prompt design

To evaluate LLMs across a range of production activities, a suite of prompts are designed to exercise key dimensions across: Factual retrieval (look up trained knowledge); Output length (directly determines total token generation time); Reasoning depth (as “thinking” models use far more tokens); Context/input length (stresses memory bandwidth and KV cache); Task type (various decoding patterns such as code, prose, and structures); Domain (knowledge retrieval vs. reasoning); Instruction complexity (multiple constraints to stress instruction-following).

4.2.1 Warm-up prompt (disregarded in stats)

“Write a simple Python function that calculates the factorial of a given integer. Omit explanations, bare code only.”

After this initial warm-up, the structured prompts (P₁–P₁₀) shown in Table 6 are applied.

Table 6

LLM capability test suite: ten prompt workload dimensions (P₁–P₁₀) used for inference benchmarking.

Dimensions are grouped by the primary inference behavior under stress. P₉ token generation rate is recorded independently at turns 1, 3, and 5 (denoted P₉-T₁, P₉-T₃, P₉-T₅) to quantify KV cache accumulation effects.

4.3 Results

This study evaluates local LLM deployment along two axes treated as equally important throughout: inference throughput and stability under realistic workloads (Section 4.3.1), and output correctness against established public benchmarks with gold answers (Section 4.3.3). Neither axis alone characterizes practical utility as a workflow assistant, so both are reported together and interpreted jointly. Table 7 presents throughput and stability first, since these determine which models are viable candidates at all; Section 4.3.3 then reports correctness for the same set.

Table 7

Quantitative evaluation of local inference performance across all ten benchmarked prompt dimensions (N = 5).

Processing velocity () is in tokens per second; execution time () in seconds. Statistical dispersion (σ) reflects variance across five independent runs on the evaluation host (Dell-T7810; NVIDIA GeForce RTX 3060, 12 GB VRAM), using Ollama v0.24.0 (Ollama, 2026b). Quantisation levels are the defaults Ollama serves for each unmodified model tag. Full per-run data, including per-turn multi-turn detail (P₉-T₁, P₉-T₃), is deposited as described in the Data availability statement.

4.3.1 Evaluation of inference performance and stability

4.3.1.1 Measurement procedure and settings

Eight open-weight models were benchmarked across ten workload dimensions (P₁ through P₁₀). Two were included as boundary cases: deepseek-r1:1.5b, whose limited reasoning capability informal use had already indicated, and gpt-oss:20b, whose 20-billion-parameter weights would clearly exceed the 12 GB VRAM budget. They establish the boundaries within which the remaining six perform reliably. Performance metrics were captured directly from the Ollama daemon (eval_count and eval_duration), eliminating network and inter-process overhead. Each model underwent an unrecorded warm-up run to promote complete VRAM allocation, followed by five (N = 5) timed trials. Table 7 reports the mean token generation rate () and its standard deviation (σ_TPS), with mean wall-clock elapsed time () per prompt.

Figure 2 shows the same means ranked by throughput, with error bars giving the standard deviation of the per-prompt means. Two features are clearer there than in the tabulated figures. The eleven-fold spread between fastest and slowest is dominated by a single outlier rather than distributed across the group, and the six stable models span only a factor of two. Within that cluster, on-disk footprint does not predict throughput: the largest model of the six (gemma4:e4b, 9.6 GB) is also the fastest, and the smallest (granite4.1:8b, 5.3 GB) is second.

Figure 2

All throughput measurements were taken with Ollama's per-model default generation settings. The benchmark client issues chat requests without an explicit options payload, so sampling temperature, context window, and any reasoning or “thinking” behavior follow each model's own packaged defaults rather than values chosen for this study. This reflects the configuration a practitioner obtains by pulling a model and running it unmodified, which is the deployment scenario this study addresses. One consequence follows and is stated explicitly. Reasoning-capable models are not constrained to minimal output on short prompts. Measured at default settings, gemma3:12b, granite4.1:8b, phi4:14b and mistral-nemo:12b answer P₁ in 24–52 generated tokens, whereas qwen3:14b and deepseek-r1:1.5b emit 558 and 508 tokens, respectively for the same one-line answer, and gemma4:e4b 71. P₁ therefore constitutes a minimal-output baseline only for the four models that do not reason by default, and the P₁ figures for the other three should not be read as measuring minimal-output throughput. On P₅, the context-saturating summarisation prompt, the same models receive approximately 3,290 prompt tokens and generate between 89 and 631: granite4.1:8b 89, phi4:14b 124, mistral-nemo:12b 161, gemma3:12b 177, deepseek-r1:1.5b 382, qwen3:14b 429 and gemma4:e4b 631. The ordering is not that of P₁: the two models that reason by default are no longer the most verbose, because a task genuinely requiring a long answer elicits one from the others too, so the penalty those models pay is specific to short prompts rather than general. Counts are means of three runs at default settings; gpt-oss:20b could not be served at the time these were measured. The correctness evaluation of Section 4.3.3 used explicit settings by contrast—greedy decoding, chat template applied, one request at a time, and a generation budget stated in that section—because those benchmarks require deterministic scoring. The context window was left at Ollama's default there as well.

Three terms are used throughout with specific meanings. A model is stable if its token-generation rate varies by less than 2.0 tokens/s (σ_TPS) across all twelve measurement points, the criterion that separates the six GPU-resident models from gpt-oss:20b, whose spilling produced a coefficient of variation as high as 0.43. A model is GPU-resident if its weights and KV cache fit entirely within VRAM at the tested context length, without CPU offload. Production-ready denotes the conjunction of these two properties and is a claim about throughput and stability alone; Section 4.3.3 shows that output correctness varies substantially within the set of models meeting it.

4.3.1.2 Production-ready cluster: stable, GPU-resident models

Six of the eight evaluated models— gemma4:e4b (Google), gemma3:12b (Google), qwen3:14b (Alibaba), granite4.1:8b (IBM), phi4:14b (Microsoft), and mistral-nemo:12b (Mistral)— exhibit characteristics consistent with dependable production deployment on throughput and stability grounds. Each fits entirely within the 12 GB VRAM budget of the evaluation GPU, and this containment is directly reflected in their statistical stability: σ_TPS remains below 2.0 tokens/s across all twelve measurement points for every model in the group. Throughput scales inversely with parameter count within this tier, from qwen3:14b and phi4:14b at approximately 29–36 tokens/s, through gemma3:12b and mistral-nemo:12b at 34–46 tokens/s, to granite4.1:8b at 51–58 tokens/s and gemma4:e4b at 59–65 tokens/s (ranges over the ten single-turn prompts; the multi-turn prompt P₉ is treated separately below)—a pattern consistent with established GPU memory-bandwidth constraints on token processing.

4.3.1.3 Degradation under resource pressure

Performance degrades predictably and modestly under the two principal resource stresses tested. Under the long-context summarisation workload (P₅), which maximizes KV cache occupancy, throughput decreases by 7%–12% relative to factual retrieval (P₁) across all six models, from gemma4:e4b's 63.4–59.1 tokens/s (−7%) to granite4.1:8b's 57.9–51.1 tokens/s (−12%). Under multi-turn dialogue accumulation (P₉), throughput declines by 6%–10% from turn 1 to turn 5, reflecting the progressive growth of the KV cache as conversational context fills the available context window. Both degradation patterns are smooth, bounded, and predictable—characteristics essential for operators planning hardware capacity for production workloads.

4.3.1.4 deepseek-r1:1.5b: a speed outlier with important caveats

deepseek-r1:1.5b is the performance outlier of the evaluation, achieving 190–201 tokens/s—approximately three to four times the throughput of the next fastest model. This result reflects the model's very small parameter count of 1.5 billion, which occupies proportionally less GPU memory bandwidth per forward pass. However, the same small parameter count imposes a correspondingly limited knowledge capacity, constraining the factual depth and contextual reasoning the model can reliably access. The R1 architecture employs extended chain-of-thought reasoning whose reliability is known to be highly sensitive to model scale; at 1.5 billion parameters, outputs on knowledge-intensive tasks warrant independent verification before use.

The benchmark further reveals a statistically significant instability on the code generation prompt (P₄). Four of the five timed runs completed normally with elapsed times of 10–16 s; the fifth required 88.3 s—a factor of six above the expected range. This single outlier inflates the mean elapsed time for P₄ to 28.8 s and the standard deviation to 33.4 s, producing the condition discussed in Section 4. The cause is consistent with a transient context-length overflow or runtime garbage collection event. The limited knowledge base and the observed runtime unpredictability together position deepseek-r1:1.5b as a performance reference point, suitable for narrow, well-defined tasks but not for general-purpose production deployment.

4.3.1.5 gpt-oss:20b: VRAM exhaustion and CPU spilling

The gpt-oss:20b (OpenAI) model presents the contrasting failure mode. At twenty billion parameters its weights require approximately 14 GB of storage, exceeding the physical 12 GB VRAM limit of the evaluation GPU. Ollama responds by partitioning the model, loading as many layers onto the GPU as VRAM allows and offloading the remainder to host RAM for execution on the CPU. The resulting throughput of 17–19 tokens/s is a direct consequence of this hybrid path—roughly half the throughput of the slowest fully GPU-resident model (qwen3:14b at 29–32 tokens/s) and approximately one-third of the group mean.

CPU spilling also produces markedly elevated elapsed-time variance: the coefficient of variation () reaches 0.43 on P₁ and 0.41 on P₅ for gpt-oss:20b, against below 0.05 for all six GPU-resident models, arising from non-deterministic PCIe transfer timing that is sensitive to system load rather than to prompt characteristics. Counter-intuitively, on P₉ the model holds flat throughput across turns one, three and five (17.9, 17.7 and 17.8 tokens/s) against the 6%–10% decline in GPU-resident models, consistent with VRAM already being saturated at load time.

The evaluation thus confirms 12 GB as sufficient for reliable, stable inference on models in the 12–14 billion parameter range. The one model tested above that range exceeded the VRAM budget and showed reduced, unstable throughput. That follows from exceeding VRAM rather than from parameter count, and one case does not establish how larger models behave in general. Organizations needing models of this scale should choose a GPU that holds the weights and KV cache in full (Section 4.1), or adopt the multi-GPU configuration of Section 3.3.

4.3.2 Cost implications of measured throughput

The cost estimate in Section 3.4 assumes a single, model-independent loaded generation time of 40 s per prompt (Table 2). The measured throughput data in Table 7 allow this assumption to be refined for the two boundary-case models identified above. Approximating loaded generation time as the 1,500-token output length divided by measured throughput gives ≈7.9 s/prompt for deepseek-r1:1.5b and ≈83.8 s/prompt for gpt-oss:20b—roughly five times faster, and twice as slow, respectively, as the flat 40-s assumption used elsewhere.

At 500 prompts/day, deepseek-r1:1.5b's faster generation reduces its share of costly loaded-power operation, yielding an estimated annual electricity cost of $125 ($0.00125/prompt)—lower than the $207 flat-assumption baseline. gpt-oss:20b, by contrast, requires approximately 11.6 h of continuous loaded operation to process the same daily volume—exceeding the 10-h power-on window assumed throughout this study before any idle time is even counted; it cannot in practice sustain 500 prompts/day within a standard working day on this hardware. Computed at full utilization, its estimated annual electricity cost is approximately $335 ($0.00335/prompt).

Set against Gemini 3 Flash's $0.0005/prompt (Table 3), the fastest and slowest locally-hosted models therefore bracket a cost range of roughly 2.5 × to 6.7 × Gemini 3 Flash's per-prompt price, driven entirely by measured throughput rather than the flat assumption used for the six production-ready models. This confirms that for gpt-oss:20b, throughput—not electricity cost per se—is the binding constraint on production use: even its modest cost premium over cloud pricing is secondary to its inability to keep pace with the assumed workload at all.

4.3.3 Qualitative correctness evaluation

Throughput and elapsed time characterize how fast a model responds, not whether it is right, and local models are prone to hallucination, as noted in Section 1. This study therefore additionally evaluates each model's output correctness using the EleutherAI Language Model Evaluation Harness (Gao et al., 2024), run against three established benchmarks covering distinct domains: GSM8K (Cobbe et al., 2021) (grade-school arithmetic reasoning, scored against gold numeric answers), HumanEval (Chen et al., 2021) (Python code correctness, verified by executing each generated solution against its unit tests), and the generative variant of TruthfulQA (Lin et al., 2022) (resistance to generating plausible-but-false answers). TruthfulQA's questions are adversarially constructed, crafted so that some humans would answer falsely from a common misconception, so scores on it run far lower than the same models achieve on general factual questions: its authors report 94% human performance against 58% for the strongest model they evaluated.

4.3.3.1 Evaluation environment

This qualitative evaluation was conducted on a separate Windows 11 workstation from the throughput evaluation described above, using the same Ollama-served model tags and, by deliberate choice, the same NVIDIA RTX 3060 (12 GB VRAM) GPU, sourced specifically to support this comparison. Output correctness depends on model weights and sampling parameters rather than on the GPU executing inference, so only the throughput and elapsed-time measurements are specific to the evaluation hardware; the matching GPU additionally rules out any concern about quantised-kernel differences across architectures. gpt-oss:20b initially could not be evaluated: under Ollama v0.32.3 every request failed with a tensor-size error arising from the model's MXFP4 mixture-of-experts layout, a defect in the serving runtime rather than in the model, the hardware or the evaluation harness. It was resolved in a later Ollama release, and the results reported here were obtained under v0.32.6.

4.3.3.2 Generation settings

Decoding was greedy (do_sample=false), the chat template was applied, and requests were issued one at a time. The context window was not set, so all correctness runs used Ollama's default of 4,096 tokens, the same default discussed in Section 4.1. The requested generation budget was 4,096 output tokens for GSM8K and TruthfulQA, and 16,384 for HumanEval on every model except gemma3:12b, the larger figure having been adopted after the initial runs showed reasoning models filling the budget before emitting an answer.

That larger budget was not effective, and the reason is invisible from the scores. Ollama clamps generation to the context window irrespective of the requested budget: a request for 16,384 output tokens against a 4,096-token window returns after 4,096 tokens of prompt and completion combined, reporting a length stop. Raising the budget without raising the window therefore had no effect for any model that does not shift its context. Two of the eight do: deepseek-r1:1.5b and qwen3:14b continue past the window by discarding the oldest tokens, logging 482 and 114 such shifts, which is how they reached 16,380 generated tokens. For the other six the effective ceiling was 4,096 tokens throughout, so the distinction between the two budgets is immaterial to their scores.

4.3.3.3 A custom HumanEval adapter

HumanEval was scored using a custom evaluation task written for this study, because neither variant packaged with the harness functions correctly against a chat-completions backend. The standard humaneval task assumes a raw-completion model and terminates generation on stop sequences that a chat-tuned model does not emit at the expected position. The humaneval_instruct variant instead relies on prefilling the start of the assistant turn, which Ollama's OpenAI-compatible endpoint accepts and silently ignores. Both therefore score text the model was never prompted to produce. The task written to bridge this, humaneval_local_chat, is published in full at Curington and Lano (2026a).

Each problem is presented as a single user turn reading “Write a solution to the following problem and make sure that it passes the tests:”, followed by the problem's function stub in a fenced Python block. Generation is greedy and without a stop sequence, under the budget and context window stated above. The reply is then reduced to executable code in four steps: internal reasoning spans are discarded; the last fenced block defining the target function is selected, or failing that the last block containing any definition; top-level statements other than imports, definitions and assignments are removed; and the dataset stub is prepended. The result is executed against the problem's hidden unit tests (which the model never sees) and scored pass@1.

4.3.3.4 Why the adapter behaves as it does

Each step addresses a failure mode that produces a false negative without raising an error. Models frequently emit a solution followed by a usage example or a call to doctest. Scoring the trailing block would discard the answer, so the extractor takes the last block that defines the target function.

Models also append their own test assertions. These are not always correct, and they abort the module before the hidden test runs. Top-level statements are therefore removed, following the standard practice of scoring the completion rather than the model's own scaffolding. Reasoning models present a third problem: they return their chain of thought in a separate field that the harness does not read, so a model filling the context window while reasoning is recorded as producing an empty answer rather than as failing to terminate. No generation budget prevents this, for the reason given above: the window, not the budget, is the binding limit.

Applying this to the eight models, 64 of 1,312 responses were empty for that reason and are counted as failures in Table 8: 54 from deepseek-r1:1.5b, six from qwen3:14b and four from gemma4:e4b, which also reasons by default. In each case the reasoning field was populated—between 6,836 and 12,106 characters for the gemma4:e4b items—while the content field remained empty. Of the remaining 1,248, every one yielded extractable code, a residual parsing-failure rate of zero.

Table 8

Output correctness across all eight benchmarked models [EleutherAI lm-evaluation-harness (Gao et al., 2024)], ordered as in Table 7.

GSM8K is scored by flexible-extract exact match, HumanEval by pass@1 against each problem's unit tests, and TruthfulQA by the BLEU and ROUGE-1 “acc” metrics; Section 4.3.3 defines these and states their limitations. The final column gives the proportion judged truthful by an independent LLM judge (llama3.3:70b); it runs consistently higher than the overlap metrics because that judge is lenient rather than more accurate, as Section 4.3.3 reports. Quantisation levels are the defaults served for each unmodified model tag. The final row repeats mistral-nemo:12b at Q4_K_M as a control. The per-item generations and scores behind every figure in this table, and the judge verdicts and human adjudication behind the final column, are deposited at Curington and Lano (2026b).

4.3.3.5 What each metric measures

Each benchmark is scored differently, and each carries a different interpretation. GSM8K uses flexible-extract exact match: the final numeric value is extracted and compared with the gold answer, which suits arithmetic word problems but rewards only the final figure, taking no account of whether the reasoning was sound. HumanEval uses pass@1: each solution is executed against the problem's own unit tests and counted correct only if all pass, the strictest of the three since it tests behavior rather than resemblance, but binary and giving no credit to a near-miss. TruthfulQA uses the BLEU and ROUGE-1 “acc” metrics, the fraction of items where the answer shares more n-grams with a true reference than with any false one. These are proxies for truthfulness rather than measures of it, since they compare wording rather than meaning: an answer may be truthful while sharing little vocabulary with the reference, or share vocabulary while being false. The original benchmark uses a fine-tuned GPT-judge classifier for this, which was unavailable here, so the TruthfulQA figures should be read as a relative comparison between the models evaluated rather than against externally reported scores.

4.3.3.6 Overall correctness

Table 8 reports correctness across the three benchmarks for all eight models. GSM8K performance is high and comparatively compressed (70.4%–92.0% flexible-extract exact-match): arithmetic word-problem reasoning at this difficulty is well-handled across the group, regardless of parameter count or architecture. HumanEval is more dispersed (47.0%–91.5% pass@1) and does not follow the ordering of either GSM8K or the throughput results in Section 4.3.1, though six of the eight models fall within a narrow 83.5%–91.5% band. Code generation therefore separates the weakest models sharply while distinguishing the stronger ones only modestly. A small number of items returned no content at all and are counted as failures, at most 2.4% on GSM8K and 1.1% on TruthfulQA, so those scores are marginally conservative; the much larger effect of this on HumanEval is discussed below.

4.3.3.7 Correctness and throughput are largely independent

Figure 3 plots the two against each other, and shows no trend: within this hardware envelope, knowing how fast a model runs says little about how often it is right. Combining correctness with the throughput and stability findings of Section 4.3.1 identifies gemma4:e4b as the strongest model on both axes simultaneously: it is the fastest of the six throughput-stable models (59–65 tokens/s) and the most correct on HumanEval (91.5%), with 90.4% on GSM8K. granite4.1:8b is the next most attractive all-rounder, second fastest at 51–58 tokens/s and the strongest model on GSM8K (92.0%), at 83.5% HumanEval. gemma3:12b and phi4:14b offer comparable correctness at more moderate throughput.

Figure 3

qwen3:14b achieves near-identical correctness to gemma4:e4b (90.2% HumanEval) but is among the slowest models measured, and its practical cost is greater than tokens per second alone conveys. Completing the same 164-problem HumanEval set took 5 h 11 m against gemma4:e4b's 1 h 28 m on identical hardware, a 3.5 × difference where the throughput figures differ by roughly 2 ×, because reasoning models emit substantially more tokens per answered task. Tokens per second understates the latency cost of reasoning models for task completion, and wall-clock time to complete a fixed workload is the more informative measure where a model reasons by default.

This has a direct consequence for the throughput-stable cluster identified in Section 4.3.1. mistral-nemo:12b, grouped there among the six stable, GPU-resident models, is the weakest of that cluster on HumanEval (62.8%) despite competitive throughput and solid GSM8K performance (76.0%). Throughput and stability describe how reliably a model responds, not whether the response is correct, so model selection should weight the relevant correctness axis alongside throughput: HumanEval for coding use, GSM8K for arithmetic and quantitative use. Re-evaluating mistral-nemo:12b at Q4_K_M, matching the quantisation served for the other models, raises its scores only marginally (77.3% GSM8K, 65.2% HumanEval, 48.8% and 52.4% TruthfulQA), so the difference is attributable to the model rather than to the quantisation its default tag happens to supply.

4.3.3.8 The boundary cases fail in opposite directions

The two models included as boundary cases fail in opposite directions, and neither failure is the one their throughput figures would suggest. deepseek-r1:1.5b, by far the fastest model measured, is the least correct on every benchmark; more revealingly, it frequently fails to answer at all, returning no solution for 33% of HumanEval problems. Of the items it did complete, 70.0% were correct. Its binding limitation on this hardware is therefore failure to return an answer rather than poor code, a distinction that a single pass@1 figure conceals. The mechanism is the interaction of a long reasoning chain with a 4,096-token context window, not the generation budget alone: once the chain exceeds the window the problem statement is discarded from context, and raising the budget cannot recover it. Section 4.1 shows that the window cannot be widened far enough on a 12 GB GPU to avoid this at these model sizes, so the practitioner-relevant result is that this model cannot be relied on to return an answer at all in this configuration, and that the failure is silent. gpt-oss:20b inverts this. It is the slowest model in the study and the only one that exceeds the VRAM budget and spills to CPU, yet it is second on HumanEval (90.9%) and the highest-scoring model on TruthfulQA (52.8% BLEU, 54.8% ROUGE-1). Excluded on throughput grounds, it would have been among the most accurate choices available. Taken together the two boundary cases make the central point of this section more sharply than the stable cluster does: within this hardware envelope, speed and correctness are close to independent.

4.3.3.9 How far the TruthfulQA figures can be trusted

TruthfulQA scores cluster more tightly than either of the other benchmarks, spanning 39.4%–55.4% across all eight models. Given the limitations of the overlap-based metrics described above, this narrow spread should not be read as evidence that the models are genuinely similar in truthfulness: it may equally reflect the limited discriminative power of the metric itself. Of the three benchmarks reported here, TruthfulQA is the one whose absolute values carry least weight.

The column also mixes two task definitions. gemma3:12b, granite4.1:8b, phi4:14b and both mistral-nemo:12b variants were scored under the harness's stock truthfulqa_gen, which caps generation at 256 tokens and stops at the first blank line; the other four were scored under truthfulqa_gen_chat, the chat-template variant written for this study, which stops only at the next question marker. Truncation therefore falls principally on the first group, whose answers average 195–361 characters against 432–1,149 for the second—a further reason to read the column comparatively rather than absolutely.

To test whether this narrow spread reflects the models or the metric, all saved TruthfulQA generations were additionally scored by an LLM judge (llama3.3:70b, chosen from a model family absent from the evaluated set, so that no model was scored by a sibling), and a blind stratified sample of 43 items across the nine model-runs was then adjudicated by hand with the judge's verdict withheld. Agreement with human adjudication was 79% for the judge, 74% for BLEU and 70% for ROUGE-1, with no pairwise difference reaching significance [exact McNemar test (McNemar, 1947), p = 0.79 and p = 0.42]. The methods do err in opposite directions, consistently across both adjudication rounds: the judge credits answers a reader rejects, whilst the overlap metrics reject answers a reader accepts. A further eight sampled items could not be adjudicated at all, because the generation configuration truncated the answer at the first paragraph break before it committed to a claim. No automated scoring method available here therefore measures truthfulness reliably, and the TruthfulQA column should be read accordingly. This adjudication was performed by one author and is not replicated, so it bounds the reliability of these metrics rather than establishing ground truth.

4.3.3.10 Caveats on the correctness results

As with the throughput results, these correctness scores are drawn from a single run per item at greedy (temperature-0) decoding, not repeated across multiple trials as the N = 5 throughput measurements in Section 4.3.1 are. Where a direct check was possible this proved reproducible: regenerating gemma4:e4b's full HumanEval set 3 weeks after the original run, under a nominally larger generation budget that the context window rendered inoperative, reproduced the identical pass or fail outcome on all 164 items. Reproducibility is weaker for models that reason at length, where the boundary between terminating and exhausting the token budget is evidently sensitive to small perturbations; Section 5 discusses this as a further consequence of the time constraints already affecting benchmark repeatability in this study.

5 Limitations

5.1 Study scope and provenance

This study was conducted independently, motivated by personal interest rather than institutional direction from an employer, university, or funding body, and reflects the first author's initial work within the AI/LLM domain. The results are intended to be of practical value both to the home-lab community of practitioners who share hardware and software configurations for running local AI workloads—a tradition also reflected in individually-authored practitioner and evaluation accounts of local LLM deployment (Falkingham, 2025; Kivimäki, 2025)—and to academic and commercial readers evaluating similar on-premises deployments.

The evaluation is deliberately scoped to a single dedicated workstation rather than a fleet of machines. This reflects both the study's origin—a single obsolete workstation repurposed after a Windows 11 hardware upgrade—and a practical constraint: the benchmarked machine has since been reassigned to another user for unrelated work and is no longer available to the authors for further data collection. Section 3.3 discusses how the approach generalizes to multi-GPU, single-chassis configurations for organizations requiring greater capacity; a multi-machine fleet study was outside the scope of this work.

5.2 Evaluation hardware

The evaluation hardware is also more capable than a typical consumer PC affected by Windows 10's end-of-life: the assumption noted in Section 4, that its server CPUs, memory and PCIe advantages matter little because most computation occurs on the GPU, has not been independently confirmed on a genuinely representative lower-specification “obsolete consumer” machine, since a second suitable GPU-equipped machine was not available to the authors. The GPU itself, however, is a consumer-tier NVIDIA RTX 3060 (12 GB VRAM)—the component most determinative of LLM inference performance per the reasoning above—rather than a workstation- or datacentre-class accelerator, which partially mitigates this concern. Only this single 12 GB configuration was measured, so the study characterizes what that capacity supports rather than what a local server minimally requires. The exact CUDA and GPU driver versions in use were not recorded at the time and cannot be retrieved retrospectively, since the hardware is no longer accessible; the benchmark date and Ollama version are recorded in Table 7.

5.3 Benchmark repeatability

Benchmark repeatability was similarly constrained by available time on the evaluation hardware: five repetitions per prompt (N = 5) already required approximately 5–6 h of continuous GPU time across the full model and prompt matrix. Increasing this to the order of N = 100 repetitions per case was not achievable within this timeframe and is no longer possible on the original hardware. Consequently, this study reports mean, variance, and standard deviation over five runs rather than higher-order percentile statistics; these indicate measurement stability under the tested conditions rather than a comprehensive statistical characterization. For the same reason, resampling deepseek-r1:1.5b and gpt-oss:20b—the fastest and slowest models measured—against multiple unique prompt instances per category, rather than repeated runs of the same fixed prompt, was not carried out; the ten prompt categories described in Section 4 provide diversity of task type, but not of phrasing within a category.

5.4 Workload scope

The benchmark design and cost model in this study assume bounded, single-turn or short multi-turn conversational interaction, matching the ten prompt dimensions evaluated in Section 4. This does not extend to agentic or autonomous tool-use workflows, such as a coding agent operating across a large codebase in an unattended mode, where a single user instruction can trigger an open-ended chain of tool calls and accumulated context with no fixed tokens-per-prompt bound. The fixed token and prompt-count assumptions in Table 2 should therefore be read as representative of conversational AI assistant use, not of agentic automation. Multi-user or batched access was not evaluated either, since the server was used by a single researcher throughout.

5.5 Correctness evaluation

The correctness evaluation carries its own constraints, described where they arise in Section 4.3.3. Scores come from a single greedy generation per item, not from repeated trials, so the fivefold repetition applied to throughput has no counterpart on that axis. The evaluation also ran on a separate workstation fitted with the same GPU model, because the throughput host had been reassigned, so the two axes were not measured on one machine. Quantisation levels are the defaults each model tag serves, and are not uniform across models.

TruthfulQA is the weakest of the three benchmarks used here. The stock task configuration truncated a substantial minority of answers at the first paragraph break, affecting four of the nine model-runs. Neither the overlap metrics nor an LLM judge agreed with human adjudication better than about three times in four, and that adjudication was performed by one author without replication.

5.6 Mitigation

To mitigate these constraints, the benchmarking suite used in this study is published under an open license (Curington and Lano, 2026a), so that other researchers and practitioners—independent or institutional, with sustained hardware access—can reproduce, extend, or scale this methodology, including larger sample sizes, additional models, or multi-machine configurations.

6 Future work

Further work is planned to investigate methods for balancing CPU/GPU loads and refining LLM system tuning. Specifically, this will investigate how model quantisation and context length affect “spilling” and how using “attention” and LLM layer allocation strategies may be employed.

7 Conclusion

This study demonstrates that obsolete Windows 10 PCs can be efficiently repurposed as on-premises LLM servers that are substantially cheaper than mid-range and premium cloud alternatives, providing a practical route for organizations to move away from subscription models whilst retaining full data sovereignty.

Benchmarking on a repurposed enterprise workstation, fitted with a single consumer-grade NVIDIA GPU, shows that this class of hardware delivers competitive inference speeds across a range of open-source models. Correctness, evaluated against established public benchmarks alongside this throughput data, does not follow the same ranking as throughput, identifying gemma4:e4b as the strongest model evaluated on this hardware across both dimensions. granite4.1:8b pairs the second-highest throughput of that group with the highest arithmetic-reasoning score of any model tested, and the slowest model evaluated, which spilled to the CPU, was second-most accurate on code generation. Selecting a model on throughput alone would therefore have discarded one of the most capable options available.

The cost-benefit analysis—encompassing the elimination of license fees, reduced reliance on external network infrastructure, and a lower estimated carbon footprint than the cloud alternatives evaluated—positions local deployment as a viable and sustainable alternative. Even against the cheapest cloud tier, where the raw cost gap narrows and briefly favors cloud, this study argues that the digital-sovereignty benefits of local control outweigh the difference for organizations facing privacy, compliance, or long-term cost-predictability requirements.

Because inference runs entirely on local hardware, no prompt or document content leaves the organization's network. That removes one common obstacle to using large language models with regulated or confidential material, although whether a particular deployment satisfies a specific regulatory obligation depends on the wider system and is not assessed here.

Statements

Author contributions

IC: Conceptualization, Investigation, Writing – original draft, Supervision, Software, Writing – review & editing, Data curation, Formal analysis, Methodology. KL: Resources, Funding acquisition, Writing – review & editing, Validation, Project administration, Supervision.

Funding

The author(s) declared that financial support was received for this work and/or its publication. Funding support was received from EPSRC grant code EP/ X525571/1 held by King's College London.

Acknowledgments

We wish to acknowledge the contributions of Harry Eggo that aided the efforts of the authors. The authors acknowledge support of EPSRC grant code EP/ X525571/1 held by King's College London.

Conflict of interest

IC was employed at Thames 3D Research.

The remaining author declares that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declared that Generative AI was used in the creation of this manuscript. The study and its early drafts are the author(s)' own work. Generative AI assistance was used in revising it. Claude Code (Anthropic; Sonnet 5 and Opus 5) was used to draft and revise passages of the manuscript, for LaTeX typesetting, citation verification and debugging. It was also used to configure and run the correctness evaluation of Section 4.3.3, using EleutherAI's lm-evaluation-harness, and to write the analysis and plotting scripts published alongside it. The carbon-footprint estimates in Section 3.4 were produced the same way, using EcoLogits. The throughput benchmark suite, its prompt design, and the throughput measurements are the author(s)' own work. The experimental goals, methodology, study design, and the conclusions drawn from the results are the author(s)', as is the decision on every point of interpretation. The author(s) have verified the factual accuracy and integrity of all AI-assisted content. The early drafts, written in Markdown before the manuscript was converted to LaTeX, are retained and can be provided on request.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1

  Anthropic (2025). System Card: Claude Opus 4 and Claude Sonnet 4. Available online at: https://www.anthropic.com/claude-4-system-card (Accessed July 31, 2026).

  • Google Scholar

• 2

  BachsteinM.GroppC.HazlewoodV.PetersonG. (2024). “Lessons from benchmarking the AI Tennessee initiative resources,” in Practice and experience in advanced research computing (PEARC '24) (Providence, RI: ACM).

  • Google Scholar

• 3

  CaddyB.JessopK. (2023). The End of Windows 10 Support Could Turn 240 Million pcs into E-Waste. Canalys. Available online at: https://www.canalys.com/insights/end-of-windows-10-support-could-turn-240-million-pcs-into-e-waste (Accessed October 27, 2024).

  • Google Scholar

• 4

  ChenM.TworekJ.JunH.YuanQ.PintoH. P. d. O.KaplanJ.et al. (2021). Evaluating large language models trained on code. arXiv [preprint]. arXiv:2107.03374. Available online at: https://arxiv.org/abs/2107.03374 (Accessed July 30, 2026).

  • Google Scholar

• 5

  Chitty-VenkataK. T.RaskarS.KaleB.FerdausF.TanikantiA.RaffenettiK.et al. (2024). “LLM-inference-bench: inference benchmarking of large language models on AI accelerators,” in Proceedings of the SC'24 workshops of the international conference on high performance computing, network, storage, and analysis (IEEE). Available online at: https://arxiv.org/abs/2411.00136 (Accessed July 25, 2026).

  • Google Scholar

• 6

  CobbeK.KosarajuV.BavarianM.ChenM.JunH.KaiserL.et al. (2021). Training verifiers to solve math word problems. arXiv [preprint]. arXiv:2110.14168. Available online at: https://arxiv.org/abs/2110.14168 (Accessed July 30, 2026).

  • Google Scholar

• 7

  CuringtonI.LanoK. (2026a). Ollama LLM Local Inference Benchmarking Suite. Version v1.0.1. Available online at: https://doi.org/10.5281/zenodo.22079049; source repository https://github.com/t3dr0/ollama_benchmark (Accessed August 24, 2026).

  • Google Scholar

• 8

  CuringtonI.LanoK. (2026b). Throughput and Output-Correctness Benchmark Data for Local LLM Inference on a Repurposed Enterprise Workstation. Dataset, Version v2. Available online at: https://doi.org/10.5281/zenodo.22748926 (Accessed September 14, 2026).

  • Google Scholar

• 9

  DatabaseMart (2026). RTX 3060 Ollama Benchmark. DatabaseMart Blog. Available online at: https://www.databasemart.com/blog/ollama-gpu-benchmark-rtx3060ti (Accessed June 2026).

  • Google Scholar

• 10

  Department for Energy Security and Net Zero (2025). 2025 Government Greenhouse Gas Conversion Factors for Company Reporting: Methodology Paper. Available online at: https://assets.publishing.service.gov.uk/media/6846b0870392ed9b784c0187/2025-GHG-CF-methodology-paper.pdf (Accessed July 30, 2026).

  • Google Scholar

• 11

  F.S.F.E. (2026). Device Neutrality Manifesto of the Free Software Foundation Europe. FSFE.org. Available online at: https://fsfe.org/activities/deviceneutrality/index (Accessed April 28, 2026).

  • Google Scholar

• 12

  FalkinghamP. (2025). Using Local AI/LLM in VS Code Without Third Party Software on the CPU, GPU or NPU. Available online at: https://peterfalkingham.com/2025/10/27/using-local-ai-llm-in-vs-code-without-third-party-software/ (Accessed April 28, 2026).

  • Google Scholar

• 13

  FinleyK. (2013). French National Police Switch 37,000 Desktop PCs to Linux. WIRED. Available online at: https://www.wired.com/2013/09/gendarmerie-linux/ (Accessed April 28, 2026).

  • Google Scholar

• 14

  FloridiL. (2020). The fight for digital sovereignty: what it is, and why it matters, especially for the EU. Philos. Technol. 33, 369–378. doi: 10.1007/s13347-020-00423-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15

  France (2026). Souveraineté numérique : l'État accélère la réduction de ses dépendances extra-européennes. Translated from the French Government Website. Available online at: https://www.numerique.gouv.fr/sinformer/espace-presse/souverainete-numerique-reduction-dependances-extra-europeennes/ (Accessed April 28, 2026).

  • Google Scholar

• 16

  GaoL.TowJ.AbbasiB.BidermanS.BlackS.DiPofiA.et al. (2024). The Language Model Evaluation Harness. Available online at: https://doi.org/10.5281/zenodo.12608602 (Accessed July 30, 2026).

  • Google Scholar

• 17

  Google (2026). Introducing Gemini 3 Flash: Benchmarks, Global Availability. Available online at: https://blog.google/products/gemini/gemini-3-flash/ (Accessed July 31, 2026).

  • Google Scholar

• 18

  GouldP.SongG.ZhuT. (2024). Environmental and Economic Impact of I/O Device Obsolescence. Available online at: https://arxiv.org/abs/2412.20655 (Accessed June 10, 2026).

  • Google Scholar

• 19

• 20

  KhalilA.HeillesG.ParragaM.HeillesS. (2025). Viability and performance of a private LLM server for SMBs: a benchmark analysis of Qwen3-30B on consumer-grade hardware. arXiv [preprint]. arXiv:2512.23029. Available online at: https://arxiv.org/abs/2512.23029

  • Google Scholar

• 21

  KivimäkiT. (2025). Usability Evaluation of the Local Large Language Models. UTU Pub Finland. Available online at: https://www.utupub.fi/handle/10024/182260 (Accessed October 27, 2025).

  • Google Scholar

• 22

  KnoopJ.HoltmannH. (2026). Private LLM inference on consumer blackwell GPUs: a practical guide for cost-effective local deployment in SMEs. arXiv [preprint]. arXiv:2601.09527. Available online at: https://arxiv.org/abs/2601.09527

  • Google Scholar

• 23

  LazukaM.AnghelA.ParnellT. (2024). “LLM-pilot: characterize and optimize performance of your LLM inference services,” in Proceedings of the international conference for high performance computing, networking, storage and analysis (SC'24) (IEEE). Available online at: https://sc24.supercomputing.org/proceedings/tech_paper/tech_paper_pages/pap547.html (Accessed July 25, 2026).

  • Google Scholar

• 24

  LiJ.XuJ.HuangS.ChenY.LiW.LiuJ.et al. (2024). Large language model inference acceleration: a comprehensive hardware perspective. arXiv [preprint]. arXiv:2410.04466. Available online at: https://arxiv.org/abs/2410.04466

  • Google Scholar

• 25

  LinS.HiltonJ.EvansO. (2022). “TruthfulQA: measuring how models mimic human falsehoods,” in Proceedings of the 60th annual meeting of the association for computational linguistics (Volume 1: Long Papers) (Dublin: Association for Computational Linguistics), 3214–3252. Available online at: https://aclanthology.org/2022.acl-long.229/ (Accessed July 30, 2026).

  • Google Scholar

• 26

  McNemarQ. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika12, 153–157. doi: 10.1007/BF02295996

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27

  Microsoft (2025a). Reporting Graphics Memory. Windows Hardware Drivers Documentation. Available online at: https://learn.microsoft.com/en-us/windows-hardware/drivers/display/reporting-graphics-memory (Accessed May 23, 2026).

  • Google Scholar

• 28

  Microsoft (2025b). Windows 10 Home and Pro–Microsoft Lifecycle. Available online at: https://learn.microsoft.com/en-us/lifecycle/products/windows-10-home-and-pro (Accessed August 19, 2026).

  • Google Scholar

• 29

  MuhlgayD.RamO.MagarI.LevineY.RatnerN.BelinkovY.et al. (2024). “Generating benchmarks for factuality evaluation of language models,” in Proceedings of the 18th conference of the European chapter of the association for computational linguistics (Volume 1: Long Papers) (St. Julian's: Association for Computational Linguistics), 49–66. Available online at: https://aclanthology.org/2024.eacl-long.4/ (Accessed July 27, 2026).

  • Google Scholar

• 30

  NaS.JeongG.AhnB. H.YoungJ.KrishnaT.KimH. (2024). “Understanding performance implications of LLM inference on CPUs,” in 2024 IEEE international symposium on workload characterization (IISWC) (Vancouver, BC: IEEE), 169–180. doi: 10.1109/IISWC63097.2024.00024

  • CrossRef
  • Google Scholar

• 31

  NanoGPT (2025). AI Chat Text 400+ Models API Pricing Table. NanoGPT. Available online at: https://nano-gpt.com/pricing (Accessed September 26, 2025).

  • Google Scholar

• 32

  NaughtonJ. (2025). The Networker: AI is the New Globalisation That Will Create a World of Have-Nots and Have-Yachts. The Observer. Available online at: https://observer.co.uk/news/columnists/article/the-networker-ai-is-the-new-globalisation-that-will-create-a-world-of-have-nots-and-have-yachts (Accessed October 27, 2024).

  • Google Scholar

• 33

  NguyenT.NguyenT. (2025). An evaluation of LLMs inference on popular single-board computers. arXiv [preprint]. arXiv:2511.07425. Available online at: https://arxiv.org/abs/2511.07425

  • Google Scholar

• 34

  Ofgem (2026). Changes to Energy Price Cap Between 1 July and 30 September 2026. Available online at: https://www.ofgem.gov.uk/news/changes-energy-price-cap-between-1-july-and-30-september-2026 (Accessed July 28, 2026).

  • Google Scholar

• 35

  Ollama (2026a). Ollama Library Open-Source LLMs. Available online at: https://ollama.com/library (Accessed April 30, 2026).

  • Google Scholar

• 36

  Ollama (2026b). Ollama v0.24.0. Available online at: https://github.com/ollama/ollama/releases/tag/v0.24.0 (Accessed July 30, 2026).

  • Google Scholar

• 37

  Open WebUI (2026). Open WebUI: Self-Hosted AI Platform. Available online at: https://docs.openwebui.com/ (Accessed August 23, 2026).

  • Google Scholar

• 38

  OthmanA. (2025). The End of Windows 10 and The Rise of Digital Waste: A Global Technological Dilemma. ResearchGate. Available online at: https://www.researchgate.net/publication/397020633_The_End_of_Windows_10_and_the_Rise_of_Digital_Waste_A_Global_Technological_Dilemma (Accessed April 28, 2026).

  • Google Scholar

• 39

  PordesR.PetravickD.KramerB.OlsonD.LivnyM.RoyA.et al. (2007). The open science grid. J. Phys. Conf. Ser. 78:012057. doi: 10.1088/1742-6596/78/1/012057

  • CrossRef
  • Google Scholar

• 40

  RahmanN. A. A.ZulkornainL. H.SoonG. Y.HamzahN. H.ChaoC. (2025). NotebookLM integrated project-based learning model development for writing among undergraduates. Int. J. Res. Innov. Soc. Sci. 4:5. doi: 10.47772/IJRISS.2025.907000444

  • CrossRef
  • Google Scholar

• 41

  ReddiV. J.ChengC.KanterD.MattsonP.SchmuellingG.WuC.-J.et al. (2020). “MLPerf inference benchmark,” in 2020 ACM/IEEE 47th annual international symposium on computer architecture (ISCA) (IEEE), 446–459. Available online at: https://arxiv.org/abs/1911.02549 (Accessed July 25, 2026).

  • Google Scholar

• 42

  RenneyH.TradF.MattarockM.WoodZ. (2026). Cloud to edge: benchmarking LLM inference on hardware-accelerated single-board computers. arXiv [preprint]. arXiv:2604.24785. Available online at: https://arxiv.org/abs/2604.24785

  • Google Scholar

• 43

  RincéS.BanseA. (2026). EcoLogits: Track the Energy and Environmental Footprint of Generative AI Models Through Their APIs, Version 0.11.1. Available online at: https://github.com/genai-impact/ecologits (Accessed July 30, 2026).

  • Google Scholar

• 44

  SchneierB. (2025). What LLMs Know About Their Users. Crypto-Gram. Available online at: https://www.schneier.com/crypto-gram/archives/2025/0715.html#cg7 (Accessed October 27, 2025).

  • Google Scholar

• 45

  SrinivasA.HegdeV.AdityaG. S.SinghA. Fayaz. (2025). “Building and hosting AI locally with Ollama, Open WebUI, and stable diffusion,” in 2025 9th international conference on computational system and information technology for sustainable solutions (CSITSS) (IEEE), 1–6. doi: 10.1109/CSITSS67709.2025.11295791

  • CrossRef
  • Google Scholar

• 46

  StuhlmannL.Fadel ArgerichM.FürstJ. (2025). Bench360: benchmarking local LLM inference from 360 degrees. arXiv [preprint]. arXiv:2511.16682. Available online at: https://arxiv.org/abs/2511.16682

  • Google Scholar

• 47

  ThainD.TannenbaumT.LivnyM. (2005). Distributed computing in practice: the condor experience. Concurrency Comput. Pract. Exp. 17, 323–356. doi: 10.1002/cpe.938

  • CrossRef
  • Google Scholar

• 48

  University of Oxford IT Services Department (2022). Environmental Impact of IT: Desktops, Laptops and Screens. University of Oxford Website. Available online at: https://web.archive.org/web/20260315150016/; https://www.it.ox.ac.uk/article/environment-and-it (Accessed August 19, 2026).

  • Google Scholar

• 49

  Vaughan-NicholsS. (2026). France is Ditching Windows for Digital Sovereignty – And its New Linux Stack is Taking Shape. ZDNET. Available online at: https://www.zdnet.com/article/france-leaves-windows-for-linux-desktop/ (Accessed April 28, 2026).

  • Google Scholar

• 50

  WhiteC.DooleyS.RobertsM.PalA.FeuerB.JainS.et al. (2024). Livebench: a challenging, contamination-free LLM benchmark. arXiv [preprint]. arXiv:2406.19314. Available online at: https://livebench.ai/livebench.pdf (Accessed July 27, 2026).

  • Google Scholar

• 51

  WolfeJ.PfohlC.SubramanianH.MertogunoJ. S.AlexiouM. S. (2024). “Recent advancements in search-engine algorithms for efficiency, privacy and security,” in 2024 15th international conference on information, intelligence, systems and applications (IISA) (Chania Crete: IEEE), 1–6. doi: 10.1109/IISA62523.2024.10786705

  • CrossRef
  • Google Scholar

• 52

  ZeffM. (2025). Meta Plans to Sell Targeted ads Based on Data in Your AI Chats. Tech Crunch. Available online at: https://techcrunch.com/2025/10/01/meta-plans-to-sell-targeted-ads-based-on-data-in-your-ai-chats/ (Accessed October 27, 2025).

  • Google Scholar

Keywords

digital sovereignty, e-waste, inference, LLMS, obsolescence, online-privacy, on-premises, windows-10

Citation

Curington I and Lano K (2026) Reusing obsolete windows 10 PCs for on-premises large language model inference. Front. Comput. Sci. 8:1911927. doi: 10.3389/fcomp.2026.1911927

Received

17 June 2026

Revised

14 September 2026

Accepted

24 September 2026

Published

08 October 2026

Volume

8 - 2026

Updates

Copyright

© 2026 Curington and Lano.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Kevin Lano, kevin.lano@kcl.ac.uk

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.