用淘汰的 Windows 10 PC 做本地大语言模型推理

摘要

Windows 10 生命周期终止为将旧 PC 重新用作本地大语言模型（LLM）推理服务器提供了实质性激励。本研究评估了将配备 NVIDIA 图形处理器（GPU）的企业工作站改造为专用 AI 节点的技术可行性、经济可行性以及环境和社会问题。本研究使用 Proxmox 虚拟化、Ollama 和 Open WebUI 对多个开源 LLM 进行基准测试，结果表明，配备单块消费级 NVIDIA GPU 的改造工作站可为有界对话式编码和文档助手工作流提供 29–65 tokens/s 的稳定推理吞吐量，同时保持完整的数据主权并消除经常性订阅费用。性能分析表明，资源争用和 GPU 内存分配是吞吐量的主要决定因素。正确性根据三个具有标准答案的成熟公共基准进行评估，其排名与吞吐量并不一致；结合两个维度可确定 gemma4:e4b 是在此硬件上评估的速度和正确性均最强的模型，而测试中最慢的模型却是最准确的模型之一。成本效益模型显示，本地运行比中端和高端云替代方案便宜得多，按每次提示计算仅比最便宜的云服务层级略贵——本研究认为，这一差距被本地控制带来的数字主权优势所抵消，尤其是在适用合规驱动的数据处理约束的场景中。本研究总结了本地 AI 的关键背景动机——隐私、安全、监管风险——并估计该改造工作站使用阶段的 CO₂e 足迹低于所比较的每种云替代方案，此外还避免了新硬件的隐含制造排放。使用现成开源工具改造旧工作站，为组织在本地部署实用的 LLM 推理能力提供了一条可持续、具有成本效益的途径，为优先考虑隐私、监管合规或长期成本控制的实体提供了云服务的战略替代方案。

1 引言

微软决定于 2025 年 10 月 14 日终止对 Windows 10 的支持（Microsoft, 2025b；Kelly, 2023），影响了庞大的装机量。Canalys 估计，约五分之一的在用 PC——约 2.4 亿台——将不符合 Windows 11 的要求，因此不太可能被翻新或转售（Caddy and Jessop, 2023）。这是一项专有行业估计而非测量值：底层装机量数据和五分之一比例的依据均未公布，作者也未找到独立佐证。该估计在后续研究中被广泛引用（Gould et al., 2024），此处用于表明规模而非精确数量。随着操作系统供应商逐步淘汰旧系统，全行业范围内的过剩硬件激增正在发生。IT 管理公司 Lansweeper 发现，基于来自 60,000 个组织的约 3,000 万台 Windows 设备估计，42% 缺少获批的 CPU，14% 缺少 TPM 2.0 安全要求（Othman, 2025）。

与此同时，云托管大语言模型（LLM）服务的兴起带来了一系列新的限制，包括订阅费用、使用限额和外部数据依赖（Naughton, 2025）。这些因素共同催生了一个显著的机会：将原本闲置的 PC 改造成专用的 AI 推理机器。当然，其他选择也同样可行，例如在适当情况下将笔记本电脑和较小的 PC 转换为 Linux，作为 Windows 成熟而有效的替代方案。

对 LLM 推理性能进行基准测试本身就是一个活跃且快速发展的研究领域，涵盖大规模高性能计算（HPC）设施和消费级硬件。MLPerf Inference（Reddi et al., 2020）等行业标准套件为比较机器学习硬件和软件栈建立了严格、可复现的方法论，这一传统在各大高性能计算会议上延续。在国际高性能计算、网络、存储与分析会议（SC）上展示的最新工作包括跨 NVIDIA、AMD 和专用 AI 硬件的多加速器推理基准测试（Chitty-Venkata et al., 2024），以及跨图形处理单元（GPU）配置的 LLM 推理服务性能预测性表征（Lazuka et al., 2024）。高级研究计算实践与经验（PEARC）同样记录了专为支持 AI 工作负载而采购的配备 GPU 的 HPC 资源的获取、基准测试和功耗分析（Bachstein et al., 2024）。这些工作与本项研究一样关注严格、可复现的测量，但针对的是机构规模的专用加速器基础设施，而非本文所考虑的改造型消费级单 GPU 硬件。第 2.5 节回顾了与本项研究背景最为接近的文献。

大语言模型在商业、研究和个人生产力中日益占据核心地位，而大多数组织通过第三方云服务商访问它们——这会将敏感数据暴露给外部行为者，使工作负载受制于网络延迟，并产生在高调用量下尤为显著的订阅成本。本地 AI 服务器为组织提供了一种替代方案，可保留对数据的完全控制、消除订阅成本，并利用日益壮大的高效开源模型生态系统。通过将搭载或可轻松升级加装通用 GPU 的旧版 Windows 10 硬件转换为本地推理引擎，组织可以实现与基础商业云服务相当的性能，同时保持对数据保护法规的严格合规。

我们承认云 AI 服务和本地 AI 推理并不直接等同。云服务背后有大型数据中心资源支持，提供本地系统无法企及的服务，使用具有众多功能的超大型专有模型——尤其是最新的“前沿”模型。此外，本地 AI 推理模型的信息冻结在某一时间点，在需要最新时效性新闻或直接网络搜索的场景下可能并不适用。

此外，本地模型——尤其是最小的那些，例如本次评估中使用的 deepseek-R1:1.5b——可能显得响应很快，但生成的结果却包含错误，或完全误解上下文、忽略输入提示的部分内容——这一模式与更广泛的发现一致，即更小、更弱的模型在经真实答案验证的推理和事实性基准上得分显著更低（Muhlgay et al., 2024；White et al., 2024）。其他完全本地的模型（如 gemma4、qwen3、phi4、mistral、granite）在第 4.3.3 节所报告的严格基准评估下呈现出更多样的图景：gemma4:e4b 和 qwen3:14b 在代码生成上得分最高（pass@1 分别为 91.5% 和 90.2%），算术推理得分与组内其他模型相当，而 mistral-nemo:12b 在 HumanEval 上仅达到 62.8%，是吞吐稳定组中最弱的，尽管其吞吐量在其他方面稳定且具有竞争力。仍然必须使用严格的基准，而非仅凭非正式印象，来验证某个特定本地模型对给定任务是否合适且可靠。

Linux 是此硬件的一个可行替代操作系统，但对技术能力较弱的用户而言，在没有帮助的情况下管理它是一道陡坡，而那些已投入 Windows 生态或依赖 Photoshop、AutoCAD 等应用的人会发现迁移充其量也很困难。如果改为购买符合 Windows 11 要求的替代设备，过时 PC 的命运依然存在，而将其重新利用——用 Linux 或作为 LLM 服务器——便进入考虑范围（Othman, 2025）。此外，本研究使用的 Proxmox 容器方法可能超出较小组织或个人所具备的 IT 和 DevOps 技能。

本研究的贡献并非一种新的基准测试方法：第 2.5 节表明，对消费级硬件进行严格、多轴的本地推理基准测试已相当成熟，包括在与本文所用硬件相当的硬件上评估成本、能力以及相对于云替代方案的可行性等研究（Khalil et al., 2025；Knoop and Holtmann, 2026；Nguyen and Nguyen, 2025）。这些研究未涉及的是本研究所回应的具体从业者决策：面对一波并非因磨损或刻意购买决定而过时、而是因单一且日期明确的操作系统生命周期终止事件而过时的硬件，当下该怎么做。该支持终止日期为这些 PC 的所有者创造了一个具体、有时间限制的决策点：处置本可正常使用的硬件、将其迁移到 Linux，还是就地重新利用。本研究的贡献在于端到端地评估第三种选项，所用硬件恰好经历了这一转变，并结合了技术可行性、经济可行性以及使这一选择值得做出的电子废弃物和数字主权动机——现有基准测试文献聚焦于孤立的硬件性能，无法提供这种结合。

在核心贡献之外，正确性评估（第4.3.3节）中还有两个较窄的发现值得注意。第一，在该模型集合内，吞吐量与输出正确性并不同步：仅凭速度和稳定性被判定为可靠的模型，未必是给定任务中最正确的，这一结果是针对此处所评估的模型和硬件而言，而非普遍性结论。第二，用已确立的公开基准评估具备推理能力的本地模型时，暴露出若干在用EleutherAI的lm-evaluation-harness对聊天服务模型进行评分时的失败模式，这些失败既源于该测试框架与Ollama提供的聊天补全后端之间的不兼容，也源于为弥合二者而编写的自定义任务的设计。每一种都会静默地产生假阴性，而不会报错，因此仅凭所报告的分数无法察觉。所有这些都在第4.3.3节中描述，修正后的任务已发布在复现包中（Curington and Lano, 2026a），供其他尝试相同评估的实践者参考。

2 背景

2.1 避免电子垃圾

在设备报废后对其进行再利用和继续使用而非丢弃，是减少电子垃圾和碳排放（此处以二氧化碳当量（CO₂e）计量——所有温室气体均以CO₂的等效质量表示）极为有效的方式。

牛津大学对一台标准办公台式机和显示器进行的生命周期评估测得：活跃使用时为40 W，空闲时为15 W，关机后为2 W。仅电力一项，每个8小时工作日就产生约70 g CO₂e。在6年使用寿命内，全生命周期足迹约为621 kg CO₂e。其中约85%来自制造和运输，仅15%来自使用中消耗的电力（University of Oxford IT Services Department, 2022）。

同一来源在别处报告的年数值为778 kg CO₂e，但其自身测量并不支持这一数字。15%的使用阶段占比需要每年约13,900小时的40 W运行，而一年只有8,760小时。若将其理解为6年总量，则与本文所用的621 kg数字一致。

重用现有机器完全避免了这部分隐含碳排放，这正是本研究的关键所在。替换一台仍在工作的机器则会再次全额产生这部分排放。

85/15的划分不应被理解为直接适用于此处评估的服务器。配备GPU的工作站耗电远高于办公台式机——空闲时150 W、负载时410 W，而牛津测得活跃时为40 W——因此其使用阶段占比要大得多。隐含碳排放的节省不受影响：它在制造时即已固定，无论机器之后耗电多少。

估算电子垃圾影响，Othman（2025）估计，在全球范围内，Windows 10 生命周期终止（EOL）过渡可能因过早处置 1.3 亿至 1.55 亿台仍可正常使用的计算设备而产生 85 万至 100 万公吨电子垃圾——一项单一的企业政策决定所产生的电子垃圾量超过了主要发达经济体的年产生量。Othman 的估算基于发达经济体 60%、发展中经济体 45% 的处置率预测，得出全球加权平均值为 55%。若 2025 年至 2027 年间有 1.3 亿至 1.55 亿台设备被处置，且 PC 平均重量为 6.5 公斤，则电子垃圾总量接近 100 万公吨。然而，这是一种风险而非必然：Windows 10 将继续获得扩展安全更新直至 2027 年 10 月，而将老旧 PC 重新利用——使用 Linux，或如本研究中所考虑的那样作为本地 LLM 服务器——为 Othman 所预测的结果提供了一种非处置的替代方案。

2.2 数字主权

数字主权是指个人、组织和国家控制自身数据、软件和计算基础设施的能力，而非依赖其管辖权和商业条款不受自身控制的外部提供商（Floridi，2020）。在私有、本地部署的硬件上运行开放权重的大语言模型，会从上述各个维度增强数字主权：模型权重和推理执行始终处于运营者自身的管理之下，任何提示或响应数据都不会离开本地网络，且运行不依赖于第三方供应商的定价、可用性或服务条款。这一关切已经在塑造各国针对依赖非本国软件提供商的政策应对。

这种动态已在国家政策中显现：微软的 Windows 政策已促使各国政府采取行动。例如，在法国，国家宪兵队（相当于国家警察部队）将 37,000 台台式 PC 迁移至 Linux，实现了显著的成本节约（Finley，2013）。此外，继国家宪兵队取得成功后，法国政府已向所有政府部门发布指令，要求启动类似迁移（Vaughan-Nichols，2026）。法国公共行动和账务部长 David Amiel 明确表达了其动机，他表示

国家不能再满足于承认自身的依赖；它必须摆脱这种依赖。我们必须摆脱对美国工具的依赖，重新掌控我们的数字命运。我们不能再接受我们的数据、我们的基础设施和我们的战略决策依赖于我们既无法控制其规则、也无法控制其定价、演变或风险的解决方案。转型正在进行：我们的各部委、运营商和工业合作伙伴今天正致力于一项前所未有的倡议，以梳理我们的依赖关系并加强我们的数字主权。数字主权不是可选项。（France，2026）

这也与“设备中立性”关切相重叠，欧洲自由软件基金会的一份声明对此有所强调：

控制设备的制造商、供应商和平台可能会以类似于控制互联网接入网关的互联网接入连接提供商的方式履行守门人职能。设备功能和组件对终端用户越重要，守门人在数字市场中的地位就可能越稳固。（F.S.F.E.，2026）

2.3 隐私与聊天监控

除了操作系统的选择之外，使用云端 AI 工具还引发了隐私担忧。AI 提示交互和网页搜索摘要工具所暴露的个人信息远多于传统网页搜索（Schneier, 2025）。在监控资本主义时代，这些系统为云服务提供商提供了一座新的金矿，使其能够收集关于个人的高度细粒度数据（Schneier, 2025）。针对此类数据收集的选择退出机制往往有限或根本不存在。例如，Meta 已宣布，从用户与其 AI 产品交互中收集的数据将用于销售定向广告——通过隐私政策更新——且不向全球用户提供选择退出选项，除非在某些法律要求提供此类选择退出的地区（Zeff, 2025）。

将 AI 推理保持在本地可确保敏感信息留在本地系统内，无需将数据传输到外部服务器。这对于处理机密数据的医疗和金融组织，以及任何必须遵守严格数据隐私法规的机构而言尤为重要。诸如欧洲《通用数据保护条例》（GDPR）等法规对企业施加了数据处理责任，而私人家庭用户也能从面向隐私的 AI 环境中受益（Kivimäki, 2025）。

在本地——无论是在家中还是在办公室——运行一台私密、安全的服务器，可消除与外部提供商共享数据的需要。近期的硬件和软件进步使得在本地托管 AI 服务器既可行又具有成本效益。重要的是，本地 AI 系统不会查询互联网来回答问题；它仅依赖其预训练的模型数据，并且在标准配置下不需要网络连接。

2.4 非本地替代方案

与云服务相关的一些隐私担忧可以通过对提示进行匿名化的中间层来解决。其中一项服务是“Duck.AI”，它旨在通过将用户提示经由受管理的应用程序编程接口（API）路由到四个主流 AI 云模型之一，来最大限度地减少隐私担忧（Wolfe et al., 2024）。

另一种方法是基于 Gemini AI 云服务的 Google NotebookLM。该服务提供了一项隐私政策，禁止利用用户数据进行训练或将其重新利用；其界面将项目组织为包含用户上传参考材料的笔记本（Rahman et al., 2025）。

然而，这两种替代方案都未能完全消除隐私担忧，也没有解决缺乏 Windows 11 兼容性的 PC 被重新利用的问题，或实现不受限制的本地使用。

2.5 基准测试方法

大语言模型（LLM）部署正转向本地化、具有成本效益的推理。对本地 LLM 的评估需要在使用方法与硬件限制之间取得平衡。本节回顾了当代与本地 AI 基准测试相关的做法，以及在消费级硬件上的实证性能评估。

2.5.1 运行时环境

评估本地化 LLM 性能需要结构化框架，而非简单的单轴指标。Stuhlmann 等人（2025）提出了 Bench360，这是一个综合框架，跨多种硬件配置和推理引擎，对四项核心任务——通用知识与推理、问答、摘要和 Text-to-SQL——进行本地推理基准测试。他们的研究结果强调，由于没有单一硬件-引擎配置占主导地位，评估本地部署需要多维度的方法。同样，本研究中的基准设计也扩展至涵盖十个提示维度。

为解决运行一致性问题，近期研究强调统计重复和提示多样性。在高度受限的环境中，Nguyen 和 Nguyen（2025）利用 Ollama（2026a）生态系统，在单板计算机（SBC）上评估了 25 个量化开源 LLM。至关重要的是，他们的方法通过对每个模型-设备组合执行每个提示四次来减轻运行间差异，以计算稳定的平均 token 生成速度，同时部署多样化的提示工作负载以模拟真实的运行多样性。本研究采用相同方法，使用五次重复和十个提示来测量性能均值和方差。

除原始 token 速度外，现代基准测试框架还纳入整体硬件指标。Renney 等人（2026）扩展了边缘推理评估，建立了一个严格的框架，在严格的生成上限下跟踪吞吐量、首 token 时间（TTFT）和能耗（以每百万 token 的 MJ 计），以确保基线可比性。如果评估需要更详细的信息，未来工作会考虑此方法。

此外，本地部署评估越来越多地结合计算效率和模型智能。Khalil 等人（2025）通过一个双轴框架考察了面向中小型企业（SMB）的私有 LLM 服务器的可行性，该框架同时衡量负载下的服务器性能和本地化模型能力。通过利用标准化的公共基准、单个消费级 GPU 和单个 LLM（Qwen），他们的方法能够直接比较本地开源权重与专有云生态系统。这一结果验证了本研究的目标，此处扩展至涵盖一套 LLM。

2.5.2 成本估算

自托管推理在财务和运营主权方面的优势推动了对消费级硬件的广泛评估。Knoop 和 Holtmann（2026）对消费级 Blackwell GPU 进行了大规模评估，涵盖 79 种不同配置，修改了量化、上下文长度（8k–64k）和高并发工作负载。他们严格的成本模型表明，自托管消费级基础设施将运营成本限制在电力消耗范围内（每百万 token 0.001 至 0.04 美元），与预算云 API 相比成本降低 40–200 倍，并在 4 个月内实现硬件盈亏平衡。这一结果证实了本研究中发现的可行性和成本效率特征。

为了在更广泛的架构格局中审视消费级能力，Li et al. (2024) 提供了关于 LLM 加速的全面硬件视角。他们的综述指出，FP16 性能扩展主要取决于 GPU Tensor Core 的数量和代际层级，并注意到不同层级之间存在显著的性能差异，例如 RTX 4090 台式机在 Llama-2-13B 上达到 110 tokens/s，而笔记本级 RTX 4070 仅为 33 tokens/s。虽然使用的相关 GPU 与本研究的替代方案不同，但它证实了大致相似的结果。

对于中端 NVIDIA Ampere 级 GPU 这一特定架构类别，行业实证基准测试补充了学术研究。在流行部署栈中对消费级硬件进行测试，DatabaseMart (2026) 的一项实证评估表明，配备 8 GB 显存（VRAM）的 RTX 3060 Ti 通过 Ollama 运行 Mistral (7B) 时，持续达到超过 70 tokens/s，同时保持 98% 的 GPU 利用率。这表明，参数低于 13B 的模型可以在广泛可获得的消费级硬件上实现极具竞争力、低资源消耗的效率，且使用的显存少于本研究（本研究使用 12 GB 显存）。

对当前文献的回顾凸显了一个明确的共识：本地 LLM 基准测试不能依赖单次运行、单一提示类型或孤立指标。现有框架成功地将多次运行的统计平均值、多样化的工作负载维度和能效结合起来，以恰当地刻画本地性能。正确性是本研究视为与吞吐量不可分割的另一个维度：快速生成但错误的响应没有实际价值，因此第 4.3 节对同一组模型同时报告吞吐量/稳定性和输出正确性，而不是用其中一个替代另一个。本研究在方法论上（多次运行的统计平均值、多种提示维度）建立在这些见解之上，但目标不同：上述回顾的研究均未专门评估因操作系统生命周期终止过渡而被淘汰的硬件，也未将其性能发现与第 2.1、2.2 节讨论的电子垃圾和数字主权动机联系起来。本研究的贡献在于这种联系，而非新的基准测试技术。

2.6 权衡

本节探讨在实际中运行本地 AI 服务器意味着什么，评估 表 1 中总结的关键权衡。投资回报率（ROI）比较可以从典型的 LLM API 订阅成本（NanoGPT, 2025）与重新配置硬件所需的时间和精力开始。

表 1

本地 LLM 的权衡，列出主要优点和缺点。

其中若干权衡值得进一步评论。维护与系统正常运行时间在两栏中彼此对立：本地部署运营方控制硬件升级、操作系统打补丁和软件更新的时间，而不受供应商发布计划约束，但作为交换，需承担保持系统运行的全部责任，一旦本地硬件故障、网络中断或停电，没有服务级别协议或内置冗余。同样，机构知识的权衡也是双向的：内部构建和运营这一技术栈可在整个组织内培养可迁移的 AI 基础设施专业知识，这是云订阅做不到的，但它需要对员工培训和技能维持进行持续投入，而托管云服务的设计正是要完全消除这种投入。

3 本地路径

3.1 专用服务器——重新利用闲置计算系统

利用因 Windows 10 生命周期终止（Gould et al., 2024）问题而退役的企业工作站，除内存升级外几乎不需要硬件改造。在本研究中，一台可从翻新系统供应商处广泛获得的典型工作站，搭配一块消费级二手 GPU。

3.2 主机操作系统选项

给定一台配备消费级 GPU（例如具有足够 VRAM 的 NVIDIA RTX）的合适台式计算机，可以直接在主机操作系统（Linux、MacOS 或 Windows）中安装本地 AI 环境，而无需底层虚拟化（如 Proxmox）。Proxmox 虚拟化带来的开销最小，整个 GPU 专供 LLM 独占使用。

在 Linux 上安装 Ollama (2026a)，并注意避免并发应用程序，相比 Proxmox 带来的开销不到 2%；其显著优势是管理复杂度低。

在 Windows 上并发运行 LLM 的主要缺点是资源争用。在 Windows 11 工作站上，加载任何 LLM 之前，空闲时就会消耗 0.3–0.8 GB VRAM，而无头 Proxmox 配置上几乎为零，这一差异可归因于 Windows 桌面窗口管理器和 CUDA 显示上下文（Microsoft, 2025a）。

在大多数情况下，关键因素是专用 GPU 显存（VRAM）的容量，以及选择适合该限制的 LLM，因为它必须容纳 LLM 权重和上下文窗口才能获得合理性能。

3.3 多 GPU 配置

对于需要超出单块 GPU 推理能力的组织，存在两大类架构方案：在单台工作站内使用多块 GPU，以及跨多台联网机器进行分布式推理。两者在成本、复杂度和实际可及性方面各有不同的权衡。

单机多 GPU 配置在家庭实验室和小型企业环境中已相当成熟，通常使用两到四块 GPU。主要的硬件要求是足够大的机箱，以及一块提供多个全带宽 PCIe × 16 插槽、且卡与卡之间具有充足物理间距的主板。现代推理框架，包括 Ollama 和 llama.cpp，能够有效地将模型层分布到多块 GPU 上，并透明地池化 VRAM。主要的内部瓶颈是 PCIe 总线本身；更高端的 NVIDIA 数据中心 GPU 通过 NVLink 这一专有的高带宽直接互连来解决该问题。然而，本研究所使用的那类消费级 GeForce GPU 并不具备 NVLink，这意味着在这一硬件层级上 PCIe 仍是唯一的互连通路。

跨多台独立机器进行分布式推理在技术上可行，但复杂度要高得多。关键约束是节点间带宽：标准千兆以太网完全不足以应对推理过程中所需的持续大容量张量数据传输，因此专用高带宽网络——如 InfiniBand 或支持高速 RDMA 的以太网——成为先决条件。用于编排跨节点分布式推理的工具也比单机部署专用得多，这使得此类配置超出了大多数小型组织的能力范围，除非具备专门的基础设施专业知识或外部专家支持。在机器集群中回收闲置算力是高吞吐计算领域一门成熟的学科，使用 HTCondor（Thain et al., 2005）或 Open Science Grid（Pordes et al., 2007）等系统，通过排队、优先级和容错机制在共享资源集群上调度批处理和并行作业。这解决的是与本文所考虑的问题不同的问题。本研究中的单用户、单工作站部署直接服务交互式提示，无需管理争用或批处理调度。只有当推理扩展到上文讨论的分布式多机配置时，此类工具才会变得相关。

对于大多数考虑本地 LLM 部署的组织而言，单机多 GPU 方案是更实用的扩展路径。将资本支出集中于单台机器内可获得的最高 VRAM 容量——无论是 1 块、2 块还是 4 块 GPU——能带来最佳的性价比，避免分布式部署的网络约束，并且仍可使用本研究中描述的同一套工具链进行管理。

3.4 云与本地成本分析

运行本地 AI 服务器的财务成本可以与使用类似的云托管服务进行比较。由于两者的使用模式可能差异很大，因此使用本文所述的具体本地研究配置进行比较。工作站本身已经可用，因 Windows 10 到 Windows 11 的过渡而腾出，因此下面的比较考虑的是持续能耗，而非机器成本。购买了一个组件：用于本工作的二手 NVIDIA RTX 3060 以约 282 美元（210 英镑）购得。这笔支出不会改变后续的比较。与中端和高端云服务层级相比，它可在下文报告的年度节省中约 3 个月内收回；而与最便宜的层级相比，它扩大了本地运行仅凭能耗就已落后的差距。由于是二手的，它也不带有新的隐含制造足迹，即第 2.1 节中讨论的不对称性。我们特别考虑运行本地 AI 服务器的能源成本是否比使用远程商业 AI 云服务更贵或更便宜。

关于本地 AI 服务器工作站的假设，由单个研究人员在家庭环境中使用，见 表 2。能源成本数字是 Ofgem 对 2026 年第三季度受监管价格上限下的英国国内电力单位费率：26.11 便士/千瓦时，含增值税（Ofgem, 2026）。按 2026 年平均英镑/美元汇率 1.3443 换算为美元。使用受监管费率而非单一家庭账单，以便分析反映全国代表性电价，而非某一处房产的电价。

表 2

本地 LLM 使用和成本假设，用作能源和成本分析的基础。

服务器功耗是直接测量的，而非估算的，使用 Lowenergie 插入式市电电能监测器在墙式插座处测量。该仪器在 100 W 以下额定为 ±2 W，在 100 至 3,000 W 范围内为 ±2%（BS 1363 / BS 5733）。它记录到机器空闲时为 150 W，在推理负载下持续平均为 410 W。该容差在加载数值下约为 ±8 W，或在推导的能源成本上约为 ±2%。与使用假设本身的不确定性相比，两者都很小。

这些是墙式插座处的全系统数值。它们涵盖双处理器工作站、其内存和存储以及 GPU 整体，而非仅 GPU 的数值或组件额定值之和。因此第 2.1 节中的比较是同类比较：牛津的 40 W 也是整机活动测量值。

每天 500 条提示词的平均值代表中等偏重的单用户工作负载，设定在合理个人使用量的高端，以免低估本地运营成本：若均匀分布在 10 小时工作日内，大约相当于每 72 秒一条提示词，与第 4 节所述结合编码、编辑和文档任务的 AI 辅助工作流一致，而非孤立的偶发查询。每条提示词 40 秒的加载时间估算近似于假设的 1,500 token 响应长度的生成时间；这与第 4 节所述六款生产就绪模型的总体平均吞吐量（在所报告的提示词类型上约为 43 token/s）一致，该吞吐量生成此长度的响应约需 35 秒，从而验证 40 秒是一个合理且略偏保守的整数估算。

对于此估算，我们假设工作站仅在工作日的工作时段供电。将空闲功耗需求加上本地 AI 评估提示词时的额外负载，即可得出功耗估算。假设按这些活动水平每日使用，可将本地能源成本与相同提示词-响应活动的云服务成本进行比较。为提供潜在的云成本范围，考虑了三种模型——一款小型高效开源模型（与本地 AI 服务器匹配）、一款中端商业模型和一款高端顶级模型，详见表 3。这些模型的成本为按使用量付费，基于提示词数量和每条提示词的 token 数。

表 3

代表性云 LLM 的成本指标，将小型、中型和大型模型与本地能源支出进行比较。

每条提示词定价来源于 NanoGPT 汇总的多提供商定价表（NanoGPT, 2025），访问日期 2025-09-26。

本地 AI 服务器 1 年的能源成本为 207 美元。如果全天 24 小时供电而非仅工作时段，成本将增至每年 354 美元。相比之下，对于每年发出 100,000 条提示词的组织，小型模型的云成本（48.75 美元）比本地服务器的电力足迹低约四倍。然而在中端，经济动态发生逆转；云 API 成本（1,562 美元）约为本地能源成本的 7.6 倍，每年可节省超过 1,350 美元的运营费用。在高端，云成本（12,000 美元）变得极其高昂，约为本地电力成本的 58 倍。

尽管小模型在原则上仍然更便宜，而且 Gemini 3 Flash 的原始能力也可能与之相当或更强，但成本和能力并不是决定哪个选项合适的唯一因素。这一点得到了各提供商自己发布的评估结果的佐证——Gemini 3 Flash 在 GPQA Diamond 上报告为 90.4%，在 SWE-bench Verified 上为 78%（Google, 2026），Claude 4 Opus 在 SWE-bench Verified 和 Terminal-Bench 上也报告了强劲结果（Anthropic, 2025）——不过，无法与本研究自身的 GSM8K/HumanEval/TruthfulQA 结果（第 4.3.3 节）进行基准匹配的直接比较：两家提供商都已针对这些模型代际转向更难、更新的基准套件，且均未公布此处使用的经典 GSM8K/HumanEval/TruthfulQA 测试集的分数。所引用的数字仅作为独立证据，表明这些是能力很强的当代系统，而非同类比较的分数对比。对于 Claude Pro 等固定费率订阅产品，这一点更加成立；如上文所述，对于合适的使用模式，这类产品可能比此处使用的按量计费比较更具能力且仍然更便宜。

如第 2.2 节所论述，本地部署从三个维度增强了数字主权——谁控制模型执行、数据是否离开本地网络，以及对供应商定价、可用性和服务条款的独立性——当隐私、监管合规或长期成本可预测性是决定性要求而非单纯的每美元性能时，任何云替代方案，无论多么便宜或强大，都无法替代这些维度。这一论证刻意基于风险和价值观，而非量化的成本比较：与本节其他地方的美元和碳足迹数字不同，主权的价值没有天然的共同单位，其实际权重将因读者自身的监管风险和风险承受能力而异。第 2.2 节的例子——GDPR 下的合规义务，以及各国政府对供应商依赖的政策回应——表明这一权重可能相当大，有时还带有真实、可量化的利害关系（合规罚款、供应商锁定的转换成本），尽管本研究并未尝试直接为其定价。

对于这些限制不适用的用例，云选项可能确实是更明智的选择；本研究的贡献在于描述本地替代方案何时变得可行且更可取，而非断言它总是如此。请注意，中端和高端模型的全部功能无法在低成本本地 AI 服务器硬件上复制。运行工作站的持续能源成本与资本支出或维护服务相比可以忽略不计——这些因素未在本评估中考虑。

因此，重新利用的回报取决于哪个云服务层级是现实可行的替代方案。在日均500条提示、每条提示执行时间40秒的情况下，本地运行比中端层级便宜得多（约7.6倍），比高端层级便宜约58倍，相比前者每年可节省超过1350美元。但面对最便宜的按量计费层级则不然：该选项仍比本地服务器的电费本身便宜约四倍，即使在最大利用率下也仍然更便宜。重新利用的理由建立在与中端和高端层级的比较之上，以及本节其他部分所述的主权、隐私和碳考量之上，而非在价格上击败所有云选项。其中哪些适用取决于使用模式、硬件可用性，以及对数据隐私和控制的重视程度。

另一个值得量化的情景是最大理论利用率：如果工作站以满负载连续运行、零空闲时间，按上述每条提示40秒的加载时间计算，它每天将处理86,400秒/40秒 = 2,160条提示，或在365.25天的一年内处理约788,940条提示。在持续满负载下（410瓦，无空闲贡献），这产生的年电费约为1,262美元——而相同提示量通过Gemini 3 Flash的费用约为394美元（表3），因此即使在这种极端利用率下，云定价仍便宜约3.2倍。然而在这种规模下，每条提示的成本几乎完全由电价本身主导，因为空闲开销变得微不足道：同一情景在与Gemini 3 Flash定价打平的电价约为0.11美元/千瓦时，远低于上文使用的英国电价，并且在许多低成本电力地区可以实现。这一情景对于持续饱和硬件而非遵循交互式、工作时间使用模式的批处理或半监督学习工作负载来说并不牵强，并表明在电力便宜的地方，持续、高利用率的本地部署甚至可以与最便宜的按量计费云层级在成本上竞争。

第 2.1 节中提出的另一个考量是本地推理与云端推理的碳足迹，这与成本不同。将英国政府公布的电网电力碳强度因子（0.196 kg CO₂e/kWh，发电与输电损耗合计）（Department for Energy Security and Net Zero, 2025）应用于本地服务器在同样每年 100,000 次提示量下的年度使用阶段电力消耗（589.7 kWh），得出估计为 115 kg CO₂e/年。每个云模型的对应数值则使用 EcoLogits（Rincé and Banse, 2026）估算，这是一种开源方法，根据模型架构、实测 token 吞吐量以及各提供商的数据中心位置、电能利用效率和电力结构来建模每次请求的能耗与排放。在此使用量下，所得估计值约为：Gemini 3 Flash 为 170 kg CO₂e/年（范围 113–228），Gemini 2.5 Pro 为 1,132 kg CO₂e/年（范围 674–1,590），Claude 4 Opus 为 1,387 kg CO₂e/年（范围 890–1,884）——这意味着本地服务器自身使用阶段的碳足迹甚至低于此处比较的最便宜云层级，约为其 1.5 倍，而相对于中端和高端层级则升至约 10–12 倍。这些数据汇总于表 4。

表 4

在同样每年 100,000 次提示量下（表 2），本地服务器与表 3中三个代表性云模型的估计年度使用阶段碳足迹对比。

云端数值使用 EcoLogits（Rincé and Banse, 2026）排放估算方法计算；本地数值使用英国政府公布的电网电力碳强度因子（Department for Energy Security and Net Zero, 2025）。

这些云端估算存在若干注意事项。三家提供商均未披露底层模型架构；因此 EcoLogits 对闭源模型（Claude 4 Opus 和 Gemini 2.5 Pro 获得相同的建模架构，该工具自身将其标记为低置信度替代）使用通用参数数量估计，而 Gemini 3 Flash 在撰写时在 EcoLogits 数据库中尚无生产模型条目，因此估算改用其预览版代理条目。数据中心位置、电能利用效率和水利用效率同样是提供商级别的默认值，而非对具体请求路径的实测。因此，这些估算应被视为数量级比较，而非精确数值。

该比较还遗漏了双方在实体制造足迹方面的差异：第2.1节的生命周期研究结论指出，一台典型台式机约85%的终身排放来自制造与运输（University of Oxford IT Services Department, 2022），这一结论完全适用于任何新的云端硬件，而本研究的本地服务器则不产生任何此类足迹，因为它复用了已存在的硬件——上文遗漏的这一不对称性只会进一步拉大差距，使本地服务器更为有利。表3中的云定价以及此处使用的EcoLogits模型数据库条目均为各自访问日期（2025-09-26和2026-07-30）的快照；云服务提供商会频繁调整定价并发布新模型，因此在本研究发表后较长时间才做出部署决策时，应依据当前数据重新核实这些比较结果，方可加以依赖。

上述云成本比较使用的是按量计费的 API 定价，这适合本研究稳定、脚本化的提示量作为基准。面向消费者的包月订阅（例如月度 AI 助手套餐）代表不同的商业层级：提供商根据众多订阅者的预期平均使用量来定价，因此重度用户消耗的等效按量 API 价值可能达到其订阅费用的许多倍，才会触及此类套餐通常设有的公平使用或速率限制。因此，对于合适的使用模式，包月订阅可能比按量 API 定价便宜得多，但受这些限制约束，且并非为本研究所建模的持续、脚本化、高负载工作负载而设计。

4 评估方法

本研究从零开始搭建 AI 推理服务器，使用自定义基准测试套件（Curington and Lano, 2026a）。此处描述的技术与工具链格局正在快速演变；作者承认，到发表时此示例可能已经过时。用于评估的工作站曾多年同时作为 Windows 桌面和虚拟 Linux 服务使用。在擦除 Windows 10 启动盘后，加装了一块二手消费级 GPU。这台专用 AI 服务器硬件是 Dell Precision T7810 塔式工作站，双 Xeon 12 核 CPU（48 线程）主频 2.6 GHz，192 GB ECC RAM，1 TB NVMe SSD 存储，以及一块 12 GB VRAM 的 NVIDIA GeForce RTX 3060 GPU。该服务器未连接显示器，因为 GPU 纯粹用于计算。本研究承认这台高端企业级工作站比典型的过时消费级 PC 更强大，但服务器 CPU、内存和 PCIe 总线的优势很小，因为大多数 LLM 计算预计发生在 GPU 上。

服务器的组织方式如图 1所示，Proxmox 9 采用 LXC 虚拟化，支持 Ollama 和 Open WebUI LLM 组件，均按照已发布的指南搭建（Srinivas et al., 2025）。Proxmox 是基础虚拟化控制层，支持多个服务和应用程序实例。在 Proxmox 中启用了多个轻量级虚拟 Linux 容器（LXC）。其中一个 LXC 容器被配置为托管 Ollama，负责维护本地 LLM 模型集合并动态响应输入提示——将 LLM 输出返回给请求进程。Open WebUI（Open WebUI, 2026）使用第二个 LXC 容器为控制 Ollama 提供客户端前端。

图 1

主要的资源约束是 12 GB 显存，因此 LLM 的选择专门针对这一水平。该 GPU 的最大额定功耗为 170 W；全程将其限制在 140 W，因此本文报告的所有吞吐量数据均是在该限制下获得的。由于未在移除限制的情况下进行测量，因此未单独量化该限制对吞吐量的影响。因此，这样的服务器在家庭环境中不需要异常高的功耗。本研究的结果（第 4.3.1 节）证实，12 GB 显存足以支持在四位量化下最多约 140 亿参数的模型。这并不是最低要求：据报道，7B 级模型在 8 GB 上运行良好（DatabaseMart, 2026）。显存需求随模型大小、量化和上下文长度而变化。

服务器通过 Microsoft Visual Studio Code 进行代码开发访问（Falkingham, 2025），并通过 Web 浏览器访问 Open WebUI 进行聊天会话。

模型功能受到有限硬件资源的严重制约：在实践中，两个关键的计算约束是上下文窗口大小和分配给模型权重的物理内存（Na et al., 2024）。该服务器提供了基本但足够的 GPU，以及足够的 CPU 和内存来运行一系列模型，不过本评估中未考虑的调优可能会产生显著影响。第 4.1 节详细讨论了这两个约束，包括看似适合 GPU 内存的模型仍会溢出到 CPU 的情况；其影响在下面的评估结果中可见。

4.1 上下文约束

本地 GPU 的物理显存容量是实践中 LLM 运行的主要约束。权重规模超过可用显存的模型会溢出到主机系统 RAM 和 CPU 计算，从而带来严重的性能损失。因此，选择权重完全适合显存的模型是配置中最关键的第一步。

第二个不太明显的约束是 token 上下文窗口（Na et al., 2024）。存储活动上下文的键值（KV）缓存也驻留在显存中，其大小随上下文长度线性增长。无论底层模型架构支持什么，Ollama 都保守地默认使用 4,096 token 的上下文窗口。例如，Gemma4:e4b 支持高达 128k token 的上下文长度，但除非显式重新配置，Ollama 将仅使用 4k。

表 5 展示了 Gemma4:e4b 在 12 GB GPU 上三种上下文长度下的 VRAM 分配情况。模型权重占用 9.6 GB，Ollama 运行时约 0.4 GB，剩余约 2.0 GB 可用于 KV 缓存。在默认 4k 上下文下，KV 缓存需要约 0.5 GB，剩余 1.5 GB 的充裕余量。将上下文翻倍至 8k 会使 KV 缓存增至约 1.0 GB，空闲余量降至 1.0 GB——这是附加源文件或文档作为上下文时稳定运行的最低推荐值。增加到 16k token 会完全耗尽剩余空间，使系统处于溢出阈值。与模型加载时 VRAM 耗尽立即且可预测不同，上下文驱动的溢出会在会话中途随着上下文窗口填满而发生，若不主动监控则难以察觉。

表 5

Gemma4:e4b 在 12 GB GPU 上三种上下文长度下的 VRAM 分配。

模型权重（9.6 GB）和 Ollama 运行时（约 0.4 GB）是固定的；只有 KV 缓存随上下文长度扩展。KV 缓存大小为近似值。8k 配置经实证确认仍在 GPU VRAM 范围内；16k 处于溢出阈值。溢出发生在会话中途上下文填满时，而非模型加载时。

Practitioners should therefore treat context length as a tunable parameter requiring explicit validation rather than accepting the Ollama default uncritically.

4.2 Structured 10-prompt design

To evaluate LLMs across a range of production activities, a suite of prompts are designed to exercise key dimensions across: Factual retrieval (look up trained knowledge); Output length (directly determines total token generation time); Reasoning depth (as “thinking” models use far more tokens); Context/input length (stresses memory bandwidth and KV cache); Task type (various decoding patterns such as code, prose, and structures); Domain (knowledge retrieval vs. reasoning); Instruction complexity (multiple constraints to stress instruction-following).

4.2.1 Warm-up prompt (disregarded in stats)

“Write a simple Python function that calculates the factorial of a given integer. Omit explanations, bare code only.”

After this initial warm-up, the structured prompts (P₁–P₁₀) shown in Table 6 are applied.

Table 6

LLM capability test suite: ten prompt workload dimensions (P₁–P₁₀) used for inference benchmarking.

Dimensions are grouped by the primary inference behavior under stress. P₉ token generation rate is recorded independently at turns 1, 3, and 5 (denoted P₉-T₁, P₉-T₃, P₉-T₅) to quantify KV cache accumulation effects.

4.3 Results

This study evaluates local LLM deployment along two axes treated as equally important throughout: inference throughput and stability under realistic workloads (Section 4.3.1), and output correctness against established public benchmarks with gold answers (Section 4.3.3). Neither axis alone characterizes practical utility as a workflow assistant, so both are reported together and interpreted jointly. Table 7 presents throughput and stability first, since these determine which models are viable candidates at all; Section 4.3.3 then reports correctness for the same set.

Table 7

Quantitative evaluation of local inference performance across all ten benchmarked prompt dimensions (N = 5).

Processing velocity () is in tokens per second; execution time () in seconds. Statistical dispersion (σ) reflects variance across five independent runs on the evaluation host (Dell-T7810; NVIDIA GeForce RTX 3060, 12 GB VRAM), using Ollama v0.24.0 (Ollama, 2026b). Quantisation levels are the defaults Ollama serves for each unmodified model tag. Full per-run data, including per-turn multi-turn detail (P₉-T₁, P₉-T₃), is deposited as described in the Data availability statement.

4.3.1 Evaluation of inference performance and stability

4.3.1.1 Measurement procedure and settings

Eight open-weight models were benchmarked across ten workload dimensions (P₁ through P₁₀). Two were included as boundary cases: deepseek-r1:1.5b, whose limited reasoning capability informal use had already indicated, and gpt-oss:20b, whose 20-billion-parameter weights would clearly exceed the 12 GB VRAM budget. They establish the boundaries within which the remaining six perform reliably. Performance metrics were captured directly from the Ollama daemon (eval_count and eval_duration), eliminating network and inter-process overhead. Each model underwent an unrecorded warm-up run to promote complete VRAM allocation, followed by five (N = 5) timed trials. Table 7 reports the mean token generation rate () and its standard deviation (σ_TPS), with mean wall-clock elapsed time () per prompt.

Figure 2 shows the same means ranked by throughput, with error bars giving the standard deviation of the per-prompt means. Two features are clearer there than in the tabulated figures. The eleven-fold spread between fastest and slowest is dominated by a single outlier rather than distributed across the group, and the six stable models span only a factor of two. Within that cluster, on-disk footprint does not predict throughput: the largest model of the six (gemma4:e4b, 9.6 GB) is also the fastest, and the smallest (granite4.1:8b, 5.3 GB) is second.

Figure 2

All throughput measurements were taken with Ollama's per-model default generation settings. The benchmark client issues chat requests without an explicit options payload, so sampling temperature, context window, and any reasoning or “thinking” behavior follow each model's own packaged defaults rather than values chosen for this study. This reflects the configuration a practitioner obtains by pulling a model and running it unmodified, which is the deployment scenario this study addresses. One consequence follows and is stated explicitly. Reasoning-capable models are not constrained to minimal output on short prompts. Measured at default settings, gemma3:12b, granite4.1:8b, phi4:14b and mistral-nemo:12b answer P₁ in 24–52 generated tokens, whereas qwen3:14b and deepseek-r1:1.5b emit 558 and 508 tokens, respectively for the same one-line answer, and gemma4:e4b 71. P₁ therefore constitutes a minimal-output baseline only for the four models that do not reason by default, and the P₁ figures for the other three should not be read as measuring minimal-output throughput. On P₅, the context-saturating summarisation prompt, the same models receive approximately 3,290 prompt tokens and generate between 89 and 631: granite4.1:8b 89, phi4:14b 124, mistral-nemo:12b 161, gemma3:12b 177, deepseek-r1:1.5b 382, qwen3:14b 429 and gemma4:e4b 631. The ordering is not that of P₁: the two models that reason by default are no longer the most verbose, because a task genuinely requiring a long answer elicits one from the others too, so the penalty those models pay is specific to short prompts rather than general. Counts are means of three runs at default settings; gpt-oss:20b could not be served at the time these were measured. The correctness evaluation of Section 4.3.3 used explicit settings by contrast—greedy decoding, chat template applied, one request at a time, and a generation budget stated in that section—because those benchmarks require deterministic scoring. The context window was left at Ollama's default there as well.

Three terms are used throughout with specific meanings. A model is stable if its token-generation rate varies by less than 2.0 tokens/s (σ_TPS) across all twelve measurement points, the criterion that separates the six GPU-resident models from gpt-oss:20b, whose spilling produced a coefficient of variation as high as 0.43. A model is GPU-resident if its weights and KV cache fit entirely within VRAM at the tested context length, without CPU offload. Production-ready denotes the conjunction of these two properties and is a claim about throughput and stability alone; Section 4.3.3 shows that output correctness varies substantially within the set of models meeting it.

4.3.1.2 Production-ready cluster: stable, GPU-resident models

Six of the eight evaluated models— gemma4:e4b (Google), gemma3:12b (Google), qwen3:14b (Alibaba), granite4.1:8b (IBM), phi4:14b (Microsoft), and mistral-nemo:12b (Mistral)— exhibit characteristics consistent with dependable production deployment on throughput and stability grounds. Each fits entirely within the 12 GB VRAM budget of the evaluation GPU, and this containment is directly reflected in their statistical stability: σ_TPS remains below 2.0 tokens/s across all twelve measurement points for every model in the group. Throughput scales inversely with parameter count within this tier, from qwen3:14b and phi4:14b at approximately 29–36 tokens/s, through gemma3:12b and mistral-nemo:12b at 34–46 tokens/s, to granite4.1:8b at 51–58 tokens/s and gemma4:e4b at 59–65 tokens/s (ranges over the ten single-turn prompts; the multi-turn prompt P₉ is treated separately below)—a pattern consistent with established GPU memory-bandwidth constraints on token processing.

4.3.1.3 Degradation under resource pressure

Performance degrades predictably and modestly under the two principal resource stresses tested. Under the long-context summarisation workload (P₅), which maximizes KV cache occupancy, throughput decreases by 7%–12% relative to factual retrieval (P₁) across all six models, from gemma4:e4b's 63.4–59.1 tokens/s (−7%) to granite4.1:8b's 57.9–51.1 tokens/s (−12%). Under multi-turn dialogue accumulation (P₉), throughput declines by 6%–10% from turn 1 to turn 5, reflecting the progressive growth of the KV cache as conversational context fills the available context window. Both degradation patterns are smooth, bounded, and predictable—characteristics essential for operators planning hardware capacity for production workloads.

4.3.1.4 deepseek-r1:1.5b: a speed outlier with important caveats

deepseek-r1:1.5b is the performance outlier of the evaluation, achieving 190–201 tokens/s—approximately three to four times the throughput of the next fastest model. This result reflects the model's very small parameter count of 1.5 billion, which occupies proportionally less GPU memory bandwidth per forward pass. However, the same small parameter count imposes a correspondingly limited knowledge capacity, constraining the factual depth and contextual reasoning the model can reliably access. The R1 architecture employs extended chain-of-thought reasoning whose reliability is known to be highly sensitive to model scale; at 1.5 billion parameters, outputs on knowledge-intensive tasks warrant independent verification before use.

The benchmark further reveals a statistically significant instability on the code generation prompt (P₄). Four of the five timed runs completed normally with elapsed times of 10–16 s; the fifth required 88.3 s—a factor of six above the expected range. This single outlier inflates the mean elapsed time for P₄ to 28.8 s and the standard deviation to 33.4 s, producing the condition discussed in Section 4. The cause is consistent with a transient context-length overflow or runtime garbage collection event. The limited knowledge base and the observed runtime unpredictability together position deepseek-r1:1.5b as a performance reference point, suitable for narrow, well-defined tasks but not for general-purpose production deployment.

4.3.1.5 gpt-oss:20b: VRAM exhaustion and CPU spilling

The gpt-oss:20b (OpenAI) model presents the contrasting failure mode. At twenty billion parameters its weights require approximately 14 GB of storage, exceeding the physical 12 GB VRAM limit of the evaluation GPU. Ollama responds by partitioning the model, loading as many layers onto the GPU as VRAM allows and offloading the remainder to host RAM for execution on the CPU. The resulting throughput of 17–19 tokens/s is a direct consequence of this hybrid path—roughly half the throughput of the slowest fully GPU-resident model (qwen3:14b at 29–32 tokens/s) and approximately one-third of the group mean.

CPU spilling also produces markedly elevated elapsed-time variance: the coefficient of variation () reaches 0.43 on P₁ and 0.41 on P₅ for gpt-oss:20b, against below 0.05 for all six GPU-resident models, arising from non-deterministic PCIe transfer timing that is sensitive to system load rather than to prompt characteristics. Counter-intuitively, on P₉ the model holds flat throughput across turns one, three and five (17.9, 17.7 and 17.8 tokens/s) against the 6%–10% decline in GPU-resident models, consistent with VRAM already being saturated at load time.

The evaluation thus confirms 12 GB as sufficient for reliable, stable inference on models in the 12–14 billion parameter range. The one model tested above that range exceeded the VRAM budget and showed reduced, unstable throughput. That follows from exceeding VRAM rather than from parameter count, and one case does not establish how larger models behave in general. Organizations needing models of this scale should choose a GPU that holds the weights and KV cache in full (Section 4.1), or adopt the multi-GPU configuration of Section 3.3.

4.3.2 Cost implications of measured throughput

The cost estimate in Section 3.4 assumes a single, model-independent loaded generation time of 40 s per prompt (Table 2). The measured throughput data in Table 7 allow this assumption to be refined for the two boundary-case models identified above. Approximating loaded generation time as the 1,500-token output length divided by measured throughput gives ≈7.9 s/prompt for deepseek-r1:1.5b and ≈83.8 s/prompt for gpt-oss:20b—roughly five times faster, and twice as slow, respectively, as the flat 40-s assumption used elsewhere.

At 500 prompts/day, deepseek-r1:1.5b's faster generation reduces its share of costly loaded-power operation, yielding an estimated annual electricity cost of $125 ($0.00125/prompt)—lower than the $207 flat-assumption baseline. gpt-oss:20b, by contrast, requires approximately 11.6 h of continuous loaded operation to process the same daily volume—exceeding the 10-h power-on window assumed throughout this study before any idle time is even counted; it cannot in practice sustain 500 prompts/day within a standard working day on this hardware. Computed at full utilization, its estimated annual electricity cost is approximately $335 ($0.00335/prompt).

Set against Gemini 3 Flash's $0.0005/prompt (Table 3), the fastest and slowest locally-hosted models therefore bracket a cost range of roughly 2.5 × to 6.7 × Gemini 3 Flash's per-prompt price, driven entirely by measured throughput rather than the flat assumption used for the six production-ready models. This confirms that for gpt-oss:20b, throughput—not electricity cost per se—is the binding constraint on production use: even its modest cost premium over cloud pricing is secondary to its inability to keep pace with the assumed workload at all.

4.3.3 Qualitative correctness evaluation

Throughput and elapsed time characterize how fast a model responds, not whether it is right, and local models are prone to hallucination, as noted in Section 1. This study therefore additionally evaluates each model's output correctness using the EleutherAI Language Model Evaluation Harness (Gao et al., 2024), run against three established benchmarks covering distinct domains: GSM8K (Cobbe et al., 2021) (grade-school arithmetic reasoning, scored against gold numeric answers), HumanEval (Chen et al., 2021) (Python code correctness, verified by executing each generated solution against its unit tests), and the generative variant of TruthfulQA (Lin et al., 2022) (resistance to generating plausible-but-false answers). TruthfulQA's questions are adversarially constructed, crafted so that some humans would answer falsely from a common misconception, so scores on it run far lower than the same models achieve on general factual questions: its authors report 94% human performance against 58% for the strongest model they evaluated.

4.3.3.1 Evaluation environment

This qualitative evaluation was conducted on a separate Windows 11 workstation from the throughput evaluation described above, using the same Ollama-served model tags and, by deliberate choice, the same NVIDIA RTX 3060 (12 GB VRAM) GPU, sourced specifically to support this comparison. Output correctness depends on model weights and sampling parameters rather than on the GPU executing inference, so only the throughput and elapsed-time measurements are specific to the evaluation hardware; the matching GPU additionally rules out any concern about quantised-kernel differences across architectures. gpt-oss:20b initially could not be evaluated: under Ollama v0.32.3 every request failed with a tensor-size error arising from the model's MXFP4 mixture-of-experts layout, a defect in the serving runtime rather than in the model, the hardware or the evaluation harness. It was resolved in a later Ollama release, and the results reported here were obtained under v0.32.6.

4.3.3.2 Generation settings

Decoding was greedy (do_sample=false), the chat template was applied, and requests were issued one at a time. The context window was not set, so all correctness runs used Ollama's default of 4,096 tokens, the same default discussed in Section 4.1. The requested generation budget was 4,096 output tokens for GSM8K and TruthfulQA, and 16,384 for HumanEval on every model except gemma3:12b, the larger figure having been adopted after the initial runs showed reasoning models filling the budget before emitting an answer.

That larger budget was not effective, and the reason is invisible from the scores. Ollama clamps generation to the context window irrespective of the requested budget: a request for 16,384 output tokens against a 4,096-token window returns after 4,096 tokens of prompt and completion combined, reporting a length stop. Raising the budget without raising the window therefore had no effect for any model that does not shift its context. Two of the eight do: deepseek-r1:1.5b and qwen3:14b continue past the window by discarding the oldest tokens, logging 482 and 114 such shifts, which is how they reached 16,380 generated tokens. For the other six the effective ceiling was 4,096 tokens throughout, so the distinction between the two budgets is immaterial to their scores.

4.3.3.3 A custom HumanEval adapter

HumanEval was scored using a custom evaluation task written for this study, because neither variant packaged with the harness functions correctly against a chat-completions backend. The standard humaneval task assumes a raw-completion model and terminates generation on stop sequences that a chat-tuned model does not emit at the expected position. The humaneval_instruct variant instead relies on prefilling the start of the assistant turn, which Ollama's OpenAI-compatible endpoint accepts and silently ignores. Both therefore score text the model was never prompted to produce. The task written to bridge this, humaneval_local_chat, is published in full at Curington and Lano (2026a).

Each problem is presented as a single user turn reading “Write a solution to the following problem and make sure that it passes the tests:”, followed by the problem's function stub in a fenced Python block. Generation is greedy and without a stop sequence, under the budget and context window stated above. The reply is then reduced to executable code in four steps: internal reasoning spans are discarded; the last fenced block defining the target function is selected, or failing that the last block containing any definition; top-level statements other than imports, definitions and assignments are removed; and the dataset stub is prepended. The result is executed against the problem's hidden unit tests (which the model never sees) and scored pass@1.

4.3.3.4 Why the adapter behaves as it does

Each step addresses a failure mode that produces a false negative without raising an error. Models frequently emit a solution followed by a usage example or a call to doctest. Scoring the trailing block would discard the answer, so the extractor takes the last block that defines the target function.

Models also append their own test assertions. These are not always correct, and they abort the module before the hidden test runs. Top-level statements are therefore removed, following the standard practice of scoring the completion rather than the model's own scaffolding. Reasoning models present a third problem: they return their chain of thought in a separate field that the harness does not read, so a model filling the context window while reasoning is recorded as producing an empty answer rather than as failing to terminate. No generation budget prevents this, for the reason given above: the window, not the budget, is the binding limit.

Applying this to the eight models, 64 of 1,312 responses were empty for that reason and are counted as failures in Table 8: 54 from deepseek-r1:1.5b, six from qwen3:14b and four from gemma4:e4b, which also reasons by default. In each case the reasoning field was populated—between 6,836 and 12,106 characters for the gemma4:e4b items—while the content field remained empty. Of the remaining 1,248, every one yielded extractable code, a residual parsing-failure rate of zero.

Table 8

Output correctness across all eight benchmarked models [EleutherAI lm-evaluation-harness (Gao et al., 2024)], ordered as in Table 7.

GSM8K is scored by flexible-extract exact match, HumanEval by pass@1 against each problem's unit tests, and TruthfulQA by the BLEU and ROUGE-1 “acc” metrics; Section 4.3.3 defines these and states their limitations. The final column gives the proportion judged truthful by an independent LLM judge (llama3.3:70b); it runs consistently higher than the overlap metrics because that judge is lenient rather than more accurate, as Section 4.3.3 reports. Quantisation levels are the defaults served for each unmodified model tag. The final row repeats mistral-nemo:12b at Q4_K_M as a control. The per-item generations and scores behind every figure in this table, and the judge verdicts and human adjudication behind the final column, are deposited at Curington and Lano (2026b).

4.3.3.5 What each metric measures

Each benchmark is scored differently, and each carries a different interpretation. GSM8K uses flexible-extract exact match: the final numeric value is extracted and compared with the gold answer, which suits arithmetic word problems but rewards only the final figure, taking no account of whether the reasoning was sound. HumanEval uses pass@1: each solution is executed against the problem's own unit tests and counted correct only if all pass, the strictest of the three since it tests behavior rather than resemblance, but binary and giving no credit to a near-miss. TruthfulQA uses the BLEU and ROUGE-1 “acc” metrics, the fraction of items where the answer shares more n-grams with a true reference than with any false one. These are proxies for truthfulness rather than measures of it, since they compare wording rather than meaning: an answer may be truthful while sharing little vocabulary with the reference, or share vocabulary while being false. The original benchmark uses a fine-tuned GPT-judge classifier for this, which was unavailable here, so the TruthfulQA figures should be read as a relative comparison between the models evaluated rather than against externally reported scores.

4.3.3.6 Overall correctness

Table 8 reports correctness across the three benchmarks for all eight models. GSM8K performance is high and comparatively compressed (70.4%–92.0% flexible-extract exact-match): arithmetic word-problem reasoning at this difficulty is well-handled across the group, regardless of parameter count or architecture. HumanEval is more dispersed (47.0%–91.5% pass@1) and does not follow the ordering of either GSM8K or the throughput results in Section 4.3.1, though six of the eight models fall within a narrow 83.5%–91.5% band. Code generation therefore separates the weakest models sharply while distinguishing the stronger ones only modestly. A small number of items returned no content at all and are counted as failures, at most 2.4% on GSM8K and 1.1% on TruthfulQA, so those scores are marginally conservative; the much larger effect of this on HumanEval is discussed below.

4.3.3.7 Correctness and throughput are largely independent

Figure 3 plots the two against each other, and shows no trend: within this hardware envelope, knowing how fast a model runs says little about how often it is right. Combining correctness with the throughput and stability findings of Section 4.3.1 identifies gemma4:e4b as the strongest model on both axes simultaneously: it is the fastest of the six throughput-stable models (59–65 tokens/s) and the most correct on HumanEval (91.5%), with 90.4% on GSM8K. granite4.1:8b is the next most attractive all-rounder, second fastest at 51–58 tokens/s and the strongest model on GSM8K (92.0%), at 83.5% HumanEval. gemma3:12b and phi4:14b offer comparable correctness at more moderate throughput.

Figure 3

qwen3:14b achieves near-identical correctness to gemma4:e4b (90.2% HumanEval) but is among the slowest models measured, and its practical cost is greater than tokens per second alone conveys. Completing the same 164-problem HumanEval set took 5 h 11 m against gemma4:e4b's 1 h 28 m on identical hardware, a 3.5 × difference where the throughput figures differ by roughly 2 ×, because reasoning models emit substantially more tokens per answered task. Tokens per second understates the latency cost of reasoning models for task completion, and wall-clock time to complete a fixed workload is the more informative measure where a model reasons by default.

This has a direct consequence for the throughput-stable cluster identified in Section 4.3.1. mistral-nemo:12b, grouped there among the six stable, GPU-resident models, is the weakest of that cluster on HumanEval (62.8%) despite competitive throughput and solid GSM8K performance (76.0%). Throughput and stability describe how reliably a model responds, not whether the response is correct, so model selection should weight the relevant correctness axis alongside throughput: HumanEval for coding use, GSM8K for arithmetic and quantitative use. Re-evaluating mistral-nemo:12b at Q4_K_M, matching the quantisation served for the other models, raises its scores only marginally (77.3% GSM8K, 65.2% HumanEval, 48.8% and 52.4% TruthfulQA), so the difference is attributable to the model rather than to the quantisation its default tag happens to supply.

4.3.3.8 The boundary cases fail in opposite directions

The two models included as boundary cases fail in opposite directions, and neither failure is the one their throughput figures would suggest. deepseek-r1:1.5b, by far the fastest model measured, is the least correct on every benchmark; more revealingly, it frequently fails to answer at all, returning no solution for 33% of HumanEval problems. Of the items it did complete, 70.0% were correct. Its binding limitation on this hardware is therefore failure to return an answer rather than poor code, a distinction that a single pass@1 figure conceals. The mechanism is the interaction of a long reasoning chain with a 4,096-token context window, not the generation budget alone: once the chain exceeds the window the problem statement is discarded from context, and raising the budget cannot recover it. Section 4.1 shows that the window cannot be widened far enough on a 12 GB GPU to avoid this at these model sizes, so the practitioner-relevant result is that this model cannot be relied on to return an answer at all in this configuration, and that the failure is silent. gpt-oss:20b inverts this. It is the slowest model in the study and the only one that exceeds the VRAM budget and spills to CPU, yet it is second on HumanEval (90.9%) and the highest-scoring model on TruthfulQA (52.8% BLEU, 54.8% ROUGE-1). Excluded on throughput grounds, it would have been among the most accurate choices available. Taken together the two boundary cases make the central point of this section more sharply than the stable cluster does: within this hardware envelope, speed and correctness are close to independent.

4.3.3.9 How far the TruthfulQA figures can be trusted

TruthfulQA scores cluster more tightly than either of the other benchmarks, spanning 39.4%–55.4% across all eight models. Given the limitations of the overlap-based metrics described above, this narrow spread should not be read as evidence that the models are genuinely similar in truthfulness: it may equally reflect the limited discriminative power of the metric itself. Of the three benchmarks reported here, TruthfulQA is the one whose absolute values carry least weight.

The column also mixes two task definitions. gemma3:12b, granite4.1:8b, phi4:14b and both mistral-nemo:12b variants were scored under the harness's stock truthfulqa_gen, which caps generation at 256 tokens and stops at the first blank line; the other four were scored under truthfulqa_gen_chat, the chat-template variant written for this study, which stops only at the next question marker. Truncation therefore falls principally on the first group, whose answers average 195–361 characters against 432–1,149 for the second—a further reason to read the column comparatively rather than absolutely.

To test whether this narrow spread reflects the models or the metric, all saved TruthfulQA generations were additionally scored by an LLM judge (llama3.3:70b, chosen from a model family absent from the evaluated set, so that no model was scored by a sibling), and a blind stratified sample of 43 items across the nine model-runs was then adjudicated by hand with the judge's verdict withheld. Agreement with human adjudication was 79% for the judge, 74% for BLEU and 70% for ROUGE-1, with no pairwise difference reaching significance [exact McNemar test (McNemar, 1947), p = 0.79 and p = 0.42]. The methods do err in opposite directions, consistently across both adjudication rounds: the judge credits answers a reader rejects, whilst the overlap metrics reject answers a reader accepts. A further eight sampled items could not be adjudicated at all, because the generation configuration truncated the answer at the first paragraph break before it committed to a claim. No automated scoring method available here therefore measures truthfulness reliably, and the TruthfulQA column should be read accordingly. This adjudication was performed by one author and is not replicated, so it bounds the reliability of these metrics rather than establishing ground truth.

4.3.3.10 Caveats on the correctness results

As with the throughput results, these correctness scores are drawn from a single run per item at greedy (temperature-0) decoding, not repeated across multiple trials as the N = 5 throughput measurements in Section 4.3.1 are. Where a direct check was possible this proved reproducible: regenerating gemma4:e4b's full HumanEval set 3 weeks after the original run, under a nominally larger generation budget that the context window rendered inoperative, reproduced the identical pass or fail outcome on all 164 items. Reproducibility is weaker for models that reason at length, where the boundary between terminating and exhausting the token budget is evidently sensitive to small perturbations; Section 5 discusses this as a further consequence of the time constraints already affecting benchmark repeatability in this study.

5 Limitations

5.1 Study scope and provenance

This study was conducted independently, motivated by personal interest rather than institutional direction from an employer, university, or funding body, and reflects the first author's initial work within the AI/LLM domain. The results are intended to be of practical value both to the home-lab community of practitioners who share hardware and software configurations for running local AI workloads—a tradition also reflected in individually-authored practitioner and evaluation accounts of local LLM deployment (Falkingham, 2025; Kivimäki, 2025)—and to academic and commercial readers evaluating similar on-premises deployments.

The evaluation is deliberately scoped to a single dedicated workstation rather than a fleet of machines. This reflects both the study's origin—a single obsolete workstation repurposed after a Windows 11 hardware upgrade—and a practical constraint: the benchmarked machine has since been reassigned to another user for unrelated work and is no longer available to the authors for further data collection. Section 3.3 discusses how the approach generalizes to multi-GPU, single-chassis configurations for organizations requiring greater capacity; a multi-machine fleet study was outside the scope of this work.

5.2 Evaluation hardware

The evaluation hardware is also more capable than a typical consumer PC affected by Windows 10's end-of-life: the assumption noted in Section 4, that its server CPUs, memory and PCIe advantages matter little because most computation occurs on the GPU, has not been independently confirmed on a genuinely representative lower-specification “obsolete consumer” machine, since a second suitable GPU-equipped machine was not available to the authors. The GPU itself, however, is a consumer-tier NVIDIA RTX 3060 (12 GB VRAM)—the component most determinative of LLM inference performance per the reasoning above—rather than a workstation- or datacentre-class accelerator, which partially mitigates this concern. Only this single 12 GB configuration was measured, so the study characterizes what that capacity supports rather than what a local server minimally requires. The exact CUDA and GPU driver versions in use were not recorded at the time and cannot be retrieved retrospectively, since the hardware is no longer accessible; the benchmark date and Ollama version are recorded in Table 7.

5.3 Benchmark repeatability

Benchmark repeatability was similarly constrained by available time on the evaluation hardware: five repetitions per prompt (N = 5) already required approximately 5–6 h of continuous GPU time across the full model and prompt matrix. Increasing this to the order of N = 100 repetitions per case was not achievable within this timeframe and is no longer possible on the original hardware. Consequently, this study reports mean, variance, and standard deviation over five runs rather than higher-order percentile statistics; these indicate measurement stability under the tested conditions rather than a comprehensive statistical characterization. For the same reason, resampling deepseek-r1:1.5b and gpt-oss:20b—the fastest and slowest models measured—against multiple unique prompt instances per category, rather than repeated runs of the same fixed prompt, was not carried out; the ten prompt categories described in Section 4 provide diversity of task type, but not of phrasing within a category.

5.4 Workload scope

The benchmark design and cost model in this study assume bounded, single-turn or short multi-turn conversational interaction, matching the ten prompt dimensions evaluated in Section 4. This does not extend to agentic or autonomous tool-use workflows, such as a coding agent operating across a large codebase in an unattended mode, where a single user instruction can trigger an open-ended chain of tool calls and accumulated context with no fixed tokens-per-prompt bound. The fixed token and prompt-count assumptions in Table 2 should therefore be read as representative of conversational AI assistant use, not of agentic automation. Multi-user or batched access was not evaluated either, since the server was used by a single researcher throughout.

5.5 Correctness evaluation

The correctness evaluation carries its own constraints, described where they arise in Section 4.3.3. Scores come from a single greedy generation per item, not from repeated trials, so the fivefold repetition applied to throughput has no counterpart on that axis. The evaluation also ran on a separate workstation fitted with the same GPU model, because the throughput host had been reassigned, so the two axes were not measured on one machine. Quantisation levels are the defaults each model tag serves, and are not uniform across models.

TruthfulQA is the weakest of the three benchmarks used here. The stock task configuration truncated a substantial minority of answers at the first paragraph break, affecting four of the nine model-runs. Neither the overlap metrics nor an LLM judge agreed with human adjudication better than about three times in four, and that adjudication was performed by one author without replication.

5.6 Mitigation

To mitigate these constraints, the benchmarking suite used in this study is published under an open license (Curington and Lano, 2026a), so that other researchers and practitioners—independent or institutional, with sustained hardware access—can reproduce, extend, or scale this methodology, including larger sample sizes, additional models, or multi-machine configurations.

6 Future work

Further work is planned to investigate methods for balancing CPU/GPU loads and refining LLM system tuning. Specifically, this will investigate how model quantisation and context length affect “spilling” and how using “attention” and LLM layer allocation strategies may be employed.

7 Conclusion

This study demonstrates that obsolete Windows 10 PCs can be efficiently repurposed as on-premises LLM servers that are substantially cheaper than mid-range and premium cloud alternatives, providing a practical route for organizations to move away from subscription models whilst retaining full data sovereignty.

Benchmarking on a repurposed enterprise workstation, fitted with a single consumer-grade NVIDIA GPU, shows that this class of hardware delivers competitive inference speeds across a range of open-source models. Correctness, evaluated against established public benchmarks alongside this throughput data, does not follow the same ranking as throughput, identifying gemma4:e4b as the strongest model evaluated on this hardware across both dimensions. granite4.1:8b pairs the second-highest throughput of that group with the highest arithmetic-reasoning score of any model tested, and the slowest model evaluated, which spilled to the CPU, was second-most accurate on code generation. Selecting a model on throughput alone would therefore have discarded one of the most capable options available.

The cost-benefit analysis—encompassing the elimination of license fees, reduced reliance on external network infrastructure, and a lower estimated carbon footprint than the cloud alternatives evaluated—positions local deployment as a viable and sustainable alternative. Even against the cheapest cloud tier, where the raw cost gap narrows and briefly favors cloud, this study argues that the digital-sovereignty benefits of local control outweigh the difference for organizations facing privacy, compliance, or long-term cost-predictability requirements.

Because inference runs entirely on local hardware, no prompt or document content leaves the organization's network. That removes one common obstacle to using large language models with regulated or confidential material, although whether a particular deployment satisfies a specific regulatory obligation depends on the wider system and is not assessed here.

Statements

Author contributions

IC: Conceptualization, Investigation, Writing – original draft, Supervision, Software, Writing – review & editing, Data curation, Formal analysis, Methodology. KL: Resources, Funding acquisition, Writing – review & editing, Validation, Project administration, Supervision.

Funding

The author(s) declared that financial support was received for this work and/or its publication. Funding support was received from EPSRC grant code EP/ X525571/1 held by King's College London.

Acknowledgments

We wish to acknowledge the contributions of Harry Eggo that aided the efforts of the authors. The authors acknowledge support of EPSRC grant code EP/ X525571/1 held by King's College London.

Conflict of interest

IC was employed at Thames 3D Research.

The remaining author declares that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declared that Generative AI was used in the creation of this manuscript. The study and its early drafts are the author(s)' own work. Generative AI assistance was used in revising it. Claude Code (Anthropic; Sonnet 5 and Opus 5) was used to draft and revise passages of the manuscript, for LaTeX typesetting, citation verification and debugging. It was also used to configure and run the correctness evaluation of Section 4.3.3, using EleutherAI's lm-evaluation-harness, and to write the analysis and plotting scripts published alongside it. The carbon-footprint estimates in Section 3.4 were produced the same way, using EcoLogits. The throughput benchmark suite, its prompt design, and the throughput measurements are the author(s)' own work. The experimental goals, methodology, study design, and the conclusions drawn from the results are the author(s)', as is the decision on every point of interpretation. The author(s) have verified the factual accuracy and integrity of all AI-assisted content. The early drafts, written in Markdown before the manuscript was converted to LaTeX, are retained and can be provided on request.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1

  Anthropic (2025). System Card: Claude Opus 4 and Claude Sonnet 4. Available online at: https://www.anthropic.com/claude-4-system-card (Accessed July 31, 2026).

  • Google Scholar

• 2

  BachsteinM.GroppC.HazlewoodV.PetersonG. (2024). “Lessons from benchmarking the AI Tennessee initiative resources,” in Practice and experience in advanced research computing (PEARC '24) (Providence, RI: ACM).

  • Google Scholar

• 3

  CaddyB.JessopK. (2023). The End of Windows 10 Support Could Turn 240 Million pcs into E-Waste. Canalys. Available online at: https://www.canalys.com/insights/end-of-windows-10-support-could-turn-240-million-pcs-into-e-waste (Accessed October 27, 2024).

  • Google Scholar

• 4

  ChenM.TworekJ.JunH.YuanQ.PintoH. P. d. O.KaplanJ.et al. (2021). Evaluating large language models trained on code. arXiv [preprint]. arXiv:2107.03374. Available online at: https://arxiv.org/abs/2107.03374 (Accessed July 30, 2026).

  • Google Scholar

• 5

  Chitty-VenkataK. T.RaskarS.KaleB.FerdausF.TanikantiA.RaffenettiK.et al. (2024). “LLM-inference-bench: inference benchmarking of large language models on AI accelerators,” in Proceedings of the SC'24 workshops of the international conference on high performance computing, network, storage, and analysis (IEEE). Available online at: https://arxiv.org/abs/2411.00136 (Accessed July 25, 2026).

  • Google Scholar

• 6

  CobbeK.KosarajuV.BavarianM.ChenM.JunH.KaiserL.et al. (2021). Training verifiers to solve math word problems. arXiv [preprint]. arXiv:2110.14168. Available online at: https://arxiv.org/abs/2110.14168 (Accessed July 30, 2026).

  • Google Scholar

• 7

  CuringtonI.LanoK. (2026a). Ollama LLM Local Inference Benchmarking Suite. Version v1.0.1. Available online at: https://doi.org/10.5281/zenodo.22079049; source repository https://github.com/t3dr0/ollama_benchmark (Accessed August 24, 2026).

  • Google Scholar

• 8

  CuringtonI.LanoK. (2026b). Throughput and Output-Correctness Benchmark Data for Local LLM Inference on a Repurposed Enterprise Workstation. Dataset, Version v2. Available online at: https://doi.org/10.5281/zenodo.22748926 (Accessed September 14, 2026).

  • Google Scholar

• 9

  DatabaseMart (2026). RTX 3060 Ollama Benchmark. DatabaseMart Blog. Available online at: https://www.databasemart.com/blog/ollama-gpu-benchmark-rtx3060ti (Accessed June 2026).

  • Google Scholar

• 10

  Department for Energy Security and Net Zero (2025). 2025 Government Greenhouse Gas Conversion Factors for Company Reporting: Methodology Paper. Available online at: https://assets.publishing.service.gov.uk/media/6846b0870392ed9b784c0187/2025-GHG-CF-methodology-paper.pdf (Accessed July 30, 2026).

  • Google Scholar

• 11

  F.S.F.E. (2026). Device Neutrality Manifesto of the Free Software Foundation Europe. FSFE.org. Available online at: https://fsfe.org/activities/deviceneutrality/index (Accessed April 28, 2026).

  • Google Scholar

• 12

  FalkinghamP. (2025). Using Local AI/LLM in VS Code Without Third Party Software on the CPU, GPU or NPU. Available online at: https://peterfalkingham.com/2025/10/27/using-local-ai-llm-in-vs-code-without-third-party-software/ (Accessed April 28, 2026).

  • Google Scholar

• 13

  FinleyK. (2013). French National Police Switch 37,000 Desktop PCs to Linux. WIRED. Available online at: https://www.wired.com/2013/09/gendarmerie-linux/ (Accessed April 28, 2026).

  • Google Scholar

• 14

  FloridiL. (2020). The fight for digital sovereignty: what it is, and why it matters, especially for the EU. Philos. Technol. 33, 369–378. doi: 10.1007/s13347-020-00423-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15

  France (2026). Souveraineté numérique : l'État accélère la réduction de ses dépendances extra-européennes. Translated from the French Government Website. Available online at: https://www.numerique.gouv.fr/sinformer/espace-presse/souverainete-numerique-reduction-dependances-extra-europeennes/ (Accessed April 28, 2026).

  • Google Scholar

• 16

  GaoL.TowJ.AbbasiB.BidermanS.BlackS.DiPofiA.et al. (2024). The Language Model Evaluation Harness. Available online at: https://doi.org/10.5281/zenodo.12608602 (Accessed July 30, 2026).

  • Google Scholar

• 17

  Google (2026). Introducing Gemini 3 Flash: Benchmarks, Global Availability. Available online at: https://blog.google/products/gemini/gemini-3-flash/ (Accessed July 31, 2026).

  • Google Scholar

• 18

  GouldP.SongG.ZhuT. (2024). Environmental and Economic Impact of I/O Device Obsolescence. Available online at: https://arxiv.org/abs/2412.20655 (Accessed June 10, 2026).

  • Google Scholar

• 19

• 20

  KhalilA.HeillesG.ParragaM.HeillesS. (2025). Viability and performance of a private LLM server for SMBs: a benchmark analysis of Qwen3-30B on consumer-grade hardware. arXiv [preprint]. arXiv:2512.23029. Available online at: https://arxiv.org/abs/2512.23029

  • Google Scholar

• 21

  KivimäkiT. (2025). Usability Evaluation of the Local Large Language Models. UTU Pub Finland. Available online at: https://www.utupub.fi/handle/10024/182260 (Accessed October 27, 2025).

  • Google Scholar

• 22

  KnoopJ.HoltmannH. (2026). Private LLM inference on consumer blackwell GPUs: a practical guide for cost-effective local deployment in SMEs. arXiv [preprint]. arXiv:2601.09527. Available online at: https://arxiv.org/abs/2601.09527

  • Google Scholar

• 23

  LazukaM.AnghelA.ParnellT. (2024). “LLM-pilot: characterize and optimize performance of your LLM inference services,” in Proceedings of the international conference for high performance computing, networking, storage and analysis (SC'24) (IEEE). Available online at: https://sc24.supercomputing.org/proceedings/tech_paper/tech_paper_pages/pap547.html (Accessed July 25, 2026).

  • Google Scholar

• 24

  LiJ.XuJ.HuangS.ChenY.LiW.LiuJ.et al. (2024). Large language model inference acceleration: a comprehensive hardware perspective. arXiv [preprint]. arXiv:2410.04466. Available online at: https://arxiv.org/abs/2410.04466

  • Google Scholar

• 25

  LinS.HiltonJ.EvansO. (2022). “TruthfulQA: measuring how models mimic human falsehoods,” in Proceedings of the 60th annual meeting of the association for computational linguistics (Volume 1: Long Papers) (Dublin: Association for Computational Linguistics), 3214–3252. Available online at: https://aclanthology.org/2022.acl-long.229/ (Accessed July 30, 2026).

  • Google Scholar

• 26

  McNemarQ. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika12, 153–157. doi: 10.1007/BF02295996

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27

  Microsoft (2025a). Reporting Graphics Memory. Windows Hardware Drivers Documentation. Available online at: https://learn.microsoft.com/en-us/windows-hardware/drivers/display/reporting-graphics-memory (Accessed May 23, 2026).

  • Google Scholar

• 28

  Microsoft (2025b). Windows 10 Home and Pro–Microsoft Lifecycle. Available online at: https://learn.microsoft.com/en-us/lifecycle/products/windows-10-home-and-pro (Accessed August 19, 2026).

  • Google Scholar

• 29

  MuhlgayD.RamO.MagarI.LevineY.RatnerN.BelinkovY.et al. (2024). “Generating benchmarks for factuality evaluation of language models,” in Proceedings of the 18th conference of the European chapter of the association for computational linguistics (Volume 1: Long Papers) (St. Julian's: Association for Computational Linguistics), 49–66. Available online at: https://aclanthology.org/2024.eacl-long.4/ (Accessed July 27, 2026).

  • Google Scholar

• 30

  NaS.JeongG.AhnB. H.YoungJ.KrishnaT.KimH. (2024). “Understanding performance implications of LLM inference on CPUs,” in 2024 IEEE international symposium on workload characterization (IISWC) (Vancouver, BC: IEEE), 169–180. doi: 10.1109/IISWC63097.2024.00024

  • CrossRef
  • Google Scholar

• 31

  NanoGPT (2025). AI Chat Text 400+ Models API Pricing Table. NanoGPT. Available online at: https://nano-gpt.com/pricing (Accessed September 26, 2025).

  • Google Scholar

• 32

  NaughtonJ. (2025). The Networker: AI is the New Globalisation That Will Create a World of Have-Nots and Have-Yachts. The Observer. Available online at: https://observer.co.uk/news/columnists/article/the-networker-ai-is-the-new-globalisation-that-will-create-a-world-of-have-nots-and-have-yachts (Accessed October 27, 2024).

  • Google Scholar

• 33

  NguyenT.NguyenT. (2025). An evaluation of LLMs inference on popular single-board computers. arXiv [preprint]. arXiv:2511.07425. Available online at: https://arxiv.org/abs/2511.07425

  • Google Scholar

• 34

  Ofgem (2026). Changes to Energy Price Cap Between 1 July and 30 September 2026. Available online at: https://www.ofgem.gov.uk/news/changes-energy-price-cap-between-1-july-and-30-september-2026 (Accessed July 28, 2026).

  • Google Scholar

• 35

  Ollama (2026a). Ollama Library Open-Source LLMs. Available online at: https://ollama.com/library (Accessed April 30, 2026).

  • Google Scholar

• 36

  Ollama (2026b). Ollama v0.24.0. Available online at: https://github.com/ollama/ollama/releases/tag/v0.24.0 (Accessed July 30, 2026).

  • Google Scholar

• 37

  Open WebUI (2026). Open WebUI: Self-Hosted AI Platform. Available online at: https://docs.openwebui.com/ (Accessed August 23, 2026).

  • Google Scholar

• 38

  OthmanA. (2025). The End of Windows 10 and The Rise of Digital Waste: A Global Technological Dilemma. ResearchGate. Available online at: https://www.researchgate.net/publication/397020633_The_End_of_Windows_10_and_the_Rise_of_Digital_Waste_A_Global_Technological_Dilemma (Accessed April 28, 2026).

  • Google Scholar

• 39

  PordesR.PetravickD.KramerB.OlsonD.LivnyM.RoyA.et al. (2007). The open science grid. J. Phys. Conf. Ser. 78:012057. doi: 10.1088/1742-6596/78/1/012057

  • CrossRef
  • Google Scholar

• 40

  RahmanN. A. A.ZulkornainL. H.SoonG. Y.HamzahN. H.ChaoC. (2025). NotebookLM integrated project-based learning model development for writing among undergraduates. Int. J. Res. Innov. Soc. Sci. 4:5. doi: 10.47772/IJRISS.2025.907000444

  • CrossRef
  • Google Scholar

• 41

  ReddiV. J.ChengC.KanterD.MattsonP.SchmuellingG.WuC.-J.et al. (2020). “MLPerf inference benchmark,” in 2020 ACM/IEEE 47th annual international symposium on computer architecture (ISCA) (IEEE), 446–459. Available online at: https://arxiv.org/abs/1911.02549 (Accessed July 25, 2026).

  • Google Scholar

• 42

  RenneyH.TradF.MattarockM.WoodZ. (2026). Cloud to edge: benchmarking LLM inference on hardware-accelerated single-board computers. arXiv [preprint]. arXiv:2604.24785. Available online at: https://arxiv.org/abs/2604.24785

  • Google Scholar

• 43

  RincéS.BanseA. (2026). EcoLogits: Track the Energy and Environmental Footprint of Generative AI Models Through Their APIs, Version 0.11.1. Available online at: https://github.com/genai-impact/ecologits (Accessed July 30, 2026).

  • Google Scholar

• 44

  SchneierB. (2025). What LLMs Know About Their Users. Crypto-Gram. Available online at: https://www.schneier.com/crypto-gram/archives/2025/0715.html#cg7 (Accessed October 27, 2025).

  • Google Scholar

• 45

  SrinivasA.HegdeV.AdityaG. S.SinghA. Fayaz. (2025). “Building and hosting AI locally with Ollama, Open WebUI, and stable diffusion,” in 2025 9th international conference on computational system and information technology for sustainable solutions (CSITSS) (IEEE), 1–6. doi: 10.1109/CSITSS67709.2025.11295791

  • CrossRef
  • Google Scholar

• 46

  StuhlmannL.Fadel ArgerichM.FürstJ. (2025). Bench360: benchmarking local LLM inference from 360 degrees. arXiv [preprint]. arXiv:2511.16682. Available online at: https://arxiv.org/abs/2511.16682

  • Google Scholar

• 47

  ThainD.TannenbaumT.LivnyM. (2005). Distributed computing in practice: the condor experience. Concurrency Comput. Pract. Exp. 17, 323–356. doi: 10.1002/cpe.938

  • CrossRef
  • Google Scholar

• 48

  University of Oxford IT Services Department (2022). Environmental Impact of IT: Desktops, Laptops and Screens. University of Oxford Website. Available online at: https://web.archive.org/web/20260315150016/; https://www.it.ox.ac.uk/article/environment-and-it (Accessed August 19, 2026).

  • Google Scholar

• 49

  Vaughan-NicholsS. (2026). France is Ditching Windows for Digital Sovereignty – And its New Linux Stack is Taking Shape. ZDNET. Available online at: https://www.zdnet.com/article/france-leaves-windows-for-linux-desktop/ (Accessed April 28, 2026).

  • Google Scholar

• 50

  WhiteC.DooleyS.RobertsM.PalA.FeuerB.JainS.et al. (2024). Livebench: a challenging, contamination-free LLM benchmark. arXiv [preprint]. arXiv:2406.19314. Available online at: https://livebench.ai/livebench.pdf (Accessed July 27, 2026).

  • Google Scholar

• 51

  WolfeJ.PfohlC.SubramanianH.MertogunoJ. S.AlexiouM. S. (2024). “Recent advancements in search-engine algorithms for efficiency, privacy and security,” in 2024 15th international conference on information, intelligence, systems and applications (IISA) (Chania Crete: IEEE), 1–6. doi: 10.1109/IISA62523.2024.10786705

  • CrossRef
  • Google Scholar

• 52

  ZeffM. (2025). Meta Plans to Sell Targeted ads Based on Data in Your AI Chats. Tech Crunch. Available online at: https://techcrunch.com/2025/10/01/meta-plans-to-sell-targeted-ads-based-on-data-in-your-ai-chats/ (Accessed October 27, 2025).

  • Google Scholar

Keywords

digital sovereignty, e-waste, inference, LLMS, obsolescence, online-privacy, on-premises, windows-10

Citation

Curington I and Lano K (2026) Reusing obsolete windows 10 PCs for on-premises large language model inference. Front. Comput. Sci. 8:1911927. doi: 10.3389/fcomp.2026.1911927

Received

17 June 2026

Revised

14 September 2026

Accepted

24 September 2026

Published

08 October 2026

Volume

8 - 2026

Updates

Copyright

© 2026 Curington and Lano.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Kevin Lano, kevin.lano@kcl.ac.uk

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.