RadarAI· MarkTechPost（RSS）·· 4 小时前AI 评分36
开发者实测指南：用公开判定数据重建 Datalab OmniParseBench 排行榜并复测 Tesseract 与 Indra OCR
AI 导读
开发者基于 Datalab 公开的 27 个解析器判定数据重建 OmniParseBench 排行榜，该基准用 16,288 个通过或失败测试对 2,937 页文档打分。复测发现排名前两位的解析器在统计上持平，多数文档切片下榜首会发生变化，且一项发布数字与已发布文件不符。作者还在 50 页样本上自行运行了 Tesseract 与 Indra OCR。
正文 · AI 翻译
MarkTechPost· Asif Razzaq·2026-10-23 00:11· 2 小时前AI 评分55
OmniParseBench 开发者指南：排行榜基准与 OCR 测试
AI 导读
这篇教程实测 Datalab 的 OmniParseBench 文档解析基准，该基准用 16,288 个通过或失败测试对 2,937 页文档给 27 个解析器打分。
正文 · 原文
在本教程中，我们实现 OmniParseBench，这是 Datalab 面向文档解析器的开放基准，它用 16,288 个小型通过或失败测试对 2,937 页文档（从扫描表单和手写内容到 arXiv 数学公式和密集表格）的解析器输出进行评分。我们没有直接相信排行榜，而是根据 Datalab 为 27 个解析器发布的判定结果重建了它，并先向评分器输入一个手工制作的页面，观察哪些错误会翻转判定。随后我们衡量排名的确定性，按文档类型切片，测试卡片中列为帮助构建测试的两个模型在模型修正过的测试上是否表现不同，并自行在 50 页样本上运行 Tesseract 和 Indra OCR。在此过程中，我们发现排名前两位的解析器在统计上并列，领先者在大多数切片上会发生变化，且一个发布数字与已发布文件不符。
Copy CodeCopiedUse a different Browser
import os
import re
import sys
import json
import time
import html
import shutil
import platform
import traceback
import subprocess
import collections
import urllib.request
from pathlib import Path
RESULTS = {} # step -> one-line result, for the summary table
CLAIMS = [] # (step, claim, held)
CURRENT = ["1"]
def banner(title):
print("\n" + "=" * 78)
print(title)
print("=" * 78)
def section(name):
def wrap(fn):
def run(*a, **kw):
CURRENT[0] = name.split(".")[0]
banner(name)
t0 = time.time()
try:
out = fn(*a, **kw)
RESULTS[name] = out if isinstance(out, str) else "ok"
return out
except Exception as e:
RESULTS[name] = f"SKIPPED / FAILED -> {type(e).__name__}: {e}"
print(f"\n[!] {name} did not complete: {type(e).__name__}: {e}")
traceback.print_exc(limit=3)
return None
finally:
print(f"\n[{time.time() - t0:.1f} s]")
return run
return wrap
def claim(text, held):
CLAIMS.append((CURRENT[0], text, bool(held)))
return held
banner("1. Install omni-parse-bench 1.0.0rc1 and Tesseract, and pin the dataset revision")
SYSTEM = platform.system()
OPB_VERSION = "1.0.0rc1" # a pre-release: a bare `pip install omni-parse-bench` resolves to 0.2.0, whose API differs
REPO = "datalab-to/omni_parse_bench"
REV = "961c0f89a2da701f5ce53afbd5e80aa469b0369a" # the dataset with the published verdicts of 27 parsers
INDRA = ["indra-ocr==0.3.2", "numpy==2.5.4", "onnxruntime==1.31.0", "opencv-python-headless==5.0.0.93",
"pypdfium2==5.14.0", "huggingface_hub==2.2.0"] # the pins of the package's own Indra server image
WORK = Path(os.environ.get("OPB_WORK") or ("/content/opb" if os.path.isdir("/content") else "opb_work")).resolve()
WORK.mkdir(parents=True, exist_ok=True)
subprocess.run([sys.executable, "-m", "pip", "install", "-q", f"omni-parse-bench[benchmark]=={OPB_VERSION}", "uv"], check=True)
if SYSTEM == "Linux" and shutil.which("tesseract") is None:
scripts = ("arab armn beng cans cher cyrl deva ethi frak geor grek gujr guru hang hang-vert hans hans-vert hant "
"hant-vert hebr jpan jpan-vert knda khmr laoo latn mlym mymr orya sinh syrc taml telu thaa thai tibt viet").split()
subprocess.run(["apt-get", "-qq", "update"], check=True)
subprocess.run(["apt-get", "-qq", "install", "-y", "tesseract-ocr", "tesseract-ocr-eng", "tesseract-ocr-osd"]
+ [f"tesseract-ocr-script-{s}" for s in scripts], check=True, stdout=subprocess.DEVNULL)
import numpy as np
import pandas as pd
import importlib.metadata as md
from huggingface_hub import snapshot_download
from omni_parse_bench import benchmark, data, harness, markdown, vocab
from omni_parse_bench.scorers import unit_tests
print(f"omni-parse-bench {md.version('omni-parse-bench')}, Python {platform.python_version()}, {SYSTEM} {platform.machine()}, "
f"{os.cpu_count()} CPU cores")
if SYSTEM == "Linux":
print("os:", open("/etc/os-release").read().split('PRETTY_NAME="')[1].split('"')[0])
tess = subprocess.run(["tesseract", "--version"], capture_output=True, text=True)
print("tesseract:", (tess.stdout or tess.stderr).splitlines()[0])
TESS_MAJOR = int((tess.stdout or tess.stderr).split()[1].split(".")[0])
print("test families:", {f: [t for t, g in vocab.FAMILIES.items() if g == f] for f in dict.fromkeys(vocab.FAMILIES.values())})
print(f"dataset {REPO} @ {REV[:7]}; work folder {WORK}")
RESULTS["1. Install omni-parse-bench 1.0.0rc1 and Tesseract, and pin the dataset revision"] = (
f"omni-parse-bench {md.version('omni-parse-bench')}, tesseract {(tess.stdout or tess.stderr).split()[1]}, dataset @ {REV[:7]}")
我们安装固定到 1.0.0rc1 的 omni-parse-bench，因为直接 pip install 仍会解析到 0.2.0，这是一个 API 不同的旧版本。benchmark extra 会带来解析器适配器、Hugging Face 客户端以及用于清单的 pyarrow，在 Colab 上我们通过 apt 添加 Tesseract 及其脚本模型。我们还将数据集固定到一个提交，因为仓库会随着解析器的添加而变化。打印 families 会显示基准所依赖的结构：present、order 和 repeat 测试构成 text 家族；table_cell 构成 tables 家族；layout_kind 构成 layout 家族。
Copy CodeCopiedUse a different Browser
@section("2. How the scorer decides: one hand-built page, five test types, one change at a time")
def scorer_anatomy():
T = lambda i, tt, out, tags, args, st=None: {"id": i, "test_type": tt, "output_type": out, "derivation": "authored",
"tags": tags, "args": args, "settings": st or {}}
exact = lambda s: {"text": s, "max_diffs": 0}
gold = [
T("title", "present", "html", ["heading"], {"content": exact("Quarterly Report")}),
T("total", "present", "html", ["eng"], {"content": exact("Total due")}),
T("growth", "present", "html", ["eng"], {"content": {"number": "-4.8%"}}),
T("amount", "present", "html", ["eng"], {"content": {"number": "1,240.00"}}),
T("eq_ratio", "present", "html", ["math"], {"content": {"latex": r"\alpha_{1} + \beta"}}),
T("eq_half", "present", "html", ["math"], {"content": {"latex": r"\frac{1}{2} \to x"}}),
T("order", "order", "html", ["eng"], {"before": exact("Introduction: this report covers the second quarter."),
"after": exact("Conclusion: the plan holds.")}),
T("repeat", "repeat", "html", ["eng"], {"content": exact("Revenue")}, {"expected_count": 2}),
T("cell_top", "table_cell", "html", ["table"], {"cell": exact("1,240"), "top_heading": exact("2024")}),
T("cell_left", "table_cell", "html", ["table"], {"cell": exact("1,240"), "left_heading": exact("Income")}),
T("cell_price", "table_cell", "html", ["table"], {"cell": exact("9.50"), "top_heading": exact("Price"),
"left": exact("Widget")}),
T("block_title", "layout_kind", "blocks", ["heading"],
{"content": dict(exact("Quarterly Report"), place=[[0.10, 0.04, 0.90, 0.09]])}, {"kind": "heading"}),
T("block_total", "layout_kind", "blocks", ["eng"],
{"content": dict(exact("Total due"), place=[[0.10, 0.12, 0.90, 0.16]])}, {"kind": "text"}),
]
gold_file = WORK / "fixture" / "gold" / "demo-p1.json" # a gold file is named after its page
gold_file.parent.mkdir(parents=True, exist_ok=True)
gold_file.write_text(json.dumps({"tests": gold}, indent=1))
page = data.Page(page_id="demo-p1", document="demo", suite="demo", doc_path=WORK / "fixture" / "demo-p1.pdf",
gt_path=gold_file, outputs=("blocks", "html"))
tests = unit_tests.gold(page)
table_a = ("<table><tr><th></th><th>2023</th><th>2024</th></tr><tr><th>Income</th><td>1,000</td><td>1,240</td></tr>"
"<tr><th>Costs</th><td>600</td><td>700</td></tr></table>")
table_b = "<table><tr><th>Product</th><th>Price</th></tr><tr><td>Widget</td><td>9.50</td></tr></table>"
md_table_b = "| Product | Price |\n|---|---|\n| Widget | 9.50 |"
parts = {"title": "<h1>Quarterly Report</h1>", "total": "<p>Total due: 1,240.00 USD.</p>",
"growth": "<p>Growth was -4.8% year on year.</p>", "ratio": r"<p>Ratio: 1+ holds.</p>",
"half": r"<p>Half: 12x in the model.</p>",
"intro": "<p>Introduction: this report covers the second quarter.</p>",
"revenue": "<p>Revenue is shown in thousands. Revenue grew in the second quarter.</p>",
"table_a": table_a, "conclusion": "<p>Conclusion: the plan holds.</p>", "table_b": table_b}
blocks = [{"label": "heading", "bbox": [0.10, 0.04, 0.90, 0.09], "text": "Quarterly Report"},
{"label": "text", "bbox": [0.10, 0.12, 0.90, 0.16], "text": "Total due: 1,240.00 USD."},
{"label": "table", "bbox": [0.10, 0.40, 0.90, 0.60], "text": "Income 1,000 1,240 Costs 600 700"}]
def pred(change=None, order=None, new_blocks=None):
p = dict(parts, **(change or {}))
return {"html": "\n".join(p[k] for k in (order or list(parts))), "blocks": blocks if new_blocks is None else new_blocks}
base = {v.id: v.result for v in unit_tests.score(pred(), tests)}
print(f"base prediction: {sum(r == 'pass' for r in base.values())} of {len(tests)} tests pass\n")
swapped = ["title", "total", "growth", "ratio", "half", "conclusion", "intro", "revenue", "table_a", "table_b"]
variants = [
("flip", "a typo: 'Total dve'", pred({"total": "<p>Total dve: 1,240.00 USD.</p>"}), ()),
("flip", "the minus sign dropped: 4.8%", pred({"growth": "<p>Growth was 4.8% year on year.</p>"}), ()),
("flip", "another symbol: alpha_2", pred({"ratio": r"<p>Ratio: 2+ holds.</p>"}), ()),
("flip", "conclusion before introduction", pred(order=swapped), ()),
("flip", "'Revenues' for the second 'Revenue'",
pred({"revenue": "<p>Revenue is shown in thousands. Revenues grew in the second quarter.</p>"}), ()),
("flip", "2023 and 2024 headings swapped", pred({"table_a": table_a.replace("<th>2023</th><th>2024</th>",
"<th>2024</th><th>2023</th>")}), ()),
("flip", "row label 'Margin' for 'Income'", pred({"table_a": table_a.replace("<th>Income</th>", "<th>Margin</th>")}), ()),
("flip", "a markdown table left in the html", pred({"table_b": md_table_b}), ()),
("flip", "the title's block labelled text", pred(new_blocks=[dict(blocks[0], label="text")] + blocks[1:]), ()),
("control", r"\dfrac12\rightarrow for \frac{1}{2}\to", pred({"half": r"<p>Half: $\dfrac12\rightarrow x$ in the model.</p>"}), ()),
("control", "'TOTAL DUE' and '1240.00'", pred({"total": "<p>TOTAL DUE: 1240.00 USD.</p>"}), ()),
("control", "markdown table through markdown.to_html", pred({"table_b": markdown.to_html(md_table_b)}), ()),
("unsupported", "no blocks, declared unsupported", {"html": pred()["html"]}, ("blocks",)),
]
as_expected = 0
for kind, name, p, unsupported in variants:
changed = [v for v in unit_tests.score(p, tests, unsupported) if v.result != base[v.id]]
if kind == "flip":
ok = len(changed) == 1 and changed[0].result == "fail"
elif kind == "control":
ok = not changed
else:
ok = bool(changed) and all(v.result == "unsupported" for v in changed)
as_expected += ok
shown = "; ".join(f"{v.id} -> {v.result}" + (f" ({v.why})" if v.result == "fail" else "") for v in changed) or "no verdict changes"
print(f"{kind:11s} {name:42s} {shown}")
s = unit_tests.summarize(unit_tests.lines(unit_tests.score(variants[0][2], tests), tests, page))
print(f"\nsummary of the typo variant: headline {s['headline']:.4f} = mean of "
+ ", ".join(f"{k} {v['score']:.3f} ({v['tests']} tests)" for k, v in s["families"].items()))
claim("every one-change variant flips exactly one verdict to fail, and no control flips any", as_expected == len(variants))
return f"{as_expected} of {len(variants)} hand-built variants behave as expected (9 flips, 3 controls, 1 unsupported)"
scorer_anatomy()
在信任分数之前，我们想看看判定意味着什么，因此我们手工编写了一个 gold 文件，在一个虚构页面上包含十三个测试，并用 unit_tests.score（基准本身使用的函数）对预测进行评分。基础预测通过所有测试，每个变体只改变一件事。一个字母的拼写错误、丢失的负号、方程中不同的符号、交换的标题和错误标记的块，每个都恰好使一个测试翻转为失败，并附带原因。对照组仍然通过：大写字母、去掉的千位分隔符以及用 dfrac 代替 frac，因为文本会被折叠，方程在 KaTeX 渲染后进行比较。
Copy CodeCopiedUse a different Browser
@section("3. The dataset: 16,288 tests, what they check, and how each answer was established")
def dataset_step():
global DATA, PAGES, TESTS, CARD_TEXT
snapshot_download(REPO, repo_type="dataset", revision=REV, local_dir=str(WORK / "dataset"),
allow_patterns=["manifest.parquet", "gold/*", "results.parquet", "README.md"])
DATA, PAGES = data.read_manifest(str(WORK / "dataset"))
rows = []
for p in PAGES:
for t in unit_tests.gold(p):
rows.append(dict(test_id=t.id, page_id=p.page_id, suite=p.suite, document=p.document, test_type=t.test_type,
derivation=t.derivation, tags=list(t.tags), pdf=p.doc_path.suffix == ".pdf"))
TESTS = pd.DataFrame(rows)
tags = collections.Counter(g for ts in TESTS.tags for g in ts)
langs = {g for g in tags if re.fullmatch(r"[a-z]{3}", g)}
scripts = {g for g in tags if re.fullmatch(r"[A-Z][a-z]{3}", g)}
n_pdf = sum(p.doc_path.suffix == ".pdf" for p in PAGES)
print(f"{len(TESTS):,} tests on {TESTS.page_id.nunique():,} pages from {TESTS.document.nunique():,} documents in "
f"{TESTS.suite.nunique()} suites; {n_pdf:,} PDF pages and {len(PAGES) - n_pdf:,} images")
print(f"tags: {len(langs)} language codes ({len(langs - {'und', 'zxx'})} without und and zxx), {len(scripts)} scripts")
counts = lambda col: {k: int(v) for k, v in col.value_counts().items()}
print("\ntests by type: ", counts(TESTS.test_type))
print("tests by family: ", counts(TESTS.test_type.map(vocab.FAMILIES)))
print("structure and role tags:", {g: n for g, n in tags.most_common() if g not in langs | scripts})
print("\nhow each test's answer was established (derivation):")
for d, n in TESTS.derivation.value_counts().items():
print(f" {d:22s} {n:6,d} {', '.join(f'{k} {v}' for k, v in TESTS[TESTS.derivation == d].test_type.value_counts().items())}")
CARD_TEXT = (DATA / "README.md").read_text()
claim("16,288 tests on 2,937 pages from 2,343 documents in 22 suites, as the card says",
(len(TESTS), TESTS.page_id.nunique(), TESTS.document.nunique(), TESTS.suite.nunique()) == (16288, 2937, 2343, 22))
return f"{len(TESTS):,} tests, {TESTS.page_id.nunique():,} pages, {TESTS.document.nunique():,} documents, {len(langs - {'und', 'zxx'})} languages"
dataset_step()
理解了评分器后，我们在固定修订版下载清单、gold 文件和已发布的判定结果，跳过 1.5 GB 的页面图像，并通过 unit_tests.gold 读取每一页的测试。计数与数据集卡片一致，包括在移除未确定和无语言代码后的 92 种语言。有两个字段值得注意。handwriting、rotated 或 table 等标签从不决定判定；它们只让我们对结果进行切片。Derivation 记录每个答案是如何确立的，从出版商标记和文本层到模型面板和人工审核，这让我们之后可以测试测试构建是否偏向任何解析器。
Copy CodeCopiedUse a different Browser
COLS = ["page_id", "suite", "document", "test_id", "test_type", "output_type", "derivation", "tags", "result", "why"]
STARRED = ["Gemini 3.8 Flash", "Claude Sonnet 5.5"] # the card: models "used in creating and verifying at least some tests"
@section("4. Recompute the published leaderboard from 439,776 verdicts")
def recompute_table():
global VERDICTS, TABLE
VERDICTS = pd.read_parquet(DATA / "results.parquet")
VERDICTS["output_type"] = VERDICTS.test_type.map(vocab.READS)
VERDICTS["tags"] = VERDICTS.tags.apply(list)
VERDICTS["family"] = VERDICTS.test_type.map(vocab.FAMILIES)
rows = []
for system, g in VERDICTS.groupby("system", sort=False):
s = unit_tests.summarize(g[COLS].to_dict("records"))
fam = {k: (np.nan if v["score"] is None else 100 * v["score"]) for k, v in s["families"].items()}
scored = g[g.result != "unsupported"]
rows.append(dict(system=system, headline=np.nan if s["headline"] is None else 100 * s["headline"],
text=fam.get("text"), tables=fam.get("tables"), layout=fam.get("layout"),
pooled=100 * (scored.result == "pass").mean()))
TABLE = pd.DataFrame(rows).set_index("system")
card = {}
for line in CARD_TEXT.splitlines():
cells = [c.strip().strip("*").strip() for c in line.strip().strip("|").split("|")]
if len(cells) == 6 and cells[1] in ("open", "closed"):
card[cells[0]] = [None if c in ("none", "unsupported") else float(c) for c in cells[2:]]
mismatch = []
for name, vals in card.items():
mine = [None if pd.isna(x) else round(x, 1) for x in TABLE.loc[name, ["headline", "text", "tables", "layout"]]]
mismatch += [(name, a, b) for a, b in zip(mine, vals) if a != b]
print(f"{len(VERDICTS):,} verdicts = {VERDICTS.system.nunique()} parsers x {VERDICTS.test_id.nunique():,} tests: "
f"{ {k: int(v) for k, v in VERDICTS.result.value_counts().items()} }")
numeric = sum(v is not None for vals in card.values() for v in vals)
print(f"the card's table: {len(card)} parsers, {4 * len(card)} entries ({numeric} numbers, the rest none or unsupported); "
f"recomputed with unit_tests.summarize, {len(mismatch)} differ {mismatch[:3] if mismatch else ''}\n")
show = TABLE.sort_values("headline", ascending=False, na_position="last")
f = lambda x: "none" if pd.isna(x) else f"{x:.2f}"
print(f"{'parser':24s} {'headline':>9s} {'text':>7s} {'tables':>7s} {'layout':>7s} {'pooled':>7s}")
for name, r in show.iterrows():
if name in show.index[:8] or pd.isna(r.headline):
print(f"{name + ('*' if name in STARRED else ''):24s} {f(r.headline):>9s} {f(r.text):>7s} {f(r.tables):>7s} "
f"{f(r.layout):>7s} {r.pooled:7.2f}")
a, g = TABLE.loc["Datalab accurate", "headline"], TABLE.loc["Gemini 3.8 Flash", "headline"]
ranked = TABLE.dropna(subset=["headline"])
print(f"\nDatalab accurate {a:.3f} vs Gemini 3.8 Flash {g:.3f}: both {a:.1f} at one decimal, {a - g:+.3f} points apart")
print("top 3 by headline (mean of the three families):", list(ranked.headline.nlargest(3).index))
print("top 3 by pooled pass rate over every scored test:", list(ranked.pooled.nlargest(3).index))
claim("every number in the card's results table is reproduced from results.parquet", not mismatch)
return f"{4 * len(card) - len(mismatch)} of {4 * len(card)} card entries reproduced; Datalab accurate {a:.2f}, Gemini {g:.2f}"
recompute_table()
因为 results.parquet 中每个解析器在每个测试中占一行，我们可以通过将每个解析器的行传给 unit_tests.summarize 来重建排行榜。总评分是文本、表格和布局通过率的平均值，其中不支持的测试被排除在外，因此像 Tesseract 这样无法返回布局块的解析器根本得不到总评分。卡片表格中的全部 108 个条目都与其一位小数值相符。重新计算还暴露出榜首的近乎平局：Datalab accurate 和 Gemini 3.8 Flash 都四舍五入为 92.9，相差 0.024 分，而将所有测试汇总而非按家族平均则让 Gemini 位居第一。
def per_document(df, systems, groups):
"""For each system: an array (documents x groups x [passes, scored tests]); `groups` maps a name to a row mask."""
docs = {d: i for i, d in enumerate(sorted(df.document.unique()))}
s = df[df.result != "unsupported"]
out = {}
for name in systems:
g = s[s.system == name]
arr = np.zeros((len(docs), len(groups), 2))
for k, mask in enumerate(groups.values()):
sel = g[mask(g)]
d = sel.document.map(docs).to_numpy()
arr[:, k, 0] = np.bincount(d, weights=(sel.result == "pass").to_numpy(), minlength=len(docs))
arr[:, k, 1] = np.bincount(d, minlength=len(docs))
out[name] = arr
return out, len(docs)
def resample(n_docs, B, seed):
"""B bootstrap resamples of the documents, each as a count per document."""
rng = np.random.default_rng(seed)
return np.stack([np.bincount(rng.integers(0, n_docs, n_docs), minlength=n_docs) for _ in range(B)]).astype(float)
@section("5. How sure is the order at the top? A document-level bootstrap")
def bootstrap_step():
top = list(TABLE.headline.nlargest(6).index)
families = {f: (lambda g, f=f: g.family == f) for f in ("text", "tables", "layout")}
arrs, n_docs = per_document(VERDICTS[VERDICTS.system.isin(top)], top, families)
W = resample(n_docs, 2000, seed=20261022)
head = {}
for s in top:
tot = np.tensordot(W, arrs[s], axes=([1], [0])) # resamples x families x [passes, tests]
head[s] = 100 * (tot[..., 0] / tot[..., 1]).mean(axis=1)
point = {s: 100 * (arrs[s].sum(0)[:, 0] / arrs[s].sum(0)[:, 1]).mean() for s in top}
print(f"2,000 resamples of {n_docs:,} documents; a document's tests stay together\n")
print(f"{'parser':22s} {'headline':>9s} {'95% interval':>15s} {'gap to ' + top[0]:>26s} {'resamples where it leads':>24s}")
leader = np.argmax(np.stack([head[x] for x in top]), axis=0)
for s in top:
lo, hi = np.percentile(head[s], [2.5, 97.5])
d = head[top[0]] - head[s]
gap = "" if s == top[0] else f"{point[top[0]] - point[s]:+.2f} [{np.percentile(d, 2.5):+.2f}, {np.percentile(d, 97.5):+.2f}]"
print(f"{s:22s} {point[s]:9.2f} {lo:6.2f} - {hi:6.2f} {gap:>26s} {100 * np.mean(leader == top.index(s)):23.1f}%")
d = head[top[0]] - head[top[1]]
lo, hi = np.percentile(d, [2.5, 97.5])
claim(f"{top[0]} and {top[1]} cannot be told apart (the 95% interval of their gap includes zero)", lo < 0 < hi)
return f"{top[0]} vs {top[1]}: {point[top[0]] - point[top[1]]:+.2f} points, 95% interval [{lo:+.2f}, {hi:+.2f}]"
bootstrap_step()
如此小的差距引出了一个问题：这个排序中有多少是噪声。针对单个文档的测试往往一起通过或失败，因此我们重采样文档而非测试：2,000 次抽取中的每一次都有放回地选取 2,343 个文档，并通过一次矩阵乘法从每个文档的通过计数重新计算每个总评分。前两名之间的差距范围约为 -0.9 到 +0.9 分，且各自在约一半的抽取中领先，因此诚实的解读是平局。落后领先者 1.35 分的 Datalab balanced 与其明显分离，其下的每个解析器也是如此。
def slice_headline(df):
"""Per system: family pass rates on a slice of tests and their mean, as the benchmark's headline is built."""
s = df[df.result != "unsupported"]
fam = s.groupby(["system", "family"]).result.apply(lambda r: 100 * (r == "pass").mean()).unstack()
return fam.mean(axis=1), fam
@section("6. Slices where a different parser comes first")
def slices_step():
ranked = TABLE.dropna(subset=["headline"]).sort_values("headline", ascending=False)
rank = {s: i + 1 for i, s in enumerate(ranked.index)}
V = VERDICTS[VERDICTS.system.isin(ranked.index)]
whole, _ = slice_headline(V)
assert np.allclose(whole[ranked.index], ranked.headline), "slice_headline disagrees with unit_tests.summarize"
print(f"{'tag':13s} {'tests':>6s} {'docs':>5s} {'first (overall rank)':34s} {'second':26s} families")
leaders = collections.Counter()
for tag in ("handwriting", "tiny", "rotated", "degraded", "math", "table", "form", "multi_column",
"Grek", "Hani", "Cyrl", "Armn", "Laoo"):
sl = V[V.tags.apply(lambda t: tag in t)]
head, fam = slice_headline(sl)
order = head.sort_values(ascending=False)
(a, x), (b, y) = list(order.items())[:2]
docs = sl.document.nunique()
leaders[a] += 1
note = (" (tie for first)" if x == y else "") + (" (10 or fewer documents)" if docs <= 10 else "")
print(f"{tag:13s} {sl.test_id.nunique():6d} {docs:5d} {f'{a} {x:.1f} (#{rank[a]})':34s} {f'{b} {y:.1f}':26s} "
f"{'+'.join(c for c in ('text', 'tables', 'layout') if c in fam.columns)}{note}")
first = ranked.index[0]
claim("the overall leader is not first on every slice", leaders[first] < 13)
return f"{first} is first on {leaders[first]} of 13 slices; {len(leaders) - (first in leaders)} other parsers lead the rest"
slices_step()
按标签切片显示了单一总评分掩盖了我们实际拥有的文档的哪些情况。我们用相同的家族平均规则重新计算手写、旋转页面、表格、单一文字系统及其他切片的总评分，此前已通过断言确认该规则在完整集合上复现了官方总评分。总体领先者仅在十三个切片中的三个上位居第一。Gemini 3.8 Flash 在手写、数学和表单上领先；Claude Sonnet 5.5 在旋转页面上领先；Mistral OCR 在希腊语上领先。亚美尼亚语和老挝语分别仅基于六个和五个文档，因此我们将其视为提示而非排名。
@section("7. Do the two models named on the card hold up better on tests a model corrected? A difference-in-differences check")
def creation_check():
others = [s for s in TABLE.dropna(subset=["headline"]).index if s not in STARRED]
V = VERDICTS[VERDICTS.system.isin(STARRED + others)]
print(f"starred: {STARRED}; compared with the {len(others)} other parsers that have a headline")
print("DiD = (starred: pass rate on the split - elsewhere) - (others: the same gap); 1,000 document resamples\n")
print(f"{'tests established by':21s} {'tests':>6s} {'starred in / out':>17s} {'others in / out':>16s} {'DiD [95% interval]':>23s}")
W = None
out = {}
for deriv, note in (("model_review", "a vision model corrected the test"),
("model_panel", "independent models agreed"),
("human_review", "placebo: a person decided")):
groups = {"in": lambda g, d=deriv: g.derivation == d, "out": lambda g, d=deriv: g.derivation != d}
arrs, n_docs = per_document(V, STARRED + others, groups)
W = resample(n_docs, 1000, seed=7) if W is None else W
def gaps(weights):
r = {}
for s, a in arrs.items():
tot = np.tensordot(weights, a, axes=([-1], [0]))
with np.errstate(invalid="ignore", divide="ignore"):
r[s] = 100 * tot[..., 0] / tot[..., 1]
did = (np.nanmean([r[s][..., 0] - r[s][..., 1] for s in STARRED], axis=0)
- np.nanmean([r[s][..., 0] - r[s][..., 1] for s in others], axis=0))
return r, did
r, d = gaps(np.ones(n_docs))
_, boot = gaps(W)
lo, hi = np.nanpercentile(boot, [2.5, 97.5])
si, so = np.mean([r[s][0] for s in STARRED]), np.mean([r[s][1] for s in STARRED])
oi, oo = np.mean([r[s][0] for s in others]), np.mean([r[s][1] for s in others])
out[deriv] = (d, lo, hi)
print(f"{deriv:21s} {(TESTS.derivation == deriv).sum():6,d} {si:7.1f} / {so:6.1f} {oi:6.1f} / {oo:6.1f} "
f"{d:+6.1f} [{lo:+5.1f}, {hi:+5.1f}] {note}")
panel = TESTS[TESTS.derivation == "model_panel"]
layout = TESTS[TESTS.test_type == "layout_kind"]
print(f"\nmodel_panel tests are {100 * (panel.test_type == 'layout_kind').mean():.0f}% layout tests, and "
f"{100 * (layout.derivation == 'model_panel').mean():.0f}% of layout tests are model_panel")
review = VERDICTS[VERDICTS.derivation == "model_review"]
rate = lambda s: 100 * (review[review.system == s].result == "pass").mean()
print(f"pass rate on the {(TESTS.derivation == 'model_review').sum()} model_review tests: "
+ ", ".join(f"{s} {rate(s):.1f}%" for s in STARRED) + f", median of the others {np.median([rate(s) for s in others]):.1f}%")
claim("the starred models gain more on model_review tests than on the placebo split", out["model_review"][1] > out["human_review"][2])
claim("the placebo split shows no gap (its interval includes zero)", out["human_review"][1] < 0 < out["human_review"][2])
return (f"DiD model_review {out['model_review'][0]:+.1f}, model_panel {out['model_panel'][0]:+.1f}, "
f"placebo human_review {out['human_review'][0]:+.1f} points")
creation_check()
数据集卡片将 Gemini 3.8 Flash 和 Claude Sonnet 5.5 标记为用于创建和验证某些测试，但未说明是哪些，因此我们用双重差分提出一个更窄的问题：相对于解析器在其他地方的表现，其在某一衍生划分上的通过率变化，对这对星标模型是否不同？在由视觉模型校正的 102 个 model_review 测试上，两组的通过率都远低于其他地方，但星标对的下降约少 25 分。human_review 安慰剂显示出同方向但更小的差距，而 model_panel 则相反，因此我们报告一个标记，而非证据。
def predict_pages(run, pages, alive=lambda: True):
"""Predict every page the run still lacks. A failed call is recorded, not raised, so we count failures ourselves."""
t0, n, failed = time.time(), 0, []
for p in pages:
if not alive():
raise RuntimeError(f"{run.provider}: the server stopped after {n} pages")
if need := benchmark.needs(run, p):
got = harness.predict(run.provider, p.doc_path, need, timeout=900, **run.options)
benchmark.record(run, p.page_id, got)
n += 1
if any(c.get("failure") for c in got["calls"]):
failed.append(f"{p.page_id}: {got['errors']}")
if failed:
print(f"{len(failed)} of {n} pages failed, e.g. {failed[0][:160]}")
if 2 * len(failed) >= n:
raise RuntimeError(f"{run.provider} failed on {len(failed)} of {n} pages")
return n, (time.time() - t0) / max(n, 1)
@section("8. Run Tesseract ourselves on 50 pages that span all 22 suites")
def tesseract_step():
global SAMPLE
if TESS_MAJOR < 5:
raise RuntimeError(f"tesseract {TESS_MAJOR}.x is installed; this tutorial was verified with tesseract 5")
SAMPLE = data.select(PAGES, sample=50)
snapshot_download(REPO, repo_type="dataset", revision=REV, local_dir=str(DATA),
allow_patterns=sorted(str(p.doc_path.relative_to(DATA)) for p in SAMPLE))
n_pdf = sum(p.doc_path.suffix == ".pdf" for p in SAMPLE)
print(f"sample: {len(SAMPLE)} pages from {len({p.suite for p in SAMPLE})} suites, {n_pdf} PDF and {len(SAMPLE) - n_pdf} images, "
f"{sum(p.doc_path.stat().st_size for p in SAMPLE) / 1e6:.1f} MB, {sum(len(unit_tests.gold(p)) for p in SAMPLE)} tests")
# The adapter asks Tesseract for '-l script/<Name>'. Homebrew keeps those models in a script/ folder; Ubuntu installs
# them flat, so on Linux we point Tesseract at a folder of symlinks with the layout it expects.
listing = subprocess.run(["tesseract", "--list-langs"], capture_output=True, text=True)
listing = listing.stdout + listing.stderr
if "script/Latin" not in listing:
src = Path(re.search(r'"(.+?)"', listing).group(1))
dst = WORK / "tessdata"
(dst / "script").mkdir(parents=True, exist_ok=True)
for f in src.glob("*.traineddata"):
for link in [dst / f.name] + ([dst / "script" / f.name] if f.name[0].isupper() else []):
if not link.exists():
link.symlink_to(f)
os.environ["TESSDATA_PREFIX"] = str(dst)
print(f"script models linked under {dst}")
run = benchmark.Run("tesseract", {}, WORK / "runs" / "tesseract")
benchmark.prepare(run, str(DATA), 900)
n, per_page = predict_pages(run, SAMPLE)
s = benchmark.grade(run, SAMPLE, say=lambda *_: None)
fam = s["families"]
print(f"\nTesseract: {n} pages read, {per_page:.1f} s per page")
for k, v in fam.items():
print(f" {k:7s} " + ("unsupported (no blocks output)" if v["score"] is None else f"{100 * v['score']:5.1f}% of {v['tests']} tests"))
print(f" headline {'none' if s['headline'] is None else round(100 * s['headline'], 1)} (a family it cannot produce leaves no headline)")
return f"Tesseract on 50 pages: text {100 * fam['text']['score']:.1f}%, tables {100 * fam['tables']['score']:.1f}%, {per_page:.1f} s/page"
tesseract_step()
从已发布的判定重建排行榜是一回事；自己生成判定则是另一回事。data.select 选取分布在全部 22 个套件上的固定 50 页样本，我们只下载这些页面。循环是基准自身的：needs 列出页面仍缺少的输出，harness.predict 运行解析器；record 保存预测并对文件夹评分，因此中断的运行可以恢复。在 Ubuntu 上，我们还将 Tesseract 的文字系统模型链接到适配器期望的布局中。Tesseract 通过样本文本测试的 30.4%，而在完整集合上为 17.6%，这一差距取决于采样了哪些页面。
def agreement(folder, published):
"""Our verdicts against the published ones for the same tests, where both scored the test."""
ours = pd.read_json(folder / "scores.jsonl", lines=True)[["test_id", "result", "why"]]
pub = VERDICTS[VERDICTS.system == published][["test_id", "result"]]
m = ours.merge(pub, on="test_id", suffixes=("_ours", "_pub"))
m = m[(m.result_ours != "unsupported") & (m.result_pub != "unsupported")]
return m, int((m.result_ours == m.result_pub).sum()), len(m)
@section("9. Indra OCR on the same pages, served on the CPU from the package's own server code")
def indra_step():
if sys.version_info < (3, 12):
raise RuntimeError("the Indra pins (numpy 2.5.4) need Python 3.12 or newer")
venv = WORK / "indra-venv"
py = venv / "bin" / "python"
if not py.exists():
uv = shutil.which("uv") or str(Path(sys.executable).parent / "uv")
subprocess.run([uv, "venv", "-q", "--python", sys.executable, str(venv)], check=True)
subprocess.run([uv, "pip", "install", "-q", "--python", str(py)] + INDRA, check=True)
server = Path(harness.__file__).parent / "providers" / "oss" / "indra" / "server.py"
# server.py counts cores with os.sched_getaffinity, which macOS lacks; on Linux the shim changes nothing
launcher = WORK / "indra_launch.py"
launcher.write_text("import os, runpy, sys\n"
"if not hasattr(os, 'sched_getaffinity'):\n"
" os.sched_getaffinity = lambda pid: set(range(os.cpu_count()))\n"
"sys.argv = ['server.py', '--port', '8001', '--workers', '2']\n"
f"runpy.run_path({str(server)!r}, run_name='__main__')\n")
log = open(WORK / "indra_server.log", "w")
proc = subprocess.Popen([str(py), str(launcher)], stdout=log, stderr=subprocess.STDOUT)
t0 = time.time()
while True:
try:
urllib.request.urlopen("http://127.0.0.1:8001/health", timeout=2)
break
except Exception:
if proc.poll() is not None or time.time() - t0 > 600:
proc.terminate()
raise RuntimeError("the Indra server did not start; see " + log.name)
time.sleep(2)
print(f"server up in {time.time() - t0:.0f} s: {INDRA[0]}, 2 worker processes")
try:
run = benchmark.Run("indra-ocr", {"base_url": "http://127.0.0.1:8001"}, WORK / "runs" / "indra-ocr")
benchmark.prepare(run, str(DATA), 900)
n, per_page = predict_pages(run, SAMPLE, alive=lambda: proc.poll() is None)
s = benchmark.grade(run, SAMPLE, say=lambda *_: None)
finally:
proc.terminate()
fam = {k: 100 * v["score"] for k, v in s["families"].items()}
print(f"Indra OCR: {n} pages, {per_page:.2f} s per page; text {fam['text']:.1f}%, tables {fam['tables']:.1f}%, "
f"layout {fam['layout']:.1f}%; headline on these pages {100 * s['headline']:.1f}\n")
ids = {p.page_id for p in SAMPLE}
fmt = lambda x: "none" if x is None else f"{100 * x:.1f}"
print(f"{'on the 50 pages':22s} {'headline':>9s} {'text':>6s} {'tables':>7s} {'layout':>7s}")
for name, folder in (("Tesseract", WORK / "runs" / "tesseract"), ("Indra OCR", WORK / "runs" / "indra-ocr")):
pub = unit_tests.summarize(VERDICTS[(VERDICTS.system == name) & VERDICTS.page_id.isin(ids)][COLS].to_dict("records"))
ours = unit_tests.summarize(benchmark.read_scores(benchmark.Run(name, {}, folder)))
for who, x in (("ours", ours), ("published", pub)):
f = x["families"]
print(f"{name + ', ' + who:22s} {fmt(x['headline']):>9s} " + " ".join(f"{fmt(f[k]['score']) if k in f else 'none':>6s}"
for k in ("text", "tables", "layout")))
print()
for name, folder in (("Tesseract", WORK / "runs" / "tesseract"), ("Indra OCR", WORK / "runs" / "indra-ocr")):
m, same, total = agreement(folder, name)
diff = m[m.result_ours != m.result_pub]
print(f"{name:10s} our verdicts equal the published ones on {same} of {total} tests ({100 * same / total:.1f}%); "
f"of the {len(diff)} that differ, {(diff.result_ours == 'pass').sum()} pass only for us and "
f"{(diff.result_ours == 'fail').sum()} only in the published run")
if len(diff):
by_page = diff.test_id.map(TESTS.set_index("test_id").page_id).value_counts()
print(" differing tests by type:", {k: int(v) for k, v in diff.test_id.map(TESTS.set_index("test_id").test_type).value_counts().items()},
f"| on {len(by_page)} pages, {by_page.iloc[0]} of them on {by_page.index[0]}")
claim(f"{name}: our verdicts equal the published ones on at least 95% of the sample's tests", same >= 0.95 * total)
return f"Indra OCR on 50 pages: headline {100 * s['headline']:.1f} (text {fam['text']:.1f}, tables {fam['tables']:.1f}, layout {fam['layout']:.1f}), {per_page:.2f} s/page"
indra_step()
Indra OCR 是一个开放的 CPU 模型，其软件包附带了适配器所通信的 HTTP 服务器，因此我们将该服务器 Dockerfile 中固定版本的依赖安装到单独环境中，并在后台启动它。对照同一批页面上已公布的判定结果，Tesseract 在每个计分测试上都一致，因此其样本得分与公布值相符。Indra 在 221 个测试中有 203 个一致，且其所有表格差异都集中在同一页面上，这正好解释了其公布表格得分 36.1 与我们得到的 69.4 之间的全部差距。一个页面就能让一个模型系列产生如此大的偏移，这是重新采样文档最有力的论据。
@section("10. Bring your own parser: grade a PDF text-layer baseline with the same code")
def byo_step():
import pypdfium2 as pdfium
def text_layer(path):
if path.suffix.lower() != ".pdf":
return "" # an image has no text layer to read
doc = pdfium.PdfDocument(str(path))
try:
return "<pre>" + html.escape(doc[0].get_textpage().get_text_range()) + "</pre>"
finally:
doc.close()
out = WORK / "runs" / "text-layer"
(out / "predictions").mkdir(parents=True, exist_ok=True)
for p in SAMPLE:
(out / "predictions" / f"{p.page_id}.json").write_text(json.dumps({"html": text_layer(p.doc_path)}))
s = benchmark.grade(benchmark.Run("text-layer", {}, out), SAMPLE, say=lambda *_: None, settings={"unsupported": ["blocks"]})
kind = {p.page_id: "PDF" if p.doc_path.suffix == ".pdf" else "image" for p in SAMPLE}
print(f"{'parser on the same 50 pages':28s} {'text: PDF':>10s} {'images':>8s} {'all':>6s} {'tables':>8s}")
for name in ("text-layer", "tesseract", "indra-ocr"):
sc = pd.read_json(WORK / "runs" / name / "scores.jsonl", lines=True)
sc = sc[sc.result.isin(["pass", "fail"])]
txt = sc[sc.test_type.map(vocab.FAMILIES) == "text"]
by = txt.groupby(txt.page_id.map(kind)).result.apply(lambda r: 100 * (r == "pass").mean())
tab = sc[sc.test_type == "table_cell"]
print(f"{name:28s} {by.get('PDF', np.nan):10.1f} {by.get('image', np.nan):8.1f} "
f"{100 * (txt.result == 'pass').mean():6.1f} {100 * (tab.result == 'pass').mean():8.1f}")
fam = s["families"]
return f"text layer, no OCR: text {100 * fam['text']['score']:.1f}%, tables {100 * fam['tables']['score']:.1f}% on the 50 pages"
byo_step()
任何解析器都可以在没有适配器的情况下评分：我们为每个页面写一个预测文件并调用 benchmark.grade，将我们无法产生的输出声明为不支持。我们的解析器使用 pypdfium2 读取 PDF 的内嵌文本层，对图像则返回空字符串，因为图像没有文本层。在 PDF 页面上，它通过的文本测试数量与 Tesseract 完全相同，各为 42.3%，且无需任何 OCR；而在图像上它一无所获，Indra OCR 仍能通过 43.1%。纯文本无法通过任何表格测试，因为通过需要单元格结构，因此每当比较两个解析器时，PDF 与图像的混合比例都很重要。
@section("11. The launch claims, checked against the files")
def claims_step():
a, t = TABLE.loc["Datalab accurate"], TABLE.loc["Tesseract"]
g = TABLE.loc["Gemini 3.8 Flash", "headline"]
c = TABLE.loc["Claude Sonnet 5.5"]
tags = TESTS.tags
langs = {x for ts in tags for x in ts if re.fullmatch("[a-z]{3}", x)} - {"und", "zxx"}
pkg = Path(vocab.__file__).parent # the scorer and the modules it reads; data.py and the harness only fetch and run
scoring = sorted(pkg.glob("scorers/**/*.py")) + [pkg / f"{m}.py" for m in ("normalize", "tables", "equations", "views", "vocab", "markdown")]
clients = [f.name for f in scoring if re.search(r"^\s*(import|from)\s+(httpx|requests|urllib|huggingface_hub|openai|anthropic|google"
r"|transformers|torch|omni_parse_bench\.harness)\b", f.read_text(), re.M)]
n_eng, n_table = tags.apply(lambda x: "eng" in x).sum(), tags.apply(lambda x: "table" in x).sum()
rows = [
("16,288 tests, 2,937 pages, 2,343 documents", f"{len(TESTS):,} / {TESTS.page_id.nunique():,} / {TESTS.document.nunique():,}",
(len(TESTS), TESTS.page_id.nunique(), TESTS.document.nunique()) == (16288, 2937, 2343)),
("92 languages", f"{len(langs)} codes besides und and zxx", len(langs) == 92),
("10,089 English tests, 6,767 table tests", f"{n_eng:,} and {n_table:,}", (n_eng, n_table) == (10089, 6767)),
("Datalab accurate scores 92.7", f"{a.headline:.2f} in results.parquet", round(a.headline, 1) == 92.7),
("Datalab accurate ranks second, behind Gemini", f"{a.headline:.3f} vs {g:.3f}", a.headline < g),
("Claude Sonnet 5.5: text 93.1, tables 95.4, layout 77.5", f"{c.text:.1f} / {c.tables:.1f} / {c.layout:.1f}",
[round(c[k], 1) for k in ("text", "tables", "layout")] == [93.1, 95.4, 77.5]),
("Tesseract: no headline, text 17.6, tables 0.0",
f"{'none' if pd.isna(t.headline) else round(t.headline, 1)} / {t.text:.1f} / {t.tables:.1f}",
pd.isna(t.headline) and round(t.text, 1) == 17.6 and round(t.tables, 1) == 0.0),
("no model judges the outputs", f"{len(clients)} of {len(scoring)} scorer modules import a model or HTTP client", not clients),
]
print(f"{'claim (launch article and blog)':56s} {'what the files say':38s} verdict")
for text, found, ok in rows:
print(f"{text:56s} {found:38s} {'holds' if ok else 'does not hold'}")
claim(text, ok)
print("\nnot checkable from these files: latency, price per 1,000 pages, the other benchmarks' test counts")
return f"{sum(bool(ok) for *_, ok in rows)} of {len(rows)} launch claims hold against the files"
claims_step()
我们没有重复发布时的声明，而是逐项对照文件进行核查。各项计数完全吻合，从测试和页面到语言和子集，Claude 和 Tesseract 两行也相符。Datalab 自身那一行则不符：发布文章和博客给 Datalab 的准确率为 92.7，将其排在 Gemini 之后位列第二，而发布的判定结果给出的是 92.92 对 92.89，即四舍五入后数值相同的平局。无评判模型的声明对评分器本身成立，其模块不导入任何模型或 HTTP 客户端；KaTeX 在嵌入式 JavaScript 引擎中运行以比较公式。延迟和价格无法从这些文件中核查。
banner("SUMMARY")
for name, res in RESULTS.items():
print(f" {name:<100s} {res}")
print("\nchecks decided in this run:")
for step, text, held in CLAIMS:
print(f" [{'held' if held else 'did not hold'}] step {step}: {text}")
failed = [k for k, v in RESULTS.items() if str(v).startswith("SKIPPED")]
print(f"\n{sum(h for *_, h in CLAIMS)} of {len(CLAIMS)} checks held; steps failed or skipped: {failed or 'none'}")
print(f"omni-parse-bench {md.version('omni-parse-bench')}, dataset @ {REV[:7]}, Python {platform.python_version()}, {SYSTEM}")
print("not run here: the hosted parsers (paid API keys), the GPU models, latency and cost, all 2,937 pages")
摘要会打印每个步骤的一行结果以及本次运行所判定的每一项检查，并标记哪些成立、哪些不成立：17 项中有 13 项成立，不成立的 4 项属于发现而非失败，即安慰剂差距、Indra 的复现以及 Datalab 的两个发布数字。它还列出了我们在此未运行的内容：需要付费密钥的托管解析器、GPU 模型、延迟与成本，以及全部 2,937 个页面。
总之，OmniParseBench 是一个我们可以端到端审计的基准：评分器是不调用任何模型的开源代码，判定结果已发布，卡片上的每个条目都可以从中重建。排名前两位的解析器在统计上持平，合并测试而非按模型系列分组会改变排名，而在手写、旋转页面、表格和多种文字上，另一个解析器位居第一。卡片上提到的两个模型在经过模型纠正的测试上下降幅度小于其他模型，我们将其作为标记而非证据来报告。Tesseract 的公布判定结果在我们的样本上完全复现，而 Indra OCR 的判定结果在 92% 的测试上一致。实际经验是：将基准切分到我们自己的文档、报告置信区间，并固定每个版本。
查看 Notebook 和 Repo。所有功劳归功于本项目的研究者。另外，欢迎在 Twitter 上关注我们，别忘了加入我们的 150k+ML SubReddit 并订阅 我们的 Newsletter。等等！你在 Telegram 上吗？现在你也可以在 Telegram 上加入我们了。
需要与我们合作推广你的 GitHub Repo 或 Hugging Face 页面或产品发布或网络研讨会等？联系我们
文章 A Developer’s Guide to OmniParseBench: Leaderboard Benchmarks and OCR Testing 首次发表于 MarkTechPost。
来源：MarkTechPost · marktechpost.com
来源：RadarAI · aihot.news