Skynet Countdown 开源 Python 评分工具：本地复现时钟评分方法

An open-source Python companion to Skynet Countdown. Bring your own collected stories, inspect the model's evidence judgements, and reproduce the published scoring calculation locally.

This project starts after collection and scraping. It does not discover stories, scrape Reddit or articles, reconcile duplicate events, manage assessment history, allocate CVE identifiers, or publish to SQL, Sheets or Slack. It writes one local JSON result. The index is editorial: its clock is symbolic, not a forecast or a probability of catastrophe.

METHODOLOGY.md is the scoring authority: methodology v1.0, with calibration effective 7 October 2026. The original n8n workflow provides the relevance, criterion-assessment and editorial stages. Python derives the published totals and clock values from the component ratings; retained n8n clock recommendations never set the clock.

Quick start

Use Python 3.13 or newer and uv. Run commands from the repository root:

uv sync
uv run python -m skynet_countdown --input examples/stories.json --validate-only

Validation checks the dataset and configuration without making API calls. The example stories are explicitly fictional and demonstrate the input format, not real incident assessments.

Copy .env.example to .env and set OPENAI_API_KEY. Then run:

uv run --env-file .env python -m skynet_countdown --input examples/stories.json --output results.json

Evaluation sends supplied story fields to OpenAI and incurs API charges. The default model is gpt-5.4 with high reasoning effort; adjust config.toml to suit your needs and model access. Existing environment variables work too: the application itself does not load .env.

The output path must be new and its parent directory must exist. Existing files are never overwritten. Use --config path/to/config.toml to select another configuration. This is a small uv project run directly from its checkout, with the OpenAI SDK and Pydantic as its only runtime dependencies.

Supply stories

The input is a UTF-8 JSON object containing a stories array:

{
  "stories": [
    {
      "id": "my-event-001",
      "title": "A fictional AI deployment",
      "source_url": "https://example.com/fictional-deployment",
      "published_date": "2026-09-28",
      "summary": "A fictional company deployed an agent with per-action human approval.",
      "content": "Optional: the already-collected source text and supporting evidence.",
      "metadata": {"fictional": true}
    }
  ]
}

id, title, source_url, published_date and summary are required. Use the event's canonical stable slug as id; it is also the tie-break key for events recorded on the same date. Dates use YYYY-MM-DD; URLs must be HTTP(S). content, reddit_post_url and subreddit are optional and may be omitted or null. Put any additional JSON data under the optional metadata object. Unknown top-level or story fields are rejected, as are invalid types, blank required strings and duplicate IDs.

Supply one selected source per explicitly grouped event. The caller owns event grouping, canonical source selection and assessment history; do not pass revisions or repeated coverage as separate events. Different articles about the same event are not automatically recognised as duplicates, even with distinct IDs. A source URL is attribution, not fetched content: provide the original source evidence in summary, content and metadata, rather than treating Reddit titles or comments as the basis for ratings. Without article content, the model works from the remaining supplied evidence and records its limitations.

This companion generates fresh assessments and reports. It does not ingest legacy n8n assessment exports or enforce their database schema. In particular, its calculation.total_score is the published weighted score; it must not be copied into n8n's retained unweighted total_score column. The distinct calculation.raw_source_total exposes that 0–13 sum for the editorial gate and integrations. A legacy exporter must also derive and validate the raw classification and other required source fields described in methodology section 11.

Evaluation stages

1. Relevance chain: produces a relevance flag, score from 1 to 10, rationale and primary dimension. A story advances only when the flag is true and its score meets minimum_relevance_score.
2. Criterion assessment: rates the three binary Trifecta checks and five amplifiers from 0 to 2, with a rationale for each, uncertainties and a separate leading-indicator flag. The model returns no totals, severity or clock calculations.
3. Python calculation: derives the weighted score, severity and clock movement from those eight ratings.
4. Editorial chain: writes a headline, metadata line, factual account and interpretation. Python attaches the stable story ID, severity and clock labels.

The relevance and writer stages are structured LLM chains without tools. The scorer is also a chain by default; optional hosted web search can check the supplied story. There is no agent framework or custom tool loop. All model outputs use strict JSON schemas and are validated with Pydantic.

Stories are processed by recorded date, then canonical event ID. No publication-date cutoff or automatic rebasing applies. The default relevance threshold is 2, matching the n8n filter actually executed; its unused configuration value of 5 is not adopted. The rubric marks scores 1–2 as irrelevant, and contradictory positive flags trigger a correction attempt. The default report threshold is 2 raw source points, matching the saved writer branch. Every successfully assessed story contributes its published points even when below that reporting threshold or when report writing is disabled. Zero-score assessments remain in the output and do not move the clock.

Deterministic scoring

Each Trifecta check is 0 or 1: ungoverned system access (T1), untrusted input exposure (T2), and permission and capability to act externally without per-action approval (T3).

Add the five amplifier ratings: governance vacuum, autonomy without oversight, human capability erosion, emergent agency signals, and physical weaponisation. Each contributes 0, 1 or 2. The legacy sentience_self_preservation key remains for compatibility; behavioural agency signals do not establish consciousness. Controlled civilian embodiment can earn one physical point. The default maximum is 17. T3 alone does not earn autonomy amplifier points; that amplifier needs separate evidence about the extent, duration or scale of operation beyond meaningful review.

With cumulative evidence points B, incident score s, initial seconds T and half-scale H:

remaining_seconds(B) = T × 2^(-B / H)
incident_delta_seconds = remaining_seconds(B) - remaining_seconds(B + s)
gap_closed_percent = 100 × (1 - 2^(-s / H))
evidence_pressure_percent = 100 × (1 - 2^(-B / H))

Defaults are T = 900 seconds (15:00) and H = 1000: every 1,000 points halve the remaining time. The initial display reference is borrowed once from IMD's September 2026 AI Safety Clock reading, as explained in methodology section 7; it does not add points or create a date cutoff. Movement is calculated as remaining_seconds(B) × gap_closed_percent / 100 using expm1, preserving tiny changes instead of subtracting nearly equal numbers. Evidence pressure measures the share of the initial symbolic window consumed, not the percentage of human control lost.

clock.initial_evidence_points starts a run from an existing cumulative total. Supplying that total does not provide history or prevent recounting earlier events. For corrections, rerun the complete caller-selected ledger from zero; a corrected score or grouping can move the recalculated clock in either direction. Scores never subtract evidence, grant recovery credit or decay over time.

Calculations retain unrounded values and reject evidence totals beyond 2**53 - 1 for compatibility with the official dataset engine. Floating-point underflow retains the smallest positive time and positive-event movement; a zero score remains exactly zero. format_remaining_time rounds to the nearest second and displays values below one second as <00:01. format_movement_seconds distinguishes NO MOVEMENT from <1 SECOND CLOSER. format_evidence_pressure uses one decimal and displays values at or above 99.95 as <100.

The official 113-point example displays 13:52. With 191 prior points, a full Trifecta plus autonomy = 2 scores 9 / CRITICAL, closes 0.62% of the remaining gap, moves 4.903 seconds, and results in 13:03. These examples are covered by offline regression tests.

The arithmetic is deterministic for the same validated ratings, configuration and event order. Model judgements and editorial text can vary between runs. The pure functions in scoring.py, including calculate_score, remaining_seconds, gap_closed_percent and evidence_pressure_percent, can be used independently of the API; pipeline.py exposes evaluate(dataset, config, client=...) for integrations. Criterion contracts live in models.py, rubric instructions in prompts/, and published defaults in config.py and config.toml.

Configuration

config.toml contains all defaults and is read with Python's standard TOML parser. Unknown keys and invalid values fail validation.

max_iterations counts structured-output attempts, including the initial request. Further attempts correct schema or validation failures; refusals and incomplete responses fail that stage. It does not count hosted search calls or SDK transport retries, which have their own settings. max_tool_calls applies separately to each request, including a correction request.

Choose a model that supports the Responses API and structured outputs, plus web search if enabled. Reasoning values and tool support depend on the model. Set reasoning_effort = "omit" to leave out that API parameter. Changing scoring constants is supported, but produces a variant of the published default methodology; the derived maximum adjusts with the Trifecta table.

Inspect the result

The JSON retains each input story alongside relevance, assessment, calculation, report, status, error and per-stage runs. Stages not reached are null. Run records include response IDs, returned model names, status, token usage, citations and web-search calls when available. The file also records the resolved configuration, prompt SHA-256 hashes, schema version, methodology version, calibration reference date, generation time, cumulative points, remaining seconds and evidence pressure.

methodology_profile is published when the scoring table, severity thresholds, starting window, sensitivity and relevance/report thresholds match the official defaults; otherwise it is custom. Model settings, disabling reports and supplying prior evidence do not change this profile. calibration_effective_date identifies the official reference calibration; the resolved configuration records any custom constants actually used.

Result schema 2.0 uses internal severity NO_MOVEMENT, displayed as “NO MOVEMENT”; adds raw_source_total, evidence pressure and methodology metadata; and applies the raw editorial gate and canonical-ID ordering. Integrations consuming schema 1.0 should account for those changes. Report prose still separates factual story and interpretation; metadata must be one lowercase line containing exactly three nonblank fragments separated by ·.

Excluded stories stay visible. A stage failure is recorded against its story, later stories still run, and the file has complete: false; the CLI exits with a nonzero status. No replacement ratings are invented. If writing fails after a successful calculation, that assessment and its points remain in the result. Review failed records before treating a run as complete.

Contribute

Start with CONTRIBUTING.md. The offline test suite uses the standard library and mocked API responses:

uv run python -m unittest discover -s tests -v

It makes no paid API calls. Changes to criteria, prompts or score calculations should include a concrete example and explain how they relate to the reference methodology. Licensed under MIT.