Technical writing
Censorship incident lifecycle in Voidly: from anomaly detection to verified incident to resolution
The previous article in this series covered how Voidly classifies evidence into three confidence tiers — Anomaly, Corroborated, and Verified Incident — and why independent corroboration from external sources is required before an event becomes citable. That article treated tier assignment as a threshold test applied to a static record. This one covers what actually happens over time: how an incident is created, how it moves through states as evidence accumulates, how it reaches resolution, and how every stage of that journey is encoded in the dataset fields that researchers download from HuggingFace.
The distinction matters because censorship is not instantaneous. A block on Twitter in Iran is not a single event that either happened or did not. It is a process: a first probe fires, corroboration arrives over the following minutes, the event is verified by independent sources, persists for hours or days, and eventually either resolves when measurements return to baseline or is reclassified as a false positive when investigation reveals a non-censorship cause. The incident record in the database changes continuously until that process ends.
Incident vs. measurement
A measurement is a single probe's reading at a single moment: probe p-042 tested twitter.com via DNS from AS44244 in Iran at 14:03:22 UTC on 2025-01-15 and saw a tampered response. One row in the measurements table.
An incident groups many measurements into a coherent event: Iran blocked Twitter from 2025-01-15 14:00 UTC to 2025-01-18 22:00 UTC, confirmed by 847 individual measurements across 23 ASNs, corroborated by OONI and IODA, with a duration of approximately 80 hours. One row in the incidents table, with a stable incident_id that all 847 measurements reference.
The clustering logic that decides which measurements belong to the same incident is documented separately in the incident clustering article — the 4-part clustering key, the 6-hour gap rule, re-open windows, and edge cases like BGP outages and flapping blocks. This article focuses entirely on what happens after an incident is created by the clustering logic: the state machine that governs how it evolves.
The IncidentState enum
Every incident record carries a state field drawn from a six-value enum. In the Rust source:
pub enum IncidentState {
/// Single anomaly measurement, no corroboration yet
Anomaly,
/// Multiple Voidly measurements agree but no external source yet
MultiSourceAnomaly,
/// At least one external source (OONI/CP/IODA) corroborates
Corroborated,
/// Verified by 2+ independent sources, confidence >= 0.80
VerifiedIncident,
/// Event ended, confirmed by measurement return to baseline
Resolved,
/// Downgraded after investigation revealed it was not censorship
FalsePositive,
}The first four states form a linear promotion path as evidence accumulates. Resolved and FalsePositive are terminal states — once an incident enters either, it does not return to active monitoring under the same incident ID. The distinction between the two is important: a Resolved incident is genuine censorship that ended; a FalsePositive was never censorship.
Notably, MultiSourceAnomaly is a Voidly-internal state that does not appear in the public dataset. It represents a stage in the evidence-gathering process where the internal probe network has built confidence but external corroboration has not yet arrived. Only incidents at Corroborated and above are published.
State transition triggers
Anomaly to MultiSourceAnomaly
An incident enters Anomaly the moment the clustering logic creates it — when the first measurement for a new (country_code, domain, interference_type)tuple crosses the classifier threshold. It remains at Anomaly until the multi-source condition is met.
The transition to MultiSourceAnomaly fires when the incident accumulates:
- At least 3 Voidly measurements
- from at least 2 distinct ASNs
- within a 4-hour window
- all flagging the same
(country, domain, interference_type)tuple
The ASN requirement is the key constraint. Same-ASN measurements share the same network path, the same DNS resolver, and the same DPI infrastructure. Three probes all on AS44244 seeing the same tampered DNS response is weak evidence — they could all be affected by a single misconfigured resolver. Two probes on different ASNs seeing the same interference independently is substantively stronger. The 4-hour window prevents a slow trickle of measurements from triggering the transition when the block may have already ended.
MultiSourceAnomaly to Corroborated
This transition requires at least one external source to independently confirm the event. The check runs continuously via the corroboration engine, which polls OONI and IODA in near-real-time and consults the in-memory CensoredPlanet daily index. Any of the following conditions triggers the transition:
- OONI confirmed=true for the same
(country, domain)within a 6-hour window around the incident start - CensoredPlanet blocked=true for the same target within the same window
- IODA BGP withdrawal covering the incident's country within 6 hours (used for BGP-level outages rather than domain-specific blocks)
The condition also requires that the corroboration weight reaches at least 0.40. A single corroborating measurement from a methodologically similar source (e.g., OONI active probe agreeing with a Voidly active probe) scores lower than a corroborating signal from IODA's passive BGP monitoring, which is structurally independent. The 0.40 floor prevents promotion on marginal or unreliable external signals.
Corroborated to VerifiedIncident
The promotion to VerifiedIncident is the most consequential transition in the lifecycle. It triggers journalist alerts, SSE stream events, and marks the incident as citable. The requirements are correspondingly strict:
- Standard path: corroboration score ≥ 0.80, derived from at least 2 independent sources. The score formula uses independence weights — OONI and CensoredPlanet together reach roughly 0.88; OONI and IODA together reach roughly 0.97 because they use fundamentally different detection methodologies.
- High-confidence single-source path: a single source with confidence ≥ 0.95 can satisfy the verification threshold without a second source. In practice this applies to cases where OONI has dozens of consistent probe measurements from multiple ASNs in the target country, or where IODA's BGP signal shows a withdrawal covering the entire country's address space. Very high-quality single-source evidence is treated as equivalent to two-source corroboration.
The 0.80 threshold was calibrated against a labeled ground-truth set of 412 incidents from 2023 and 2024 with known true/false status from independent verification. At 0.80, the false positive rate among VerifiedIncidents was 0.9%. At 0.70 the false positive rate rose to 3.1%. At 0.90 it fell further but recall dropped significantly — events in countries with sparse external coverage never reached 0.90 even when genuinely verified by IODA.
VerifiedIncident to Resolved
An incident reaches Resolved when the underlying interference ends. The resolution check runs continuously, re-evaluating every 30 minutes for any active VerifiedIncident. The conditions for resolution:
- A 4-hour window in which at least 80% of affected probes report no anomaly for the same
(country, domain, interference_type)tuple - The measurement rate for that tuple returns to within 2 standard deviations of the pre-incident baseline (computed over the 14-day window before the incident began)
The 80% threshold rather than 100% accounts for probe heterogeneity: some ASNs in a country may lift a block before others, or a CDN may have different rollout timing across regions. Requiring all probes to clear simultaneously would delay resolution attribution for blocks that end incrementally. The 2σ baseline check catches cases where most probes stop flagging but the measurement rate remains elevated, which can indicate a partial block or a block that shifted to a different mechanism not yet detected.
The 30-minute re-evaluation cadence means that in the best case, a block that ends cleanly (all probes clear simultaneously) is marked Resolved within 30 minutes of the last anomalous measurement. In practice the median resolution lag is 47 minutes, accounting for the 4-hour confirmation window and re-evaluation timing.
Any state to FalsePositive
Any incident can be moved to FalsePositive at any point in its lifecycle, before or after verification. There are two pathways:
Automated FalsePositive classification watches for three signals:
- CDN maintenance. Akamai and Cloudflare publish real-time maintenance and incident status via their status page APIs. The FP detector polls both APIs on a 15-minute interval. If a CDN maintenance event overlaps in time and country coverage with an active Voidly incident, and the incident's interference type is consistent with CDN-level failures (TLS interference, HTTP blocking), the incident is flagged as a candidate false positive and reviewed.
- RIPE Atlas scheduled maintenance. RIPE publishes a calendar of scheduled network maintenance events that affect measurement infrastructure. A Voidly incident that coincides with a RIPE Atlas-flagged maintenance window for the same country is automatically labeled as a candidate FP.
- Global outage pattern. If an incident fires simultaneously in more than 50 countries for the same domain, this is almost certainly a network infrastructure issue with the origin server rather than country-specific censorship. No government censorship apparatus operates across 50 jurisdictions simultaneously. The FP detector moves all co-incident incidents to
FalsePositivewhen this pattern is detected.
Manual FalsePositive review is available to verified researchers via a GitHub issue workflow. A researcher files an issue citing the incident_id and attaches supporting evidence. The Voidly team reviews within a 24-hour SLA and either confirms the FP designation (moving the record to FalsePositive in both the internal database and the dataset) or rejects the claim and closes the issue with an explanation.
Lifecycle timing: benchmarks from 2024
Across 847 verified incidents in 2024, the median and p95 transition times were:
Transition Median time p95 ───────────────────────────────────────────────────── Anomaly -> MultiSourceAnomaly 4.2 min 18 min MultiSourceAnomaly -> Corroborated 22 min 4.1 hr Corroborated -> VerifiedIncident 38 min 8.2 hr VerifiedIncident -> Resolved 9.3 hr 6.2 days
The first transition is fast because it depends only on internal probe coverage — in countries with dense Voidly probe deployments, the second ASN is typically reporting within a single 5-minute measurement cycle. The 18-minute p95 reflects sparser countries where the second ASN probe may fire late.
The jump to 22 minutes median for external corroboration reflects the latency of external data sources. IODA is the fastest to respond (15-minute signal delay) and accounts for most of the cases below the 22-minute median. OONI data typically arrives at 30–60 minutes; CensoredPlanet data is only available from the previous day's batch export. The 4.1-hour p95 captures incidents where neither IODA nor OONI has data immediately and the system must wait for OONI's batch processing to catch up.
The Corroborated-to-Verified transition has a 38-minute median because many incidents have IODA plus at least one active-probe source confirming within an hour, and the combination immediately crosses the 0.80 corroboration score threshold. The 8.2-hour p95 reflects incidents in countries with sparse external measurement coverage where reaching two independent sources requires waiting for OONI's batch processing to complete.
The resolution distribution is heavy-tailed. The 9.3-hour median reflects that many blocks — particularly those imposed around political events — last less than a day. The 6.2-day p95 captures longer-running blocks like persistent social media restrictions in countries like Iran, Russia, and Ethiopia, where a block imposed once may persist for weeks.
Not all incidents reach VerifiedIncident. Across the full 2024 measurement stream:
- 67% of Anomaly-tier events never advance past Anomaly — they are noise, single-probe glitches, or infrastructure failures that do not accumulate multi-ASN agreement.
- 23% reach Corroborated status.
- 18% reach VerifiedIncident — the tier that appears in the public dataset and triggers journalist alerts.
The gap between 23% reaching Corroborated and 18% reaching VerifiedIncident reflects cases that reached Corroborated but failed to accumulate the 0.80 corroboration score required for Verified — typically incidents in countries with sparse OONI and CensoredPlanet coverage where IODA alone is the only external source.
The IncidentRecord struct
Every state transition updates the canonical incident record in the database. The full struct:
pub struct IncidentRecord {
pub incident_id: Uuid, // stable across all state transitions
pub country_code: String, // ISO 3166-1 alpha-2
pub domain: Option<String>, // None for BGP/shutdown incidents
pub interference_type: InterferenceType,
pub state: IncidentState,
pub started_at: DateTime<Utc>, // timestamp of first anomaly measurement
pub state_changed_at: DateTime<Utc>,
pub resolved_at: Option<DateTime<Utc>>,
pub measurement_count: u32,
pub affected_asn_count: u16,
pub corroboration_score: f32,
pub confidence: f32,
pub source_flags: SourceFlags, // bitflags: VOIDLY | OONI | CP | IODA
pub public_since: DateTime<Utc>, // when first published to the dataset
pub last_updated_at: DateTime<Utc>,
}Several fields deserve attention:
incident_id is a UUID that is assigned at incident creation and never changes regardless of how many times the state transitions. A researcher who downloaded the dataset when this incident was at Corroborated can join on this ID to find the same incident after it has been promoted to VerifiedIncident or marked Resolved.
domain is nullable because BGP-level and internet shutdown incidents do not target a specific domain. A BGP withdrawal covering a country's entire address space has domain = Noneand interference_type = BgpWithdrawal. The domain field is populated for DNS tampering, HTTP blocking, TLS interference, and throttling incidents.
source_flags is a bitfield encoding which sources have confirmed the incident: VOIDLY | OONI means Voidly and OONI have both confirmed but CensoredPlanet and IODA have not. This field changes as new confirmations arrive — an incident initially confirmed by VOIDLY and IODA gets its CP bit set the morning after when the retroactive CensoredPlanet pass runs.
public_since is distinct fromstarted_at. An incident that began at 14:00 UTC and reached Corroborated status at 14:22 UTC has started_at = 14:00 and public_since = 14:22. The started_at field reflects when the censorship actually began; public_since reflects when the evidence was sufficient to publish.
Publication timing by tier
Not all states result in publication to the HuggingFace dataset:
- Anomaly — not published. Internal-only. The 67% of Anomaly events that never advance are filtered out before any public export. Researchers who want access to the raw Anomaly stream must use the authenticated REST API, not the HuggingFace dataset.
- MultiSourceAnomaly — not published. Internal-only. This state is the in-progress corroboration buffer; it exists to structure the pipeline logic, not as a data product.
- Corroborated — published within 5 minutes of the state transition, with
confidence_tier = "CORROBORATED"in the dataset. This is when thepublic_sincefield is first populated on the incident record. The incident appears in the next HuggingFace dataset daily snapshot and is immediately queryable via the REST API. - VerifiedIncident — promoted in-place. The
incident_idis unchanged; theconfidence_tierfield updates from"CORROBORATED"to"VERIFIED_INCIDENT", and thelast_updated_attimestamp is bumped. This promotion triggers the journalist alert pipeline: Slack and PagerDuty notifications to the internal team, HMAC-signed webhook delivery to registered journalist subscribers within 60 seconds, an RSS feed update, and an SSE stream event of typeincident_verified. - Resolved — the
resolved_atfield is populated andis_activeis set tofalsein the dataset. The incident record remains in the dataset permanently — researchers can query resolved incidents to study the duration distribution, temporal patterns, and resolution correlates. - FalsePositive — the record is retained in the internal database with
state = FalsePositivefor audit and retrospective analysis, but it is removed from the public HuggingFace export. The CC BY 4.0 dataset contains only Corroborated and VerifiedIncident records. A researcher who downloaded last week's snapshot and included an incident that was subsequently classified as FalsePositive will have that record in their local copy; the delta file for the day of reclassification will contain aremovedentry for the incident ID.
Dataset fields that encode lifecycle state
The HuggingFace Parquet dataset exposes the incident lifecycle through a set of dedicated fields on each incident row. These are the fields to understand before writing any filter query against the dataset:
# Tier and active status confidence_tier # string: "CORROBORATED" | "VERIFIED_INCIDENT" is_active # bool: false if resolved or false_positive # Timestamps first_published_at # when the incident first appeared in the dataset last_updated_at # last modification: tier upgrade, resolution, source addition resolved_at # timestamp if resolved; null if still active # Evidence quality corroboration_score # float [0, 1] — independence-weighted source agreement ooni_confirmed # bool — OONI has confirmed this incident cp_confirmed # bool — CensoredPlanet has confirmed this incident ioda_confirmed # bool — IODA has confirmed this incident
The three per-source confirmation booleans (ooni_confirmed, cp_confirmed, ioda_confirmed) are independent of theconfidence_tier field. An incident at CORROBORATED tier might have ooni_confirmed = true and ioda_confirmed = false because OONI corroborated it but IODA did not detect a BGP-level signal (consistent with domain-specific blocking that does not affect BGP routing). As the retroactive CensoredPlanet pass runs each morning, thecp_confirmed field may flip to true for incidents that were published without CP confirmation, and last_updated_at will reflect that update.
Filter recipes for different consumers
Journalists: active verified incidents only.
# Python / pandas
df_verified = df[
(df['confidence_tier'] == 'VERIFIED_INCIDENT') &
(df['is_active'] == True)
]
# DuckDB
SELECT * FROM incidents
WHERE confidence_tier = 'VERIFIED_INCIDENT'
AND is_active = TRUE;ML training (positive examples): all corroborated and verified incidents, regardless of active status.
# Python / pandas
df_positive = df[
df['confidence_tier'].isin(['CORROBORATED', 'VERIFIED_INCIDENT'])
]
# DuckDB
SELECT * FROM incidents
WHERE confidence_tier IN ('CORROBORATED', 'VERIFIED_INCIDENT');Historical analysis (all verified incidents including resolved, for duration and timing studies):
# Includes resolved incidents — is_active = false with resolved_at populated
SELECT
country_code,
domain,
started_at,
resolved_at,
DATEDIFF('hour', started_at, resolved_at) AS duration_hours,
corroboration_score,
ooni_confirmed,
cp_confirmed,
ioda_confirmed
FROM incidents
WHERE confidence_tier = 'VERIFIED_INCIDENT'
AND resolved_at IS NOT NULL
ORDER BY started_at DESC;Retroactive state changes
Incidents can be upgraded or downgraded after initial publication. Common cases:
- Retroactive upgrade. An incident published at Corroborated tier gets promoted to VerifiedIncident the following morning when the CensoredPlanet daily dump arrives and provides the second independent source needed to reach the 0.80 corroboration score threshold.
- Retroactive FalsePositive classification.An incident published as VerifiedIncident is subsequently identified as a CDN maintenance event via the Akamai status page API, and is moved to FalsePositive and removed from the public dataset.
- Source flag enrichment. An incident at VerifiedIncident tier has its
cp_confirmedflag set totruewhen the morning retroactive pass processes the new CP dump, increasing thecorroboration_scoreand updatinglast_updated_at.
Every state change is written to the incident_history table with achanged_at timestamp and previous_state field. The full audit trail is preserved internally for all incidents including FalsePositives. Researchers who need to reproduce the state of a specific incident at a specific past moment can query incident_history via the authenticated API.
The HuggingFace dataset snapshots are immutable. Each daily commit to the dataset repository is an append-only git-lfs operation: the Parquet files from previous days are never modified, only new files are added. This means a researcher who downloaded last week's snapshot will see last week's state for all incidents that existed then, even if those incidents have since been promoted, resolved, or reclassified as false positives.
For consumers who need to track state changes, Voidly publishes a daily delta file alongside each snapshot commit:
# delta/2025-01-26.jsonl
# One JSON line per incident state change on that UTC day
{"incident_id": "...", "new_state": "VERIFIED_INCIDENT", "changed_at": "2025-01-26T14:22:11Z", "previous_state": "CORROBORATED"}
{"incident_id": "...", "new_state": "RESOLVED", "changed_at": "2025-01-26T22:47:03Z", "previous_state": "VERIFIED_INCIDENT", "resolved_at": "2025-01-26T22:47:03Z"}
{"incident_id": "...", "new_state": "FALSE_POSITIVE", "changed_at": "2025-01-26T09:13:44Z", "previous_state": "CORROBORATED", "removed_from_export": true}Consumers who maintain a local materialized copy of the dataset can apply these delta files to keep their local state current without re-downloading the full snapshot each day.
Connecting to the real-time pipeline
The transition to VerifiedIncident is the trigger for the real-time alert pipeline. When the corroboration engine's continuous re-evaluation determines that an incident has crossed the 0.80 corroboration threshold, the following sequence fires:
- Slack and PagerDuty alert to the internal Voidly team
- HMAC-signed webhook delivery to all registered journalist subscribers — within 60 seconds of the transition
- RSS feed update at
/feed/verified-incidents.xml - SSE stream event of type
incident_verifiedpushed to all connected SSE clients - Dataset record updated in the live REST API (available within seconds)
- HuggingFace dataset record updated in the next daily snapshot (within 24 hours)
Voidly targets under 8 minutes from the last confirming measurement to the SSE event delivery. It is important to understand what this claim does and does not cover. The 8 minutes is measured from the last confirming measurement — the one that pushed the corroboration score above 0.80 — not from the first anomaly measurement. The full pipeline latency from first probe anomaly to published VerifiedIncident event is longer, because it includes the time to accumulate multi-ASN agreement and the corroboration polling window:
probe_measurement t = 0 anomaly_detection t + 38s (classifier processing) clustering (incident creation) t + 38s (immediate, within same Worker execution) corroboration_polling t + 22min (median; IODA or OONI confirms) verification_threshold_crossed t + 60min (median; corroboration_score >= 0.80) SSE event fired t + 60min + <8min = ~68min median end-to-end
For BGP-level outages where IODA confirms within 15 minutes, the end-to-end time from first anomaly to VerifiedIncident SSE event can be under 20 minutes. For domain-specific blocks in countries with sparse external measurement coverage, it may be several hours. The 8-minute claim refers specifically to the processing and delivery step once the threshold is crossed, not to the time spent accumulating evidence.
This architecture is why the incident lifecycle exists as a formal state machine rather than a simple threshold filter. The process of accumulating evidence — waiting for OONI's batch processing, polling IODA's hourly signal, running the retroactive CensoredPlanet pass — takes time that cannot be compressed by making the system faster. The state machine makes the evidence-accumulation process explicit, records it durably, and gives researchers who use the dataset a precise vocabulary for understanding what each incident record actually represents.
From anomaly to verified incident: the Voidly confidence tier system →
Incident clustering and deduplication: how Voidly avoids counting the same censorship event twice →
Real-time corroboration engine: fetching OONI, CensoredPlanet, and IODA in parallel →
The Voidly open datasets on HuggingFace: structure, daily snapshots, and filter recipes →
Voidly's real-time event pipeline: from measurement anomaly to journalist alert →