صفرthe sifr prize

Honorable mention · The Sifr Prize 2026

The Claim Extraction Problem

What Should Exist for Arabic Crisis Verification

Published
On this page

I am writing from Beirut in May 2026. Since hostilities escalated on March 2, more than a million people have been driven from their homes. On April 8, Human Rights Watch reported that more than 100 Israeli strikes across Lebanon, including Beirut, killed over 300 people and damaged the last main bridge linking the south to the rest of the country. These are the conditions in which people must decide, in minutes, what to trust. In one such moment, students around me received an unverified warning that our university might be targeted, and the decision was no longer abstract: should we go to campus, stay home, or trust a message no one could trace back to any source?

In Lebanon, crisis information rarely arrives through one official, reliable channel. It arrives as a WhatsApp message forwarded by a relative, a Telegram post copied into a family group, a screenshot of an alleged evacuation order, or a voice note with no traceable origin. People need answers to practical, immediate questions: Is this road open? Is this evacuation notice real? Is this hospital still receiving patients? Is this video from tonight or from a different war entirely?

The danger is not hypothetical. After the Ain Saade strike that killed Lebanese politician Pierre Moawad, a media-monitoring report identified a fabricated voice note that circulated with his photograph, implying he had warned the apartment owner. It spread through public posts and private WhatsApp groups before correction could catch up. No deployed Arabic crisis-verification system was available to flag it in real time.

The Arab world has fact-checkers, journalists, civil society monitors, and increasingly capable Arabic language models. What it lacks is the technical layer that connects them to real crisis communication: a system that can extract and verify factual claims in real time from dialectal Arabic text under crisis constraints. This is not only a media or policy problem. It is a scientific and engineering problem with a concrete technical core.

The Problem, Precisely

Arabic NLP has made genuine progress. AraBERT delivers strong performance on Arabic language understanding tasks; MARBERT was trained on large-scale Arabic social media text; and CAMeL Tools provides preprocessing, dialect identification, morphological analysis, and named entity recognition for Arabic. On standard benchmarks, these tools perform well.

Crisis verification is categorically different from standard Arabic text classification. A model can perform well on a curated dataset and still fail on a forwarded voice note during an airstrike. The reasons are specific and technical.

Failure 1: The dialect problem is not what most papers assume it is. Arabic NLP often treats dialect as a classification category: a label attached before routing text to the right model. In a crisis, that framing collapses. The messages that circulate during an emergency are not cleanly dialectal or cleanly formal. They are mixed: a South Lebanese speaker writing in haste on a phone, switching into English or French for a street name, abbreviating because time is short, and using spellings shaped by region, keyboard, and habit. The same road-closure claim can appear under several spellings within a single Telegram channel. None may match the tokenization or normalization assumptions of a model trained on cleaner text. The problem is not simply dialect identification. It is robustness to informal writing under pressure, and it has not been properly benchmarked because the right data does not exist yet.

Failure 2: Claim extraction from crisis text is still underdeveloped. Before a system can verify anything, it must identify what needs to be verified. This is claim detection: distinguishing checkable factual assertions from opinion, instruction, rumor, and emotional expression inside the same short message. In English, claim detection for fact-checking is a more mature task. In dialectal Arabic crisis text, it remains far less developed. The challenge is structural. Lebanese and Levantine Arabic use implied subjects, compressed syntax, local references, and emotionally inflected phrasing that may still carry factual content. A sentence like “They hit the corner building, do not go out” contains at least one checkable event claim and possibly a location claim, but it is expressed in a form that ordinary fake-news classifiers are not designed to parse. The first step should not be a true/false classifier. It should be a claim-extraction system that turns messy messages into structured objects: claim type, location, time, evidence needed, and confidence — something a journalist, NGO monitor, or civil society verifier can act on.

Failure 3: The field has optimized for the wrong environment. Most Arabic NLP benchmarks assume stable internet, cloud computing, and clean text. Crisis conditions often weaken all three assumptions at once. On April 8 alone, over 100 Israeli strikes hit Lebanon, including Beirut, and damaged the last main bridge linking the south to the rest of the country. During the moments when verification matters most, users may have only a mid-range phone, a limited battery, unstable connectivity, and delayed access to trusted sources. Large transformer models that perform best in controlled settings are not always deployable under those conditions. Cloud-only inference is incomplete when the user is in a shelter, a newsroom with unstable electricity, or a field office with intermittent connectivity. The missing research question is simple and measurable: what level of Arabic crisis-NLP accuracy is achievable on a mid-range phone, under degraded bandwidth, with a compressed model? The core contribution needed is not another fake-news classifier. It is a benchmark and system architecture for dialectal Arabic crisis NLP under field constraints: noisy input, time-sensitive claims, degraded connectivity, mobile inference, and explicit uncertainty.

What the R&D Would Take

The work can be decomposed into three phases, each producing a standalone contribution and enabling the next.

Phase 1: A crisis-domain dialectal claim corpus. The foundational bottleneck is data. No existing Arabic dataset captures what crisis communication looks like: informal spelling variation, panic-register vocabulary, compressed syntax, local place names, code-switching, and content derived from audio, such as voice notes transcribed and forwarded as text. A first corpus should draw on publicly accessible Arabic-language posts from Lebanon and comparable regional crises, annotated at the sentence or message level for dialect and register, claim presence and type, temporal grounding, geographic grounding, and veracity where external evidence permits. Claim types should begin narrowly: road access, evacuation notices, hospital reachability, school or university closure, infrastructure status, and media authenticity.

The annotation challenge is itself a research problem. Informal Levantine Arabic has no single spelling standard or universal tokenization strategy, and annotators from different sub-regions will disagree on dialect and claim boundaries unless guidelines are carefully designed. The goal is not to flatten dialectal Arabic into Modern Standard Arabic, which may destroy the signal the model needs to learn. The goal is to preserve the messiness while making it learnable. Data collection must follow clear ethics rules: public posts under documented governance, no private messages without informed consent, and removal of personal identifiers before annotation.

Phase 2: A lightweight, dialect-robust claim extraction model. The modeling goal is not to build the largest Arabic model. It is to test a specific empirical hypothesis: can a smaller model, fine-tuned on a carefully constructed crisis-domain corpus, outperform larger general-purpose Arabic models on crisis claim extraction? A realistic experimental design would compare MARBERT and AraBERT baselines against a fine-tuned BERT-base-scale model, with distilled and quantized mobile versions tested in parallel. The primary result would not be only a model. It would answer the scale-versus-specificity tradeoff: when does domain-specific data compensate for model size, and when does compression become unsafe?

The model should produce structured claim representations, not binary labels. A message claiming a road closure should yield claim type, location confidence, time reference, missing information, evidence requirement, and uncertainty. That output is more useful than a true/false label because crisis information is often incomplete rather than simply false.

Phase 3: A field-realistic evaluation benchmark. Arabic crisis NLP needs a benchmark that evaluates deployment conditions rather than ideal ones. This phase would measure accuracy, inference latency, memory footprint, battery cost, and bandwidth requirements together, across different model sizes and compression levels, on dialectal crisis content. The key output would be an accuracy-latency-bandwidth curve: a performance frontier showing how much accuracy is lost as a model is compressed toward field deployability, and where the curve bends enough to make further compression unsafe. This benchmark would be valuable even if future teams use different architectures, because it would define what crisis-readiness means for Arabic NLP.

How a Small Team Would Start

A team of three could begin immediately: an Arabic NLP researcher with dialect expertise, a software engineer, and a domain expert from journalism, crisis response, or digital rights. In the first three months, they would define the annotation schema, ethics protocol, claim taxonomy, and pilot dataset. By month six, they would annotate 15,000 to 20,000 crisis-domain sentences. By month twelve, they would train and compare claim-extraction models. By month eighteen, they would publish the field-realistic benchmark and build a working verifier prototype.

The prototype should not begin as a mass-use consumer app. It should first serve trained verifiers: journalists, NGO monitors, civil society organizations, and emergency information teams. A user would input a message and receive the extracted claim, missing information, possible evidence sources, confidence level, and a warning if the claim is under-specified or outdated.

Lebanon is the right pilot case because the conditions are demanding: dialect variation, code-switching, fragmented information channels, infrastructure stress, and a population that knows the cost of trusting the wrong message at the wrong moment. A system that works under these conditions could later be adapted to Palestine, Sudan, Syria, Yemen, and other Arab contexts where civilians rely on fragmented information under pressure.

Why This Has Not Been Done

The honest answer is misalignment between where knowledge lives and where research gets built. Many Arabic NLP systems are developed far from the environments they are meant to serve. The people who know what a Lebanese family WhatsApp group sounds like during an airstrike, who recognize a genuine warning versus a fabricated one, and who understand how claims move between public posts and private groups are rarely the people setting benchmark tasks.

The result is a body of work that can score well on available data while failing silently on the data that matters. Organizations such as SMEX and the Samir Kassir Foundation have documented fabricated evacuation threats, misleading crisis narratives, and wartime information disorder. Their public advice is often: verify before you share. But advice is not infrastructure.

What Should Exist

What should exist is an Arabic crisis-verification layer that understands dialectal language, extracts checkable claims, grounds them in time and place, retrieves evidence, and communicates uncertainty honestly.

The Arab world does not need another generic AI slogan. It needs the corpus, benchmark, and empirical work required to make crisis verification possible in the region’s language, infrastructure, and uncertainty.

The cost of not building it is not abstract. It is a fabricated voice note forwarded faster than the correction, a road-closure claim no one can verify in time, and an evacuation notice that people cannot tell is real or fake. What should exist is not an AI system that claims to know the truth. It is an Arabic crisis-verification system honest enough to say what it knows, what it does not know, and what evidence is missing — fast enough to matter.