Clash of the Queries dataset

The document collection and question pool for the head-to-head between our two question-answering platforms. Both teams get exactly these files. This is version 1, and it is now frozen.

Downloads

clash-dataset-v1-part-01.zip Documents, first half 7058fde6436fac83f7ef3fda6900bb33ebf58684c1d300f5dc175fdf8cea9400 3.8 GB Download
clash-dataset-v1-part-02.zip Documents, second half ca0a87a8775c0c7faee08fcc84b63c26dcd9f15ce133451aeb73563e45864552 1.6 GB Download
clash-dataset-v1-meta.zip Manifest, questions, guide and build tools f6d4b97ae3aec360fa95c1f778c14ffbd64a9e1528dc04e2ed9dca82fa6d53b6 9 MB Download

Check your downloads against SHA256SUMS. The links work until 16 December 2026.

What is in it

Documents
39,641 files: reports, laws, statistics tables, invoices, slide decks, scanned pages and photos
Languages
About 70% English and 30% Norwegian text, including Norwegian questions about English documents
Formats
PDF (born-digital and scanned without a text layer), TIFF, PNG and JPG, Excel, Word, PowerPoint, HTML and Markdown
Size
5.4 GB to download, 9.4 GB unpacked, about 79,000 pages and 166 million tokens of text
Questions
57,363, each with its answer and the exact page, cell or passage it comes from
Sources
Public only: Stortinget, Lovdata, Statistics Norway, regjeringen.no, SEC filings, synthetic business documents, web PDFs and established benchmark sets

Getting started

  1. Put the three zip files and SHA256SUMS in one folder and check them:
    shasum -a 256 -c SHA256SUMS
    On Linux, use sha256sum -c SHA256SUMS.
  2. Unzip all three into the same place. They share one root folder, clash-dataset-v1:
    for z in clash-dataset-v1-*.zip; do unzip -q "$z"; done
  3. Fetch the 9,985 files we may not redistribute ourselves, DocVQA scans and Docmatix web PDFs. The script downloads them at fixed versions and checks every file, so you end up with exactly what we have. It needs uv, downloads about 6 GB and takes around ten minutes:
    cd clash-dataset-v1/tools && uv sync && cd ..
    uv run --project tools python fetch_recipes.py
  4. Read HOW_TO_USE.md, which is also in the folder. It explains the manifest, the question format and how the day works.

How the questions are used

Both teams get the whole question pool to test against. On the day, most questions come from a pack that a neutral person draws from the pool with a seed of their own choosing. Neither team sees the pack in advance, and the draw can be repeated afterwards to show it was not hand-picked. Board members may also ask their own questions.

The collection was built from public sources by the open script in tools/, so every step can be checked. If something needs fixing, version 1.1 goes to both teams at the same time.