The corpus.

Cito does not crawl the web or generate any records itself. The corpus is built from two open scholarly datasets, mirrored locally and kept in sync with their upstream releases.

Sources

• Semantic Scholar (Allen Institute for AI) provides the backbone: paper metadata, abstracts, citation counts and the SPECTER2 embeddings behind semantic search. Result IDs are Semantic Scholar CorpusIDs.
• OpenAlex fills in what open access adds after publication: direct PDF links for papers with a free, legal copy, plus abstracts for papers where Semantic Scholar has none. Records are joined by DOI.

Coverage

Honest limits: abstract coverage is partial (some publishers do not license abstracts for redistribution), citation counts lag the very newest papers, DOIs are missing on much older and informally published work, and Cito has no full text. The pdf_url field links to the copy at its origin; nothing is proxied or re-hosted.

Publication years

Fields of study

Semantic Scholar's coarse subject labels. Multi-label: a paper spanning two disciplines counts under both, so shares add up to more than 100%.

Publication types

Document types from Semantic Scholar. Multi-label, so a paper can carry more than one tag.

Freshness and update schedule

The corpus is currently serving Semantic Scholar release 2026-09-29, applied on 3 October 2026. Every search response echoes both, as corpus_release and corpus_synced_at, so a client can judge a time-sensitive query without a second call; GET /status returns the same pair on its own. Nothing in the query path calls upstream, so no paper published after that timestamp can appear in results at all.

• Within a day of each Semantic Scholar release: new, updated and withdrawn papers are merged into all indexes. Semantic Scholar publishes bulk releases roughly weekly, and a nightly job applies each one as it appears, so in practice the corpus tracks their release cadence. Freshly added papers are enriched with OpenAlex open-access data in the same pass.
• Quarterly: recent papers that still lack a PDF link get re-checked against OpenAlex. Open-access records improve after the fact (a repository copy often appears months after publication), so a paper that had no free PDF at ingest may gain one later.

The remaining lag is upstream, not here: Semantic Scholar's bulk releases trail their own live API by up to a week, so expect a paper to be findable in Cito roughly one to two weeks after it lands in Semantic Scholar. Closing that gap would mean calling the upstream API per query, which is the rate limit Cito exists to escape. If you need same-day coverage of brand-new preprints, query the preprint server directly; Cito optimizes for breadth, ranking quality and throughput rather than same-day recall.

Reproducibility

Search ordering is deterministic within a snapshot, and the published_before parameter gives a leak-safe date cutoff (papers whose date cannot prove precedence are excluded, not guessed). Together with corpus_release, this makes runs replayable: useful for benchmarks, agent evaluations and citation audits.