We Tested Jev on 1,711 Gaokao Questions
A model that makes judgments instead of generating text answered 90.8% of 1,711 exam questions correctly. Seventeen essays offered a different test.
In this essay
On September 15, 2026, TypeSafe AI released Jev, describing it as the first System One Model. [1]
Three features stand out in TypeSafe’s introduction. [1][2]
First, Jev takes unstructured states as input and returns typed, probabilistic decisions.
unstructured state in, typed probabilistic decisions out
Unlike the large language models we are familiar with, Jev can process unstructured input such as text, but it does not generate text. It makes judgments: whether something is true, which option to choose, or which predefined rating to assign.
Second, Jev cannot return a result outside the specified type. TypeSafe therefore says it cannot hallucinate. [1]
Jev gives up string generation, it’s optimized for structured outputs and can’t hallucinate.
Here, “cannot hallucinate” does not mean its judgments are infallible. It means that the question and the possible answers are defined in advance, and its result must conform to those types.
Third, Jev is fast.
40x-200x faster
TypeSafe says Jev can achieve intelligence comparable to existing large language models on System One tasks. It reports end-to-end latency of 70–500 milliseconds, potentially 40–200 times faster on comparable tasks. Jev uses a new model architecture and parallel sampler: where a language model generates tokens sequentially, Jev produces probabilities and confidence for its judgments in parallel. [1]
That prompted a technical hypothesis: might Jev achieve this efficiency by trading a smaller model and a narrower range of capabilities for speed?
What we really wanted to know was how broad its knowledge is, and whether that breadth could support fast, accurate judgments across different fields.
To investigate, we needed questions spanning a wide range of knowledge, with reasonably clear answers. China’s university entrance examination, the gaokao, was the first thing that came to mind.
01 Of 1,711 objective questions, Jev answered 1,553 correctly
The questions came from two public datasets, GAOKAO-Bench and GAOKAO-Bench-Updates. They cover Chinese, mathematics, English, physics, chemistry, biology, history, and politics exams from 2010 to 2024. [3][4]
We assembled 1,711 single-answer multiple-choice questions and tested Jev zero-shot. For each question, we supplied the subject, question text, and four options, asking it to select A, B, C, or D directly.
The result exceeded our expectations: Jev answered 1,553 questions correctly, for an overall accuracy of 90.8%.
Biology and politics both approached 98%. English, history, and Chinese exceeded 93%. Physics and mathematics were comparatively weaker.
Jev also returns a probability for each option. We selected the option with the highest probability for every question, then grouped the questions by that probability. The higher the maximum probability, the higher the observed accuracy.
Among the 1,294 questions whose highest option probability was at least 80%, actual accuracy reached 98.4%. The probabilities corresponded closely to observed accuracy and offered information about the reliability of different judgments.
02 After the multiple-choice questions, we gave Jev 17 essays
The objective questions showed that Jev has substantial general knowledge even though it does not generate text. We wanted to take the question further: how would it handle tasks without a single correct answer, where an overall evaluation is required?
Essay assessment already uses score bands. If we first give the model explicit criteria and then ask it to choose among ordered levels, the task still takes the form of the rating or selection problems Jev is designed for.
We collected 17 essays and supplied Jev with the writing prompt, band criteria, and essay text, asking it to assign each essay to a broad band. The collection included four sample essays used to train Shanghai gaokao markers and 13 essays from Haidian District’s second mock examination for final-year high-school students in 2025. Both sets came with publicly available scores and bands. [5][6]
Jev agreed with the published bands on 11 essays and disagreed on six.
We reread all six disputed essays and made our own judgments. In two cases, we preferred Jev’s band; in the other four, we preferred the published one. By our own review, Jev judged 13 of the 17 essays correctly.
Disagreement with an existing band does not always mean the model is wrong. Sometimes the disagreement is precisely the reason to read an essay again.
03 What, then, can Jev do?
This new kind of model also suggested possible applications. In examinations, it could provide an independent judgment alongside an existing assessment.
Exams are an easy example to understand. Many other jobs also begin with explicit criteria and then require someone to select, rate, or review complex information. Jev may prove useful there as well.
Sources:
[1] TypeSafe AI: Introducing System One Models & Jev, September 15, 2026.
[2] Cloudflare AI Docs: Jev (typesafe).
[4] OpenLMLab: GAOKAO-Bench-Updates.
[5] East China Normal University: How gaokao essay rating criteria are developed (Chinese).
[6] Beijing Gaokao Online: Essays from Haidian’s second mock examination, 2025 (Chinese).
AI contribution
The author formulated the test questions, assembled the datasets, ran the evaluations, reviewed the disputed essays, and made the final judgments. ChatGPT in the browser and Codex helped organize sources, discuss structure, revise the prose, and produce the charts. The author was responsible for fact-checking, selecting the arguments, and finalizing the Chinese article. Codex translated it into English.
