Local models against verified Jev cases

INTERIM · Jev 1.13.0 is a historical published API reference.
231 public items match the publisher's file hashes, questions, option order and per-case outcome IDs. Local partial tiers are excluded from the tables. This is not the official JevBench ranking, which also includes sealed tasks.

We verified source integrity and input equality. We have not independently adjudicated every answer key. Public benchmark exposure remains possible. Accuracy differences and their paired intervals are descriptive; many model comparisons increase the chance of a misleading apparent winner.

Easy · 48 cases

Local model	Local accuracy	Published Jev	Local minus Jev	Local-only / Jev-only correct	Invalid local outputs
CLM 8B INT8	33/48 · 68.8%	48/48 · 100.0%	-31.2 pp95% interval -43.8 to -18.8	0 / 15	0
Cygnet 12B NF4 (local variant)	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Decider 0.8B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Decider 2B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Decider 4B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Decision 4B v1.2	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Decision 2.0 Eos 0.8B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Decision 2.0 Kai 0.6B	47/48 · 97.9%	48/48 · 100.0%	-2.1 pp95% interval -6.2 to +0.0	0 / 1	0
Decision 2.0 Nox 4B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Decision 2.0 Sol 2B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
GLiNER Decide 1B	47/48 · 97.9%	48/48 · 100.0%	-2.1 pp95% interval -6.2 to +0.0	0 / 1	0
GLiNER Decide multilingual	47/48 · 97.9%	48/48 · 100.0%	-2.1 pp95% interval -6.2 to +0.0	0 / 1	0
GLiNER Decide 340M	47/48 · 97.9%	48/48 · 100.0%	-2.1 pp95% interval -6.2 to +0.0	0 / 1	0
Imajev 2B	47/48 · 97.9%	48/48 · 100.0%	-2.1 pp95% interval -6.2 to +0.0	0 / 1	1
Imajev 4B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Intern-Decision 4B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Jev-Omni 12B NF4 (local variant)	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
JevK5 2B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
JevK5 4B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Julia 144M	35/48 · 72.9%	48/48 · 100.0%	-27.1 pp95% interval -39.6 to -14.6	0 / 13	0
Kev 0.8B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Kev 4B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Laya multilingual	43/48 · 89.6%	48/48 · 100.0%	-10.4 pp95% interval -18.8 to -2.1	0 / 5	0
Laya	46/48 · 95.8%	48/48 · 100.0%	-4.2 pp95% interval -10.4 to +0.0	0 / 2	0
Nev 2B Q8	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Nimble 9B NF4	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Plumb 4B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Tev1 4B	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Von	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Wald 4B v1.2	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0
Winnow 12B Q8 (Windows CUDA)	48/48 · 100.0%	48/48 · 100.0%	+0.0 pp95% interval +0.0 to +0.0	0 / 0	0

Original · 72 cases

Local model	Local accuracy	Published Jev	Local minus Jev	Local-only / Jev-only correct	Invalid local outputs
CLM 8B INT8	25/72 · 34.7%	71/72 · 98.6%	-63.9 pp95% interval -77.8 to -50.0	1 / 47	0
Cygnet 12B NF4 (local variant)	65/72 · 90.3%	71/72 · 98.6%	-8.3 pp95% interval -16.7 to -1.4	0 / 6	0
Decider 0.8B	61/72 · 84.7%	71/72 · 98.6%	-13.9 pp95% interval -25.0 to -5.6	1 / 11	0
Decider 2B	64/72 · 88.9%	71/72 · 98.6%	-9.7 pp95% interval -19.4 to -1.4	1 / 8	0
Decider 4B	71/72 · 98.6%	71/72 · 98.6%	+0.0 pp95% interval -4.2 to +4.2	1 / 1	0
Decision 4B v1.2	69/72 · 95.8%	71/72 · 98.6%	-2.8 pp95% interval -9.7 to +2.8	1 / 3	0
Decision 2.0 Eos 0.8B	64/72 · 88.9%	71/72 · 98.6%	-9.7 pp95% interval -18.1 to -2.7	1 / 8	0
Decision 2.0 Kai 0.6B	61/72 · 84.7%	71/72 · 98.6%	-13.9 pp95% interval -23.6 to -5.6	1 / 11	0
Decision 2.0 Nox 4B	68/72 · 94.4%	71/72 · 98.6%	-4.2 pp95% interval -13.9 to +2.8	1 / 4	0
Decision 2.0 Sol 2B	67/72 · 93.1%	71/72 · 98.6%	-5.6 pp95% interval -13.9 to +1.4	1 / 5	0
GLiNER Decide 1B	35/72 · 48.6%	71/72 · 98.6%	-50.0 pp95% interval -63.9 to -34.7	0 / 36	0
GLiNER Decide multilingual	42/72 · 58.3%	71/72 · 98.6%	-40.3 pp95% interval -56.9 to -25.0	1 / 30	0
GLiNER Decide 340M	48/72 · 66.7%	71/72 · 98.6%	-31.9 pp95% interval -44.4 to -19.4	1 / 24	0
Imajev 2B	65/72 · 90.3%	71/72 · 98.6%	-8.3 pp95% interval -16.7 to -2.8	0 / 6	1
Imajev 4B	70/72 · 97.2%	71/72 · 98.6%	-1.4 pp95% interval -4.2 to +0.0	0 / 1	1
Intern-Decision 4B	71/72 · 98.6%	71/72 · 98.6%	+0.0 pp95% interval -4.2 to +4.2	1 / 1	0
Jev-Omni 12B NF4 (local variant)	69/72 · 95.8%	71/72 · 98.6%	-2.8 pp95% interval -6.9 to +0.0	0 / 2	0
JevK5 2B	57/72 · 79.2%	71/72 · 98.6%	-19.4 pp95% interval -30.6 to -8.3	0 / 14	0
JevK5 4B	68/72 · 94.4%	71/72 · 98.6%	-4.2 pp95% interval -12.5 to +2.8	1 / 4	0
Julia 144M	34/72 · 47.2%	71/72 · 98.6%	-51.4 pp95% interval -66.7 to -36.1	1 / 38	0
Kev 0.8B	58/72 · 80.6%	71/72 · 98.6%	-18.1 pp95% interval -30.6 to -6.9	1 / 14	0
Kev 4B	67/72 · 93.1%	71/72 · 98.6%	-5.6 pp95% interval -15.3 to +1.4	1 / 5	0
Laya multilingual	29/72 · 40.3%	71/72 · 98.6%	-58.3 pp95% interval -70.8 to -45.8	0 / 42	0
Laya	52/72 · 72.2%	71/72 · 98.6%	-26.4 pp95% interval -40.3 to -13.9	1 / 20	0
Nev 2B Q8	63/72 · 87.5%	71/72 · 98.6%	-11.1 pp95% interval -19.4 to -4.2	1 / 9	0
Nimble 9B NF4	68/72 · 94.4%	71/72 · 98.6%	-4.2 pp95% interval -13.9 to +2.8	1 / 4	0
Plumb 4B	70/72 · 97.2%	71/72 · 98.6%	-1.4 pp95% interval -6.9 to +2.8	1 / 2	0
Tev1 4B	69/72 · 95.8%	71/72 · 98.6%	-2.8 pp95% interval -9.7 to +2.8	1 / 3	0
Von	46/72 · 63.9%	71/72 · 98.6%	-34.7 pp95% interval -50.0 to -20.8	1 / 26	0
Wald 4B v1.2	72/72 · 100.0%	71/72 · 98.6%	+1.4 pp95% interval +0.0 to +4.2	1 / 0	0
Winnow 12B Q8 (Windows CUDA)	69/72 · 95.8%	71/72 · 98.6%	-2.8 pp95% interval -8.3 to +0.0	0 / 2	0

Hard · 111 cases

Local model	Local accuracy	Published Jev	Local minus Jev	Local-only / Jev-only correct	Invalid local outputs
CLM 8B INT8	38/111 · 34.2%	81/111 · 73.0%	-38.7 pp95% interval -51.4 to -25.2	12 / 55	0
Cygnet 12B NF4 (local variant)	82/111 · 73.9%	81/111 · 73.0%	+0.9 pp95% interval -8.1 to +9.9	13 / 12	0
Decider 0.8B	43/111 · 38.7%	81/111 · 73.0%	-34.2 pp95% interval -44.1 to -23.4	6 / 44	0
Decider 2B	63/111 · 56.8%	81/111 · 73.0%	-16.2 pp95% interval -26.1 to -6.3	9 / 27	0
Decider 4B	72/111 · 64.9%	81/111 · 73.0%	-8.1 pp95% interval -17.1 to +0.0	7 / 16	0
Decision 4B v1.2	86/111 · 77.5%	81/111 · 73.0%	+4.5 pp95% interval -3.6 to +12.6	12 / 7	0
Decision 2.0 Eos 0.8B	51/111 · 45.9%	81/111 · 73.0%	-27.0 pp95% interval -37.8 to -16.2	6 / 36	0
Decision 2.0 Kai 0.6B	44/111 · 39.6%	81/111 · 73.0%	-33.3 pp95% interval -43.2 to -23.4	4 / 41	0
Decision 2.0 Nox 4B	69/111 · 62.2%	81/111 · 73.0%	-10.8 pp95% interval -18.9 to -3.6	3 / 15	0
Decision 2.0 Sol 2B	51/111 · 45.9%	81/111 · 73.0%	-27.0 pp95% interval -36.9 to -17.1	4 / 34	0
GLiNER Decide 1B	39/111 · 35.1%	81/111 · 73.0%	-37.8 pp95% interval -47.7 to -27.9	2 / 44	0
GLiNER Decide multilingual	42/111 · 37.8%	81/111 · 73.0%	-35.1 pp95% interval -45.9 to -23.4	9 / 48	0
GLiNER Decide 340M	36/111 · 32.4%	81/111 · 73.0%	-40.5 pp95% interval -50.5 to -30.6	3 / 48	0
Imajev 2B	62/111 · 55.9%	81/111 · 73.0%	-17.1 pp95% interval -27.0 to -7.2	7 / 26	2
Imajev 4B	80/111 · 72.1%	81/111 · 73.0%	-0.9 pp95% interval -7.2 to +5.4	6 / 7	0
Intern-Decision 4B	82/111 · 73.9%	81/111 · 73.0%	+0.9 pp95% interval -8.1 to +9.9	12 / 11	0
Jev-Omni 12B NF4 (local variant)	81/111 · 73.0%	81/111 · 73.0%	+0.0 pp95% interval -8.1 to +8.1	10 / 10	0
JevK5 2B	68/111 · 61.3%	81/111 · 73.0%	-11.7 pp95% interval -21.6 to -1.8	9 / 22	0
JevK5 4B	86/111 · 77.5%	81/111 · 73.0%	+4.5 pp95% interval -2.7 to +11.7	12 / 7	0
Julia 144M	40/111 · 36.0%	81/111 · 73.0%	-36.9 pp95% interval -48.6 to -24.3	10 / 51	0
Kev 0.8B	41/111 · 36.9%	81/111 · 73.0%	-36.0 pp95% interval -46.0 to -25.2	5 / 45	0
Kev 4B	60/111 · 54.1%	81/111 · 73.0%	-18.9 pp95% interval -27.0 to -10.8	3 / 24	0
Laya multilingual	37/111 · 33.3%	81/111 · 73.0%	-39.6 pp95% interval -50.5 to -27.9	10 / 54	0
Laya	37/111 · 33.3%	81/111 · 73.0%	-39.6 pp95% interval -52.3 to -27.0	10 / 54	0
Nev 2B Q8	75/111 · 67.6%	81/111 · 73.0%	-5.4 pp95% interval -14.4 to +3.6	10 / 16	0
Nimble 9B NF4	65/111 · 58.6%	81/111 · 73.0%	-14.4 pp95% interval -22.5 to -6.3	4 / 20	0
Plumb 4B	89/111 · 80.2%	81/111 · 73.0%	+7.2 pp95% interval +0.9 to +14.4	12 / 4	0
Tev1 4B	57/111 · 51.4%	81/111 · 73.0%	-21.6 pp95% interval -29.7 to -13.5	3 / 27	0
Von	50/111 · 45.0%	81/111 · 73.0%	-27.9 pp95% interval -38.7 to -16.2	8 / 39	0
Wald 4B v1.2	77/111 · 69.4%	81/111 · 73.0%	-3.6 pp95% interval -12.6 to +5.4	11 / 15	10
Winnow 12B Q8 (Windows CUDA)	82/111 · 73.9%	81/111 · 73.0%	+0.9 pp95% interval -6.3 to +9.0	10 / 9	0

All public cases · 231

Local model	Local accuracy	Published Jev	Local minus Jev	Local-only / Jev-only correct	Invalid local outputs
CLM 8B INT8	96/231 · 41.6%	200/231 · 86.6%	-45.0 pp95% interval -53.2 to -37.2	13 / 117	0
Cygnet 12B NF4 (local variant)	195/231 · 84.4%	200/231 · 86.6%	-2.2 pp95% interval -6.9 to +2.2	13 / 18	0
Decider 0.8B	152/231 · 65.8%	200/231 · 86.6%	-20.8 pp95% interval -26.8 to -14.7	7 / 55	0
Decider 2B	175/231 · 75.8%	200/231 · 86.6%	-10.8 pp95% interval -16.5 to -5.2	10 / 35	0
Decider 4B	191/231 · 82.7%	200/231 · 86.6%	-3.9 pp95% interval -8.2 to +0.4	8 / 17	0
Decision 4B v1.2	203/231 · 87.9%	200/231 · 86.6%	+1.3 pp95% interval -3.0 to +5.6	13 / 10	0
Decision 2.0 Eos 0.8B	163/231 · 70.6%	200/231 · 86.6%	-16.0 pp95% interval -21.6 to -10.4	7 / 44	0
Decision 2.0 Kai 0.6B	152/231 · 65.8%	200/231 · 86.6%	-20.8 pp95% interval -26.4 to -15.2	5 / 53	0
Decision 2.0 Nox 4B	185/231 · 80.1%	200/231 · 86.6%	-6.5 pp95% interval -10.8 to -2.6	4 / 19	0
Decision 2.0 Sol 2B	166/231 · 71.9%	200/231 · 86.6%	-14.7 pp95% interval -19.9 to -9.5	5 / 39	0
GLiNER Decide 1B	121/231 · 52.4%	200/231 · 86.6%	-34.2 pp95% interval -40.7 to -27.7	2 / 81	0
GLiNER Decide multilingual	131/231 · 56.7%	200/231 · 86.6%	-29.9 pp95% interval -37.2 to -22.1	10 / 79	0
GLiNER Decide 340M	131/231 · 56.7%	200/231 · 86.6%	-29.9 pp95% interval -36.4 to -23.4	4 / 73	0
Imajev 2B	174/231 · 75.3%	200/231 · 86.6%	-11.3 pp95% interval -16.9 to -6.1	7 / 33	4
Imajev 4B	198/231 · 85.7%	200/231 · 86.6%	-0.9 pp95% interval -3.9 to +2.2	6 / 8	1
Intern-Decision 4B	201/231 · 87.0%	200/231 · 86.6%	+0.4 pp95% interval -3.5 to +4.8	13 / 12	0
Jev-Omni 12B NF4 (local variant)	198/231 · 85.7%	200/231 · 86.6%	-0.9 pp95% interval -4.8 to +3.0	10 / 12	0
JevK5 2B	173/231 · 74.9%	200/231 · 86.6%	-11.7 pp95% interval -17.7 to -6.1	9 / 36	0
JevK5 4B	202/231 · 87.4%	200/231 · 86.6%	+0.9 pp95% interval -3.5 to +4.8	13 / 11	0
Julia 144M	109/231 · 47.2%	200/231 · 86.6%	-39.4 pp95% interval -46.8 to -31.6	11 / 102	0
Kev 0.8B	147/231 · 63.6%	200/231 · 86.6%	-22.9 pp95% interval -29.0 to -16.9	6 / 59	0
Kev 4B	175/231 · 75.8%	200/231 · 86.6%	-10.8 pp95% interval -16.0 to -6.1	4 / 29	0
Laya multilingual	109/231 · 47.2%	200/231 · 86.6%	-39.4 pp95% interval -45.9 to -32.0	10 / 101	0
Laya	135/231 · 58.4%	200/231 · 86.6%	-28.1 pp95% interval -35.1 to -20.8	11 / 76	0
Nev 2B Q8	186/231 · 80.5%	200/231 · 86.6%	-6.1 pp95% interval -10.8 to -1.3	11 / 25	0
Nimble 9B NF4	181/231 · 78.4%	200/231 · 86.6%	-8.2 pp95% interval -13.4 to -3.5	5 / 24	0
Plumb 4B	207/231 · 89.6%	200/231 · 86.6%	+3.0 pp95% interval -0.4 to +6.9	13 / 6	0
Tev1 4B	174/231 · 75.3%	200/231 · 86.6%	-11.3 pp95% interval -16.5 to -6.9	4 / 30	0
Von	144/231 · 62.3%	200/231 · 86.6%	-24.2 pp95% interval -31.2 to -16.9	9 / 65	0
Wald 4B v1.2	197/231 · 85.3%	200/231 · 86.6%	-1.3 pp95% interval -5.6 to +3.0	12 / 15	10
Winnow 12B Q8 (Windows CUDA)	199/231 · 86.1%	200/231 · 86.6%	-0.4 pp95% interval -4.8 to +3.5	10 / 11	0

Exact 300-case pilot

100 news, 100 emotion and 100 banking-intent cases. Identical published manifest, original instructions, opaque label IDs, full option descriptions and order. The banking slice has 72 options. 0 local model runs remain incomplete. Natural-label results from the main comparison are a separate prompt diagnostic.

Local model	Task	Local accuracy	Published Jev	Difference	Valid requests
CLM 8B INT8	agnews	47.0%	91.0%	-44.0 pp	100/100
CLM 8B INT8	emotiondair	22.0%	48.0%	-26.0 pp	100/100
CLM 8B INT8	banking77	8.0%	87.0%	-79.0 pp	100/100
Cygnet 12B NF4 (local variant)	agnews	88.0%	91.0%	-3.0 pp	100/100
Cygnet 12B NF4 (local variant)	emotiondair	43.0%	48.0%	-5.0 pp	100/100
Cygnet 12B NF4 (local variant)	banking77	0.0%	87.0%	-87.0 pp	0/100
Decider 0.8B	agnews	81.0%	91.0%	-10.0 pp	100/100
Decider 0.8B	emotiondair	61.0%	48.0%	+13.0 pp	100/100
Decider 0.8B	banking77	78.0%	87.0%	-9.0 pp	100/100
Decider 2B	agnews	93.0%	91.0%	+2.0 pp	100/100
Decider 2B	emotiondair	76.0%	48.0%	+28.0 pp	100/100
Decider 2B	banking77	78.0%	87.0%	-9.0 pp	100/100
Decider 4B	agnews	96.0%	91.0%	+5.0 pp	100/100
Decider 4B	emotiondair	71.0%	48.0%	+23.0 pp	100/100
Decider 4B	banking77	89.0%	87.0%	+2.0 pp	100/100
Decision 4B v1.2	agnews	91.0%	91.0%	+0.0 pp	100/100
Decision 4B v1.2	emotiondair	51.0%	48.0%	+3.0 pp	100/100
Decision 4B v1.2	banking77	0.0%	87.0%	-87.0 pp	0/100
Decision 2.0 Eos 0.8B	agnews	73.0%	91.0%	-18.0 pp	100/100
Decision 2.0 Eos 0.8B	emotiondair	49.0%	48.0%	+1.0 pp	100/100
Decision 2.0 Eos 0.8B	banking77	90.0%	87.0%	+3.0 pp	100/100
Decision 2.0 Kai 0.6B	agnews	83.0%	91.0%	-8.0 pp	100/100
Decision 2.0 Kai 0.6B	emotiondair	53.0%	48.0%	+5.0 pp	100/100
Decision 2.0 Kai 0.6B	banking77	81.0%	87.0%	-6.0 pp	100/100
Decision 2.0 Nox 4B	agnews	85.0%	91.0%	-6.0 pp	100/100
Decision 2.0 Nox 4B	emotiondair	51.0%	48.0%	+3.0 pp	100/100
Decision 2.0 Nox 4B	banking77	86.0%	87.0%	-1.0 pp	100/100
Decision 2.0 Sol 2B	agnews	80.0%	91.0%	-11.0 pp	100/100
Decision 2.0 Sol 2B	emotiondair	52.0%	48.0%	+4.0 pp	100/100
Decision 2.0 Sol 2B	banking77	89.0%	87.0%	+2.0 pp	100/100
GLiNER Decide 1B	agnews	74.0%	91.0%	-17.0 pp	100/100
GLiNER Decide 1B	emotiondair	48.0%	48.0%	+0.0 pp	100/100
GLiNER Decide 1B	banking77	44.0%	87.0%	-43.0 pp	100/100
GLiNER Decide multilingual	agnews	78.0%	91.0%	-13.0 pp	100/100
GLiNER Decide multilingual	emotiondair	49.0%	48.0%	+1.0 pp	100/100
GLiNER Decide multilingual	banking77	61.0%	87.0%	-26.0 pp	100/100
GLiNER Decide 340M	agnews	87.0%	91.0%	-4.0 pp	100/100
GLiNER Decide 340M	emotiondair	56.0%	48.0%	+8.0 pp	100/100
GLiNER Decide 340M	banking77	75.0%	87.0%	-12.0 pp	100/100
Imajev 2B	agnews	83.0%	91.0%	-8.0 pp	100/100
Imajev 2B	emotiondair	45.0%	48.0%	-3.0 pp	86/100
Imajev 2B	banking77	77.0%	87.0%	-10.0 pp	100/100
Imajev 4B	agnews	86.0%	91.0%	-5.0 pp	100/100
Imajev 4B	emotiondair	36.0%	48.0%	-12.0 pp	61/100
Imajev 4B	banking77	88.0%	87.0%	+1.0 pp	100/100
Intern-Decision 4B	agnews	92.0%	91.0%	+1.0 pp	100/100
Intern-Decision 4B	emotiondair	47.0%	48.0%	-1.0 pp	100/100
Intern-Decision 4B	banking77	0.0%	87.0%	-87.0 pp	0/100
Jev-Omni 12B NF4 (local variant)	agnews	85.0%	91.0%	-6.0 pp	100/100
Jev-Omni 12B NF4 (local variant)	emotiondair	46.0%	48.0%	-2.0 pp	100/100
Jev-Omni 12B NF4 (local variant)	banking77	52.0%	87.0%	-35.0 pp	100/100
JevK5 2B	agnews	85.0%	91.0%	-6.0 pp	100/100
JevK5 2B	emotiondair	46.0%	48.0%	-2.0 pp	100/100
JevK5 2B	banking77	61.0%	87.0%	-26.0 pp	100/100
JevK5 4B	agnews	86.0%	91.0%	-5.0 pp	100/100
JevK5 4B	emotiondair	49.0%	48.0%	+1.0 pp	100/100
JevK5 4B	banking77	76.0%	87.0%	-11.0 pp	100/100
Julia 144M	agnews	94.0%	91.0%	+3.0 pp	100/100
Julia 144M	emotiondair	86.0%	48.0%	+38.0 pp	100/100
Julia 144M	banking77	0.0%	87.0%	-87.0 pp	0/100
Kev 0.8B	agnews	92.0%	91.0%	+1.0 pp	100/100
Kev 0.8B	emotiondair	45.0%	48.0%	-3.0 pp	100/100
Kev 0.8B	banking77	87.0%	87.0%	+0.0 pp	100/100
Kev 4B	agnews	95.0%	91.0%	+4.0 pp	100/100
Kev 4B	emotiondair	45.0%	48.0%	-3.0 pp	100/100
Kev 4B	banking77	88.0%	87.0%	+1.0 pp	100/100
Laya multilingual	agnews	91.0%	91.0%	+0.0 pp	100/100
Laya multilingual	emotiondair	22.0%	48.0%	-26.0 pp	100/100
Laya multilingual	banking77	2.0%	87.0%	-85.0 pp	100/100
Laya	agnews	97.0%	91.0%	+6.0 pp	100/100
Laya	emotiondair	40.0%	48.0%	-8.0 pp	100/100
Laya	banking77	2.0%	87.0%	-85.0 pp	100/100
Nev 2B Q8	agnews	88.0%	91.0%	-3.0 pp	100/100
Nev 2B Q8	emotiondair	48.0%	48.0%	+0.0 pp	100/100
Nev 2B Q8	banking77	0.0%	87.0%	-87.0 pp	0/100
Nimble 9B NF4	agnews	94.0%	91.0%	+3.0 pp	100/100
Nimble 9B NF4	emotiondair	45.0%	48.0%	-3.0 pp	100/100
Nimble 9B NF4	banking77	84.0%	87.0%	-3.0 pp	100/100
Plumb 4B	agnews	89.0%	91.0%	-2.0 pp	100/100
Plumb 4B	emotiondair	47.0%	48.0%	-1.0 pp	100/100
Plumb 4B	banking77	77.0%	87.0%	-10.0 pp	100/100
Tev1 4B	agnews	96.0%	91.0%	+5.0 pp	100/100
Tev1 4B	emotiondair	48.0%	48.0%	+0.0 pp	100/100
Tev1 4B	banking77	0.0%	87.0%	-87.0 pp	0/100
Von	agnews	89.0%	91.0%	-2.0 pp	100/100
Von	emotiondair	71.0%	48.0%	+23.0 pp	100/100
Von	banking77	78.0%	87.0%	-9.0 pp	100/100
Wald 4B v1.2	agnews	83.0%	91.0%	-8.0 pp	100/100
Wald 4B v1.2	emotiondair	43.0%	48.0%	-5.0 pp	100/100
Wald 4B v1.2	banking77	84.0%	87.0%	-3.0 pp	100/100
Winnow 12B Q8 (Windows CUDA)	agnews	88.0%	91.0%	-3.0 pp	100/100
Winnow 12B Q8 (Windows CUDA)	emotiondair	52.0%	48.0%	+4.0 pp	100/100
Winnow 12B Q8 (Windows CUDA)	banking77	0.0%	87.0%	-87.0 pp	0/100

Published Jev pilot reference: news 91%, emotion 48%, banking 87%. Its per-case predictions are not public, so this panel shows aggregate differences without a paired significance claim. The publisher's scoring code is reused; hosted and local timings are not ranked together.

Evidence

Pinned public Jev per-case outcomes · Pinned 300-case pilot report · Local source verification · Case-by-case comparison (local-only evidence) · Full metrics

Paired intervals resample the same local/Jev cases together, stratify by tier, and keep supplied paraphrase groups together. Missing or invalid local answers count as incorrect. Jev public reference: 200/231 correct. Public correctness outcomes do not provide the Jev probability vectors required for a paired calibration comparison.