231 public JevBench cases; original scoring. Separate from the sealed leaderboard and from full English/Norwegian evaluation. Results use the same pinned inputs and retain native rejections as failures. Scores appear only when the complete suite has been attempted.
All four native models completed all five frozen fixtures: 70,304 saved case predictions, 0 final saved prediction errors. Windows allocation/kernel failures interrupted 4 process attempts; exact-runtime retries resumed saved IDs. Independent result verification and crash history.
Decision 4B v1.2 is the separate FlyMy model previously benchmarked here.
| Model | Suite | Cases | Exact-match accuracy | Coverage |
|---|
| Decision 2.0 Kai 0.6B | jev_verified/agnews | 100 | 83.0% | 100.0% |
| Decision 2.0 Kai 0.6B | jev_verified/emotiondair | 100 | 53.0% | 100.0% |
| Decision 2.0 Kai 0.6B | jev_verified/banking77 | 100 | 81.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fresh/news | 200 | 83.5% | 100.0% |
| Decision 2.0 Kai 0.6B | fresh/emotion | 200 | 57.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fresh/spam | 200 | 54.5% | 100.0% |
| Decision 2.0 Kai 0.6B | fresh/phishing | 200 | 68.5% | 100.0% |
| Decision 2.0 Kai 0.6B | fresh_business/routing/en | 12 | 58.3% | 100.0% |
| Decision 2.0 Kai 0.6B | fresh_business/routing/nb | 12 | 16.7% | 100.0% |
| Decision 2.0 Kai 0.6B | fresh_business/documents/en | 12 | 91.7% | 100.0% |
| Decision 2.0 Kai 0.6B | fresh_business/documents/nb | 12 | 91.7% | 100.0% |
| Decision 2.0 Kai 0.6B | industry/routing/en | 40 | 87.5% | 100.0% |
| Decision 2.0 Kai 0.6B | industry/routing/en/reversed | 40 | 87.5% | 100.0% |
| Decision 2.0 Kai 0.6B | industry/routing/nb | 40 | 77.5% | 100.0% |
| Decision 2.0 Kai 0.6B | industry/routing/nb/reversed | 40 | 77.5% | 100.0% |
| Decision 2.0 Kai 0.6B | industry/documents/en | 28 | 100.0% | 100.0% |
| Decision 2.0 Kai 0.6B | industry/documents/en/reversed | 28 | 100.0% | 100.0% |
| Decision 2.0 Kai 0.6B | industry/documents/nb | 28 | 92.9% | 100.0% |
| Decision 2.0 Kai 0.6B | industry/documents/nb/reversed | 28 | 96.4% | 100.0% |
| Decision 2.0 Kai 0.6B | jev_native/agnews | 100 | 82.0% | 100.0% |
| Decision 2.0 Kai 0.6B | jev_native/emotiondair | 100 | 53.0% | 100.0% |
| Decision 2.0 Kai 0.6B | jev_native/banking77 | 100 | 81.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/agent_handoff | 100 | 58.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/banking_intent | 100 | 55.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/benefits_request | 100 | 49.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/clinic_request | 100 | 41.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/document_type | 100 | 80.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/email_triage | 100 | 8.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/news_topic | 100 | 57.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/paper_field | 100 | 42.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/product_feedback | 100 | 4.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/restaurant_review | 100 | 13.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/review_sentiment | 100 | 68.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/screen_tags | 100 | 15.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/sports_recap | 100 | 40.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/support_intent | 100 | 60.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/support_topic | 100 | 40.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/ticket_route | 100 | 4.0% | 100.0% |
| Decision 2.0 Kai 0.6B | fast_decisions_dev/travel_request | 100 | 60.0% | 100.0% |
| Decision 2.0 Kai 0.6B | laya_claim/news.replication | 400 | 83.8% | 100.0% |
| Decision 2.0 Kai 0.6B | laya_claim/emotion.replication | 400 | 55.0% | 100.0% |
| Decision 2.0 Kai 0.6B | laya_claim/spam.replication | 400 | 48.0% | 100.0% |
| Decision 2.0 Kai 0.6B | laya_claim/phishing.replication | 400 | 71.5% | 100.0% |
| Decision 2.0 Kai 0.6B | laya_claim/jailbreaking | 400 | 87.5% | 100.0% |
| Decision 2.0 Kai 0.6B | laya_claim/toxicity | 400 | 65.8% | 100.0% |
| Decision 2.0 Kai 0.6B | laya_claim/intent | 300 | 85.7% | 100.0% |
| Decision 2.0 Kai 0.6B | laya_claim/scenario | 300 | 66.3% | 100.0% |
| Decision 2.0 Kai 0.6B | massive_en | 2948 | 72.9% | 100.0% |
| Decision 2.0 Kai 0.6B | massive_nb | 2948 | 54.7% | 100.0% |
| Decision 2.0 Kai 0.6B | norec_no | 1173 | 57.7% | 100.0% |
| Decision 2.0 Kai 0.6B | intern_claim/typed_decisions-test | 400 | 3.8% | 100.0% |
| Decision 2.0 Eos 0.8B | jev_verified/agnews | 100 | 73.0% | 100.0% |
| Decision 2.0 Eos 0.8B | jev_verified/emotiondair | 100 | 49.0% | 100.0% |
| Decision 2.0 Eos 0.8B | jev_verified/banking77 | 100 | 90.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fresh/news | 200 | 82.5% | 100.0% |
| Decision 2.0 Eos 0.8B | fresh/emotion | 200 | 51.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fresh/spam | 200 | 69.5% | 100.0% |
| Decision 2.0 Eos 0.8B | fresh/phishing | 200 | 76.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fresh_business/routing/en | 12 | 50.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fresh_business/routing/nb | 12 | 41.7% | 100.0% |
| Decision 2.0 Eos 0.8B | fresh_business/documents/en | 12 | 83.3% | 100.0% |
| Decision 2.0 Eos 0.8B | fresh_business/documents/nb | 12 | 83.3% | 100.0% |
| Decision 2.0 Eos 0.8B | industry/routing/en | 40 | 85.0% | 100.0% |
| Decision 2.0 Eos 0.8B | industry/routing/en/reversed | 40 | 82.5% | 100.0% |
| Decision 2.0 Eos 0.8B | industry/routing/nb | 40 | 65.0% | 100.0% |
| Decision 2.0 Eos 0.8B | industry/routing/nb/reversed | 40 | 67.5% | 100.0% |
| Decision 2.0 Eos 0.8B | industry/documents/en | 28 | 100.0% | 100.0% |
| Decision 2.0 Eos 0.8B | industry/documents/en/reversed | 28 | 100.0% | 100.0% |
| Decision 2.0 Eos 0.8B | industry/documents/nb | 28 | 89.3% | 100.0% |
| Decision 2.0 Eos 0.8B | industry/documents/nb/reversed | 28 | 89.3% | 100.0% |
| Decision 2.0 Eos 0.8B | jev_native/agnews | 100 | 82.0% | 100.0% |
| Decision 2.0 Eos 0.8B | jev_native/emotiondair | 100 | 49.0% | 100.0% |
| Decision 2.0 Eos 0.8B | jev_native/banking77 | 100 | 84.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/agent_handoff | 100 | 65.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/banking_intent | 100 | 54.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/benefits_request | 100 | 44.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/clinic_request | 100 | 41.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/document_type | 100 | 76.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/email_triage | 100 | 5.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/news_topic | 100 | 56.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/paper_field | 100 | 35.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/product_feedback | 100 | 4.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/restaurant_review | 100 | 11.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/review_sentiment | 100 | 69.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/screen_tags | 100 | 19.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/sports_recap | 100 | 41.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/support_intent | 100 | 58.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/support_topic | 100 | 39.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/ticket_route | 100 | 5.0% | 100.0% |
| Decision 2.0 Eos 0.8B | fast_decisions_dev/travel_request | 100 | 67.0% | 100.0% |
| Decision 2.0 Eos 0.8B | laya_claim/news.replication | 400 | 79.0% | 100.0% |
| Decision 2.0 Eos 0.8B | laya_claim/emotion.replication | 400 | 52.8% | 100.0% |
| Decision 2.0 Eos 0.8B | laya_claim/spam.replication | 400 | 75.0% | 100.0% |
| Decision 2.0 Eos 0.8B | laya_claim/phishing.replication | 400 | 76.5% | 100.0% |
| Decision 2.0 Eos 0.8B | laya_claim/jailbreaking | 400 | 92.5% | 100.0% |
| Decision 2.0 Eos 0.8B | laya_claim/toxicity | 400 | 70.0% | 100.0% |
| Decision 2.0 Eos 0.8B | laya_claim/intent | 300 | 84.7% | 100.0% |
| Decision 2.0 Eos 0.8B | laya_claim/scenario | 300 | 76.7% | 100.0% |
| Decision 2.0 Eos 0.8B | massive_en | 2948 | 67.5% | 100.0% |
| Decision 2.0 Eos 0.8B | massive_nb | 2948 | 52.6% | 100.0% |
| Decision 2.0 Eos 0.8B | norec_no | 1173 | 53.4% | 100.0% |
| Decision 2.0 Eos 0.8B | intern_claim/typed_decisions-test | 400 | 2.0% | 100.0% |
| Decision 2.0 Sol 2B | jev_verified/agnews | 100 | 80.0% | 100.0% |
| Decision 2.0 Sol 2B | jev_verified/emotiondair | 100 | 52.0% | 100.0% |
| Decision 2.0 Sol 2B | jev_verified/banking77 | 100 | 89.0% | 100.0% |
| Decision 2.0 Sol 2B | fresh/news | 200 | 83.5% | 100.0% |
| Decision 2.0 Sol 2B | fresh/emotion | 200 | 29.0% | 100.0% |
| Decision 2.0 Sol 2B | fresh/spam | 200 | 74.5% | 100.0% |
| Decision 2.0 Sol 2B | fresh/phishing | 200 | 86.0% | 100.0% |
| Decision 2.0 Sol 2B | fresh_business/routing/en | 12 | 58.3% | 100.0% |
| Decision 2.0 Sol 2B | fresh_business/routing/nb | 12 | 58.3% | 100.0% |
| Decision 2.0 Sol 2B | fresh_business/documents/en | 12 | 100.0% | 100.0% |
| Decision 2.0 Sol 2B | fresh_business/documents/nb | 12 | 100.0% | 100.0% |
| Decision 2.0 Sol 2B | industry/routing/en | 40 | 87.5% | 100.0% |
| Decision 2.0 Sol 2B | industry/routing/en/reversed | 40 | 77.5% | 100.0% |
| Decision 2.0 Sol 2B | industry/routing/nb | 40 | 75.0% | 100.0% |
| Decision 2.0 Sol 2B | industry/routing/nb/reversed | 40 | 70.0% | 100.0% |
| Decision 2.0 Sol 2B | industry/documents/en | 28 | 100.0% | 100.0% |
| Decision 2.0 Sol 2B | industry/documents/en/reversed | 28 | 96.4% | 100.0% |
| Decision 2.0 Sol 2B | industry/documents/nb | 28 | 100.0% | 100.0% |
| Decision 2.0 Sol 2B | industry/documents/nb/reversed | 28 | 96.4% | 100.0% |
| Decision 2.0 Sol 2B | jev_native/agnews | 100 | 80.0% | 100.0% |
| Decision 2.0 Sol 2B | jev_native/emotiondair | 100 | 49.0% | 100.0% |
| Decision 2.0 Sol 2B | jev_native/banking77 | 100 | 90.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/agent_handoff | 100 | 75.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/banking_intent | 100 | 55.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/benefits_request | 100 | 50.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/clinic_request | 100 | 38.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/document_type | 100 | 80.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/email_triage | 100 | 7.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/news_topic | 100 | 58.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/paper_field | 100 | 41.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/product_feedback | 100 | 3.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/restaurant_review | 100 | 19.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/review_sentiment | 100 | 76.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/screen_tags | 100 | 18.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/sports_recap | 100 | 44.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/support_intent | 100 | 55.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/support_topic | 100 | 49.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/ticket_route | 100 | 11.0% | 100.0% |
| Decision 2.0 Sol 2B | fast_decisions_dev/travel_request | 100 | 66.0% | 100.0% |
| Decision 2.0 Sol 2B | laya_claim/news.replication | 400 | 82.0% | 100.0% |
| Decision 2.0 Sol 2B | laya_claim/emotion.replication | 400 | 31.5% | 100.0% |
| Decision 2.0 Sol 2B | laya_claim/spam.replication | 400 | 80.0% | 100.0% |
| Decision 2.0 Sol 2B | laya_claim/phishing.replication | 400 | 85.2% | 100.0% |
| Decision 2.0 Sol 2B | laya_claim/jailbreaking | 400 | 91.8% | 100.0% |
| Decision 2.0 Sol 2B | laya_claim/toxicity | 400 | 79.0% | 100.0% |
| Decision 2.0 Sol 2B | laya_claim/intent | 300 | 84.7% | 100.0% |
| Decision 2.0 Sol 2B | laya_claim/scenario | 300 | 72.3% | 100.0% |
| Decision 2.0 Sol 2B | massive_en | 2948 | 73.4% | 100.0% |
| Decision 2.0 Sol 2B | massive_nb | 2948 | 64.4% | 100.0% |
| Decision 2.0 Sol 2B | norec_no | 1173 | 57.2% | 100.0% |
| Decision 2.0 Sol 2B | intern_claim/typed_decisions-test | 400 | 6.2% | 100.0% |
| Decision 2.0 Nox 4B | jev_verified/agnews | 100 | 85.0% | 100.0% |
| Decision 2.0 Nox 4B | jev_verified/emotiondair | 100 | 51.0% | 100.0% |
| Decision 2.0 Nox 4B | jev_verified/banking77 | 100 | 86.0% | 100.0% |
| Decision 2.0 Nox 4B | fresh/news | 200 | 87.0% | 100.0% |
| Decision 2.0 Nox 4B | fresh/emotion | 200 | 59.0% | 100.0% |
| Decision 2.0 Nox 4B | fresh/spam | 200 | 93.0% | 100.0% |
| Decision 2.0 Nox 4B | fresh/phishing | 200 | 92.5% | 100.0% |
| Decision 2.0 Nox 4B | fresh_business/routing/en | 12 | 83.3% | 100.0% |
| Decision 2.0 Nox 4B | fresh_business/routing/nb | 12 | 83.3% | 100.0% |
| Decision 2.0 Nox 4B | fresh_business/documents/en | 12 | 91.7% | 100.0% |
| Decision 2.0 Nox 4B | fresh_business/documents/nb | 12 | 100.0% | 100.0% |
| Decision 2.0 Nox 4B | industry/routing/en | 40 | 95.0% | 100.0% |
| Decision 2.0 Nox 4B | industry/routing/en/reversed | 40 | 95.0% | 100.0% |
| Decision 2.0 Nox 4B | industry/routing/nb | 40 | 87.5% | 100.0% |
| Decision 2.0 Nox 4B | industry/routing/nb/reversed | 40 | 85.0% | 100.0% |
| Decision 2.0 Nox 4B | industry/documents/en | 28 | 100.0% | 100.0% |
| Decision 2.0 Nox 4B | industry/documents/en/reversed | 28 | 96.4% | 100.0% |
| Decision 2.0 Nox 4B | industry/documents/nb | 28 | 100.0% | 100.0% |
| Decision 2.0 Nox 4B | industry/documents/nb/reversed | 28 | 100.0% | 100.0% |
| Decision 2.0 Nox 4B | jev_native/agnews | 100 | 88.0% | 100.0% |
| Decision 2.0 Nox 4B | jev_native/emotiondair | 100 | 53.0% | 100.0% |
| Decision 2.0 Nox 4B | jev_native/banking77 | 100 | 87.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/agent_handoff | 100 | 81.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/banking_intent | 100 | 59.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/benefits_request | 100 | 47.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/clinic_request | 100 | 48.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/document_type | 100 | 80.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/email_triage | 100 | 8.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/news_topic | 100 | 67.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/paper_field | 100 | 43.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/product_feedback | 100 | 13.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/restaurant_review | 100 | 42.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/review_sentiment | 100 | 87.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/screen_tags | 100 | 19.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/sports_recap | 100 | 48.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/support_intent | 100 | 69.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/support_topic | 100 | 44.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/ticket_route | 100 | 12.0% | 100.0% |
| Decision 2.0 Nox 4B | fast_decisions_dev/travel_request | 100 | 67.0% | 100.0% |
| Decision 2.0 Nox 4B | laya_claim/news.replication | 400 | 85.5% | 100.0% |
| Decision 2.0 Nox 4B | laya_claim/emotion.replication | 400 | 58.0% | 100.0% |
| Decision 2.0 Nox 4B | laya_claim/spam.replication | 400 | 93.5% | 100.0% |
| Decision 2.0 Nox 4B | laya_claim/phishing.replication | 400 | 92.0% | 100.0% |
| Decision 2.0 Nox 4B | laya_claim/jailbreaking | 400 | 78.8% | 100.0% |
| Decision 2.0 Nox 4B | laya_claim/toxicity | 400 | 84.0% | 100.0% |
| Decision 2.0 Nox 4B | laya_claim/intent | 300 | 90.3% | 100.0% |
| Decision 2.0 Nox 4B | laya_claim/scenario | 300 | 75.0% | 100.0% |
| Decision 2.0 Nox 4B | massive_en | 2948 | 85.2% | 100.0% |
| Decision 2.0 Nox 4B | massive_nb | 2948 | 80.3% | 100.0% |
| Decision 2.0 Nox 4B | norec_no | 1173 | 72.0% | 100.0% |
| Decision 2.0 Nox 4B | intern_claim/typed_decisions-test | 400 | 19.0% | 100.0% |
Fresh suite accuracies exclude development records. Score classes use the modal level; native expected scores are retained in raw results. Multilabel tasks use separate native Yes/No questions. Authored Norwegian/business suites are diagnostics.
Windows NVIDIA CUDA/Transformers/FLA environment differs from publisher ROCm latency environment. Public benchmark training contamination and model-selection exposure are not independently established. No model weights, prompts or test labels were tuned during this run. Decision 2.0 Lux 9B and Vega 27B: native precision exceeds local GPU budget