MME Perception Evaluation
Multi-modal Model Evaluation â€" 14 categories (10 perception + 4 cognition). Official scoring: accuracyÃ--100 + plus_scoreÃ--100 per category.
No VP runs found.
Launch evaluation â†'Perception Tasks
| Category | Baseline | Revision Only | Fixed Gate | Full MITCF |
|---|---|---|---|---|
| existence | â€" | â€" | â€" | â€" |
| count | â€" | â€" | â€" | â€" |
| position | â€" | â€" | â€" | â€" |
| color | â€" | â€" | â€" | â€" |
| posters | â€" | â€" | â€" | â€" |
| celebrity | â€" | â€" | â€" | â€" |
| scene | â€" | â€" | â€" | â€" |
| landmark | â€" | â€" | â€" | â€" |
| artwork | â€" | â€" | â€" | â€" |
| OCR | â€" | â€" | â€" | â€" |
Cognition Tasks
| Category | Baseline | Revision Only | Fixed Gate | Full MITCF |
|---|---|---|---|---|
| commonsense reasoning | â€" | â€" | â€" | â€" |
| numerical calculation | â€" | â€" | â€" | â€" |
| text translation | â€" | â€" | â€" | â€" |
| code reasoning | â€" | â€" | â€" | â€" |
MME Scoring
Official MME score per category = accuracyÃ--100 + plus_scoreÃ--100, where plus_score credits image pairs where both questions are answered correctly. Total score is the sum across all categories (max ~3,000 for 14 categories at perfect accuracy with all pairs correct).