AI evaluation for Latin America and the Caribbean
Created by Adrian Dunkley. Model comparisons should help a reader choose a defensible test, not turn uncertain numbers into a procurement recommendation.
Current publication status
The earlier model leaderboard, LAC composite scores and country-readiness tiers are withheld because the supporting evidence needed to verify them was not available during this review. This is an evidence-availability limitation, not a finding that the underlying research did not occur. Do not cite the withheld tables as verified results.
- Proposed LAC Benchmark method: regional languages, context, business tasks, cost and constrained-connectivity testing.
- Reading global benchmarks: what to verify before comparing published scores.
- Tool-selection guide: local eligibility, cost, data handling and practical trials.
Compare evidence before scores
Check the exact model version, test date, dataset version, prompting, tools, inference budget and scoring procedure. Keep provider-reported results separate from independently reproduced results. Missing evidence should be labelled missing, not treated as a zero or silently estimated.
Test a bounded task with representative local-language examples and a suitable reviewer. Include failures, refusals, uncertainty and escalation behaviour. Measure human checking time and error consequences as well as successful completions.
Proposed country-readiness framework
The earlier design grouped indicators into seven dimensions. These remain proposals, not measured rankings or claims about any country’s current position.
| Proposed dimension | Proposed weight | Evidence that would be needed |
|---|---|---|
| Infrastructure | 15% | Dated, comparable connectivity and service-access indicators |
| Talent | 20% | Clearly defined education, workforce and research measures |
| Policy and governance | 15% | Current primary legal and institutional sources, distinguishing proposals from laws in force |
| Investment | 15% | Comparable funding definitions, periods and documented amounts |
| Adoption | 20% | Representative surveys with methods, denominators and uncertainty |
| Ethics and inclusion | 10% | Defined access, accessibility and language-coverage indicators |
| Economic impact | 5% | A defensible method separating measured association from causal effects |
Do not infer readiness from a law’s existence, a funding announcement or a vendor’s presence alone. Document missing data, country comparability, weighting choices and sensitivity before calculating a composite. No release date or data-collection progress is promised here.
Using and citing this section
Cite the proposed method as a proposal. A methodology citation is not a source for a model score or country ranking. Published results should be restored only with sufficient evidence for readers to inspect the claims and limitations.
The original LAC framework carries a Creative Commons Attribution 4.0 notice with attribution to Adrian Dunkley. This does not cover third-party benchmark data unless its own licence permits reuse.