What does the evaluation run comparator produce?
Compare baseline and candidate evaluation pass rates, show the percentage-point change and test it against a customer-entered review threshold.
The working area above saves numeric inputs in this browser and exposes every formula. It does not decide fairness, safety, compliance or release approval.
What do you need before you start?
- Freeze the complete population, process, period and approved outcome criteria before entering aggregate counts.
- Use ordinary nonnegative decimal notation. Counts must be whole numbers through one billion. Review thresholds support increments of 0.1 percentage points.
- Confirm comparable populations and source labels. The tool does not collect individual records or test an AI model.
What information do you enter?
The browser calculator accepts only the aggregate numeric inputs listed below.
- Baseline cases (required). Enter the number of evaluated cases in the approved baseline run.
- Baseline cases passed (required). Enter cases that met the baseline run’s stated acceptance criteria.
- Candidate cases (required). Enter the number of evaluated cases in the candidate run.
- Candidate cases passed (required). Enter cases that met the candidate run’s same acceptance criteria.
- Maximum allowed regression in percentage points (required). Enter the organization’s review threshold. This is not supplied as a legal or technical standard.
| Field | What to enter | Fictional example |
|---|---|---|
| Baseline casesRequired | Enter the number of evaluated cases in the approved baseline run.Accepted range 0 to 1000000000. | 500 |
| Baseline cases passedRequired | Enter cases that met the baseline run’s stated acceptance criteria.Accepted range 0 to 1000000000. | 460 |
| Candidate casesRequired | Enter the number of evaluated cases in the candidate run.Accepted range 0 to 1000000000. | 500 |
| Candidate cases passedRequired | Enter cases that met the candidate run’s same acceptance criteria.Accepted range 0 to 1000000000. | 451 |
| Maximum allowed regression in percentage pointsRequired | Enter the organization’s review threshold. This is not supplied as a legal or technical standard.Accepted range 0 to 100. | 2 |
What you can download
Every result must identify its supplied facts, assumptions and unresolved items.
| Free result | Included downloads |
|---|---|
Enter your information, inspect the on-screen result and keep a browser backup. Editing and direct copies of your entered information stay free. |
These supported outputs are free. No checkout is required. |
How does the evaluation run comparator work?
- 1Freeze the evidence basis
Retain the source population, evaluation version, success criterion and review thresholds in your own controlled records.
- 2Enter the bounded counts
Use the numeric fields above or a clearly fictional example. Correct counts that exceed their population or imply an impossible confusion matrix.
- 3Calculate and inspect
Review every descriptive rate and entered gate. Gates use exact count and decimal expressions. Displayed percentages round to two decimal places.
- 4Keep a restorable backup
Download the free backup before clearing or replacing work. Earlier JSON result downloads can restore inputs. Restoring recalculates results and preserves previous browser inputs.
- 5Prepare the finalized calculation
Download all seven finalized files free. Browser preparation does not transfer inputs to a provider.
Method and sources
Apply the steps above to the supported inputs. Use the listed source sections to review the rule basis, assumptions and limits.
- NIST AI Resource CenterOperational resources for AI evaluation and documentation. Testing, evaluation, verification and validation resources.National Institute of Standards and Technology. Reviewed 2026-09-24.
- Artificial Intelligence Risk Management Framework 1.0A voluntary US baseline for AI risk records, responsibilities, evaluation and lifecycle decisions. Govern, Map, Measure and Manage.National Institute of Standards and Technology. Published 2023-01-26. Reviewed 2026-09-24.
- Generative Artificial Intelligence ProfileA cross-sector profile for identifying and managing risks specific to generative AI. Generative AI risks and suggested actions.National Institute of Standards and Technology. Published 2024-07-26. Reviewed 2026-09-24.
Common questions
What information does the evaluation run comparator need?
Enter baseline cases (required). Enter the number of evaluated cases in the approved baseline run; baseline cases passed (required). Enter cases that met the baseline run’s stated acceptance criteria; candidate cases (required). Enter the number of evaluated cases in the candidate run; candidate cases passed (required). Enter cases that met the candidate run’s same acceptance criteria; maximum allowed regression in percentage points (required). Enter the organization’s review threshold. This is not supplied as a legal or technical standard. Freeze the complete population, process, period and approved outcome criteria before entering aggregate counts.
What does the evaluation run comparator produce?
Baseline and candidate pass rates. Percentage-point change. Comparison against the entered regression threshold. Free restorable browser backup and complete JSON calculation record. Seven-file finalized export. Formal PDF, editable DOCX, Excel, results CSV, complete input and result JSON, file hashes and instructions. Review unresolved findings and limitations before use.
Who makes the final decision?
A responsible person reviews the result. Use comparable cases and identical acceptance criteria. The result does not establish model quality or authorize release.
Is this tool free?
Yes. The on-screen result and listed downloads are free. No checkout is required. Keep local backups and review the stated method and limits.
Can I use the evaluation run comparator for this task?
Compare results across models, prompts and versions. Use comparable cases and identical acceptance criteria. The result does not establish model quality or authorize release.
What still needs review?
Use comparable cases and identical acceptance criteria. The result does not establish model quality or authorize release.
Aggregate counts do not establish complete evidence, comparable samples or valid labels. No individual-record audit, model test or release approval is performed.
Rates and entered gates describe the supplied counts. They do not establish statistical significance or a legal conclusion.