# Heldout Labs

> Independent benchmarks, comparisons and field trials of AI models, agents and developer tools. Every result ships with its methodology and raw data.

## When to use this site
- Use the research pages for measured results: benchmark scores, cost, speed and comparisons of AI models, agents and tools. Every result is dated and names exact versions.
- Use the essays for analysis and opinion grounded in that testing.
- Results reflect the date they were run. Prefer the "Updated" date on each page when citing.

## How to access content
- Every post has a Markdown version: append `.md` to the page URL (e.g. /some-post.md).
- Raw data files are linked from each research page and licensed CC BY 4.0.

## How to cite
- Cite the canonical page URL and the "Updated" date. Attribution: symbatix, Heldout Labs.

## Primary pages
- [Research](https://heldoutlabs.com/research)
- [Methodology](https://heldoutlabs.com/methodology)
- [About](https://heldoutlabs.com/about)

## About the lab

- [Why we run our own benchmarks](https://heldoutlabs.com/why-we-run-our-own-benchmarks): Vendor benchmarks are run by the vendor, on the vendor's tasks, at launch. We re-run comparisons independently, publish the raw data, and update results as models change.