# Proctor

> Evaluation harness for on-device system LLMs: one eval suite for Gemini Nano and Apple Foundation Models, from Kotlin Multiplatform.

## Docs

- [Introduction](https://proctor-docs-phi.vercel.app/): Evaluation harness for the language models built into Android and iOS, written once in Kotlin Multiplatform.
- [Getting started](https://proctor-docs-phi.vercel.app/getting-started): Add Proctor, define a suite, run it and build a report.
- [Prior art](https://proctor-docs-phi.vercel.app/prior-art): What exists for evaluating LLMs and on-device models, and what Proctor adds.
- [Proctor and Apple Evaluations](https://proctor-docs-phi.vercel.app/apple-evaluations): When to use Apple's Evaluations framework and when to use Proctor.
- [Changelog](https://proctor-docs-phi.vercel.app/changelog): Every release of Proctor, newest first.

## Guides

- [Writing suites](https://proctor-docs-phi.vercel.app/guides/writing-suites): Prompt versions, cases, repeats, sampling and the pass condition.
- [Graders](https://proctor-docs-phi.vercel.app/guides/graders): Exact match, regex, JSON schema, semantic similarity, LLM judge and custom graders.
- [Running on Android](https://proctor-docs-phi.vercel.app/guides/running-on-android): Gemini Nano through the ML Kit Prompt API, the foreground and quota rules, and the Developer Preview.
- [Running on iOS](https://proctor-docs-phi.vercel.app/guides/running-on-ios): Apple Foundation Models through a Swift bridge, on a device or a simulator.
- [Reading reports](https://proctor-docs-phi.vercel.app/guides/reading-reports): The result files, the report sections, and what "not graded" means.
- [CI setup](https://proctor-docs-phi.vercel.app/guides/ci): Use the report as a pass threshold, and what CI can and cannot run.
- [Privacy](https://proctor-docs-phi.vercel.app/guides/privacy): Where datasets and outputs go.

## Agent skills

- [proctor](https://proctor-docs-phi.vercel.app/.well-known/agent-skills/proctor/SKILL.md): Read the Proctor docs instead of answering from memory. Evaluation harness for on-device system LLMs: one eval suite for Gemini Nano and Apple Foundation Models, from Kotlin Multiplatform. Use when a task involves Proctor: how it works, how to set it up or configure it, or its API.

## Agent resources

- [llms-full.txt](https://proctor-docs-phi.vercel.app/llms-full.txt): The full Markdown of every page in one file.
- [Page Markdown](https://proctor-docs-phi.vercel.app/index.md): Append `.md` to any page URL to fetch that page as raw Markdown.
- [JSON API](https://proctor-docs-phi.vercel.app/api/docs/pages.json): Page index of the JSON docs API; each entry links the page's JSON and Markdown forms. Described by the OpenAPI document at https://proctor-docs-phi.vercel.app/openapi.json.
- [Agent skills](https://proctor-docs-phi.vercel.app/.well-known/agent-skills/index.json): Agent Skills discovery index of the skills this site publishes.
- [API catalog](https://proctor-docs-phi.vercel.app/.well-known/api-catalog): RFC 9727 linkset of the APIs documented here.
- [AI catalog](https://proctor-docs-phi.vercel.app/.well-known/ai-catalog.json): ARD manifest of the agent-facing resources on this site (MCP server, skills, APIs).
- [agent-readability.json](https://proctor-docs-phi.vercel.app/agent-readability.json): Manifest of every agent-facing artifact on this site.
- [Sitemap](https://proctor-docs-phi.vercel.app/sitemap.xml): Every indexable page URL with its last-modified date.