Security
What happens to your code when you run an evaluation
Isolated single-use workspaces, encryption in transit and at rest, and a firm rule that customer data is never used to train models.
When you submit code to Evals.sh or point it at a running app, you are trusting us with work that is often unreleased, sometimes graded, and occasionally commercially sensitive. This post explains exactly what happens to it: what we create, what we delete, what we keep, and who can see it.
The short version
- Code you submit is placed in a single-use workspace for one job and deleted when that job ends.
- Browser evaluations run in an isolated container that is removed when the job finishes.
- Evaluation results (scores, reasoning, step logs) and UI screenshots are stored in your account, so you can see your history and share reports. They are kept for a limited retention period, and you can ask us to delete them at any time.
- Your data is never used to train AI models, ours or anyone else's.
- Traffic is encrypted in transit with TLS, and stored data is encrypted at rest.
The rest of this post goes through each of those in more detail.
Code review: a single-use workspace
When you submit code for review, by uploading files or folders or pasting code, it is placed in a workspace directory created for that job and no other. Files that would land outside that directory are rejected, and submissions over a size limit are refused before anything is written.
The review agent then works inside that workspace with read-only tools: it can list directories, read files and search the code. It cannot run your code, install packages or reach other jobs' files.
When the job ends, the workspace is deleted. That happens in a cleanup step that runs whether the review succeeded, failed or hit a time limit, and it also runs if preparing the workspace fails partway through. We don't keep a working copy of your code after the job; each new review starts from a fresh workspace.
Browser sessions
UI evaluations drive a headless browser against the URL you provide. Each job gets its own browser in its own isolated container, with no cookies, cache or sessions carried over from any other job. When the job finishes, the container is stopped and removed and its scratch directory is deleted. If you gave the agent test credentials in your instructions, they exist in that browser only for the length of the job.
We go into the mechanics in Inside our computer-use agent.
What we keep, and why
We do store data, and we think it's better to say so plainly. An evaluation is only useful if you can come back to it, compare it with the last one and send it to someone else. So the results of each job are saved to your account:
| Data | Kept? | Why |
|---|---|---|
| Submitted code (working copy) | No, deleted at job end | Only needed while the agent reads them |
| Browser container and session | No, removed at job end | Only needed while the agent tests the app |
| Scores and per-criterion reasoning | Yes, in your account | Your evaluation history and reports |
| Step logs and agent trace | Yes, with the result | A record of each step behind a score |
| UI evaluation screenshots | Yes, in your account | Evidence attached to the report |
| Inputs you submitted (URLs, instructions, rubric) | Yes, in your account | To show what was evaluated and to re-run it |
Results can include short excerpts of your code. A review rationale might quote the line where a secret is hard-coded, or name the file where validation is missing. Those excerpts live inside the result, alongside the rest of the report.
Everything in the “Yes” rows is kept for a limited retention period so your history stays useful, and you can request deletion at any time.
Results are private to your account by default. When you create a share link for a report, anyone with that link can view that report. That is a deliberate action, and the link applies to that one result.
AI models and training
Evaluations are carried out by large language models, which means the content being evaluated (the files the review agent reads, the page snapshots the browser agent sees) is sent to the model provider as part of each request. That is the only way the model can reason about it.
We also treat what the models see as untrusted input. Instructions embedded in submitted code or a web page are recorded as evidence, not followed, so a submission can't talk the grader into a better score.
Encryption and access
All traffic between your browser, our service and the model providers is encrypted in transit with TLS. Data we store, including results and screenshots, is encrypted at rest by our cloud infrastructure.
Access to your account data requires authentication, and results are scoped to the account that created them. Internally, access to production systems is limited to the people and services that need it to operate the product, on a least-privilege basis.
More detail on how we secure the platform is on our security page, and the formal terms are in our privacy policy.
Deleting your data
You can delete individual evaluation results from the app at any time. If you want everything removed (your results, screenshots and account), contact support and we'll delete it.
If anything here is unclear, or you need a specific answer for a procurement or ethics review, ask us. We would rather give you a precise answer than a reassuring one.