Blog
Notes on building evaluation agents
Engineering deep-dives, practical guides and product updates from the team behind Evals.sh.
Inside our computer-use agent: evaluating apps through a real browser
How Evals.sh drives an isolated, headless browser to click, type and verify user flows, and why we capture evidence for every step.
- Security
What happens to your code when you run an evaluation
Isolated single-use workspaces, encryption in transit and at rest, and a firm rule that customer data is never used to train models.
- Product
Matching the model to the evaluation
Browser agents, repository review and bulk grading stress models differently. How we think about speed, depth and context when choosing one.
- Engineering
Agentic API testing: beyond the 200 OK
Status codes tell you an endpoint answered. An agent can tell you whether it answered correctly, consistently and safely.
- Guides
Writing rubrics an AI grader can apply consistently
Observable criteria, explicit weights and clear failure states: the habits that make automated grading repeatable and fair.
- Guides
What a frontend audit actually measures
Core Web Vitals, accessibility, SEO and security headers: what each metric means, and which fixes move the score most.
- Engineering
Code review at cohort scale
How bulk code review clones, reads and scores dozens of repositories in parallel, and how to read the results without drowning in them.
- Engineering
Smoke-testing every release with a browser agent
Scripted end-to-end tests break on every redesign. How engineering teams use plain-language instructions and the Evals.sh API to check critical flows after each deploy.
- Education
The autonomous classroom: scaling technical mentorship with AI
When verification is automated, instructors get their time back for the part of teaching that doesn't scale: judgement, feedback and mentorship.
- Education
The evaluation bottleneck: when students ship faster than you can grade
AI coding assistants have multiplied how much students build. Grading capacity hasn't kept up. Here's what changes when evaluation is automated.
- Guides
Grading a full cohort in one session with Groups
A practical walkthrough of bulk evaluation: preparing a CSV of submissions, writing one rubric, and reviewing results side by side.
- Guides
Choosing a project evaluation tool in 2026: a buyer's framework
Unit-test autograders, static analysers and agentic evaluators solve different problems. A framework for picking the right one for your course or team.
- Engineering
Agentic evaluation vs. manual grading: how Evals.sh works
A look at the perceive → reason → act loop behind our evaluation agents, and why we separate evidence gathering from scoring.