evals.sh
ProductModelsSecurityPricingDocsBlog
Sign inGet started
Blog

Guides

Grading a full cohort in one session with Groups

A practical walkthrough of bulk evaluation: preparing a CSV of submissions, writing one rubric, and reviewing results side by side.

Sajan R.·April 2, 2026·8 min read

Evaluating one project is easy to set up. Evaluating sixty the night after a deadline is where most tooling falls over. Groups is the part of Evals.sh built for that second case: one configuration, many submissions, and a single place to review the results. This guide walks through a full run.

When to use Groups

A Group is a batch of evaluations of the same type that share one set of instructions. You can create one for any of the main evaluation types:

  • Project evaluation: an agent opens each live URL in a headless browser and checks it against your instructions.
  • Code review: upload each submission’s project files or folder; each is copied into its own single-use workspace and reviewed.
  • Frontend audit: performance, accessibility, SEO and security headers for each URL.
  • Assignment grading: written submissions graded against a rubric.

If you are grading a cohort, re-checking last week’s submissions after students pushed fixes, or auditing a set of sites you maintain, a Group saves you from configuring the same thing over and over.

Prepare the submissions

For project evaluations and frontend audits, you can paste URLs into the form one at a time, but for a class it is faster to upload a CSV. (For a code review group, you upload each submission’s files or folder instead.) The importer reads the first column of each row as the URL. A header row containing the word “url” is skipped, duplicate links are ignored, and anything that is not a valid URL is reported and left out, so a messy export from your LMS will not silently produce bad tasks.

submissions.csv
url,student
https://notes-app-amara.example.app,Amara O.
https://ben-taskboard.example.app,Ben K.
https://cohort4-team7.example.app,Team 7

Only the first column is used. Keeping a second column with the student’s name costs nothing and makes it easy to match results back to your roster later.

Before you import, it is worth sending one reminder to students: check that your link loads in a private browser window. The agent evaluates the deployment it is given. If the app has gone to sleep, requires a login you did not provide, or crashes on load, that is what the report will show.

Write one set of instructions

Every task in a Group uses the same instructions, so this is where the quality of the whole batch is decided. For a project evaluation, describe the behaviour you expect in plain language, as specifically as you can.

Evaluation instructions
A user can sign up with an email and password, then log in.
After logging in, they can create a note with a title and body.
Notes can be edited and deleted.
Submitting a note with an empty title shows a validation message.
Notes are still listed after a full page reload.

The agent turns this into a test plan for each submission, so it will find the right buttons and fields even though every student named and styled them differently. What it needs from you is the behaviour, not the selectors.

If every project sits behind a login, you can mark the batch as protected and supply a test username and password that the agent will use. This works best when you give students a fixed test account to seed in their app.

Tip

Run the instructions against one or two submissions on their own before launching the full batch. If the reports test the wrong thing, it is much cheaper to adjust the wording now than after sixty runs.

Run the batch

A list or CSV of URLs becomes a group with one shared instruction. Each URL runs as its own task, and results collect in one table that can be exported.01 import02 configure03 run04 reviewurls.csvgroupone instructiontask 01donetask 02donetask 03runningtask 04queuedresultscsv · xlsx · pdfeach task has its own full report
Each URL becomes its own task with its own isolated environment. Scores collect in the group view, and every row links to a full report.
  1. 01

    Create the group

    Give it a name you will recognise later, such as the course and assignment, and pick the evaluation type.
  2. 02

    Add targets and instructions

    Upload the CSV or paste the list, then add the shared instructions and any test credentials.
  3. 03

    Start and walk away

    Tasks run in the background, several at a time, each in its own isolated environment. You can turn on an email notification so you hear when the batch is finished rather than watching the page.

The group view updates live as tasks move from queued to running to completed. If a task fails for an operational reason, such as a site that timed out, you can restart just that task, or restart all failed tasks in the group in one action, without re-running the ones that succeeded.

Review results

When the batch finishes, you get one table with a row per submission and its score. That table can be exported as CSV, Excel or PDF, which is usually the easiest way to get numbers into a gradebook.

The table is a starting point, not the grade. Each row links to the full report for that submission: the test plan the agent derived, the step log for each feature, the screenshots it took and the reasoning behind each criterion score. A sensible review order is:

  1. The lowest scores first. Check whether the project is genuinely incomplete or whether something like a broken deploy got in the way.
  2. Anything near your pass mark. These are the cases where your judgement changes an outcome.
  3. A sample of the rest. Reading a few high-scoring reports is a quick way to confirm the instructions were interpreted the way you meant.

Individual reports can be shared with a link, so students see the same evidence you saw. That tends to make feedback conversations shorter and more concrete: instead of “the login didn’t work,” you can both look at the step where it failed.

A short checklist

  • Ask students to verify their links load in a private window before the deadline.
  • Export submissions as a CSV with the URL in the first column.
  • Write instructions as observable behaviours, one per line.
  • Test the instructions on one or two submissions first.
  • Provide a shared test account if apps require login.
  • Turn on the completion email and come back later.
  • Review low and borderline scores by hand, then export.

For background on what the agent is doing inside each task, see how Evals.sh works.

On this page

When to use GroupsPrepare the submissionsWrite one set of instructionsRun the batchReview resultsA short checklist

Try it on your own project

Free credits every month. No card required.

Get started

Keep reading

Guides

Writing rubrics an AI grader can apply consistently

October 23, 2026

Guides

What a frontend audit actually measures

October 23, 2026

Guides

Choosing a project evaluation tool in 2026: a buyer's framework

April 2, 2026

evals.sh

Autonomous evaluation for software teams and educators. Agents that review code, drive real browsers and grade work against your rubric.

Product

  • Project evaluation
  • Code review
  • Frontend audit
  • API testing
  • Assignment grading

Platform

  • Models
  • Security
  • Pricing
  • Documentation

Resources

  • Blog
  • Guides
  • Contact

Legal

  • Privacy policy
  • Terms of service
  • Data handling

© 2026 Evals.sh. All rights reserved.

Customer data is never used to train AI models.