An agent can now build a working multi-view dashboard in an afternoon, and it can also write a confident review of that dashboard. Neither output is reliable on its own. The builder fills gaps in your request with defaults you did not choose, and the critic misreads values, describes elements that are not there, and offers generic advice that sounds right but does not address the dashboard’s purpose. In practice, the person responsible for a dashboard is the one who can tell which parts of the agent’s work hold up. This assignment practices that supervision. You will specify a dashboard, have an agent build it, critique it yourself, have a separate agent critique it, and then reconcile the two critiques: where the agent was right, where it was wrong, and how you know. The revision you make at the end has to follow from that reconciliation.

The Data

You will work from the U.S. Department of Education’s College Scorecard, institution-level data. The data has categorical, quantitative, and geographic fields, enrollment that spans several orders of magnitude, and substantial missingness. How your dashboard treats each is a decision you make.

Getting Started

We will create a starter repository for you under https://github.com/dataviscourse2026. If you can’t find your repository there, contact course staff as soon as possible. It contains:

index.html                 # your name, email, UID; links to your written work; v1 and v2 embedded
data/                      # contains the data you use
step1-brief.md
step2-v1/                  # the agent-built dashboard, build transcript, screenshots
step3-critique-own.md
step4-critique-agent/      # transcript and the exact inputs you gave the agent
step5-reconciliation.md
step6-v2/                  # the revised dashboard, screenshots
step6-revision.md
ai-use.md                  # AI use disclosure

Add your name, email, and UID at the top of index.html. It is the front page of your submission: it links to each step’s written work and embeds v1 and v2 in iframes, each with a link to open it in its own tab. Each markdown file has prompts in it marking what belongs in each section.

As in Homework 1, serve the repository rather than opening HTML files directly:

$ cd path/to/repo
$ python -m http.server 8000

Then view your submission, with both dashboards embedded, at http://localhost:8000/index.html.

Several steps below require a commit before the next step begins.

Steps

1. Write the brief

Before asking an agent for anything, write step1-brief.md. A brief is what you would hand a contractor; it should be specific enough that two different builders would produce recognizably similar dashboards.

  1. User. One specific person who will use this dashboard and what decision they are making. “A high school counselor in Utah advising first-generation students on in-state versus out-of-state options” is a user. “People interested in college” is not.
  2. Tasks. Exactly three questions the user needs to answer with the dashboard. Each must be answerable by looking at the dashboard, and at least one must require using two views together.
  3. Views. At least three coordinated views. For each: the marks, the encoding channels, and one line of perceptual justification for each channel, as in Homework 1.
  4. Interaction. What the user can select, filter, or brush, which views respond, and what a hover shows. At least one selection must update every other view.
  5. Data decisions. How you will treat different types of missing data, how you will handle enrollment’s range, and which rows (if any) you exclude and why.
  6. Toolchain. D3, Vega-Lite, or a mix, and one or two sentences on why, drawing on what you learned in Homework 1.
  7. Acceptance criteria. Four to six statements a person could check by looking at the finished dashboard, for example “a user can identify the three for-profit institutions with the highest median debt in Utah without reading a table.” These are what you will judge v1 and v2 against.

Commit step1-brief.md before any dashboard code exists in the repository. Do not edit it afterward; if your thinking changes, record that in step6-revision.md.

2. Build v1 with an agent

Give the agent your brief and the data, and have it build the dashboard in step2-v1/, with step2-v1/index.html as its entry point so that it loads in the iframe in index.html. You may iterate with it as much as you like, and you may edit code yourself. When v1 meets the brief as well as you can get it to, stop, and commit v1. Do not change the dashboard code in step2-v1/ after that commit.

Save the full build transcript to step2-v1/build-transcript.md. Then take screenshots of v1 into step2-v1/screenshots/: the default state, and one screenshot for each of your three tasks showing the dashboard in the state where that task’s answer is visible.

3. Critique v1 yourself

Before you show v1 to any agent for critique, write step3-critique-own.md. It has two parts.

Task walkthrough. For each of the three tasks in your brief, use v1 to answer it. Verify the answers are correct but looking at the original data. Note any task where the dashboard’s answer was wrong, slow to reach, or ambiguous. This is your ground truth for Step 5.

Critique. Assess v1 against your acceptance criteria and against the principles from lecture: expressiveness and effectiveness of the encodings, color, readability, and whether any view misrepresents the data. Order your points by how much each one would affect your user’s decision, not by how easy each is to fix. Cite a screenshot for each point.

Commit step3-critique-own.md before Step 4.

4. Have a separate agent critique v1

Start a fresh agent session with no access to the build session. Agents that built something tend to defend it. Give it:

  • your step1-brief.md,
  • the screenshots from step2-v1/screenshots/, and
  • the following requests, in your own words:
    1. Critique the dashboard against the brief’s acceptance criteria and against general visualization principles.
    2. Answer each of the three tasks by reading the screenshots. Report the specific values it reads.
    3. Describe each view: its marks, its channels, and what each channel encodes.

You may give it the code or a running copy as well, but record what you gave it. Save the full, unedited transcript to step4-critique-agent/transcript.md and copy every input you provided into step4-critique-agent/inputs/. If your agent lets you choose models, record which one.

5. Reconcile the two critiques

This is the core of the assignment. In step5-reconciliation.md, go through every claim the agent made in Step 4: each critique point, each value it read, and each view description. For each one, record:

  • The claim, quoted.
  • Verdict: correct, partly correct, wrong, or unverifiable.
  • Evidence: how you checked it. Point to a screenshot, a computed value from your Step 3 walkthrough, the code, or a principle from lecture. “I disagree” without evidence is not a verdict.
  • Kind of error, if it was wrong: misread value, element that does not exist, misidentified encoding, advice that ignores the brief’s user or tasks, or correct observation with the wrong priority.

Then write three short syntheses:

  1. Where the agent was wrong. Which kinds of error it made, and whether they follow a pattern (for example, reliable about color but unreliable at reading values from small marks). Expect to find some errors. If after checking every claim you find fewer than three, say so directly, and show the checks that confirmed the rest.
  2. What the agent caught that you missed. Compare against step3-critique-own.md. If it caught nothing new, say so.
  3. What you caught that the agent missed, and why you think it missed it.

6. Revise to v2

Copy the v1 dashboard to step6-v2/ and revise it. An agent may do the implementation. In step6-revision.md:

  • List each change. For each, cite the step5-reconciliation.md or step3-critique-own.md entry that motivated it, and include a before/after screenshot pair (step2-v1/screenshots/... and step6-v2/screenshots/...).
  • List at least any critique points you chose not to act on, and why: the point was wrong, out of scope for the user, or not worth its cost.
  • Repeat the task walkthrough from Step 3 on v2 and report whether each answer is now faster, clearer, or more accurate.
  • Note any place where your thinking now differs from step1-brief.md.

Make at least three substantive changes. Changing colors and font sizes alone does not count as three.

Deliverables

Your repository, containing:

  • index.html — name, email, UID, links to the written work, and v1 and v2 embedded as iframes
  • step1-brief.md — user, tasks, views, interaction, data decisions, toolchain, and acceptance criteria, committed before any dashboard code
  • step2-v1/ — the agent-built dashboard, frozen at the v1 frozen commit, with build-transcript.md and screenshots/
  • step3-critique-own.md — task walkthrough with ground truth and your critique, committed before the agent critique
  • step4-critique-agent/ — the full critique transcript and every input you gave the agent
  • step5-reconciliation.md — a verdict with evidence for every agent claim, and the three syntheses
  • step6-v2/ — the revised dashboard, with screenshots/
  • step6-revision.md — changes with sources and before/after pairs, points not acted on, and the v2 task walkthrough
  • ai-use.md — the AI use disclosure (see below)

Submission

Submit by committing and pushing to your GitHub repository. Your last commit before the deadline is your submission. There is no separate Canvas upload.

Grading

Every line below is graded from your written files, screenshots, running dashboards, and commit history. The TA will not read your code.

  • 15% — Brief: a specific user, three answerable tasks, encoding choices with perceptual justification, an interaction spec, explicit data decisions, and checkable acceptance criteria, all committed before any dashboard code (verified against commit history)
  • 10% — v1: runs, has the views and linked interaction specified in the brief, frozen at the v1 frozen commit with a build transcript
  • 15% — Own critique: task walkthrough with ground-truth values shown, and a critique tied to the acceptance criteria and to lecture principles, ordered by impact on the user, committed before the agent critique
  • 5% — Agent critique setup: separate session, complete unedited transcript, all inputs recorded, all three requests made
  • 30% — Reconciliation: every agent claim addressed; each verdict supported by specific evidence; errors classified; the three syntheses are specific and consistent with the entries above them
  • 15% — Revision: at least three substantive changes, each traced to a critique entry with a before/after pair; justified decisions not to act; v2 task walkthrough
  • 10% — AI use disclosure: specific about what each agent produced and what you kept, changed, or rejected, and consistent with the transcripts

AI Use

Agents are allowed and expected throughout this assignment, for building, for critique, and for computing ground-truth values. The written reasoning in step1-brief.md, step3-critique-own.md, step5-reconciliation.md, and step6-revision.md must be your own. You may use an agent to check facts or compute values for them, but not to write them.

In ai-use.md, for each step, state which agent and model you used, what it produced, and what you kept, changed, or rejected and why. Keep it short and specific. A disclosure that says only “I used an agent to build the dashboard” is not sufficient; one that says “the agent defaulted to a rainbow color scale for completion rate; I replaced it with a single-hue sequential scale because the field is ordered” is the level of detail expected.