# Experiment Data Export (CSV) Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Add a client-side CSV export on the experiment page that produces a date×subject matrix for any chosen daily parameter (including session metrics). **Architecture:** A pure, unit-tested helper module (`frontend/src/lib/dataExport.js`) enumerates exportable parameters, pivots fetched daily statuses into a date-row × subject-column matrix, and serializes to CSV. A modal component (`ExportDataModal.jsx`) fetches the full daily-statuses set on open, drives the picker, and triggers a browser download. `ExperimentDetail.jsx` gets an "Export data" button. No backend changes — the existing `GET /:id/daily-statuses` endpoint already returns every field needed. **Tech Stack:** React 18, Jest + @testing-library, existing `experimentsApi` axios client, existing `Modal`/`Button` UI components. --- ## Reference: data shapes (read before starting) **Daily template field** (`dailyTemplate` array items): ```js { fieldId: '0000…0002', key: 'vitals', label: 'Vitals', type: 'text', builtin: true, active: true, tags: [] } ``` - `builtin: true` → value stored at `status[field.key]` (e.g. `status.vitals`). - `builtin: false` → value stored at `status.custom_fields[field.fieldId]`. **Daily status** (from `experimentsApi.getDailyStatuses(id, {})`): ```js { id, animal_id, date: '2026-07-01T00:00:00.000Z', experiment_description, vitals, treatment, notes, // builtin columns custom_fields: { '': value, … } | null, analysis_summary: { total: 12, success_rate: 0.83, counts: { Success: 10, Failure: 2 } } | null, } ``` **Animal**: `{ id, animal_name, animal_id_string, subject_info, … }`. **Parameter descriptor** (produced by `listExportableParams`, consumed everywhere): ```js { id, label, group: 'metrics' | 'daily', kind: 'total'|'success_rate'|'category'|'builtin'|'custom', category?, statusKey?, fieldId? } ``` --- ## Task 1: `extractValue` — read one parameter's value from a status **Files:** - Create: `frontend/src/lib/dataExport.js` - Test: `frontend/tests/dataExport.test.js` - [ ] **Step 1: Write the failing test** Create `frontend/tests/dataExport.test.js`: ```js import { extractValue } from '../src/lib/dataExport'; const status = { vitals: 'ok', custom_fields: { 'f-weight': 250 }, analysis_summary: { total: 12, success_rate: 0.83, counts: { Success: 10, Failure: 2 } }, }; describe('extractValue', () => { it('reads a builtin field via statusKey', () => { expect(extractValue(status, { kind: 'builtin', statusKey: 'vitals' })).toBe('ok'); }); it('reads a custom field via fieldId', () => { expect(extractValue(status, { kind: 'custom', fieldId: 'f-weight' })).toBe(250); }); it('reads session-metric total', () => { expect(extractValue(status, { kind: 'total' })).toBe(12); }); it('reads session-metric success_rate as a raw 0-1 fraction', () => { expect(extractValue(status, { kind: 'success_rate' })).toBe(0.83); }); it('reads a dynamic count category', () => { expect(extractValue(status, { kind: 'category', category: 'Failure' })).toBe(2); }); it('returns null when analysis_summary is absent', () => { expect(extractValue({ custom_fields: {} }, { kind: 'total' })).toBeNull(); expect(extractValue({}, { kind: 'category', category: 'Success' })).toBeNull(); }); it('returns null for a missing custom field', () => { expect(extractValue({ custom_fields: {} }, { kind: 'custom', fieldId: 'nope' })).toBeNull(); expect(extractValue({}, { kind: 'builtin', statusKey: 'notes' })).toBeNull(); }); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `cd frontend && npx jest tests/dataExport.test.js -t extractValue` Expected: FAIL — "extractValue is not a function" / cannot import. - [ ] **Step 3: Write minimal implementation** Create `frontend/src/lib/dataExport.js`: ```js // Pure helpers for exporting daily-parameter data as a date×subject CSV matrix. // No React, no I/O — mirrors the crossSubjectChart.js pure-helper pattern. // Read a single parameter's raw value from one daily status. // Returns null when the value is absent. Note: 0 and '' are returned as-is // (real values); only genuinely-missing lookups become null. export function extractValue(status, param) { if (!status || !param) return null; switch (param.kind) { case 'total': return status.analysis_summary?.total ?? null; case 'success_rate': return status.analysis_summary?.success_rate ?? null; case 'category': return status.analysis_summary?.counts?.[param.category] ?? null; case 'builtin': return status[param.statusKey] ?? null; case 'custom': return status.custom_fields?.[param.fieldId] ?? null; default: return null; } } ``` - [ ] **Step 4: Run test to verify it passes** Run: `cd frontend && npx jest tests/dataExport.test.js -t extractValue` Expected: PASS (7 assertions). - [ ] **Step 5: Commit** ```bash git add frontend/src/lib/dataExport.js frontend/tests/dataExport.test.js git commit -m "feat(frontend): extractValue helper for data export" ``` --- ## Task 2: `listExportableParams` — enumerate the picker options **Files:** - Modify: `frontend/src/lib/dataExport.js` - Test: `frontend/tests/dataExport.test.js` - [ ] **Step 1: Write the failing test** Append to `frontend/tests/dataExport.test.js`: ```js import { listExportableParams } from '../src/lib/dataExport'; const dailyTemplate = [ { fieldId: 'b-vitals', key: 'vitals', label: 'Vitals', builtin: true, active: true }, { fieldId: 'c-weight', key: 'weight', label: 'Weight (g)', builtin: false, active: false }, ]; const statuses = [ { analysis_summary: { counts: { Success: 1, Failure: 0 } } }, { analysis_summary: { counts: { Success: 2, Other: 1 } } }, { analysis_summary: null }, ]; describe('listExportableParams', () => { const params = listExportableParams(dailyTemplate, statuses); it('starts with the two fixed session metrics', () => { expect(params.slice(0, 2)).toEqual([ expect.objectContaining({ id: '__total__', label: 'Total attempts', group: 'metrics', kind: 'total' }), expect.objectContaining({ id: '__success_rate__', label: 'Success rate', group: 'metrics', kind: 'success_rate' }), ]); }); it('adds one metrics entry per dynamic count category, sorted', () => { const cats = params.filter((p) => p.kind === 'category').map((p) => p.label); expect(cats).toEqual(['Failure', 'Other', 'Success']); }); it('includes every template field (incl. inactive) in the daily group', () => { const daily = params.filter((p) => p.group === 'daily'); expect(daily).toEqual([ expect.objectContaining({ label: 'Vitals', kind: 'builtin', statusKey: 'vitals' }), expect.objectContaining({ label: 'Weight (g)', kind: 'custom', fieldId: 'c-weight' }), ]); }); it('gives every param a unique id', () => { const ids = params.map((p) => p.id); expect(new Set(ids).size).toBe(ids.length); }); it('tolerates empty/missing inputs', () => { expect(listExportableParams(undefined, undefined).length).toBe(2); // just the two fixed metrics }); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `cd frontend && npx jest tests/dataExport.test.js -t listExportableParams` Expected: FAIL — "listExportableParams is not a function". - [ ] **Step 3: Write minimal implementation** Append to `frontend/src/lib/dataExport.js`: ```js // Build the grouped, ordered list of exportable parameters. // Session metrics (total, success_rate, then each dynamic count category found // in the data) come first; then every field defined in the daily template, // including inactive ones (historical data may exist). export function listExportableParams(dailyTemplate, statuses) { const params = [ { id: '__total__', label: 'Total attempts', group: 'metrics', kind: 'total' }, { id: '__success_rate__', label: 'Success rate', group: 'metrics', kind: 'success_rate' }, ]; const cats = new Set(); for (const s of statuses ?? []) { const counts = s?.analysis_summary?.counts; if (counts && typeof counts === 'object') { for (const k of Object.keys(counts)) cats.add(k); } } for (const cat of [...cats].sort()) { params.push({ id: `__cat__${cat}`, label: cat, group: 'metrics', kind: 'category', category: cat }); } for (const f of dailyTemplate ?? []) { if (f.builtin) { params.push({ id: `b:${f.fieldId}`, label: f.label, group: 'daily', kind: 'builtin', statusKey: f.key }); } else { params.push({ id: `c:${f.fieldId}`, label: f.label, group: 'daily', kind: 'custom', fieldId: f.fieldId }); } } return params; } ``` - [ ] **Step 4: Run test to verify it passes** Run: `cd frontend && npx jest tests/dataExport.test.js -t listExportableParams` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add frontend/src/lib/dataExport.js frontend/tests/dataExport.test.js git commit -m "feat(frontend): listExportableParams enumerates export options" ``` --- ## Task 3: `buildMatrix` — pivot statuses into date rows × subject columns **Files:** - Modify: `frontend/src/lib/dataExport.js` - Test: `frontend/tests/dataExport.test.js` - [ ] **Step 1: Write the failing test** Append to `frontend/tests/dataExport.test.js`: ```js import { buildMatrix } from '../src/lib/dataExport'; const animals = [ { id: 'a2', animal_name: 'Beta', animal_id_string: 'R-002' }, { id: 'a1', animal_name: 'Alpha', animal_id_string: 'R-001' }, ]; const totalParam = { kind: 'total' }; describe('buildMatrix', () => { it('orders columns by animal_name and rows by date ascending', () => { const statuses = [ { animal_id: 'a1', date: '2026-07-02T00:00:00.000Z', analysis_summary: { total: 5 } }, { animal_id: 'a2', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 9 } }, ]; const m = buildMatrix(statuses, animals, totalParam); expect(m.columns.map((c) => c.header)).toEqual(['Alpha', 'Beta']); expect(m.rows.map((r) => r.date)).toEqual(['2026-07-01', '2026-07-02']); // Row for 07-01: Alpha blank, Beta 9 expect(m.rows[0].values).toEqual(['', 9]); // Row for 07-02: Alpha 5, Beta blank expect(m.rows[1].values).toEqual([5, '']); }); it('includes only dates that have at least one value (0 counts as a value)', () => { const statuses = [ { animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 0 } }, { animal_id: 'a1', date: '2026-07-03T00:00:00.000Z', analysis_summary: null }, // no value ]; const m = buildMatrix(statuses, animals, totalParam); expect(m.rows.map((r) => r.date)).toEqual(['2026-07-01']); expect(m.rows[0].values).toEqual([0, '']); }); it('disambiguates duplicate animal names with the id string', () => { const dupAnimals = [ { id: 'a1', animal_name: 'Rat', animal_id_string: 'R-001' }, { id: 'a2', animal_name: 'Rat', animal_id_string: 'R-002' }, ]; const statuses = [{ animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 1 } }]; const m = buildMatrix(statuses, dupAnimals, totalParam); expect(m.columns.map((c) => c.header)).toEqual(['Rat (R-001)', 'Rat (R-002)']); }); it('keeps the first status when a subject has duplicates on one date', () => { const statuses = [ { animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 7 } }, { animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 99 } }, ]; const m = buildMatrix(statuses, animals, totalParam); expect(m.rows[0].values).toEqual([7, '']); }); it('returns empty rows when no status has a value', () => { const m = buildMatrix([{ animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: null }], animals, totalParam); expect(m.rows).toEqual([]); expect(m.columns.length).toBe(2); }); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `cd frontend && npx jest tests/dataExport.test.js -t buildMatrix` Expected: FAIL — "buildMatrix is not a function". - [ ] **Step 3: Write minimal implementation** Append to `frontend/src/lib/dataExport.js`: ```js // A cell has a value unless it is null, undefined, or the empty string. // (0 and false are real values.) function hasValue(v) { return v !== null && v !== undefined && v !== ''; } function dateKey(date) { return String(date).slice(0, 10); // ISO datetime or date → YYYY-MM-DD } // Pivot statuses into { columns, rows }. // columns: [{ id, header }] — all animals, ordered by name, headers // disambiguated with animal_id_string on duplicate names. // rows: [{ date, values }] — one per date that has ≥1 value; values are // aligned to columns, blank cells are '' (empty string). export function buildMatrix(statuses, animals, param) { const sortedAnimals = [...(animals ?? [])].sort((a, b) => String(a.animal_name ?? '').localeCompare(String(b.animal_name ?? '')), ); const nameCounts = new Map(); for (const a of sortedAnimals) { const n = a.animal_name ?? ''; nameCounts.set(n, (nameCounts.get(n) ?? 0) + 1); } const columns = sortedAnimals.map((a) => { const name = a.animal_name ?? ''; const header = nameCounts.get(name) > 1 ? `${name} (${a.animal_id_string ?? ''})` : name; return { id: a.id, header }; }); const colIndex = new Map(columns.map((c, i) => [c.id, i])); // cells: dateKey -> Map(animalId -> value); first status per (date, animal) wins. const cells = new Map(); const datesWithData = new Set(); const ordered = [...(statuses ?? [])].sort((a, b) => dateKey(a.date).localeCompare(dateKey(b.date))); for (const s of ordered) { if (!colIndex.has(s.animal_id)) continue; const dk = dateKey(s.date); if (!cells.has(dk)) cells.set(dk, new Map()); const byAnimal = cells.get(dk); if (byAnimal.has(s.animal_id)) continue; // first wins const v = extractValue(s, param); byAnimal.set(s.animal_id, v); if (hasValue(v)) datesWithData.add(dk); } const rows = [...datesWithData].sort().map((dk) => { const byAnimal = cells.get(dk); const values = columns.map((c) => { const v = byAnimal.has(c.id) ? byAnimal.get(c.id) : null; return hasValue(v) ? v : ''; }); return { date: dk, values }; }); return { columns, rows }; } ``` - [ ] **Step 4: Run test to verify it passes** Run: `cd frontend && npx jest tests/dataExport.test.js -t buildMatrix` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add frontend/src/lib/dataExport.js frontend/tests/dataExport.test.js git commit -m "feat(frontend): buildMatrix pivots statuses into date×subject grid" ``` --- ## Task 4: `toCSV` + `csvFilename` — serialize and name the file **Files:** - Modify: `frontend/src/lib/dataExport.js` - Test: `frontend/tests/dataExport.test.js` - [ ] **Step 1: Write the failing test** Append to `frontend/tests/dataExport.test.js`: ```js import { toCSV, csvFilename } from '../src/lib/dataExport'; describe('toCSV', () => { it('writes a header row of Date + column headers, then data rows', () => { const matrix = { columns: [{ id: 'a1', header: 'Alpha' }, { id: 'a2', header: 'Beta' }], rows: [{ date: '2026-07-01', values: [5, ''] }, { date: '2026-07-02', values: ['', 9] }], }; expect(toCSV(matrix)).toBe('Date,Alpha,Beta\n2026-07-01,5,\n2026-07-02,,9'); }); it('escapes commas, quotes, and newlines per RFC 4180', () => { const matrix = { columns: [{ id: 'a1', header: 'Note, field' }], rows: [{ date: '2026-07-01', values: ['he said "hi"'] }, { date: '2026-07-02', values: ['line1\nline2'] }], }; expect(toCSV(matrix)).toBe( 'Date,"Note, field"\n2026-07-01,"he said ""hi"""\n2026-07-02,"line1\nline2"', ); }); }); describe('csvFilename', () => { it('slugifies the experiment title and parameter label', () => { expect(csvFilename('My Study #1', 'Success rate')).toBe('My-Study-1-Success-rate.csv'); }); it('falls back to "export" when both slugs are empty', () => { expect(csvFilename('', '')).toBe('export.csv'); }); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `cd frontend && npx jest tests/dataExport.test.js -t "toCSV|csvFilename"` Expected: FAIL — "toCSV is not a function". - [ ] **Step 3: Write minimal implementation** Append to `frontend/src/lib/dataExport.js`: ```js function escapeCsvCell(value) { const s = value === null || value === undefined ? '' : String(value); return /[",\n]/.test(s) ? `"${s.replace(/"/g, '""')}"` : s; } // Serialize a matrix to an RFC-4180-ish CSV string (LF line endings). // First column is the date; remaining columns follow matrix.columns order. export function toCSV(matrix) { const header = ['Date', ...matrix.columns.map((c) => c.header)].map(escapeCsvCell).join(','); const lines = matrix.rows.map((r) => [r.date, ...r.values].map(escapeCsvCell).join(',')); return [header, ...lines].join('\n'); } // Build a download filename from the experiment title and parameter label. export function csvFilename(title, label) { const slug = (s) => String(s ?? '').replace(/[^a-z0-9]+/gi, '-').replace(/^-+|-+$/g, ''); const base = [slug(title), slug(label)].filter(Boolean).join('-'); return `${base || 'export'}.csv`; } ``` - [ ] **Step 4: Run test to verify it passes** Run: `cd frontend && npx jest tests/dataExport.test.js` Expected: PASS (all describe blocks in the file). - [ ] **Step 5: Commit** ```bash git add frontend/src/lib/dataExport.js frontend/tests/dataExport.test.js git commit -m "feat(frontend): toCSV + csvFilename for data export" ``` --- ## Task 5: `ExportDataModal` component **Files:** - Create: `frontend/src/components/ExportDataModal.jsx` This component is rendered as the *content* of the shared `Modal` (same pattern as `AnimalForm`/`TemplateEditor`), so it mounts only when the modal opens — the fetch runs on mount. - [ ] **Step 1: Create the component** Create `frontend/src/components/ExportDataModal.jsx`: ```jsx import React, { useEffect, useMemo, useState } from 'react'; import { experimentsApi } from '../api/client'; import Button from './ui/Button'; import Alert from './ui/Alert'; import { listExportableParams, buildMatrix, toCSV, csvFilename } from '../lib/dataExport'; const GROUP_LABELS = { metrics: 'Session metrics', daily: 'Daily record fields' }; function triggerDownload(filename, text) { const blob = new Blob([text], { type: 'text/csv;charset=utf-8;' }); const url = URL.createObjectURL(blob); const a = document.createElement('a'); a.href = url; a.download = filename; document.body.appendChild(a); a.click(); document.body.removeChild(a); URL.revokeObjectURL(url); } export default function ExportDataModal({ experimentTitle, experimentId, dailyTemplate, animals, onClose }) { const [loading, setLoading] = useState(true); const [error, setError] = useState(null); const [statuses, setStatuses] = useState([]); const [selectedId, setSelectedId] = useState(null); useEffect(() => { let alive = true; setLoading(true); setError(null); experimentsApi.getDailyStatuses(experimentId, {}) .then((data) => { if (alive) setStatuses(data); }) .catch((err) => { if (alive) setError(err.message); }) .finally(() => { if (alive) setLoading(false); }); return () => { alive = false; }; }, [experimentId]); const params = useMemo(() => listExportableParams(dailyTemplate, statuses), [dailyTemplate, statuses]); // Default the selection to the first parameter once params are available. useEffect(() => { if (selectedId === null && params.length > 0) setSelectedId(params[0].id); }, [params, selectedId]); const selectedParam = params.find((p) => p.id === selectedId) ?? null; const matrix = useMemo( () => (selectedParam ? buildMatrix(statuses, animals, selectedParam) : { columns: [], rows: [] }), [statuses, animals, selectedParam], ); const hasData = matrix.rows.length > 0; // Group params for rendering, preserving order. const grouped = useMemo(() => { const out = []; for (const p of params) { let g = out.find((x) => x.group === p.group); if (!g) { g = { group: p.group, items: [] }; out.push(g); } g.items.push(p); } return out; }, [params]); function handleExport() { if (!hasData || !selectedParam) return; triggerDownload(csvFilename(experimentTitle, selectedParam.label), toCSV(matrix)); onClose(); } if (loading) return

Loading data…

; if (error) return ; return (

Export one parameter as a CSV file — each row is a date, each column is a subject.

{hasData ? `${matrix.rows.length} date${matrix.rows.length !== 1 ? 's' : ''} × ${matrix.columns.length} subject${matrix.columns.length !== 1 ? 's' : ''}` : 'No data for this parameter yet.'}

); } ``` - [ ] **Step 2: Verify it compiles (build)** Run: `cd frontend && npx vite build` Expected: build succeeds with no import/JSX errors. (There is no `ExportDataModal` test — its logic lives in the unit-tested `dataExport.js`; the DOM download path is verified manually in Task 7.) - [ ] **Step 3: Commit** ```bash git add frontend/src/components/ExportDataModal.jsx git commit -m "feat(frontend): ExportDataModal — pick a parameter and download CSV" ``` --- ## Task 6: Wire the "Export data" button into the experiment page **Files:** - Modify: `frontend/src/pages/ExperimentDetail.jsx` - [ ] **Step 1: Import the modal** In `frontend/src/pages/ExperimentDetail.jsx`, add to the imports near the other component imports (after the `ExperimentCalendar` import on line 11): ```jsx import ExportDataModal from '../components/ExportDataModal'; ``` - [ ] **Step 2: Add modal open/close state** Immediately after the `const [showSubjectTemplate, setShowSubjectTemplate] = useState(false);` line (around line 64), add: ```jsx const [showExport, setShowExport] = useState(false); ``` - [ ] **Step 3: Add the header button** In the header actions area, replace the single Add-Animal button block: ```jsx ``` with: ```jsx
``` - [ ] **Step 4: Render the modal** Just before the closing `` and the final Edit-Animal `Modal` (after the `Edit Animal` modal block, around line 406), add: ```jsx setShowExport(false)} title="Export data" size="md"> setShowExport(false)} /> ``` - [ ] **Step 5: Build to verify wiring** Run: `cd frontend && npx vite build` Expected: build succeeds. - [ ] **Step 6: Commit** ```bash git add frontend/src/pages/ExperimentDetail.jsx git commit -m "feat(frontend): Export data button + modal on experiment page" ``` --- ## Task 7: Full-suite verification & manual smoke test **Files:** none (verification only) - [ ] **Step 1: Run the whole frontend unit suite** Run: `cd frontend && npm test` Expected: all suites PASS, including `tests/dataExport.test.js`. - [ ] **Step 2: Manual smoke test** Bring up the dev stack (`docker-compose -f docker-compose.dev.yml up` or the project's usual dev command), open an experiment that has animals and some daily statuses, then: - Click **Export data** → modal opens, parameter dropdown shows a **Session metrics** group (Total attempts, Success rate, any count categories) and a **Daily record fields** group (all template fields). - The summary line shows a plausible "N dates × M subjects". - Pick a session metric and a daily field; confirm the summary updates and **Export CSV** downloads a file. - Open the CSV: first column `Date` (YYYY-MM-DD rows ascending), one column per subject (by name), correct values, blank cells where a subject had no value, and **success rate as a raw 0–1 fraction**. - Pick a parameter with no data → summary reads "No data for this parameter yet." and **Export CSV** is disabled. - [ ] **Step 3: Final commit (if any manual fixes were needed)** ```bash git add -A git commit -m "chore(frontend): finalize experiment data export" ``` --- ## Self-review notes (for the implementer) - The whole feature is client-side; **no backend files change**. If a test asks you to touch `backend/`, stop — the plan is wrong. - `success_rate` is exported **raw (0–1)**, deliberately not multiplied to a percentage. - All daily template fields are exportable, **including inactive ones** — this is intentional (historical data). - Parameter descriptor property names (`kind`, `statusKey`, `fieldId`, `category`, `group`, `id`, `label`) must stay identical across `listExportableParams`, `extractValue`, `buildMatrix`, and `ExportDataModal`.