diff --git a/docs/superpowers/plans/2026-07-19-configurable-data-export.md b/docs/superpowers/plans/2026-07-19-configurable-data-export.md new file mode 100644 index 0000000..687ffd5 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-configurable-data-export.md @@ -0,0 +1,926 @@ +# Configurable Data Export Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Let the export modal assign any of Date/Subject/Parameter to rows and columns, pin the third dimension (shown as a top context line), and emit each subject's group in a separate cell when Subject is an axis. + +**Architecture:** Generalize the pure helpers in `frontend/src/lib/dataExport.js` around three interchangeable dimensions (date, subject, parameter). Each dimension produces an ordered list of "members"; `buildMatrix` resolves every cell's `(dateKey, animalId, param)` triple through a status index. `ExportDataModal.jsx` becomes glue over these helpers with four controls. `ExperimentDetail.jsx` passes two new props. + +**Tech Stack:** React 18, plain ES modules, Jest + `@testing-library/react` (jsdom). No new dependencies. + +**Spec:** `docs/superpowers/specs/2026-07-19-configurable-data-export-design.md` + +## Global Constraints + +- `dataExport.js` stays pure: no React, no I/O (mirrors `crossSubjectChart.js`). +- Cell value semantics: `0` and `''` are **real** values; only genuinely-missing lookups render as blank (`''`). Use the existing `hasValue` helper. +- CSV: LF (`\n`) line endings; escape every cell via the existing `escapeCsvCell`. +- Empty-member rule: **Subject members are always kept**; **Date and Parameter members are dropped when entirely blank** in the grid. +- Group field ids: `__none__` (None), `__name__` (Name), `__id__` (Subject ID), otherwise a `subject_info` field id. Missing group value renders as `—`. +- Default modal state must reproduce today's export exactly: Rows = Date, Columns = Subject, Fixed = Parameter (first parameter). +- Context line is **beginning only** (no footer); two cells: pinned dimension label, pinned value. +- Run tests from `frontend/`. Single file: `npx jest tests/dataExport.test.js`. Component file: `npx jest tests/ExportDataModal.test.jsx`. + +--- + +### Task 1: Dimension primitives + date members + status index + +**Files:** +- Modify: `frontend/src/lib/dataExport.js` (add exports; keep existing `extractValue`, `listExportableParams`, `hasValue`, `dateKey`, `escapeCsvCell`, `csvFilename`) +- Test: `frontend/tests/dataExport.test.js` + +**Interfaces:** +- Consumes: existing `dateKey(date)` (private), `hasValue(v)` (private) in `dataExport.js`. +- Produces: + - `EXPORT_DIMENSIONS: Array<{ dim: 'date'|'subject'|'parameter', label: string }>` + - `dimLabel(dim: string): string` + - `otherDimension(rowDim: string, colDim: string): string` — the leftover dimension + - `subjectGroupValue(animal: object, groupField: string): string|null` — `null` when `groupField` is falsy/`__none__`; `—` when the value is missing/empty + - `listDateMembers(statuses: array): Array<{ id, label, dateKey }>` — sorted unique dateKeys + - `buildStatusIndex(statuses: array): Map>` — `dateKey → animalId → status`, first-wins + +- [ ] **Step 1: Write the failing tests** + +Append to `frontend/tests/dataExport.test.js`: + +```js +import { + EXPORT_DIMENSIONS, dimLabel, otherDimension, subjectGroupValue, + listDateMembers, buildStatusIndex, +} from '../src/lib/dataExport'; + +describe('dimension primitives', () => { + it('exposes the three dimensions with labels', () => { + expect(EXPORT_DIMENSIONS.map((d) => d.dim)).toEqual(['date', 'subject', 'parameter']); + expect(dimLabel('date')).toBe('Date'); + expect(dimLabel('subject')).toBe('Subject'); + expect(dimLabel('parameter')).toBe('Parameter'); + }); + it('otherDimension returns the leftover', () => { + expect(otherDimension('date', 'subject')).toBe('parameter'); + expect(otherDimension('parameter', 'date')).toBe('subject'); + }); +}); + +describe('subjectGroupValue', () => { + const animal = { animal_name: 'Alpha', animal_id_string: 'R-001', subject_info: { sex: 'M', cohort: '' } }; + it('returns null when no group field', () => { + expect(subjectGroupValue(animal, '__none__')).toBeNull(); + expect(subjectGroupValue(animal, '')).toBeNull(); + }); + it('resolves name / id / subject_info fields', () => { + expect(subjectGroupValue(animal, '__name__')).toBe('Alpha'); + expect(subjectGroupValue(animal, '__id__')).toBe('R-001'); + expect(subjectGroupValue(animal, 'sex')).toBe('M'); + }); + it('renders missing or empty values as an em dash', () => { + expect(subjectGroupValue(animal, 'cohort')).toBe('—'); + expect(subjectGroupValue({}, 'sex')).toBe('—'); + }); +}); + +describe('listDateMembers', () => { + it('returns sorted unique dateKeys with id/label', () => { + const statuses = [ + { date: '2026-07-02T00:00:00.000Z' }, + { date: '2026-07-01T12:00:00.000Z' }, + { date: '2026-07-02T09:00:00.000Z' }, + ]; + expect(listDateMembers(statuses)).toEqual([ + { id: '2026-07-01', label: '2026-07-01', dateKey: '2026-07-01' }, + { id: '2026-07-02', label: '2026-07-02', dateKey: '2026-07-02' }, + ]); + }); + it('tolerates empty input', () => { + expect(listDateMembers(undefined)).toEqual([]); + }); +}); + +describe('buildStatusIndex', () => { + it('indexes by dateKey then animalId, first status wins', () => { + const statuses = [ + { animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 7 } }, + { animal_id: 'a1', date: '2026-07-01T06:00:00.000Z', analysis_summary: { total: 99 } }, + { animal_id: 'a2', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 3 } }, + ]; + const idx = buildStatusIndex(statuses); + expect(idx.get('2026-07-01').get('a1').analysis_summary.total).toBe(7); + expect(idx.get('2026-07-01').get('a2').analysis_summary.total).toBe(3); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `npx jest tests/dataExport.test.js -t "dimension primitives"` +Expected: FAIL — `EXPORT_DIMENSIONS is not defined` / imports undefined. + +- [ ] **Step 3: Write minimal implementation** + +Add to `frontend/src/lib/dataExport.js` (after the existing `listExportableParams`, before `hasValue`): + +```js +export const EXPORT_DIMENSIONS = [ + { dim: 'date', label: 'Date' }, + { dim: 'subject', label: 'Subject' }, + { dim: 'parameter', label: 'Parameter' }, +]; + +export function dimLabel(dim) { + return EXPORT_DIMENSIONS.find((d) => d.dim === dim)?.label ?? ''; +} + +export function otherDimension(rowDim, colDim) { + return EXPORT_DIMENSIONS.map((d) => d.dim).find((d) => d !== rowDim && d !== colDim) ?? null; +} + +// Resolve a subject's group value for a given group field id. +// null -> no grouping (field is falsy or '__none__') +// '—' -> grouping is on but this subject has no value +export function subjectGroupValue(animal, groupField) { + if (!groupField || groupField === '__none__') return null; + if (groupField === '__name__') return animal?.animal_name ?? '—'; + if (groupField === '__id__') return animal?.animal_id_string ?? '—'; + const v = animal?.subject_info?.[groupField]; + return v === null || v === undefined || v === '' ? '—' : v; +} + +export function listDateMembers(statuses) { + const keys = new Set(); + for (const s of statuses ?? []) { + if (s?.date) keys.add(dateKey(s.date)); + } + return [...keys].sort().map((k) => ({ id: k, label: k, dateKey: k })); +} + +// dateKey -> Map(animalId -> status); first status per (date, animal) wins, +// scanning in date order (matches the legacy buildMatrix tie-break). +export function buildStatusIndex(statuses) { + const index = new Map(); + const ordered = [...(statuses ?? [])].sort((a, b) => dateKey(a?.date).localeCompare(dateKey(b?.date))); + for (const s of ordered) { + if (!s?.date || s.animal_id === null || s.animal_id === undefined) continue; + const dk = dateKey(s.date); + if (!index.has(dk)) index.set(dk, new Map()); + const byAnimal = index.get(dk); + if (!byAnimal.has(s.animal_id)) byAnimal.set(s.animal_id, s); + } + return index; +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `npx jest tests/dataExport.test.js` +Expected: PASS (new suites green, all pre-existing suites still green). + +- [ ] **Step 5: Commit** + +```bash +git add frontend/src/lib/dataExport.js frontend/tests/dataExport.test.js +git commit -m "feat(export): dimension primitives, date members, status index" +``` + +--- + +### Task 2: Subject + parameter members, dimension dispatcher + +**Files:** +- Modify: `frontend/src/lib/dataExport.js` +- Test: `frontend/tests/dataExport.test.js` + +**Interfaces:** +- Consumes: `subjectGroupValue`, `listDateMembers`, `listExportableParams` (existing). +- Produces: + - `listSubjectMembers(animals, groupField): Array<{ id, label, animalId, group }>` — headers disambiguated by `animal_id_string` on duplicate names; clustered by group value then sorted by name/header when grouping is on, else name/header only. + - `listParameterMembers(dailyTemplate, statuses): Array<{ id, label, param, group }>` — wraps `listExportableParams`. + - `listDimensionMembers(dim, { statuses, animals, dailyTemplate, groupField }): Array` + +- [ ] **Step 1: Write the failing tests** + +Append to `frontend/tests/dataExport.test.js`: + +```js +import { listSubjectMembers, listParameterMembers, listDimensionMembers } from '../src/lib/dataExport'; + +describe('listSubjectMembers', () => { + const animals = [ + { id: 'a2', animal_name: 'Beta', animal_id_string: 'R-002', subject_info: { grp: 'Drug' } }, + { id: 'a1', animal_name: 'Alpha', animal_id_string: 'R-001', subject_info: { grp: 'Control' } }, + { id: 'a3', animal_name: 'Gamma', animal_id_string: 'R-003', subject_info: { grp: 'Control' } }, + ]; + it('orders by name and carries group=null when no group field', () => { + const m = listSubjectMembers(animals, '__none__'); + expect(m.map((s) => s.label)).toEqual(['Alpha', 'Beta', 'Gamma']); + expect(m.every((s) => s.group === null)).toBe(true); + expect(m[0]).toMatchObject({ id: 'a1', animalId: 'a1' }); + }); + it('clusters by group value then name when grouping is on', () => { + const m = listSubjectMembers(animals, 'grp'); + expect(m.map((s) => [s.group, s.label])).toEqual([ + ['Control', 'Alpha'], ['Control', 'Gamma'], ['Drug', 'Beta'], + ]); + }); + it('disambiguates duplicate names with the id string', () => { + const dup = [ + { id: 'a1', animal_name: 'Rat', animal_id_string: 'R-001' }, + { id: 'a2', animal_name: 'Rat', animal_id_string: 'R-002' }, + ]; + expect(listSubjectMembers(dup, '__none__').map((s) => s.label)).toEqual(['Rat (R-001)', 'Rat (R-002)']); + }); +}); + +describe('listParameterMembers / listDimensionMembers', () => { + const dailyTemplate = [{ fieldId: 'b-vitals', key: 'vitals', label: 'Vitals', builtin: true, active: true }]; + const statuses = [{ date: '2026-07-01T00:00:00.000Z', animal_id: 'a1', analysis_summary: { total: 1 } }]; + const animals = [{ id: 'a1', animal_name: 'Alpha', animal_id_string: 'R-001' }]; + it('wraps params with id/label/param/group', () => { + const m = listParameterMembers(dailyTemplate, statuses); + expect(m[0]).toMatchObject({ id: '__total__', label: 'Total attempts', group: 'metrics' }); + expect(m[0].param).toMatchObject({ kind: 'total' }); + }); + it('dispatches by dimension', () => { + const ctx = { statuses, animals, dailyTemplate, groupField: '__none__' }; + expect(listDimensionMembers('date', ctx).map((d) => d.id)).toEqual(['2026-07-01']); + expect(listDimensionMembers('subject', ctx).map((d) => d.id)).toEqual(['a1']); + expect(listDimensionMembers('parameter', ctx)[0].id).toBe('__total__'); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `npx jest tests/dataExport.test.js -t "listSubjectMembers"` +Expected: FAIL — `listSubjectMembers is not defined`. + +- [ ] **Step 3: Write minimal implementation** + +Add to `frontend/src/lib/dataExport.js` (after `buildStatusIndex`): + +```js +export function listSubjectMembers(animals, groupField) { + const list = [...(animals ?? [])]; + const nameCounts = new Map(); + for (const a of list) { + const n = a?.animal_name ?? ''; + nameCounts.set(n, (nameCounts.get(n) ?? 0) + 1); + } + const members = list.map((a) => { + const name = a?.animal_name ?? ''; + const label = nameCounts.get(name) > 1 ? `${name} (${a?.animal_id_string ?? ''})` : name; + return { id: a.id, label, animalId: a.id, group: subjectGroupValue(a, groupField) }; + }); + const grouped = !!groupField && groupField !== '__none__'; + members.sort((x, y) => { + if (grouped) { + const g = String(x.group ?? '').localeCompare(String(y.group ?? '')); + if (g !== 0) return g; + } + return String(x.label).localeCompare(String(y.label)); + }); + return members; +} + +export function listParameterMembers(dailyTemplate, statuses) { + return listExportableParams(dailyTemplate, statuses).map((p) => ({ + id: p.id, label: p.label, param: p, group: p.group, + })); +} + +export function listDimensionMembers(dim, { statuses, animals, dailyTemplate, groupField }) { + if (dim === 'date') return listDateMembers(statuses); + if (dim === 'subject') return listSubjectMembers(animals, groupField); + if (dim === 'parameter') return listParameterMembers(dailyTemplate, statuses); + return []; +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `npx jest tests/dataExport.test.js` +Expected: PASS (all suites green). + +- [ ] **Step 5: Commit** + +```bash +git add frontend/src/lib/dataExport.js frontend/tests/dataExport.test.js +git commit -m "feat(export): subject/parameter members + dimension dispatcher" +``` + +--- + +### Task 3: Generalized buildMatrix + +**Files:** +- Modify: `frontend/src/lib/dataExport.js` (replace the existing `buildMatrix`) +- Test: `frontend/tests/dataExport.test.js` (replace the existing `describe('buildMatrix', ...)` block) + +**Interfaces:** +- Consumes: `buildStatusIndex`, `listDimensionMembers`, `otherDimension`, `dimLabel`, `extractValue`, `hasValue`. +- Produces: + `buildMatrix(config, ctx)` where + - `config = { rowDim, colDim, pinnedMember, groupField }` + - `ctx = { statuses, animals, dailyTemplate }` + - returns `{ rowDim, colDim, pinnedDim, corner: string, context: { label, value }, groupAxis: 'row'|'col'|null, columns: Array<{ id, label, group }>, rows: Array<{ member: { id, label, group }, values: Array }> }` +- Empty-member rule: subject members always kept; date/parameter rows and columns dropped when entirely blank. +- `groupAxis`: `'row'` when `rowDim === 'subject'` and grouping on; `'col'` when `colDim === 'subject'` and grouping on; else `null`. + +- [ ] **Step 1: Replace the failing tests** + +In `frontend/tests/dataExport.test.js`, **delete the entire existing `describe('buildMatrix', ...)` block** (the one using the old `buildMatrix(statuses, animals, totalParam)` signature) and the `const totalParam = ...` line, then add: + +```js +const gAnimals = [ + { id: 'a2', animal_name: 'Beta', animal_id_string: 'R-002', subject_info: { grp: 'Drug' } }, + { id: 'a1', animal_name: 'Alpha', animal_id_string: 'R-001', subject_info: { grp: 'Control' } }, +]; +const gStatuses = [ + { animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 5, success_rate: 0.5 } }, + { animal_id: 'a2', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 9, success_rate: 0.9 } }, + { animal_id: 'a1', date: '2026-07-02T00:00:00.000Z', analysis_summary: { total: 7, success_rate: 0.7 } }, +]; +const ctx = { statuses: gStatuses, animals: gAnimals, dailyTemplate: [] }; +const totalMember = { id: '__total__', label: 'Total attempts', param: { kind: 'total' }, group: 'metrics' }; + +describe('buildMatrix', () => { + it('default preset: rows=date, cols=subject, pinned=parameter (legacy layout)', () => { + const m = buildMatrix({ rowDim: 'date', colDim: 'subject', pinnedMember: totalMember, groupField: '__none__' }, ctx); + expect(m.corner).toBe('Date'); + expect(m.context).toEqual({ label: 'Parameter', value: 'Total attempts' }); + expect(m.columns.map((c) => c.label)).toEqual(['Alpha', 'Beta']); + expect(m.rows.map((r) => r.member.label)).toEqual(['2026-07-01', '2026-07-02']); + expect(m.rows[0].values).toEqual([5, 9]); + expect(m.rows[1].values).toEqual([7, '']); // Beta blank on 07-02 + expect(m.groupAxis).toBeNull(); + }); + + it('drops blank date rows but keeps all subject columns', () => { + const statuses = [ + { animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 0 } }, // 0 is a value + { animal_id: 'a1', date: '2026-07-03T00:00:00.000Z', analysis_summary: null }, // no value + ]; + const m = buildMatrix({ rowDim: 'date', colDim: 'subject', pinnedMember: totalMember, groupField: '__none__' }, + { statuses, animals: gAnimals, dailyTemplate: [] }); + expect(m.rows.map((r) => r.member.label)).toEqual(['2026-07-01']); + expect(m.columns.length).toBe(2); + expect(m.rows[0].values).toEqual([0, '']); + }); + + it('subject as columns sets groupAxis=col and clusters subjects', () => { + const m = buildMatrix({ rowDim: 'date', colDim: 'subject', pinnedMember: totalMember, groupField: 'grp' }, ctx); + expect(m.groupAxis).toBe('col'); + expect(m.columns.map((c) => [c.group, c.label])).toEqual([['Control', 'Alpha'], ['Drug', 'Beta']]); + }); + + it('subject as rows sets groupAxis=row; pinned date; cols=parameter', () => { + const dateMember = { id: '2026-07-01', label: '2026-07-01', dateKey: '2026-07-01' }; + const m = buildMatrix({ rowDim: 'subject', colDim: 'parameter', pinnedMember: dateMember, groupField: 'grp' }, + { statuses: gStatuses, animals: gAnimals, dailyTemplate: [] }); + expect(m.corner).toBe('Subject'); + expect(m.context).toEqual({ label: 'Date', value: '2026-07-01' }); + expect(m.groupAxis).toBe('row'); + expect(m.rows.map((r) => [r.member.group, r.member.label])).toEqual([['Control', 'Alpha'], ['Drug', 'Beta']]); + // columns are parameters (Total attempts, Success rate); Alpha on 07-01: total 5, rate 0.5 + const totalCol = m.columns.findIndex((c) => c.label === 'Total attempts'); + expect(m.rows[0].values[totalCol]).toBe(5); + }); + + it('drops blank parameter columns (params always dropped when empty)', () => { + // dailyTemplate adds a custom field with no data anywhere -> its column drops + const dailyTemplate = [{ fieldId: 'c-x', key: 'x', label: 'X', builtin: false, active: true }]; + const dateMember = { id: '2026-07-01', label: '2026-07-01', dateKey: '2026-07-01' }; + const m = buildMatrix({ rowDim: 'subject', colDim: 'parameter', pinnedMember: dateMember, groupField: '__none__' }, + { statuses: gStatuses, animals: gAnimals, dailyTemplate }); + expect(m.columns.map((c) => c.label)).not.toContain('X'); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `npx jest tests/dataExport.test.js -t "buildMatrix"` +Expected: FAIL — new call shape not yet implemented (old `buildMatrix` returns `columns[].header`, no `context`). + +- [ ] **Step 3: Replace the implementation** + +In `frontend/src/lib/dataExport.js`, **delete the old `buildMatrix` function** (the `export function buildMatrix(statuses, animals, param) { ... }` block and its doc comment) and replace with: + +```js +// Pivot statuses into a 2-D matrix per the chosen row/column dimensions. +// The leftover (pinned) dimension is fixed to pinnedMember. Every cell resolves +// a (dateKey, animalId, param) triple through the status index. +export function buildMatrix(config, ctx) { + const { rowDim, colDim, pinnedMember, groupField } = config; + const pinnedDim = otherDimension(rowDim, colDim); + const index = buildStatusIndex(ctx.statuses); + const full = { ...ctx, groupField }; + const rowMembers = listDimensionMembers(rowDim, full); + const colMembers = listDimensionMembers(colDim, full); + + const cellValue = (rowM, colM) => { + const byDim = { [rowDim]: rowM, [colDim]: colM, [pinnedDim]: pinnedMember }; + const dk = byDim.date?.dateKey ?? null; + const animalId = byDim.subject?.animalId ?? null; + const param = byDim.parameter?.param ?? null; + if (dk === null || animalId === null || animalId === undefined || !param) return ''; + const status = index.get(dk)?.get(animalId); + if (!status) return ''; + const v = extractValue(status, param); + return hasValue(v) ? v : ''; + }; + + let rows = rowMembers.map((rowM) => ({ + member: rowM, + values: colMembers.map((colM) => cellValue(rowM, colM)), + })); + + // Empty-member rule: keep all subjects; drop entirely-blank date/parameter rows & columns. + if (rowDim !== 'subject') rows = rows.filter((r) => r.values.some((v) => v !== '')); + + let keptCols = colMembers.map((_, i) => i); + if (colDim !== 'subject') keptCols = keptCols.filter((i) => rows.some((r) => r.values[i] !== '')); + const columns = keptCols.map((i) => colMembers[i]); + rows = rows.map((r) => ({ member: r.member, values: keptCols.map((i) => r.values[i]) })); + + const grouped = !!groupField && groupField !== '__none__'; + const groupAxis = grouped + ? (rowDim === 'subject' ? 'row' : colDim === 'subject' ? 'col' : null) + : null; + + return { + rowDim, + colDim, + pinnedDim, + corner: dimLabel(rowDim), + context: { label: dimLabel(pinnedDim), value: pinnedMember?.label ?? '' }, + groupAxis, + columns, + rows, + }; +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `npx jest tests/dataExport.test.js` +Expected: PASS. (The `toCSV` suite still passes — it builds its own matrix literals and does not call `buildMatrix`; it is rewritten in Task 4.) + +- [ ] **Step 5: Commit** + +```bash +git add frontend/src/lib/dataExport.js frontend/tests/dataExport.test.js +git commit -m "feat(export): generalized buildMatrix over assignable dimensions" +``` + +--- + +### Task 4: toCSV with context line + group cell + +**Files:** +- Modify: `frontend/src/lib/dataExport.js` (replace `toCSV`; leave `csvFilename` unchanged) +- Test: `frontend/tests/dataExport.test.js` (replace the existing `describe('toCSV', ...)` block) + +**Interfaces:** +- Consumes: `escapeCsvCell` (existing), matrix shape from Task 3. +- Produces: `toCSV(matrix): string` — context line, blank line, optional group row (groupAxis==='col'), header row, data rows. Header/data gain a leading `Group` column when groupAxis==='row'. `csvFilename(title, label)` unchanged. + +- [ ] **Step 1: Replace the failing tests** + +In `frontend/tests/dataExport.test.js`, **delete the existing `describe('toCSV', ...)` block** and add: + +```js +describe('toCSV', () => { + it('emits context line, blank line, header, data (no group)', () => { + const matrix = { + corner: 'Date', + context: { label: 'Parameter', value: 'Total attempts' }, + groupAxis: null, + columns: [{ id: 'a1', label: 'Alpha', group: null }, { id: 'a2', label: 'Beta', group: null }], + rows: [{ member: { label: '2026-07-01' }, values: [5, ''] }, { member: { label: '2026-07-02' }, values: ['', 9] }], + }; + expect(toCSV(matrix)).toBe( + 'Parameter,Total attempts\n\nDate,Alpha,Beta\n2026-07-01,5,\n2026-07-02,,9', + ); + }); + + it('adds a Group header row when subjects are columns', () => { + const matrix = { + corner: 'Date', + context: { label: 'Parameter', value: 'Total attempts' }, + groupAxis: 'col', + columns: [{ id: 'a1', label: 'Alpha', group: 'Control' }, { id: 'a2', label: 'Beta', group: 'Drug' }], + rows: [{ member: { label: '2026-07-01' }, values: [5, 9] }], + }; + expect(toCSV(matrix)).toBe( + 'Parameter,Total attempts\n\nGroup,Control,Drug\nDate,Alpha,Beta\n2026-07-01,5,9', + ); + }); + + it('adds a leading Group column when subjects are rows', () => { + const matrix = { + corner: 'Subject', + context: { label: 'Date', value: '2026-07-01' }, + groupAxis: 'row', + columns: [{ id: 'p1', label: 'Total attempts', group: 'metrics' }], + rows: [ + { member: { label: 'Alpha', group: 'Control' }, values: [5] }, + { member: { label: 'Beta', group: 'Drug' }, values: [9] }, + ], + }; + expect(toCSV(matrix)).toBe( + 'Date,2026-07-01\n\nGroup,Subject,Total attempts\nControl,Alpha,5\nDrug,Beta,9', + ); + }); + + it('escapes commas, quotes, and newlines per RFC 4180', () => { + const matrix = { + corner: 'Date', + context: { label: 'Parameter', value: 'Note, field' }, + groupAxis: null, + columns: [{ id: 'a1', label: 'he said "hi"', group: null }], + rows: [{ member: { label: '2026-07-01' }, values: ['line1\nline2'] }], + }; + expect(toCSV(matrix)).toBe( + 'Parameter,"Note, field"\n\nDate,"he said ""hi"""\n2026-07-01,"line1\nline2"', + ); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `npx jest tests/dataExport.test.js -t "toCSV"` +Expected: FAIL — old `toCSV` emits `Date,Alpha,Beta\n...` with no context line. + +- [ ] **Step 3: Replace the implementation** + +In `frontend/src/lib/dataExport.js`, **delete the old `toCSV` function** and its doc comment, and replace with: + +```js +// Serialize a matrix to an RFC-4180-ish CSV string (LF line endings): +// , +// (blank line) +// [Group,] -- only when groupAxis === 'col' +// [,Group?], +// [,group?], +export function toCSV(matrix) { + const lines = []; + lines.push([matrix.context.label, matrix.context.value].map(escapeCsvCell).join(',')); + lines.push(''); + + if (matrix.groupAxis === 'col') { + const groupRow = ['Group', ...matrix.columns.map((c) => c.group ?? '—')]; + lines.push(groupRow.map(escapeCsvCell).join(',')); + } + + const header = matrix.groupAxis === 'row' + ? ['Group', matrix.corner, ...matrix.columns.map((c) => c.label)] + : [matrix.corner, ...matrix.columns.map((c) => c.label)]; + lines.push(header.map(escapeCsvCell).join(',')); + + for (const r of matrix.rows) { + const lead = matrix.groupAxis === 'row' + ? [r.member.group ?? '—', r.member.label] + : [r.member.label]; + lines.push([...lead, ...r.values].map(escapeCsvCell).join(',')); + } + + return lines.join('\n'); +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `npx jest tests/dataExport.test.js` +Expected: PASS (all suites, including the unchanged `csvFilename` and `extractValue` suites). + +- [ ] **Step 5: Commit** + +```bash +git add frontend/src/lib/dataExport.js frontend/tests/dataExport.test.js +git commit -m "feat(export): toCSV emits context line + subject group cell" +``` + +--- + +### Task 5: Rewrite ExportDataModal with axis + group controls + +**Files:** +- Modify: `frontend/src/components/ExportDataModal.jsx` (full rewrite of the body; keep `triggerDownload`) +- Test: `frontend/tests/ExportDataModal.test.jsx` (create) + +**Interfaces:** +- Consumes: `EXPORT_DIMENSIONS`, `dimLabel`, `otherDimension`, `listDimensionMembers`, `buildMatrix`, `toCSV`, `csvFilename` from `../lib/dataExport`; `experimentsApi.getDailyStatuses` from `../api/client`. +- New props: `subjectTemplate` (array of `{ fieldId, label, active }`), `groupField` (string, the experiment's saved default). Existing props unchanged: `experimentTitle`, `experimentId`, `dailyTemplate`, `animals`, `onClose`. +- Produces: rendered modal. Default selection reproduces the legacy layout. + +- [ ] **Step 1: Write the failing component test** + +Create `frontend/tests/ExportDataModal.test.jsx`: + +```jsx +import React from 'react'; +import { render, screen, waitFor, fireEvent } from '@testing-library/react'; +import ExportDataModal from '../src/components/ExportDataModal'; +import * as client from '../src/api/client'; + +jest.mock('../src/api/client', () => ({ + experimentsApi: { getDailyStatuses: jest.fn() }, +})); + +const animals = [ + { id: 'a1', animal_name: 'Alpha', animal_id_string: 'R-001', subject_info: { grp: 'Control' } }, + { id: 'a2', animal_name: 'Beta', animal_id_string: 'R-002', subject_info: { grp: 'Drug' } }, +]; +const statuses = [ + { animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 5 } }, + { animal_id: 'a2', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 9 } }, +]; +const subjectTemplate = [{ fieldId: 'grp', label: 'Group', active: true }]; + +beforeEach(() => { + jest.clearAllMocks(); + client.experimentsApi.getDailyStatuses.mockResolvedValue(statuses); +}); + +function renderModal(props = {}) { + return render( + , + ); +} + +describe('ExportDataModal', () => { + it('defaults to Rows=Date, Columns=Subject and shows the preview', async () => { + renderModal(); + expect(await screen.findByLabelText('Rows')).toHaveValue('date'); + expect(screen.getByLabelText('Columns')).toHaveValue('subject'); + await waitFor(() => expect(screen.getByText(/1 date.*2 subjects/i)).toBeInTheDocument()); + }); + + it('hides the Group by control until Subject is an axis', async () => { + renderModal(); + await screen.findByLabelText('Rows'); + // subject is a column by default -> Group by visible + expect(screen.getByLabelText('Group by')).toBeInTheDocument(); + // move subject off both axes: Rows=Date, Columns=Parameter -> subject pinned + fireEvent.change(screen.getByLabelText('Columns'), { target: { value: 'parameter' } }); + await waitFor(() => expect(screen.queryByLabelText('Group by')).not.toBeInTheDocument()); + }); + + it('keeps row and column dimensions distinct', async () => { + renderModal(); + const rows = await screen.findByLabelText('Rows'); + fireEvent.change(rows, { target: { value: 'subject' } }); // collides with column=subject + await waitFor(() => expect(screen.getByLabelText('Columns')).not.toHaveValue('subject')); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `npx jest tests/ExportDataModal.test.jsx` +Expected: FAIL — modal has no `Rows`/`Columns`/`Group by` labels yet. + +- [ ] **Step 3: Rewrite the component** + +Replace the body of `frontend/src/components/ExportDataModal.jsx` (keep the top `triggerDownload` helper exactly as-is) with this component: + +```jsx +import React, { useEffect, useMemo, useState } from 'react'; +import { experimentsApi } from '../api/client'; +import Button from './ui/Button'; +import Alert from './ui/Alert'; +import { + EXPORT_DIMENSIONS, dimLabel, otherDimension, + listDimensionMembers, buildMatrix, toCSV, csvFilename, +} from '../lib/dataExport'; + +const PARAM_GROUP_LABELS = { metrics: 'Session metrics', daily: 'Daily record fields' }; + +export default function ExportDataModal({ + experimentTitle, experimentId, dailyTemplate, animals, subjectTemplate = [], groupField = '__none__', onClose, +}) { + const [loading, setLoading] = useState(true); + const [error, setError] = useState(null); + const [statuses, setStatuses] = useState([]); + + const [rowDim, setRowDim] = useState('date'); + const [colDim, setColDim] = useState('subject'); + const [pinnedId, setPinnedId] = useState(null); + const [groupBy, setGroupBy] = useState(groupField ?? '__none__'); + + useEffect(() => { + let alive = true; + setLoading(true); + setError(null); + experimentsApi.getDailyStatuses(experimentId, {}) + .then((data) => { if (alive) setStatuses(data); }) + .catch((err) => { if (alive) setError(err.message); }) + .finally(() => { if (alive) setLoading(false); }); + return () => { alive = false; }; + }, [experimentId]); + + const pinnedDim = otherDimension(rowDim, colDim); + const subjectIsAxis = rowDim === 'subject' || colDim === 'subject'; + const effGroup = subjectIsAxis ? groupBy : '__none__'; + const ctx = useMemo(() => ({ statuses, animals, dailyTemplate }), [statuses, animals, dailyTemplate]); + + // Members of the pinned dimension populate its value dropdown. + const pinnedMembers = useMemo( + () => listDimensionMembers(pinnedDim, { ...ctx, groupField: effGroup }), + [pinnedDim, ctx, effGroup], + ); + const effPinnedId = pinnedId ?? pinnedMembers[0]?.id ?? null; + const pinnedMember = pinnedMembers.find((m) => m.id === effPinnedId) ?? pinnedMembers[0] ?? null; + + const matrix = useMemo( + () => (pinnedMember + ? buildMatrix({ rowDim, colDim, pinnedMember, groupField: effGroup }, ctx) + : { columns: [], rows: [], context: { label: '', value: '' }, corner: '', groupAxis: null }), + [rowDim, colDim, pinnedMember, effGroup, ctx], + ); + const hasData = matrix.rows.length > 0; + + // When the user changes an axis, keep the two distinct and reset the pinned pick. + function changeRowDim(next) { + setRowDim(next); + if (colDim === next) setColDim(otherDimension(next, null) ?? EXPORT_DIMENSIONS.find((d) => d.dim !== next).dim); + setPinnedId(null); + } + function changeColDim(next) { + setColDim(next); + if (rowDim === next) setRowDim(EXPORT_DIMENSIONS.find((d) => d.dim !== next).dim); + setPinnedId(null); + } + + function handleExport() { + if (!hasData || !pinnedMember) return; + triggerDownload(csvFilename(experimentTitle, pinnedMember.label), toCSV(matrix)); + onClose(); + } + + if (loading) return

Loading data…

; + if (error) return ( +
+ +
+
+ ); + + const selectCls = 'w-full border border-gray-200 rounded px-2 py-1.5 text-sm bg-white focus:outline-none focus:ring-1 focus:ring-indigo-400'; + const colOptions = EXPORT_DIMENSIONS.filter((d) => d.dim !== rowDim); + + return ( +
+

+ Choose what goes on the rows and columns. The remaining dimension is fixed to one value, shown at the top of the file. +

+ +
+
+ + +
+
+ + +
+
+ +
+ + +
+ + {subjectIsAxis && ( +
+ + +
+ )} + +

+ {hasData + ? `${matrix.rows.length} ${matrix.corner.toLowerCase()}${matrix.rows.length !== 1 ? 's' : ''} × ${matrix.columns.length} ${dimLabel(colDim).toLowerCase()}${matrix.columns.length !== 1 ? 's' : ''}` + : 'No data for this selection yet.'} +

+ +
+ + +
+
+ ); +} + +// Group parameter members by their .group for rendering, preserving order. +function groupParamMembers(members) { + const out = []; + for (const m of members) { + let g = out.find((x) => x.group === m.group); + if (!g) { g = { group: m.group, items: [] }; out.push(g); } + g.items.push(m); + } + return out; +} +``` + +Note: the preview test matches `/1 date.*2 subjects/i` — the corner is `Date` (→ "date"/"dates") and `dimLabel(colDim)` is `Subject` (→ "subject"/"subjects"). Confirm the string reads "1 date × 2 subjects". + +- [ ] **Step 4: Run test to verify it passes** + +Run: `npx jest tests/ExportDataModal.test.jsx` +Expected: PASS (all three cases). + +- [ ] **Step 5: Commit** + +```bash +git add frontend/src/components/ExportDataModal.jsx frontend/tests/ExportDataModal.test.jsx +git commit -m "feat(export): modal gains assignable axes + group-by control" +``` + +--- + +### Task 6: Wire new props from ExperimentDetail + +**Files:** +- Modify: `frontend/src/pages/ExperimentDetail.jsx` (the `` render, ~line 414) + +**Interfaces:** +- Consumes: `subjectTemplate` state and `groupField` state (both already present in `ExperimentDetail`). +- Produces: passes `subjectTemplate={subjectTemplate}` and `groupField={groupField}` to `ExportDataModal`. + +- [ ] **Step 1: Add the two props** + +In `frontend/src/pages/ExperimentDetail.jsx`, change the `` element to: + +```jsx + setShowExport(false)} + /> +``` + +- [ ] **Step 2: Run the full frontend test suite** + +Run: `npm test` +Expected: PASS — full suite green, including `dataExport` and `ExportDataModal`. + +- [ ] **Step 3: Manual smoke check** + +Run: `npm run dev`, open an experiment with daily data, click **Export data**. Verify: default download matches the old date×subject×parameter layout; switching Rows/Columns updates the preview; the fixed-value dropdown lists the third dimension; a group-by choice adds the Group row/column and the top context line reads e.g. `Subject,Mouse-01`. + +- [ ] **Step 4: Commit** + +```bash +git add frontend/src/pages/ExperimentDetail.jsx +git commit -m "feat(export): pass subjectTemplate + saved group field to export modal" +``` + +--- + +## Self-Review Notes + +- **Spec coverage:** assignable axes (Tasks 1–3, 5) · pinned third dimension + value picker (Tasks 3, 5) · beginning-only context line (Task 4) · subject group cell on the subject axis, clustered by group (Tasks 2–4) · Group-by dropdown defaulting to saved field, shown only when Subject is an axis (Tasks 5–6) · keep-subjects/drop-blank-date-and-param rule (Task 3) · default reproduces legacy layout (Tasks 3, 5). All covered. +- **Type consistency:** member shape `{ id, label, ... }` and matrix shape `{ context, corner, groupAxis, columns:[{id,label,group}], rows:[{member,values}] }` are used identically across Tasks 3–5. +- **Out of scope (per spec):** multi-value stacking, footer repeat, experiment-level metadata block, persisting axis choices.