experiments-database/docs/superpowers/specs/2026-07-19-configurable-data-export-design.md

Configurable Data Export — Design

Date: 2026-07-19 Status: Approved (design) Area: frontend/src/lib/dataExport.js, frontend/src/components/ExportDataModal.jsx, frontend/tests/dataExport.test.js

Problem

Today's data export has a fixed layout: each row is a date, each column is a subject, and the user picks one parameter whose value fills every cell (buildMatrix in dataExport.js). Users want to:

1. Choose which dimension goes on rows and which on columns (fully assignable axes).
2. See the pinned/third dimension labeled at the top of the file ("group information at the beginning").
3. When Subject is an axis, see each subject's group in a separate cell, similar to the cross-subject metrics view.

Model: three interchangeable dimensions

Export data spans three dimensions: Date, Subject, Parameter. A 2-D CSV grid can place two of them on the axes; the third must be pinned to a single value (a cell holds exactly one value).

Each dimension yields an ordered list of members:

• Date → sorted unique YYYY-MM-DD keys present in the data.
• Subject → animals, headers disambiguated by animal_id_string on duplicate names (existing logic). When a group field is chosen, subjects are clustered by group value, then sorted by name within each group.
• Parameter → listExportableParams(dailyTemplate, statuses) (session metrics, then daily fields — unchanged).

Every cell is identified by a full (dateKey, animalId, param) triple: two coordinates come from the row/column members, the third from the pinned value. Lookup goes through a status index Map(dateKey → Map(animalId → status)) (first-status-per-cell wins, as today), then extractValue(status, param).

Because Subject is always exactly one of the three dimensions (row, column, or pinned), the group annotation attaches to at most one axis — there is no 2-D corner-block complexity.

Modal UI (ExportDataModal.jsx)

Replaces today's single "Parameter" dropdown with four controls:

• Rows — Date / Subject / Parameter.
• Columns — the two dimensions not chosen for Rows (auto-excludes the row pick; if a change collides, the other axis auto-shifts to a still-valid dimension).
• Fixed: <leftover dim> — value dropdown for the auto-pinned third dimension. Its options are that dimension's members (a date, a subject, or a parameter, grouped like today for parameters).
• Group by — subject_info fields plus "— None —". Shown only when Subject is a row or column axis. Defaults to the experiment's saved group field (localStorage['exp-subject-group-<id>']); falls back to None when unset or the field no longer exists.

Live preview line stays: N rows × M columns (or "No data …" when empty). Default state reproduces today's export exactly: Rows = Date, Columns = Subject, Fixed = Parameter (first parameter). Export button disabled when the grid has no data rows.

CSV output format

Structure: pinned-dimension context line, blank line, optional group row, header row, data rows.

• Line 1 (context): two cells — the pinned dimension's label and its value, e.g. Parameter,Total attempts or Subject,Mouse-01. Beginning only (no footer).
• Blank line.
• Grid, with the group cell riding on whichever axis holds Subject.

Subject = columns (e.g. Rows = Date, Fixed = Parameter) → extra header row labeled Group:

Parameter,Total attempts

Group,Control,Control,Drug,Drug
Date,Mouse-01,Mouse-02,Mouse-03,Mouse-04
2026-01-01,12,10,8,9

Subject = rows (e.g. Cols = Parameter, Fixed = Date) → extra leading column labeled Group:

Date,2026-01-01

Group,Subject,Total attempts,Success rate
Control,Mouse-01,12,0.83
Drug,Mouse-03,8,0.71

Rules:

• The grid corner label is the row dimension's name (Date, Subject, or Parameter) — no longer hardcoded Date.
• Missing group value → —. When Group by = None, the group row/column is omitted entirely.
• Empty-member rule: Subject members are always kept (matches today's "keep all subject columns"). Date and Parameter members are dropped when entirely blank in the current grid.
• Cell values use existing semantics: 0 / '' are real; genuinely-missing lookups render as blank. CSV escaping via existing escapeCsvCell.

Refactor of dataExport.js

Keep extractValue and listExportableParams unchanged. Replace single-purpose buildMatrix with dimension-agnostic pieces:

• listDateMembers(statuses) → sorted unique dateKeys.
• listSubjectMembers(animals, groupField) → [{ id, header, group }], clustered by group then name; header disambiguation preserved.
• buildStatusIndex(statuses) → Map(dateKey → Map(animalId → status)), first-wins.
• buildMatrix({ rowDim, colDim, pinnedDim, pinnedMember, groupField }, { statuses, animals, dailyTemplate }) → { context, groupAxis, corner, columns, rows }. Resolves each cell's (dateKey, animalId, param) triple through the index + extractValue; applies the keep-subjects / drop-blank-date-and-param rule.
• toCSV(matrix) → emits context line, blank line, optional group row, header row, data rows, with the dynamic corner label.
• csvFilename(title, pinnedMember.label) → signature unchanged.

Testing (tests/dataExport.test.js)

Existing extractValue / listExportableParams tests stay green. Add:

• listDateMembers, listSubjectMembers (including group clustering + name tie-break + header disambiguation), buildStatusIndex (first-wins).
• buildMatrix for all three axis pairings in both orientations.
• Group row (Subject = columns) vs. group column (Subject = rows) placement; — fallback; Group = None omits it.
• Empty-member rule: blank date/parameter members dropped, subjects retained.
• toCSV snapshot including the context line and a group row.

Out of scope (YAGNI)

• Stacking multiple values of the pinned dimension into one file (chose "pick one value").
• Footer/repeat of the context line (beginning only).
• Experiment-level metadata block (pinned dimension only).
• Persisting axis choices across sessions (default reproduces current behavior; group field reuses existing saved value only).