Experiments DB Dev
87 lines
6.5 KiB
Markdown
87 lines
6.5 KiB
Markdown
# Subject-Series Data Export — Design
|
||
|
||
**Date:** 2026-07-19
|
||
**Status:** Approved (design)
|
||
**Area:** `frontend/src/lib/dataExport.js`, `frontend/src/components/ExportDataModal.jsx`, `frontend/tests/`
|
||
**Supersedes:** the assignable-axes UI from `2026-07-19-configurable-data-export-design.md` (that branch's `buildMatrix`/`toCSV`/subject-group work is reused; its rows/columns/pinned modal is replaced).
|
||
|
||
## Problem
|
||
|
||
The export modal currently offers assignable row/column/pinned axes. That is more than needed. The user wants the export to mirror the **cross-subject metrics plot**: pick an **X field** and a **data value**, with each **subject as a column series**, empty where a subject has no data point, plus each subject's **group** included. The one capability missing today is choosing the X (row) field — it is fixed to calendar Date, whereas the plot's X-axis offers **# Days Reach** and daily fields.
|
||
|
||
## Layout (fixed)
|
||
|
||
- **Columns = subjects**, one series per subject, ordered clustered-by-group then by name (existing `listSubjectMembers`).
|
||
- **Rows = the chosen X field's values.**
|
||
- **Cells = the chosen Data value** for that (subject, X); empty (`''`) when absent. `0` and `''` remain real values (existing `hasValue`/`extractValue`).
|
||
- **Group row** gives each subject's group value (existing group-row support).
|
||
|
||
There is no pinned dimension and no assignable columns.
|
||
|
||
## The three pickers
|
||
|
||
1. **X (rows).** Options: `Date`, `# Days Reach`, and each **active** daily-template field. Default `Date` (keeps the same date-rows × subject-columns grid as today; only the line-1 label changes to `Data`, see CSV output).
|
||
2. **Data (cells).** Options: `listExportableParams(dailyTemplate, statuses)` — total, success rate, each count category, each daily field. Default: first parameter (`Total attempts`).
|
||
3. **Group by.** Options: `— None —`, `Name`, `Subject ID`, and each active `subject_info` field. Defaults to the experiment's saved group field (`localStorage['exp-subject-group-<id>']`), validated against the available options and coerced to `__none__` when unset or stale. Always visible (subjects are always the columns).
|
||
|
||
Preview line: `N rows × M subjects`. Export disabled when there are no data rows.
|
||
|
||
## X-field semantics
|
||
|
||
Each X coordinate maps a `(subject, X-value)` pair to a single daily status, from which the Data value is read:
|
||
|
||
- **`Date`** → rows are the distinct calendar dates present (`YYYY-MM-DD`), sorted. Cell reads the subject's status on that date. (Today's behavior.)
|
||
- **`# Days Reach`** → rows are `1, 2, 3, …` up to the maximum, per subject, of the count of that subject's recorded daily statuses. A cell for `(subject, N)` reads the subject's **Nth recorded day**, counting **all** recorded statuses sorted ascending by date (Day 1 = earliest record; days without an analysis summary are still counted — this intentionally differs from the plot, which counts analyzed sessions only).
|
||
- **daily field** (e.g. `Weight (g)`) → rows are the distinct **numeric** values that field takes across the data (`numericFieldVal`: builtin via `status[field.key]`, custom via `status.custom_fields[field.fieldId]`, `parseFloat`, non-numeric → excluded), sorted ascending. A cell for `(subject, v)` reads that subject's status where the field equals `v`; on ties, first status by date wins.
|
||
|
||
Row members that are entirely blank across all subjects are dropped (matching Date's existing behavior); subject columns are always kept even when empty.
|
||
|
||
## CSV output
|
||
|
||
Structure (X = # Days Reach, Data = Success rate, Group by = Treatment):
|
||
|
||
```
|
||
Data,Success rate
|
||
Group,Control,Control,Drug
|
||
# Days Reach,Mouse-A,Mouse-B,Mouse-C
|
||
1,0.5,0.6,0.7
|
||
2,0.55,,0.72
|
||
3,0.6,0.65,
|
||
```
|
||
|
||
- **Line 1** names the Data metric so the file is self-describing: two cells, `Data` and the metric label.
|
||
- **Blank line.**
|
||
- **Group row** (`Group` + each subject's group), only when Group by ≠ None; missing group value → `—`.
|
||
- **Header row:** the X-field label in the corner, then subject headers.
|
||
- **Data rows:** the X value, then one cell per subject.
|
||
|
||
LF line endings; every cell escaped via existing `escapeCsvCell`. Filename: existing `csvFilename(title, dataLabel)`.
|
||
|
||
## Implementation notes
|
||
|
||
Reuse from the current branch: `extractValue`, `listExportableParams`, `subjectGroupValue`, `listSubjectMembers`, `escapeCsvCell`, `toCSV` (its `{ context, corner, groupAxis, columns, rows }` shape is kept), and `csvFilename`.
|
||
|
||
Replace: the general assignable-axes `buildMatrix(config, ctx)` and its members/dispatch helpers (`otherDimension`, `dimLabel`, `EXPORT_DIMENSIONS`, `listDateMembers`, `listDimensionMembers`, `listParameterMembers`, `buildStatusIndex` in its current form) — remove what the fixed layout no longer uses rather than leaving dead code.
|
||
|
||
Add:
|
||
- `numericFieldVal(status, field)` — mirror of the plot's helper (or import/share it).
|
||
- `listSampleMembers(statuses, xField, dailyTemplate)` → the ordered row members for the chosen X coordinate (`{ id, label, ... }`).
|
||
- `buildSampleIndex(statuses, xField, dailyTemplate)` → `Map(animalId → Map(sampleValue → status))`, first-status-per-cell wins, encoding the date/#days/field resolution above.
|
||
- `buildSubjectSeriesMatrix({ xField, dataParam, groupField }, { statuses, animals, dailyTemplate })` → `{ context: { label: 'Data', value: dataParam.label }, corner: <X label>, groupAxis: grouping ? 'col' : null, columns: <subjects>, rows: [{ member, values }] }`, consumable by the existing `toCSV`.
|
||
|
||
Simplify `ExportDataModal.jsx` to the three pickers over these helpers; drop the Rows/Columns/Fixed controls.
|
||
|
||
## Testing
|
||
|
||
- `listSampleMembers` / `buildSampleIndex` for each X coordinate: Date (unchanged), # Days Reach (all-records ordinal, per-subject alignment), daily field (distinct numeric values sorted, non-numeric dropped, tie first-by-date).
|
||
- `buildSubjectSeriesMatrix`: subjects as columns clustered by group; empty cells where a subject lacks a point; blank row dropped; `groupAxis` null when Group by None.
|
||
- `toCSV` reused; add/keep a snapshot for the new context line + group row.
|
||
- `ExportDataModal`: renders three pickers; default X = Date produces date rows × subject columns with the chosen metric in cells; switching X to # Days Reach changes rows to ordinals; group row present when a valid group field is chosen; stale/unset group field coerces to None.
|
||
|
||
## Out of scope (YAGNI)
|
||
|
||
- Assignable columns / pinned dimension (removed).
|
||
- Choosing a subset of subjects (all subjects always included).
|
||
- Matching the plot's analyzed-only day counting (we count all recorded days).
|
||
- A footer or repeated context line.
|