10 Commits

Author SHA1 Message Date
Experiments DB Dev 5314071c6b analysis(tdcs): GLM hypothesis tests + methods for the reaching study
Poisson GEE / binomial rate / learning-rate models, subject random-intercept
sensitivity, unknown-group classification and merge scenarios, plus METHODS.md
and learning-curve plots. Data reconstructed from the experiment DB and
verified against the exported matrix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 22:07:17 -04:00
Experiments DB Dev 86feec2e77 fix(export): label synthetic ordinal '# Days Reach (ordinal)' to avoid collision with same-named daily field
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 19:35:22 -04:00
Experiments DB Dev 4f3fe64cad test(export): stale-group + same-date ordinal coverage; comment/guard polish 2026-07-19 15:18:56 -04:00
Experiments DB Dev 33ee191966 refactor(export): remove dead assignable-axes helpers and their tests
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 15:12:15 -04:00
Experiments DB Dev 6f23f83bc8 feat(export): modal becomes X / Data / Group by pickers, subjects as columns 2026-07-19 15:05:38 -04:00
Experiments DB Dev 39a5a4ec0c feat(export): buildSubjectSeriesMatrix — subjects as columns over an X field 2026-07-19 14:59:42 -04:00
Experiments DB Dev 8510903771 feat(export): X-coordinate helpers (date, # days reach, daily field) 2026-07-19 14:54:25 -04:00
Experiments DB Dev 3e52117c1a docs(plan): subject-series export implementation plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 14:36:10 -04:00
Experiments DB Dev 8153777951 docs(spec): subject-series export — choose X field + data value, subjects as columns
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 14:29:20 -04:00
Experiments DB Dev 9b3d52d89b fix(export): default group-by to None when unset/stale; polish
Final code review of the configurable-data-export feature: ExperimentDetail
was forwarding its on-page groupField state (defaults to '__name__') into
ExportDataModal, leaking a clustering default into the default (untouched)
export. Now passes the raw saved localStorage value, defaulting to
'__none__'. The modal also now validates the incoming groupField against
'__none__'/'__name__'/'__id__'/active subjectTemplate fields and coerces to
'__none__' otherwise, guarding against stale/deleted subject_info fields.
Plus a stale module comment fix and optional-chaining consistency fix in
listSubjectMembers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 13:59:48 -04:00
14 changed files with 1905 additions and 342 deletions
+158
View File
@@ -0,0 +1,158 @@
# tDCS reaching study — GLM methods
This document explains the statistics in `tdcs_glm.py`: the models, the exact
formulas, how each subject's progress is accounted for, and the caveats.
## The question
Two **known** conditions anchor the performance scale and differ from each other:
- **H2 (anchor check):** `Electrode-Box-B2` performs **better** than `Electrode-Box-A2`.
Two **unknown** conditions are then **classified** against those anchors — for
each of `Electrode-Box-A` and `Right-Electrode`, is it **A2-like** or **B2-like**?
We do *not* assume either belongs to B2; each unknown is compared to *both*
anchors, and the anchor it cannot be distinguished from is its likely class.
There is also a **merged-assumption** mode (`--merge`): assume the two unknowns
resolve as `Right-Electrode == Box-B2` and `Electrode-Box-A == Box-A2`, fold them
into the anchors, and re-estimate everything with 4 subjects per group.
## Data and outcome
- **Outcome:** `success` = successful reaches in a session
(`analysis_summary.counts.Success`), a non-negative **count**.
- **Attempts:** `total` = reach attempts in the session (`analysis_summary.total`);
used as the denominator for the rate model. `success_rate == success / total`.
- **Time:** `day` = the "# Days Reach" field (training day, 0..26). Analyses use
days ≥ 0 (pre-training negative days and zero-attempt sessions are excluded;
zero-attempt sessions are undefined for the rate model).
- **Groups (subjects):** Naive (4), Box-A2 (3), Box-B2 (3), Box-A (1),
Right-Electrode (1). Under `--merge`: Box-A2 (4), Box-B2 (4), Naive (4).
**Provenance:** `tdcs_reach_data.csv` was reconstructed from the experiment
database and verified cell-by-cell against the exported matrix (71/71
unambiguous cells on days 05 matched exactly).
## The three models
All models use `Electrode-Box-B2` as the **reference** group, so each group term
is that group's contrast *versus Box-B2*. `day_c` is the centered training day and
`day_c2 = day_c²` captures the rise-then-plateau of the learning curve.
### (A) Level — count (primary)
Poisson GEE on the success counts, clustered by subject:
```
success ~ C(group, Treatment('Electrode-Box-B2')) + day_c + day_c2
family = Poisson (log link)
groups = subject # repeated-measures cluster
cov_struct = Exchangeable # working within-subject correlation
SE = robust (sandwich)
```
`exp(coef)` for a group term is an **incidence-rate ratio (IRR)**: expected
successes relative to Box-B2.
### (B) Level — rate
Binomial GLM on successes-out-of-attempts, with cluster-robust SEs by subject:
```
cbind(success, total - success) ~ C(group, Treatment('Electrode-Box-B2')) + day_c + day_c2
family = Binomial (logit link)
cov_type = cluster (groups = subject)
```
`exp(coef)` is an **odds ratio** for a successful reach relative to Box-B2. This
controls for differing numbers of attempts, so it answers "who is more accurate
per attempt?" rather than "who attempts more?"
### (C) Learning rate
Poisson GEE with a **group × day** interaction, to ask whether groups improve at
different *rates* (not just different levels):
```
success ~ C(group, Treatment('Electrode-Box-B2')) * day_c + day_c2
```
Each `group[T.X]:day_c` term is the difference in log-slope versus Box-B2; a joint
Wald test asks whether *any* group's slope differs. Large p ⇒ parallel learning.
### Anchors-only model
The A2-vs-B2 comparison, refit on **just the two anchor groups** so nothing else
influences the shared day terms or the dispersion/correlation nuisance:
```
# subset to {Box-A2, Box-B2}
success ~ C(group, Treatment('Electrode-Box-B2')) + day_c + day_c2 # count
cbind(success, total-success) ~ C(group, ...) + day_c + day_c2 # rate
```
With B2 as reference the single group term is the A2-vs-B2 effect; H2 is a
one-sided test that this coefficient is below zero.
## How each subject's progress is accounted for
Two distinct pieces:
1. **Progress over training** — the `day_c + day_c2` fixed terms model the average
learning curve, so groups are compared at comparable points in training rather
than being confounded by *when* each was measured.
2. **Repeated measures / individual baselines** — each subject contributes many
correlated sessions and has its own baseline. Handled two ways:
- **Primary (GEE):** subject is the cluster; an exchangeable working
correlation plus robust (sandwich) SEs give *population-average* group
effects whose inference is valid under within-subject correlation and
Poisson overdispersion.
- **Sensitivity (mixed model):** a Poisson model with a **per-subject random
intercept** (`(1 | subject)`) so each animal gets its own baseline level;
the group effects are estimated after allowing for that individual variation.
statsmodels has no frequentist Poisson GLMM, so this is the **MAP/Laplace**
fit (`fit_vb` diverges on these large counts). Its p-values are approximate —
read it as a direction/magnitude check that should agree with GEE.
## Classification logic (unknowns)
For each unknown group, `diff_contrast` builds a linear contrast of that group
against **each anchor** (both coded vs the B2 reference) and tests it:
- indistinguishable from B2 **and** different from A2 → **B2-like**
- indistinguishable from A2 **and** different from B2 → **A2-like**
- indistinguishable from both → **ambiguous** (report the numerically nearer one)
- different from both → **unlike both**
## Caveats
- **Tiny groups.** Box-A2/B2 have 3 subjects, Naive 4, and each unknown has
**n = 1 subject**. Classifying a one-subject condition is weak: "matches
anchor X" means "not statistically distinguishable from X," **not** proof of
equivalence.
- **Single-cluster fragility.** Cluster-robust/GEE inference with one cluster in a
group can produce artificially small SEs — most visibly the learning-rate
interaction for the n=1 groups; do not read those p-values literally.
- **Mixed-model confounding.** With a per-subject random intercept, a group made
of one subject is partly confounded with that subject's random intercept, so its
fixed effect is shrunk.
- **Rate vs count.** "Success" alone is a count; the rate model (success/attempts)
is the fairer accuracy comparison when attempt counts differ.
## Files and usage
- `tdcs_glm.py` — the analysis (run it directly).
- `tdcs_reach_data.csv` — the verified long-format data (`subject, group, day, success, total`).
- `tdcs_learning_curves*.png` — per-group learning curves (count and rate panels).
```
python3 analysis/tdcs_glm.py # all training days, 5 groups
python3 analysis/tdcs_glm.py --max-day 10 # restrict to days 010
python3 analysis/tdcs_glm.py --merge # assume Right==B2 and Box-A==A2 (3 groups)
```
Requires: pandas, numpy, scipy, statsmodels, matplotlib.
Binary file not shown.
+440
View File
@@ -0,0 +1,440 @@
#!/usr/bin/env python3
"""
tDCS reaching study — GLM hypothesis tests.
QUESTION
Two KNOWN conditions anchor the scale and differ from each other:
H2 (anchor check): Electrode-Box-B2 performs BETTER than Electrode-Box-A2.
Two UNKNOWN conditions then get CLASSIFIED against those anchors:
For "Electrode-Box-A" and for "Right-Electrode", is each one
A2-like or B2-like? (We do not assume either belongs to B2.)
Each unknown is compared to BOTH anchors; a group it cannot be
distinguished from is its likely class.
Each comparison is examined through THREE complementary GLMs:
(A) LEVEL / count Poisson GEE on `success` (successful reaches per
session), adjusting for training day. Answers
"who makes more successful reaches overall?"
(B) LEVEL / rate Binomial GLM on success / attempts
(`success` out of `total`), cluster-robust.
Controls for differing numbers of attempts —
"who is more accurate per attempt?"
(C) LEARNING RATE Poisson GEE with a group x day interaction.
Answers "do the groups improve at different
RATES?" (slope of the learning curve).
All models adjust for time with day (and day^2 for the plateau) and account
for repeated measures on each subject via GEE (subject = cluster,
exchangeable correlation, robust SE) or cluster-robust standard errors.
Group is coded with Electrode-Box-B2 as the REFERENCE, so H1 and H2 read
directly off the group terms:
Right-Electrode term -> H1 (expect ~0 / non-significant)
Electrode-Box-A2 term -> H2 (expect < 0: B2 above A2)
exp(coef) is an incidence-rate ratio (count model) or odds ratio (rate
model) relative to Box-B2.
CAVEATS
* Tiny groups: Box-A2 n=3, Box-B2 n=3, Naive n=4, and each UNKNOWN
(Electrode-Box-A, Right-Electrode) has only n=1 SUBJECT. Classifying a
1-subject condition is weak: "matches anchor X" means "not statistically
distinguishable from X", NOT proof of equivalence, and cluster-robust
inference with a single cluster in a group is fragile (can show
artificially small SEs, especially in the learning-rate interaction).
DATA PROVENANCE
analysis/tdcs_reach_data.csv was reconstructed from the experiment
database (experiment "tDCS", outcome = analysis_summary.counts.Success,
attempts = analysis_summary.total, x-axis = the "# Days Reach" field) and
verified cell-by-cell against the exported matrix (71/71 unambiguous
cells on days 0-5 matched exactly).
USAGE
python3 analysis/tdcs_glm.py
Requires: pandas, numpy, scipy, statsmodels, matplotlib.
Writes: analysis/tdcs_learning_curves.png
"""
import os
import argparse
import numpy as np
import pandas as pd
import patsy
import statsmodels.api as sm
import statsmodels.formula.api as smf
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
REF = "Electrode-Box-B2"
# Two KNOWN anchor conditions that differ (Box-B2 > Box-A2, see H2).
ANCHOR_LOW = "Electrode-Box-A2"
ANCHOR_HIGH = "Electrode-Box-B2"
# Two UNKNOWN conditions to classify: is each one A2-like or B2-like?
CANON_UNKNOWNS = ["Electrode-Box-A", "Right-Electrode"]
ALL_GROUPS = ["Naive", ANCHOR_LOW, ANCHOR_HIGH] + CANON_UNKNOWNS
# The active group config is set at runtime (after any --merge/--assign remap).
UNKNOWNS = list(CANON_UNKNOWNS)
MAIN = list(ALL_GROUPS)
OTHERS = [g for g in MAIN if g != REF]
# --merge preset: assume each unknown resolves into an anchor.
MERGE_MAP = {"Electrode-Box-A": ANCHOR_LOW, "Right-Electrode": ANCHOR_HIGH}
HERE = os.path.dirname(os.path.abspath(__file__))
DATA = os.path.join(HERE, "tdcs_reach_data.csv")
PLOT = os.path.join(HERE, "tdcs_learning_curves.png")
RHS = f"C(group, Treatment('{REF}')) + day_c + day_c2"
def gterm(name):
"""statsmodels/patsy column name for a group level (vs the reference)."""
return f"C(group, Treatment('{REF}'))[T.{name}]"
def xterm(name):
"""Interaction column name (group level x day slope)."""
return f"C(group, Treatment('{REF}'))[T.{name}]:day_c"
def load(max_day=None, mapping=None):
df = pd.read_csv(DATA)
if mapping: # remap unknown -> target group
df["group"] = df["group"].map(lambda g: mapping.get(g, g))
df = df[df["group"].isin(ALL_GROUPS)].copy() # keep canonical groups
df = df[df["day"] >= 0].copy() # training days only (day 0..)
if max_day is not None:
df = df[df["day"] <= max_day].copy() # restrict the analysis window
df["day_c"] = df["day"] - df["day"].mean() # center day
df["day_c2"] = df["day_c"] ** 2
present = [g for g in ALL_GROUPS if g != REF and g in set(df["group"])]
df["group"] = pd.Categorical(df["group"], categories=[REF] + present)
return df
def contrast(res, tname):
"""coef, robust SE, p, effect=exp(coef) and its 95% CI for one term."""
ci = res.conf_int()
lo, hi = ci.loc[tname]
return dict(coef=res.params[tname], se=res.bse[tname], p=res.pvalues[tname],
eff=np.exp(res.params[tname]), lo=np.exp(lo), hi=np.exp(hi))
def one_sided_p_below(c):
"""One-sided p for H: coef < 0 (i.e. Box-B2 above the compared group)."""
return stats.norm.cdf(c["coef"] / c["se"])
def diff_contrast(res, a, b):
"""Compare group `a` vs group `b` (both coded relative to REF) via a linear
contrast. Returns coef, robust SE, p, effect=exp(coef)=ratio a/b, and CI.
Works for any pair, including b == REF (then it is just the `a` term)."""
names = list(res.params.index)
v = np.zeros(len(names))
if a != REF:
v[names.index(gterm(a))] += 1.0
if b != REF:
v[names.index(gterm(b))] -= 1.0
tt = res.t_test(v)
coef = float(np.ravel(tt.effect)[0])
se = float(np.ravel(tt.sd)[0])
p = float(np.ravel(tt.pvalue)[0])
lo, hi = np.ravel(tt.conf_int())[:2]
return dict(coef=coef, se=se, p=p, eff=np.exp(coef),
lo=np.exp(lo), hi=np.exp(hi))
# ----------------------------------------------------------------------------- models
def fit_count_level(df):
"""(A) Poisson GEE on success counts; group main effects vs Box-B2."""
return smf.gee(f"success ~ {RHS}", groups="subject", data=df,
family=sm.families.Poisson(),
cov_struct=sm.cov_struct.Exchangeable()).fit()
def fit_rate_level(df):
"""(B) Binomial GLM on success/attempts with cluster-robust SE.
Zero-attempt sessions (total==0) carry no rate information and are dropped."""
d = df[df["total"] > 0].copy()
dropped = len(df) - len(d)
if dropped:
print(f"[rate model] dropped {dropped} zero-attempt session(s) (undefined rate)")
X = patsy.dmatrix(RHS, d, return_type="dataframe")
endog = np.column_stack([d["success"].values,
(d["total"] - d["success"]).values])
return sm.GLM(endog, X, family=sm.families.Binomial()).fit(
cov_type="cluster", cov_kwds={"groups": d["subject"].values})
def fit_learning_rate(df):
"""(C) Poisson GEE with group x day interaction (slope differences)."""
return smf.gee(
f"success ~ C(group, Treatment('{REF}')) * day_c + day_c2",
groups="subject", data=df, family=sm.families.Poisson(),
cov_struct=sm.cov_struct.Exchangeable()).fit()
def fit_mixed_count(df):
"""Subject random-intercept Poisson mixed model (MAP / Laplace).
Adds a per-subject random intercept so each subject has its own baseline
level; the group fixed effects are then estimated after allowing for that
individual variation. A subject-specific complement to the population-
average GEE. statsmodels has no frequentist Poisson GLMM, so this is the
posterior-mode (MAP) fit with a Laplace covariance for the fixed effects
(fit_vb failed to converge on these large counts; MAP is stable)."""
import warnings
from statsmodels.genmod.bayes_mixed_glm import PoissonBayesMixedGLM
vc = {"subject": "0 + C(subject)"} # random intercept per subject
model = PoissonBayesMixedGLM.from_formula(f"success ~ {RHS}", vc, df)
with warnings.catch_warnings():
# MAP stops at |gradient|~5e-5 (effectively converged); silence the
# over-strict "did not converge" notice.
warnings.filterwarnings("ignore", message="Laplace fitting did not converge")
return model.fit_map()
def mixed_contrast(res, tname):
"""coef, SD, approx p and IRR + 95% interval from a BayesMixedGLM fit."""
names = list(res.model.exog_names)
i = names.index(tname)
mean, sd = float(res.fe_mean[i]), float(res.fe_sd[i])
p = 2.0 * stats.norm.sf(abs(mean / sd))
return dict(coef=mean, se=sd, p=p, eff=np.exp(mean),
lo=np.exp(mean - 1.96 * sd), hi=np.exp(mean + 1.96 * sd))
# ----------------------------------------------------------------------------- reports
def describe(df):
print("=" * 78)
print("DESCRIPTIVES")
print("=" * 78)
grp = df.groupby("group", observed=True)
tbl = pd.DataFrame({
"n_subj": grp["subject"].nunique(),
"n_sessions": grp.size(),
"mean_success": grp["success"].mean().round(1),
"mean_rate": (grp["success"].sum() / grp["total"].sum()).round(3),
"max_day": grp["day"].max(),
})
print(tbl.to_string())
def report_level(title, res, effect_label):
print("\n" + "=" * 78)
print(title)
print("=" * 78)
print(f"Effect vs {REF} ({effect_label} relative to Box-B2):")
h = f"{'group':20s} {'effect':>7s} {'95% CI':>16s} {'coef':>8s} {'SE':>7s} {'p':>9s}"
print(h)
print("-" * len(h))
out = {}
for g in OTHERS:
c = contrast(res, gterm(g))
out[g] = c
ci = f"[{c['lo']:.2f}, {c['hi']:.2f}]"
print(f"{g:20s} {c['eff']:7.3f} {ci:>16s} {c['coef']:8.3f} {c['se']:7.3f} {c['p']:9.4f}")
return out
def report_learning(res):
print("\n" + "=" * 78)
print("(C) LEARNING RATE — Poisson GEE, group x day interaction")
print("=" * 78)
base = res.params["day_c"]
print(f"Box-B2 learning slope: {np.exp(base):.3f}x successes per training day "
f"(baseline).")
print("Slope DIFFERENCE vs Box-B2 (exp(coef) = per-day multiplier on the rate ratio):")
h = f"{'group':20s} {'slopeΔ/day':>10s} {'coef':>8s} {'SE':>7s} {'p':>9s}"
print(h)
print("-" * len(h))
for g in OTHERS:
c = contrast(res, xterm(g))
print(f"{g:20s} {c['eff']:10.3f} {c['coef']:8.3f} {c['se']:7.3f} {c['p']:9.4f}")
# Joint Wald test: do ANY groups differ from Box-B2 in learning rate?
names = list(res.params.index)
idx = [names.index(xterm(g)) for g in OTHERS]
R = np.zeros((len(idx), len(names)))
for i, j in enumerate(idx):
R[i, j] = 1.0
jt = res.wald_test(R, scalar=True)
print(f"\nJoint test (all group x day interactions = 0): "
f"chi2={float(jt.statistic):.2f}, df={len(idx)}, p={float(jt.pvalue):.4f}")
print(" -> small p = groups improve at DIFFERENT rates; large p = parallel learning.")
def report_anchor_only(df):
"""A2-vs-B2 ONLY: refit on just the two anchors so nothing else influences
the shared day terms / dispersion. B2 is the reference, so the single group
term is the A2-vs-B2 comparison."""
print("\n" + "=" * 78)
print("ANCHORS ONLY — Box-A2 vs Box-B2 (two-group models)")
print("=" * 78)
d = df[df["group"].isin([ANCHOR_LOW, ANCHOR_HIGH])].copy()
d["day_c"] = d["day"] - d["day"].mean()
d["day_c2"] = d["day_c"] ** 2
d["group"] = pd.Categorical(d["group"], categories=[ANCHOR_HIGH, ANCHOR_LOW])
n_sub = d.groupby("group", observed=True)["subject"].nunique().to_dict()
print(f"subjects: Box-B2={n_sub.get(ANCHOR_HIGH)}, Box-A2={n_sub.get(ANCHOR_LOW)}"
f" sessions: {len(d)}")
print(f"formula: success ~ C(group, Treatment('{REF}')) + day_c + day_c2")
for label, res in [("count/level (Poisson GEE)", fit_count_level(d)),
("rate/level (Binomial cluster-robust)", fit_rate_level(d))]:
c = contrast(res, gterm(ANCHOR_LOW))
p1 = one_sided_p_below(c)
tag = "SUPPORTED" if (c["coef"] < 0 and p1 < 0.05) else "not supported"
print(f" [{label}] A2/B2 = {c['eff']:.2f} [{c['lo']:.2f}, {c['hi']:.2f}] "
f"=> B2 = {1.0 / c['eff']:.2f}x A2, one-sided p={p1:.4f} -> H2 {tag}")
def report_mixed(df):
"""Subject random-intercept Poisson mixed model (sensitivity vs GEE)."""
print("\n" + "=" * 78)
print("SENSITIVITY — Poisson MIXED model, per-subject random intercept (MAP/Laplace)")
print("=" * 78)
try:
res = fit_mixed_count(df)
except Exception as exc: # VB can be finicky; degrade gracefully
print(f" (mixed model skipped: {type(exc).__name__}: {exc})")
return
print("Group effect vs Box-B2 (IRR; each subject given its own baseline):")
h = f"{'group':20s} {'IRR':>7s} {'~95% CI':>16s} {'mean':>8s} {'SD':>7s} {'~p':>9s}"
print(h)
print("-" * len(h))
for g in OTHERS:
c = mixed_contrast(res, gterm(g))
ci = f"[{c['lo']:.2f}, {c['hi']:.2f}]"
print(f"{g:20s} {c['eff']:7.3f} {ci:>16s} {c['coef']:8.3f} {c['se']:7.3f} {c['p']:9.4f}")
print(f" subject random-intercept SD = {float(np.exp(res.vcp_mean[0])):.3f} "
"(log scale); Laplace SEs, treat p-values as approximate.")
print(" Compare directions/magnitudes with the GEE table (they should agree).")
print(" Single-subject groups (Box-A, Right-Electrode) are partly confounded")
print(" with their own random intercept here, so their effects are shrunk.")
def plot_curves(df, plot_path):
fig, ax = plt.subplots(1, 2, figsize=(12, 4.8))
for g in MAIN:
s = df[df["group"] == g]
d = s.groupby("day").apply(
lambda x: pd.Series({"succ": x["success"].mean(),
"rate": x["success"].sum() / x["total"].sum()}),
include_groups=False)
ax[0].plot(d.index, d["succ"], marker="o", ms=3, label=g)
ax[1].plot(d.index, d["rate"], marker="o", ms=3, label=g)
ax[0].set(xlabel="# Days Reach (training day)", ylabel="mean successful reaches",
title="(A) Success count learning curves")
ax[1].set(xlabel="# Days Reach (training day)", ylabel="success rate (success/attempts)",
title="(B) Success-rate learning curves")
for a in ax:
a.legend(fontsize=8)
a.grid(alpha=0.3)
fig.tight_layout()
fig.savefig(plot_path, dpi=120)
print(f"\nSaved learning-curve plot -> {plot_path}")
def classify_one(res, unknown):
"""Compare an unknown group against BOTH anchors; return a label + detail."""
vlo = diff_contrast(res, unknown, ANCHOR_LOW) # unknown vs Box-A2
vhi = diff_contrast(res, unknown, ANCHOR_HIGH) # unknown vs Box-B2
like_lo = vlo["p"] >= 0.05 # indistinguishable from Box-A2
like_hi = vhi["p"] >= 0.05 # indistinguishable from Box-B2
if like_hi and not like_lo:
verdict = "B2-like (differs from A2, matches B2)"
elif like_lo and not like_hi:
verdict = "A2-like (differs from B2, matches A2)"
elif like_lo and like_hi:
# tie: point toward whichever ratio is closer to 1 on the log scale
nearer = ANCHOR_HIGH if abs(vhi["coef"]) < abs(vlo["coef"]) else ANCHOR_LOW
verdict = f"AMBIGUOUS (matches both; numerically nearer {nearer})"
else:
verdict = "UNLIKE BOTH (differs from A2 and B2)"
return vlo, vhi, verdict
def verdicts(count, rate):
print("\n" + "=" * 78)
print("VERDICTS")
print("=" * 78)
print("\nAnchor check — H2: Box-B2 BETTER than Box-A2 (the two anchors must differ)")
for label, res in [("count/level", count), ("rate/level ", rate)]:
c = contrast(res, gterm(ANCHOR_LOW))
p1 = one_sided_p_below(c)
tag = "SUPPORTED" if (c["coef"] < 0 and p1 < 0.05) else "not supported"
print(f" [{label}] Box-B2 = {1.0 / c['eff']:.2f}x Box-A2 "
f"(one-sided p={p1:.4f}) -> {tag}")
if not UNKNOWNS:
print("\n(No unknown groups — they were merged into the anchors; "
"only the H2 anchor contrast applies.)")
return
print("\nClassification — is each UNKNOWN condition A2-like or B2-like?")
print("(ratio >1 = above that anchor; p = differs from that anchor)")
for u in UNKNOWNS:
print(f"\n {u}:")
for label, res in [("count/level", count), ("rate/level ", rate)]:
vlo, vhi, verdict = classify_one(res, u)
print(f" [{label}] vs Box-A2: {vlo['eff']:.2f}x "
f"[{vlo['lo']:.2f},{vlo['hi']:.2f}] p={vlo['p']:.3f} "
f"vs Box-B2: {vhi['eff']:.2f}x [{vhi['lo']:.2f},{vhi['hi']:.2f}] "
f"p={vhi['p']:.3f}")
print(f" -> {verdict}")
print("\n NOTE: each unknown has ONLY 1 subject. 'Matches' means 'not statistically")
print(" distinguishable' — weak evidence at n=1, not proof of equivalence.")
def main():
ap = argparse.ArgumentParser(description="tDCS GLM hypothesis tests")
ap.add_argument("--max-day", type=int, default=None,
help="restrict analysis to training days 0..MAX_DAY (default: all)")
ap.add_argument("--merge", action="store_true",
help="preset: Electrode-Box-A->Box-A2 and Right-Electrode->Box-B2")
ap.add_argument("--assign", type=str, default=None,
help="custom merges 'UNKNOWN=TARGET,...', e.g. "
"'Electrode-Box-A=Electrode-Box-B2,Right-Electrode=Electrode-Box-B2'")
args = ap.parse_args()
mapping = {}
if args.merge:
mapping.update(MERGE_MAP)
if args.assign:
for pair in args.assign.split(","):
k, v = pair.split("=")
mapping[k.strip()] = v.strip()
df = load(args.max_day, mapping)
global MAIN, OTHERS, UNKNOWNS
present = set(df["group"])
MAIN = [g for g in ALL_GROUPS if g in present]
OTHERS = [g for g in MAIN if g != REF]
UNKNOWNS = [g for g in CANON_UNKNOWNS if g in present]
window = f"days 0-{args.max_day}" if args.max_day is not None else "all training days (0+)"
tag = f"_d0-{args.max_day}" if args.max_day is not None else ""
tag += "_merged" if mapping else ""
plot_path = os.path.join(HERE, f"tdcs_learning_curves{tag}.png")
mode = ("assign " + ", ".join(f"{k.replace('Electrode-','')}->{v.replace('Electrode-','')}"
for k, v in mapping.items())) if mapping \
else "5 groups (anchors + unknowns)"
print(f"ANALYSIS WINDOW: {window} | MODE: {mode} | observations: {len(df)}")
describe(df)
count = fit_count_level(df)
rate = fit_rate_level(df)
learn = fit_learning_rate(df)
report_level("(A) LEVEL / COUNT — Poisson GEE on successful reaches",
count, "incidence-rate ratio")
report_level("(B) LEVEL / RATE — Binomial GLM on success/attempts (cluster-robust)",
rate, "odds ratio")
report_learning(learn)
report_anchor_only(df)
report_mixed(df)
verdicts(count, rate)
plot_curves(df, plot_path)
print("\nDone. See the module docstring for modeling choices and caveats.")
if __name__ == "__main__":
main()
Binary file not shown.

After

Width:  |  Height:  |  Size: 161 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 154 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 133 KiB

+194
View File
@@ -0,0 +1,194 @@
subject,group,day,success,total
Vu-vuong,Naive,0,18,61
Vu-vuong,Naive,1,35,91
Vu-vuong,Naive,2,61,127
Vu-vuong,Naive,3,34,146
Vu-vuong,Naive,4,61,141
Vu-vuong,Naive,5,37,151
Vu-vuong,Naive,6,49,131
Vu-vuong,Naive,7,65,149
Vu-vuong,Naive,8,67,141
Vu-vuong,Naive,9,73,140
Vu-vuong,Naive,10,72,131
Vu-vuong,Naive,11,89,158
Vu-vuong,Naive,12,97,154
Vu-vuong,Naive,13,93,145
Vu-vuong,Naive,14,75,147
Vu-vuong,Naive,15,68,142
Vu-vuong,Naive,16,96,162
Vu-vuong,Naive,17,68,144
Vu-vuong,Naive,18,81,134
Vu-vuong,Naive,19,66,144
Vu-vuong,Naive,20,84,127
Vu-vuong,Naive,21,66,133
Khoai-tay-1,Electrode-Box-A,-2,1,18
Khoai-tay-1,Electrode-Box-A,-1,3,24
Khoai-tay-1,Electrode-Box-A,0,14,62
Khoai-tay-1,Electrode-Box-A,1,22,83
Khoai-tay-1,Electrode-Box-A,2,6,79
Khoai-tay-1,Electrode-Box-A,3,28,97
Khoai-tay-1,Electrode-Box-A,4,60,134
Khoai-tay-1,Electrode-Box-A,5,75,138
Khoai-tay-1,Electrode-Box-A,6,81,137
Khoai-tay-1,Electrode-Box-A,7,78,147
Khoai-tay-1,Electrode-Box-A,8,91,132
Khoai-tay-1,Electrode-Box-A,9,99,146
Khoai-tay-1,Electrode-Box-A,10,94,143
Khoai-tay-1,Electrode-Box-A,11,110,156
Khoai-tay-1,Electrode-Box-A,12,105,143
Khoai-tay-1,Electrode-Box-A,13,106,153
Khoai-tay-1,Electrode-Box-A,14,91,152
Banh-mi-2,Electrode-Box-A2,-1,2,42
Banh-mi-2,Electrode-Box-A2,0,25,97
Banh-mi-2,Electrode-Box-A2,1,22,101
Banh-mi-2,Electrode-Box-A2,2,22,119
Banh-mi-2,Electrode-Box-A2,3,29,118
Banh-mi-2,Electrode-Box-A2,4,27,136
Banh-mi-2,Electrode-Box-A2,5,43,146
Banh-mi-2,Electrode-Box-A2,6,74,146
Banh-mi-2,Electrode-Box-A2,7,70,148
Banh-mi-2,Electrode-Box-A2,8,65,130
Banh-mi-2,Electrode-Box-A2,9,79,151
Banh-mi-2,Electrode-Box-A2,10,93,152
Egg-tart-2,Electrode-Box-A2,0,9,32
Egg-tart-2,Electrode-Box-A2,1,2,38
Egg-tart-2,Electrode-Box-A2,2,31,93
Egg-tart-2,Electrode-Box-A2,3,44,101
Egg-tart-2,Electrode-Box-A2,4,54,131
Egg-tart-2,Electrode-Box-A2,5,84,139
Egg-tart-2,Electrode-Box-A2,6,85,145
Egg-tart-2,Electrode-Box-A2,7,79,143
Egg-tart-2,Electrode-Box-A2,8,76,131
Egg-tart-2,Electrode-Box-A2,9,88,149
Egg-tart-2,Electrode-Box-A2,10,81,151
Egg-tart-2,Electrode-Box-A2,11,78,152
Egg-tart-2,Electrode-Box-A2,12,96,155
Egg-tart-2,Electrode-Box-A2,13,84,155
Root-beer-2,Electrode-Box-A2,0,22,74
Root-beer-2,Electrode-Box-A2,1,31,87
Root-beer-2,Electrode-Box-A2,2,49,134
Root-beer-2,Electrode-Box-A2,3,31,89
Root-beer-2,Electrode-Box-A2,4,60,140
Root-beer-2,Electrode-Box-A2,5,84,147
Banh-mi-1,Electrode-Box-B2,0,13,84
Banh-mi-1,Electrode-Box-B2,1,35,86
Banh-mi-1,Electrode-Box-B2,2,47,110
Banh-mi-1,Electrode-Box-B2,3,65,140
Banh-mi-1,Electrode-Box-B2,4,70,127
Banh-mi-1,Electrode-Box-B2,5,102,142
Banh-mi-1,Electrode-Box-B2,6,90,131
Banh-mi-1,Electrode-Box-B2,7,109,148
Banh-mi-1,Electrode-Box-B2,8,104,137
Banh-mi-1,Electrode-Box-B2,9,119,150
Banh-mi-1,Electrode-Box-B2,10,121,158
Banh-mi-1,Electrode-Box-B2,11,121,148
Banh-mi-1,Electrode-Box-B2,12,120,149
Banh-mi-1,Electrode-Box-B2,13,135,154
Egg-tart-1,Electrode-Box-B2,0,7,56
Egg-tart-1,Electrode-Box-B2,1,16,78
Egg-tart-1,Electrode-Box-B2,2,23,103
Egg-tart-1,Electrode-Box-B2,3,63,120
Egg-tart-1,Electrode-Box-B2,4,69,132
Egg-tart-1,Electrode-Box-B2,5,83,136
Egg-tart-1,Electrode-Box-B2,6,71,142
Egg-tart-1,Electrode-Box-B2,7,79,138
Egg-tart-1,Electrode-Box-B2,8,98,142
Egg-tart-1,Electrode-Box-B2,9,89,139
Egg-tart-1,Electrode-Box-B2,10,96,143
Egg-tart-1,Electrode-Box-B2,11,96,148
Egg-tart-1,Electrode-Box-B2,12,101,156
Egg-tart-1,Electrode-Box-B2,13,103,152
Egg-tart-1,Electrode-Box-B2,14,97,152
Root-beer-1,Electrode-Box-B2,0,11,85
Root-beer-1,Electrode-Box-B2,1,18,76
Root-beer-1,Electrode-Box-B2,2,40,105
Root-beer-1,Electrode-Box-B2,3,55,134
Root-beer-1,Electrode-Box-B2,4,75,136
Root-beer-1,Electrode-Box-B2,5,64,133
Root-beer-1,Electrode-Box-B2,6,104,139
Root-beer-1,Electrode-Box-B2,7,98,148
Root-beer-1,Electrode-Box-B2,8,81,145
Root-beer-1,Electrode-Box-B2,9,89,156
Root-beer-1,Electrode-Box-B2,10,105,158
Khoai-lang-2,Naive,0,0,0
Khoai-lang-2,Naive,1,0,0
Khoai-lang-2,Naive,2,10,47
Khoai-lang-2,Naive,3,11,52
Khoai-lang-2,Naive,4,9,56
Khoai-lang-2,Naive,5,34,95
Khoai-lang-2,Naive,6,21,72
Khoai-lang-2,Naive,7,23,99
Khoai-lang-2,Naive,8,64,136
Khoai-lang-2,Naive,9,75,131
Khoai-lang-2,Naive,10,63,134
Khoai-lang-2,Naive,11,59,139
Khoai-lang-2,Naive,12,51,129
Khoai-lang-2,Naive,13,73,143
Khoai-lang-2,Naive,14,82,136
Khoai-lang-2,Naive,15,70,145
Khoai-lang-2,Naive,16,76,135
Khoai-lang-2,Naive,17,76,150
Khoai-lang-2,Naive,18,63,122
Khoai-lang-2,Naive,19,48,116
Khoai-lang-2,Naive,20,65,134
Khoai-lang-2,Naive,21,75,131
Khoai-lang-2,Naive,22,98,146
Khoai-lang-2,Naive,23,88,139
Khoai-lang-2,Naive,24,94,148
Khoai-lang-2,Naive,25,56,102
Khoai-lang-2,Naive,26,75,143
Khoai-tay-2,Naive,1,21,68
Khoai-tay-2,Naive,2,6,79
Khoai-tay-2,Naive,3,0,83
Khoai-tay-2,Naive,4,28,87
Khoai-tay-2,Naive,5,31,125
Khoai-tay-2,Naive,6,62,144
Khoai-tay-2,Naive,7,87,141
Khoai-tay-2,Naive,8,105,152
Khoai-tay-2,Naive,9,75,148
Khoai-tay-2,Naive,10,101,149
Khoai-tay-2,Naive,11,102,154
Khoai-tay-2,Naive,12,95,144
Khoai-tay-2,Naive,13,77,149
Khoai-tay-2,Naive,14,91,153
OM-2,Naive,-2,0,0
OM-2,Naive,-1,5,18
OM-2,Naive,0,1,7
OM-2,Naive,1,11,33
OM-2,Naive,2,19,62
OM-2,Naive,3,29,117
OM-2,Naive,4,73,126
OM-2,Naive,5,53,135
OM-2,Naive,6,73,138
OM-2,Naive,7,80,131
OM-2,Naive,8,91,141
OM-2,Naive,9,90,135
OM-2,Naive,10,95,142
OM-2,Naive,11,60,133
OM-2,Naive,12,58,142
Khoai-lang-1,Right-Electrode,-4,0,0
Khoai-lang-1,Right-Electrode,-3,2,7
Khoai-lang-1,Right-Electrode,-1,0,0
Khoai-lang-1,Right-Electrode,0,3,69
Khoai-lang-1,Right-Electrode,1,18,97
Khoai-lang-1,Right-Electrode,2,27,84
Khoai-lang-1,Right-Electrode,3,46,121
Khoai-lang-1,Right-Electrode,4,65,143
Khoai-lang-1,Right-Electrode,5,76,141
Khoai-lang-1,Right-Electrode,6,79,146
Khoai-lang-1,Right-Electrode,7,81,153
Khoai-lang-1,Right-Electrode,8,98,144
Khoai-lang-1,Right-Electrode,9,101,149
Khoai-lang-1,Right-Electrode,10,103,150
Khoai-lang-1,Right-Electrode,11,113,154
Khoai-lang-1,Right-Electrode,12,117,148
Khoai-lang-1,Right-Electrode,13,114,146
Khoai-lang-1,Right-Electrode,14,119,132
Khoai-lang-1,Right-Electrode,15,111,156
Khoai-lang-1,Right-Electrode,16,74,156
Khoai-lang-1,Right-Electrode,17,109,139
Khoai-lang-1,Right-Electrode,18,88,138
Khoai-lang-1,Right-Electrode,19,98,140
Khoai-lang-1,Right-Electrode,20,118,153
Khoai-lang-1,Right-Electrode,21,109,146
Khoai-lang-1,Right-Electrode,22,106,141
1 subject group day success total
2 Vu-vuong Naive 0 18 61
3 Vu-vuong Naive 1 35 91
4 Vu-vuong Naive 2 61 127
5 Vu-vuong Naive 3 34 146
6 Vu-vuong Naive 4 61 141
7 Vu-vuong Naive 5 37 151
8 Vu-vuong Naive 6 49 131
9 Vu-vuong Naive 7 65 149
10 Vu-vuong Naive 8 67 141
11 Vu-vuong Naive 9 73 140
12 Vu-vuong Naive 10 72 131
13 Vu-vuong Naive 11 89 158
14 Vu-vuong Naive 12 97 154
15 Vu-vuong Naive 13 93 145
16 Vu-vuong Naive 14 75 147
17 Vu-vuong Naive 15 68 142
18 Vu-vuong Naive 16 96 162
19 Vu-vuong Naive 17 68 144
20 Vu-vuong Naive 18 81 134
21 Vu-vuong Naive 19 66 144
22 Vu-vuong Naive 20 84 127
23 Vu-vuong Naive 21 66 133
24 Khoai-tay-1 Electrode-Box-A -2 1 18
25 Khoai-tay-1 Electrode-Box-A -1 3 24
26 Khoai-tay-1 Electrode-Box-A 0 14 62
27 Khoai-tay-1 Electrode-Box-A 1 22 83
28 Khoai-tay-1 Electrode-Box-A 2 6 79
29 Khoai-tay-1 Electrode-Box-A 3 28 97
30 Khoai-tay-1 Electrode-Box-A 4 60 134
31 Khoai-tay-1 Electrode-Box-A 5 75 138
32 Khoai-tay-1 Electrode-Box-A 6 81 137
33 Khoai-tay-1 Electrode-Box-A 7 78 147
34 Khoai-tay-1 Electrode-Box-A 8 91 132
35 Khoai-tay-1 Electrode-Box-A 9 99 146
36 Khoai-tay-1 Electrode-Box-A 10 94 143
37 Khoai-tay-1 Electrode-Box-A 11 110 156
38 Khoai-tay-1 Electrode-Box-A 12 105 143
39 Khoai-tay-1 Electrode-Box-A 13 106 153
40 Khoai-tay-1 Electrode-Box-A 14 91 152
41 Banh-mi-2 Electrode-Box-A2 -1 2 42
42 Banh-mi-2 Electrode-Box-A2 0 25 97
43 Banh-mi-2 Electrode-Box-A2 1 22 101
44 Banh-mi-2 Electrode-Box-A2 2 22 119
45 Banh-mi-2 Electrode-Box-A2 3 29 118
46 Banh-mi-2 Electrode-Box-A2 4 27 136
47 Banh-mi-2 Electrode-Box-A2 5 43 146
48 Banh-mi-2 Electrode-Box-A2 6 74 146
49 Banh-mi-2 Electrode-Box-A2 7 70 148
50 Banh-mi-2 Electrode-Box-A2 8 65 130
51 Banh-mi-2 Electrode-Box-A2 9 79 151
52 Banh-mi-2 Electrode-Box-A2 10 93 152
53 Egg-tart-2 Electrode-Box-A2 0 9 32
54 Egg-tart-2 Electrode-Box-A2 1 2 38
55 Egg-tart-2 Electrode-Box-A2 2 31 93
56 Egg-tart-2 Electrode-Box-A2 3 44 101
57 Egg-tart-2 Electrode-Box-A2 4 54 131
58 Egg-tart-2 Electrode-Box-A2 5 84 139
59 Egg-tart-2 Electrode-Box-A2 6 85 145
60 Egg-tart-2 Electrode-Box-A2 7 79 143
61 Egg-tart-2 Electrode-Box-A2 8 76 131
62 Egg-tart-2 Electrode-Box-A2 9 88 149
63 Egg-tart-2 Electrode-Box-A2 10 81 151
64 Egg-tart-2 Electrode-Box-A2 11 78 152
65 Egg-tart-2 Electrode-Box-A2 12 96 155
66 Egg-tart-2 Electrode-Box-A2 13 84 155
67 Root-beer-2 Electrode-Box-A2 0 22 74
68 Root-beer-2 Electrode-Box-A2 1 31 87
69 Root-beer-2 Electrode-Box-A2 2 49 134
70 Root-beer-2 Electrode-Box-A2 3 31 89
71 Root-beer-2 Electrode-Box-A2 4 60 140
72 Root-beer-2 Electrode-Box-A2 5 84 147
73 Banh-mi-1 Electrode-Box-B2 0 13 84
74 Banh-mi-1 Electrode-Box-B2 1 35 86
75 Banh-mi-1 Electrode-Box-B2 2 47 110
76 Banh-mi-1 Electrode-Box-B2 3 65 140
77 Banh-mi-1 Electrode-Box-B2 4 70 127
78 Banh-mi-1 Electrode-Box-B2 5 102 142
79 Banh-mi-1 Electrode-Box-B2 6 90 131
80 Banh-mi-1 Electrode-Box-B2 7 109 148
81 Banh-mi-1 Electrode-Box-B2 8 104 137
82 Banh-mi-1 Electrode-Box-B2 9 119 150
83 Banh-mi-1 Electrode-Box-B2 10 121 158
84 Banh-mi-1 Electrode-Box-B2 11 121 148
85 Banh-mi-1 Electrode-Box-B2 12 120 149
86 Banh-mi-1 Electrode-Box-B2 13 135 154
87 Egg-tart-1 Electrode-Box-B2 0 7 56
88 Egg-tart-1 Electrode-Box-B2 1 16 78
89 Egg-tart-1 Electrode-Box-B2 2 23 103
90 Egg-tart-1 Electrode-Box-B2 3 63 120
91 Egg-tart-1 Electrode-Box-B2 4 69 132
92 Egg-tart-1 Electrode-Box-B2 5 83 136
93 Egg-tart-1 Electrode-Box-B2 6 71 142
94 Egg-tart-1 Electrode-Box-B2 7 79 138
95 Egg-tart-1 Electrode-Box-B2 8 98 142
96 Egg-tart-1 Electrode-Box-B2 9 89 139
97 Egg-tart-1 Electrode-Box-B2 10 96 143
98 Egg-tart-1 Electrode-Box-B2 11 96 148
99 Egg-tart-1 Electrode-Box-B2 12 101 156
100 Egg-tart-1 Electrode-Box-B2 13 103 152
101 Egg-tart-1 Electrode-Box-B2 14 97 152
102 Root-beer-1 Electrode-Box-B2 0 11 85
103 Root-beer-1 Electrode-Box-B2 1 18 76
104 Root-beer-1 Electrode-Box-B2 2 40 105
105 Root-beer-1 Electrode-Box-B2 3 55 134
106 Root-beer-1 Electrode-Box-B2 4 75 136
107 Root-beer-1 Electrode-Box-B2 5 64 133
108 Root-beer-1 Electrode-Box-B2 6 104 139
109 Root-beer-1 Electrode-Box-B2 7 98 148
110 Root-beer-1 Electrode-Box-B2 8 81 145
111 Root-beer-1 Electrode-Box-B2 9 89 156
112 Root-beer-1 Electrode-Box-B2 10 105 158
113 Khoai-lang-2 Naive 0 0 0
114 Khoai-lang-2 Naive 1 0 0
115 Khoai-lang-2 Naive 2 10 47
116 Khoai-lang-2 Naive 3 11 52
117 Khoai-lang-2 Naive 4 9 56
118 Khoai-lang-2 Naive 5 34 95
119 Khoai-lang-2 Naive 6 21 72
120 Khoai-lang-2 Naive 7 23 99
121 Khoai-lang-2 Naive 8 64 136
122 Khoai-lang-2 Naive 9 75 131
123 Khoai-lang-2 Naive 10 63 134
124 Khoai-lang-2 Naive 11 59 139
125 Khoai-lang-2 Naive 12 51 129
126 Khoai-lang-2 Naive 13 73 143
127 Khoai-lang-2 Naive 14 82 136
128 Khoai-lang-2 Naive 15 70 145
129 Khoai-lang-2 Naive 16 76 135
130 Khoai-lang-2 Naive 17 76 150
131 Khoai-lang-2 Naive 18 63 122
132 Khoai-lang-2 Naive 19 48 116
133 Khoai-lang-2 Naive 20 65 134
134 Khoai-lang-2 Naive 21 75 131
135 Khoai-lang-2 Naive 22 98 146
136 Khoai-lang-2 Naive 23 88 139
137 Khoai-lang-2 Naive 24 94 148
138 Khoai-lang-2 Naive 25 56 102
139 Khoai-lang-2 Naive 26 75 143
140 Khoai-tay-2 Naive 1 21 68
141 Khoai-tay-2 Naive 2 6 79
142 Khoai-tay-2 Naive 3 0 83
143 Khoai-tay-2 Naive 4 28 87
144 Khoai-tay-2 Naive 5 31 125
145 Khoai-tay-2 Naive 6 62 144
146 Khoai-tay-2 Naive 7 87 141
147 Khoai-tay-2 Naive 8 105 152
148 Khoai-tay-2 Naive 9 75 148
149 Khoai-tay-2 Naive 10 101 149
150 Khoai-tay-2 Naive 11 102 154
151 Khoai-tay-2 Naive 12 95 144
152 Khoai-tay-2 Naive 13 77 149
153 Khoai-tay-2 Naive 14 91 153
154 OM-2 Naive -2 0 0
155 OM-2 Naive -1 5 18
156 OM-2 Naive 0 1 7
157 OM-2 Naive 1 11 33
158 OM-2 Naive 2 19 62
159 OM-2 Naive 3 29 117
160 OM-2 Naive 4 73 126
161 OM-2 Naive 5 53 135
162 OM-2 Naive 6 73 138
163 OM-2 Naive 7 80 131
164 OM-2 Naive 8 91 141
165 OM-2 Naive 9 90 135
166 OM-2 Naive 10 95 142
167 OM-2 Naive 11 60 133
168 OM-2 Naive 12 58 142
169 Khoai-lang-1 Right-Electrode -4 0 0
170 Khoai-lang-1 Right-Electrode -3 2 7
171 Khoai-lang-1 Right-Electrode -1 0 0
172 Khoai-lang-1 Right-Electrode 0 3 69
173 Khoai-lang-1 Right-Electrode 1 18 97
174 Khoai-lang-1 Right-Electrode 2 27 84
175 Khoai-lang-1 Right-Electrode 3 46 121
176 Khoai-lang-1 Right-Electrode 4 65 143
177 Khoai-lang-1 Right-Electrode 5 76 141
178 Khoai-lang-1 Right-Electrode 6 79 146
179 Khoai-lang-1 Right-Electrode 7 81 153
180 Khoai-lang-1 Right-Electrode 8 98 144
181 Khoai-lang-1 Right-Electrode 9 101 149
182 Khoai-lang-1 Right-Electrode 10 103 150
183 Khoai-lang-1 Right-Electrode 11 113 154
184 Khoai-lang-1 Right-Electrode 12 117 148
185 Khoai-lang-1 Right-Electrode 13 114 146
186 Khoai-lang-1 Right-Electrode 14 119 132
187 Khoai-lang-1 Right-Electrode 15 111 156
188 Khoai-lang-1 Right-Electrode 16 74 156
189 Khoai-lang-1 Right-Electrode 17 109 139
190 Khoai-lang-1 Right-Electrode 18 88 138
191 Khoai-lang-1 Right-Electrode 19 98 140
192 Khoai-lang-1 Right-Electrode 20 118 153
193 Khoai-lang-1 Right-Electrode 21 109 146
194 Khoai-lang-1 Right-Electrode 22 106 141
@@ -0,0 +1,668 @@
# Subject-Series Data Export Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace the export modal's assignable-axes UI with three pickers — X field (rows), Data value (cells), and Group by — where columns are always the subjects, matching the cross-subject metrics plot.
**Architecture:** Add X-coordinate helpers to `dataExport.js` (`numericFieldVal`, `listSampleMembers`, `buildSampleIndex`) and a focused `buildSubjectSeriesMatrix` that emits the existing `toCSV` shape. Rewrite `ExportDataModal.jsx` to the three pickers. Then delete the now-dead assignable-axes machinery.
**Tech Stack:** React 18, plain ES modules, Jest + `@testing-library/react` (jsdom). No new dependencies.
**Spec:** `docs/superpowers/specs/2026-07-19-subject-series-export-design.md`
## Global Constraints
- `dataExport.js` stays pure: no React, no I/O.
- Cell value semantics: `0` and `''` are **real** values; only genuinely-missing lookups render blank (`''`), via the existing `hasValue` + `extractValue`.
- CSV: LF (`\n`) line endings; escape every cell via the existing `escapeCsvCell`.
- **X = `Date`** → rows are distinct `YYYY-MM-DD` dates, sorted.
- **X = `# Days Reach`** (`'__days__'`) → rows are `1, 2, 3, …`; the max over subjects of that subject's count of recorded statuses (a status must have `animal_id` and `date`). A cell for `(subject, N)` reads the subject's Nth recorded day, counting **all** records sorted ascending by date (per-status ordinal, Day 1 = earliest).
- **X = daily field** (its `fieldId`) → rows are the distinct **numeric** values (`numericFieldVal`), sorted ascending; non-numeric excluded. A cell for `(subject, v)` reads that subject's first status (by date) whose field equals `v`.
- Columns are **always** the subjects (`listSubjectMembers`), clustered by group then name. Entirely-blank sample rows are dropped; subject columns are always kept.
- Group by defaults to the saved field (`groupField` prop), validated against `__none__`/`__name__`/`__id__`/active `subjectTemplate` fields; coerced to `__none__` when unset or stale.
- CSV: line 1 is `Data,<metric label>`; blank line; a `Group,…` row only when grouping (missing group → `—`); header `= <X label>,<subject headers…>`; data rows `= <X value>,<values…>`. Filename via existing `csvFilename(title, dataLabel)`.
- Default X = `Date`; default Data = first parameter (`Total attempts`).
- Run tests from `frontend/`: `npx jest tests/dataExport.test.js`, `npx jest tests/ExportDataModal.test.jsx`, full `npm test`.
- Two suites (`tests/ExperimentCalendar.test.jsx`, `tests/ExperimentDayView.test.jsx`, 31 tests) fail for a PRE-EXISTING, unrelated reason and must be left as-is; changes must introduce no new failures.
---
### Task 1: X-coordinate helpers
**Files:**
- Modify: `frontend/src/lib/dataExport.js` (add three exports; keep everything else for now)
- Test: `frontend/tests/dataExport.test.js`
**Interfaces:**
- Consumes: existing private `dateKey(date)`.
- Produces:
- `numericFieldVal(status, field): number|null``field.builtin ? status[field.key] : status.custom_fields[field.fieldId]`, `parseFloat`, non-numeric → null.
- `listSampleMembers(statuses, xField, dailyTemplate): Array<{ id, label, sampleValue }>` — row members for the chosen X (`xField` is `'__date__'`, `'__days__'`, or a daily `fieldId`).
- `buildSampleIndex(statuses, xField, dailyTemplate): Map<animalId, Map<sampleValue, status>>` — first status per `(subject, sampleValue)` wins, per-subject date-ordered.
- [ ] **Step 1: Write the failing tests**
Append to `frontend/tests/dataExport.test.js`:
```js
import { numericFieldVal, listSampleMembers, buildSampleIndex } from '../src/lib/dataExport';
const sDaily = [{ fieldId: 'c-w', key: 'w', label: 'Weight', builtin: false, active: true }];
const sStatuses = [
{ animal_id: 'a1', date: '2026-07-01T00:00:00Z', custom_fields: { 'c-w': '250' } },
{ animal_id: 'a1', date: '2026-07-03T00:00:00Z', custom_fields: { 'c-w': '260' } },
{ animal_id: 'a2', date: '2026-07-01T00:00:00Z', custom_fields: { 'c-w': '250' } },
];
describe('numericFieldVal', () => {
it('reads builtin via key and custom via fieldId, parsed to number', () => {
expect(numericFieldVal({ weight: '250' }, { builtin: true, key: 'weight' })).toBe(250);
expect(numericFieldVal({ custom_fields: { 'c-w': '12.5' } }, { builtin: false, fieldId: 'c-w' })).toBe(12.5);
});
it('returns null for non-numeric, missing, or no field', () => {
expect(numericFieldVal({ custom_fields: { 'c-w': 'abc' } }, { builtin: false, fieldId: 'c-w' })).toBeNull();
expect(numericFieldVal({}, { builtin: true, key: 'weight' })).toBeNull();
expect(numericFieldVal({ weight: 1 }, null)).toBeNull();
});
});
describe('listSampleMembers', () => {
it('date: distinct dates sorted', () => {
expect(listSampleMembers(sStatuses, '__date__', sDaily).map((m) => m.sampleValue)).toEqual(['2026-07-01', '2026-07-03']);
});
it('# days reach: 1..max records per subject', () => {
expect(listSampleMembers(sStatuses, '__days__', sDaily).map((m) => m.sampleValue)).toEqual([1, 2]);
});
it('daily field: distinct numeric values sorted ascending', () => {
expect(listSampleMembers(sStatuses, 'c-w', sDaily).map((m) => m.sampleValue)).toEqual([250, 260]);
});
});
describe('buildSampleIndex', () => {
it('date coord maps animalId -> dateKey -> status', () => {
const idx = buildSampleIndex(sStatuses, '__date__', sDaily);
expect(idx.get('a1').get('2026-07-03').date).toContain('2026-07-03');
});
it('# days reach assigns per-subject ordinals in date order', () => {
const idx = buildSampleIndex(sStatuses, '__days__', sDaily);
expect(idx.get('a1').get(1).date).toContain('2026-07-01');
expect(idx.get('a1').get(2).date).toContain('2026-07-03');
expect(idx.get('a2').get(1).date).toContain('2026-07-01');
expect(idx.get('a2').has(2)).toBe(false);
});
it('daily field maps numeric value to first status by date', () => {
const idx = buildSampleIndex(sStatuses, 'c-w', sDaily);
expect(idx.get('a1').get(250).date).toContain('2026-07-01');
expect(idx.get('a1').get(260).date).toContain('2026-07-03');
});
});
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `npx jest tests/dataExport.test.js -t "listSampleMembers"`
Expected: FAIL — `listSampleMembers is not defined`.
- [ ] **Step 3: Write minimal implementation**
Add to `frontend/src/lib/dataExport.js` (after `listExportableParams`, before the private `hasValue`/`dateKey` — note `dateKey` is defined lower in the file but hoisted as a function declaration, so referencing it here is fine):
```js
// Numeric value of a daily field on one status (mirrors the metrics plot's helper).
export function numericFieldVal(status, field) {
if (!field) return null;
const raw = field.builtin ? status?.[field.key] : status?.custom_fields?.[field.fieldId];
const n = parseFloat(raw);
return Number.isNaN(n) ? null : n;
}
// Ordered row members for the chosen X coordinate.
// xField: '__date__' | '__days__' | <daily fieldId>
export function listSampleMembers(statuses, xField, dailyTemplate) {
const list = statuses ?? [];
if (xField === '__date__') {
const keys = new Set();
for (const s of list) if (s?.date) keys.add(dateKey(s.date));
return [...keys].sort().map((k) => ({ id: k, label: k, sampleValue: k }));
}
if (xField === '__days__') {
const counts = new Map();
for (const s of list) {
if (s?.animal_id == null || !s?.date) continue;
counts.set(s.animal_id, (counts.get(s.animal_id) ?? 0) + 1);
}
const max = counts.size ? Math.max(...counts.values()) : 0;
return Array.from({ length: max }, (_, i) => ({ id: String(i + 1), label: String(i + 1), sampleValue: i + 1 }));
}
const field = (dailyTemplate ?? []).find((f) => f.fieldId === xField);
const vals = new Set();
for (const s of list) {
const v = numericFieldVal(s, field);
if (v !== null) vals.add(v);
}
return [...vals].sort((a, b) => a - b).map((v) => ({ id: String(v), label: String(v), sampleValue: v }));
}
// animalId -> Map(sampleValue -> status). First status per (subject, sampleValue) wins,
// scanning each subject's statuses in ascending date order.
export function buildSampleIndex(statuses, xField, dailyTemplate) {
const bySubject = new Map();
for (const s of statuses ?? []) {
if (s?.animal_id == null || !s?.date) continue;
if (!bySubject.has(s.animal_id)) bySubject.set(s.animal_id, []);
bySubject.get(s.animal_id).push(s);
}
const field = xField !== '__date__' && xField !== '__days__'
? (dailyTemplate ?? []).find((f) => f.fieldId === xField)
: null;
const index = new Map();
for (const [animalId, subjStatuses] of bySubject) {
const ordered = [...subjStatuses].sort((a, b) => dateKey(a.date).localeCompare(dateKey(b.date)));
const m = new Map();
ordered.forEach((s, i) => {
let key;
if (xField === '__date__') key = dateKey(s.date);
else if (xField === '__days__') key = i + 1;
else { const v = numericFieldVal(s, field); if (v === null) return; key = v; }
if (!m.has(key)) m.set(key, s);
});
index.set(animalId, m);
}
return index;
}
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `npx jest tests/dataExport.test.js`
Expected: PASS (new suites green; all pre-existing suites still green).
- [ ] **Step 5: Commit**
```bash
git add frontend/src/lib/dataExport.js frontend/tests/dataExport.test.js
git commit -m "feat(export): X-coordinate helpers (date, # days reach, daily field)"
```
---
### Task 2: buildSubjectSeriesMatrix
**Files:**
- Modify: `frontend/src/lib/dataExport.js` (add `buildSubjectSeriesMatrix` + private `xFieldLabel`)
- Test: `frontend/tests/dataExport.test.js`
**Interfaces:**
- Consumes: `listSubjectMembers`, `listSampleMembers`, `buildSampleIndex`, `extractValue`, private `hasValue`.
- Produces: `buildSubjectSeriesMatrix({ xField, dataParam, groupField }, { statuses, animals, dailyTemplate })``{ corner, context: { label: 'Data', value }, groupAxis: 'col'|null, columns: [{ id, label, animalId, group }], rows: [{ member: { id, label, sampleValue }, values: [value|''] }] }` — the same shape the existing `toCSV` consumes (`groupAxis: 'col'` triggers its group-row branch).
- [ ] **Step 1: Write the failing tests**
Append to `frontend/tests/dataExport.test.js`:
```js
import { buildSubjectSeriesMatrix } from '../src/lib/dataExport';
const ssAnimals = [
{ id: 'a1', animal_name: 'Alpha', animal_id_string: 'R-001', subject_info: { grp: 'Control' } },
{ id: 'a2', animal_name: 'Beta', animal_id_string: 'R-002', subject_info: { grp: 'Drug' } },
];
const ssStatuses = [
{ animal_id: 'a1', date: '2026-07-01T00:00:00Z', analysis_summary: { total: 5 } },
{ animal_id: 'a1', date: '2026-07-02T00:00:00Z', analysis_summary: { total: 7 } },
{ animal_id: 'a2', date: '2026-07-01T00:00:00Z', analysis_summary: { total: 9 } },
];
const totalParam = { id: '__total__', label: 'Total attempts', kind: 'total', group: 'metrics' };
describe('buildSubjectSeriesMatrix', () => {
it('date X: subjects as columns, date rows, blank where missing, context names the metric', () => {
const m = buildSubjectSeriesMatrix(
{ xField: '__date__', dataParam: totalParam, groupField: '__none__' },
{ statuses: ssStatuses, animals: ssAnimals, dailyTemplate: [] },
);
expect(m.corner).toBe('Date');
expect(m.context).toEqual({ label: 'Data', value: 'Total attempts' });
expect(m.groupAxis).toBeNull();
expect(m.columns.map((c) => c.label)).toEqual(['Alpha', 'Beta']);
expect(m.rows.map((r) => r.member.label)).toEqual(['2026-07-01', '2026-07-02']);
expect(m.rows[0].values).toEqual([5, 9]);
expect(m.rows[1].values).toEqual([7, '']);
});
it('# days reach X: rows align each subject by day ordinal', () => {
const m = buildSubjectSeriesMatrix(
{ xField: '__days__', dataParam: totalParam, groupField: '__none__' },
{ statuses: ssStatuses, animals: ssAnimals, dailyTemplate: [] },
);
expect(m.corner).toBe('# Days Reach');
expect(m.rows.map((r) => r.member.label)).toEqual(['1', '2']);
expect(m.rows[0].values).toEqual([5, 9]);
expect(m.rows[1].values).toEqual([7, '']);
});
it('grouping sets groupAxis col and clusters subjects', () => {
const m = buildSubjectSeriesMatrix(
{ xField: '__date__', dataParam: totalParam, groupField: 'grp' },
{ statuses: ssStatuses, animals: ssAnimals, dailyTemplate: [] },
);
expect(m.groupAxis).toBe('col');
expect(m.columns.map((c) => [c.group, c.label])).toEqual([['Control', 'Alpha'], ['Drug', 'Beta']]);
});
it('drops entirely-blank rows', () => {
const statuses = [
{ animal_id: 'a1', date: '2026-07-01T00:00:00Z', analysis_summary: { total: 5 } },
{ animal_id: 'a1', date: '2026-07-05T00:00:00Z', analysis_summary: null },
];
const m = buildSubjectSeriesMatrix(
{ xField: '__date__', dataParam: totalParam, groupField: '__none__' },
{ statuses, animals: ssAnimals, dailyTemplate: [] },
);
expect(m.rows.map((r) => r.member.label)).toEqual(['2026-07-01']);
});
});
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `npx jest tests/dataExport.test.js -t "buildSubjectSeriesMatrix"`
Expected: FAIL — `buildSubjectSeriesMatrix is not defined`.
- [ ] **Step 3: Write minimal implementation**
Add to `frontend/src/lib/dataExport.js` (after `buildSampleIndex`):
```js
function xFieldLabel(xField, dailyTemplate) {
if (xField === '__date__') return 'Date';
if (xField === '__days__') return '# Days Reach';
return (dailyTemplate ?? []).find((f) => f.fieldId === xField)?.label ?? '';
}
// Subjects as columns, X-field values as rows, one Data value per cell.
// Returns the { context, corner, groupAxis, columns, rows } shape toCSV consumes.
export function buildSubjectSeriesMatrix({ xField, dataParam, groupField }, { statuses, animals, dailyTemplate }) {
const columns = listSubjectMembers(animals, groupField);
const rowMembers = listSampleMembers(statuses, xField, dailyTemplate);
const index = buildSampleIndex(statuses, xField, dailyTemplate);
const cellValue = (sampleValue, animalId) => {
const status = index.get(animalId)?.get(sampleValue);
if (!status || !dataParam) return '';
const v = extractValue(status, dataParam);
return hasValue(v) ? v : '';
};
let rows = rowMembers.map((rm) => ({
member: rm,
values: columns.map((c) => cellValue(rm.sampleValue, c.animalId)),
}));
rows = rows.filter((r) => r.values.some((v) => v !== '')); // drop entirely-blank sample rows
const grouped = !!groupField && groupField !== '__none__';
return {
corner: xFieldLabel(xField, dailyTemplate),
context: { label: 'Data', value: dataParam?.label ?? '' },
groupAxis: grouped ? 'col' : null,
columns,
rows,
};
}
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `npx jest tests/dataExport.test.js`
Expected: PASS (all suites green).
- [ ] **Step 5: Commit**
```bash
git add frontend/src/lib/dataExport.js frontend/tests/dataExport.test.js
git commit -m "feat(export): buildSubjectSeriesMatrix — subjects as columns over an X field"
```
---
### Task 3: Rewrite ExportDataModal to the three pickers
**Files:**
- Modify: `frontend/src/components/ExportDataModal.jsx` (full body rewrite; keep `triggerDownload`)
- Test: `frontend/tests/ExportDataModal.test.jsx` (rewrite)
**Interfaces:**
- Consumes: `listExportableParams`, `buildSubjectSeriesMatrix`, `toCSV`, `csvFilename` from `../lib/dataExport`; `experimentsApi.getDailyStatuses`.
- Props unchanged: `experimentTitle`, `experimentId`, `dailyTemplate`, `animals`, `subjectTemplate` (default `[]`), `groupField` (default `'__none__'`), `onClose`.
- Produces: modal with three selects — X (rows), Data (cells), Group by.
- [ ] **Step 1: Rewrite the component test**
Replace the entire contents of `frontend/tests/ExportDataModal.test.jsx` with:
```jsx
import React from 'react';
import { render, screen, waitFor, fireEvent } from '@testing-library/react';
import ExportDataModal from '../src/components/ExportDataModal';
import * as client from '../src/api/client';
jest.mock('../src/api/client', () => ({
experimentsApi: { getDailyStatuses: jest.fn() },
}));
const animals = [
{ id: 'a1', animal_name: 'Alpha', animal_id_string: 'R-001', subject_info: { grp: 'Control' } },
{ id: 'a2', animal_name: 'Beta', animal_id_string: 'R-002', subject_info: { grp: 'Drug' } },
];
const statuses = [
{ animal_id: 'a1', date: '2026-07-01T00:00:00Z', analysis_summary: { total: 5, success_rate: 0.5 } },
{ animal_id: 'a1', date: '2026-07-02T00:00:00Z', analysis_summary: { total: 7, success_rate: 0.7 } },
{ animal_id: 'a2', date: '2026-07-01T00:00:00Z', analysis_summary: { total: 9, success_rate: 0.9 } },
];
const subjectTemplate = [{ fieldId: 'grp', label: 'Group', active: true }];
beforeEach(() => {
jest.clearAllMocks();
client.experimentsApi.getDailyStatuses.mockResolvedValue(statuses);
});
// Intercept the Blob handed to URL.createObjectURL so tests can read the real CSV.
// Also stub anchor.click() — jsdom treats a click on <a href="blob:..."> as an
// unimplemented navigation that throws async and can flake a later test.
function mockDownload() {
let blob = null;
const origCreate = global.URL.createObjectURL;
const origRevoke = global.URL.revokeObjectURL;
const origClick = window.HTMLAnchorElement.prototype.click;
global.URL.createObjectURL = jest.fn((b) => { blob = b; return 'blob:mock'; });
global.URL.revokeObjectURL = jest.fn();
window.HTMLAnchorElement.prototype.click = jest.fn();
return {
getBlob: () => blob,
restore: async () => {
await new Promise((r) => setTimeout(r, 0)); // flush handleExport's setTimeout revoke
global.URL.createObjectURL = origCreate;
global.URL.revokeObjectURL = origRevoke;
window.HTMLAnchorElement.prototype.click = origClick;
},
};
}
function readBlobText(blob) {
return new Promise((resolve, reject) => {
const fr = new FileReader();
fr.onload = () => resolve(fr.result);
fr.onerror = reject;
fr.readAsText(blob);
});
}
function renderModal(props = {}) {
return render(
<ExportDataModal
experimentId="exp-1"
experimentTitle="My Study"
dailyTemplate={[]}
animals={animals}
subjectTemplate={subjectTemplate}
groupField="__none__"
onClose={jest.fn()}
{...props}
/>,
);
}
describe('ExportDataModal', () => {
it('renders X / Data / Group by pickers; X defaults to Date', async () => {
renderModal();
expect(await screen.findByLabelText('X axis (rows)')).toHaveValue('__date__');
expect(screen.getByLabelText('Data (cells)')).toBeInTheDocument();
expect(screen.getByLabelText('Group by')).toHaveValue('__none__');
await waitFor(() => expect(screen.getByText(/2 rows × 2 subjects/i)).toBeInTheDocument());
});
it('default export (X=Date, Data=Total attempts) has subject columns, date rows, blank where missing', async () => {
const dl = mockDownload();
renderModal();
await screen.findByLabelText('X axis (rows)');
await waitFor(() => expect(screen.getByText(/2 rows/i)).toBeInTheDocument());
fireEvent.click(screen.getByText('Export CSV'));
const text = await readBlobText(dl.getBlob());
expect(text).toBe('Data,Total attempts\n\nDate,Alpha,Beta\n2026-07-01,5,9\n2026-07-02,7,');
await dl.restore();
});
it('switching X to # Days Reach makes rows the day ordinals', async () => {
const dl = mockDownload();
renderModal();
fireEvent.change(await screen.findByLabelText('X axis (rows)'), { target: { value: '__days__' } });
await waitFor(() => expect(screen.getByText(/2 rows/i)).toBeInTheDocument());
fireEvent.click(screen.getByText('Export CSV'));
const text = await readBlobText(dl.getBlob());
expect(text).toBe('Data,Total attempts\n\n# Days Reach,Alpha,Beta\n1,5,9\n2,7,');
await dl.restore();
});
it('a valid group field adds a Group row and clusters subjects', async () => {
const dl = mockDownload();
renderModal({ groupField: 'grp' });
await screen.findByLabelText('X axis (rows)');
await waitFor(() => expect(screen.getByText(/2 rows/i)).toBeInTheDocument());
fireEvent.click(screen.getByText('Export CSV'));
const text = await readBlobText(dl.getBlob());
expect(text).toMatch(/^Group,Control,Drug$/m);
await dl.restore();
});
});
```
- [ ] **Step 2: Run the test to verify it fails**
Run: `npx jest tests/ExportDataModal.test.jsx`
Expected: FAIL — old modal has no `X axis (rows)` / `Data (cells)` labels.
- [ ] **Step 3: Rewrite the component**
Replace the entire contents of `frontend/src/components/ExportDataModal.jsx` with:
```jsx
import React, { useEffect, useMemo, useState } from 'react';
import { experimentsApi } from '../api/client';
import Button from './ui/Button';
import Alert from './ui/Alert';
import { listExportableParams, buildSubjectSeriesMatrix, toCSV, csvFilename } from '../lib/dataExport';
const PARAM_GROUP_LABELS = { metrics: 'Session metrics', daily: 'Daily record fields' };
function triggerDownload(filename, text) {
const blob = new Blob([text], { type: 'text/csv;charset=utf-8;' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
setTimeout(() => URL.revokeObjectURL(url), 0);
}
export default function ExportDataModal({
experimentTitle, experimentId, dailyTemplate, animals, subjectTemplate = [], groupField = '__none__', onClose,
}) {
const [loading, setLoading] = useState(true);
const [error, setError] = useState(null);
const [statuses, setStatuses] = useState([]);
const [xField, setXField] = useState('__date__');
const [dataId, setDataId] = useState(null);
const [groupBy, setGroupBy] = useState(() => {
const valid = groupField === '__none__' || groupField === '__name__' || groupField === '__id__'
|| subjectTemplate.some((f) => f.active && f.fieldId === groupField);
return valid ? groupField : '__none__';
});
useEffect(() => {
let alive = true;
setLoading(true);
setError(null);
experimentsApi.getDailyStatuses(experimentId, {})
.then((data) => { if (alive) setStatuses(data); })
.catch((err) => { if (alive) setError(err.message); })
.finally(() => { if (alive) setLoading(false); });
return () => { alive = false; };
}, [experimentId]);
const params = useMemo(() => listExportableParams(dailyTemplate, statuses), [dailyTemplate, statuses]);
const effDataId = dataId ?? params[0]?.id ?? null;
const dataParam = params.find((p) => p.id === effDataId) ?? params[0] ?? null;
const activeDaily = useMemo(() => (dailyTemplate ?? []).filter((f) => f.active), [dailyTemplate]);
const xOptions = useMemo(() => [
{ id: '__date__', label: 'Date' },
{ id: '__days__', label: '# Days Reach' },
...activeDaily.map((f) => ({ id: f.fieldId, label: f.label })),
], [activeDaily]);
const matrix = useMemo(
() => (dataParam
? buildSubjectSeriesMatrix({ xField, dataParam, groupField: groupBy }, { statuses, animals, dailyTemplate })
: { columns: [], rows: [], context: { label: 'Data', value: '' }, corner: '', groupAxis: null }),
[xField, dataParam, groupBy, statuses, animals, dailyTemplate],
);
const hasData = matrix.rows.length > 0;
const groupedParams = useMemo(() => {
const out = [];
for (const p of params) {
let g = out.find((x) => x.group === p.group);
if (!g) { g = { group: p.group, items: [] }; out.push(g); }
g.items.push(p);
}
return out;
}, [params]);
function handleExport() {
if (!hasData || !dataParam) return;
triggerDownload(csvFilename(experimentTitle, dataParam.label), toCSV(matrix));
onClose();
}
if (loading) return <p className="text-sm text-gray-400" aria-live="polite">Loading data</p>;
if (error) return (
<div className="space-y-4">
<Alert type="error" message={error} />
<div className="flex justify-end"><Button variant="secondary" onClick={onClose}>Close</Button></div>
</div>
);
const selectCls = 'w-full border border-gray-200 rounded px-2 py-1.5 text-sm bg-white focus:outline-none focus:ring-1 focus:ring-indigo-400';
return (
<div className="space-y-4">
<p className="text-sm text-gray-500">
One value per subject over an X axis each column is a subject, each row an X value. Blank where a subject has no value.
</p>
<div>
<label htmlFor="export-x" className="block text-xs font-medium text-gray-500 mb-1">X axis (rows)</label>
<select id="export-x" value={xField} onChange={(e) => setXField(e.target.value)} className={selectCls}>
{xOptions.map((o) => <option key={o.id} value={o.id}>{o.label}</option>)}
</select>
</div>
<div>
<label htmlFor="export-data" className="block text-xs font-medium text-gray-500 mb-1">Data (cells)</label>
<select id="export-data" value={effDataId ?? ''} onChange={(e) => setDataId(e.target.value)} className={selectCls}>
{groupedParams.map((g) => (
<optgroup key={g.group} label={PARAM_GROUP_LABELS[g.group] ?? g.group}>
{g.items.map((p) => <option key={p.id} value={p.id}>{p.label}</option>)}
</optgroup>
))}
</select>
</div>
<div>
<label htmlFor="export-group" className="block text-xs font-medium text-gray-500 mb-1">Group by</label>
<select id="export-group" value={groupBy} onChange={(e) => setGroupBy(e.target.value)} className={selectCls}>
<option value="__none__"> None </option>
<option value="__name__">Name</option>
<option value="__id__">Subject ID</option>
{subjectTemplate.filter((f) => f.active).map((f) => (
<option key={f.fieldId} value={f.fieldId}>{f.label}</option>
))}
</select>
</div>
<p className="text-xs text-gray-400">
{hasData
? `${matrix.rows.length} row${matrix.rows.length !== 1 ? 's' : ''} × ${matrix.columns.length} subject${matrix.columns.length !== 1 ? 's' : ''}`
: 'No data for this selection yet.'}
</p>
<div className="flex justify-end gap-2 pt-2">
<Button variant="secondary" onClick={onClose}>Cancel</Button>
<Button onClick={handleExport} disabled={!hasData}>Export CSV</Button>
</div>
</div>
);
}
```
- [ ] **Step 4: Run the test to verify it passes**
Run: `npx jest tests/ExportDataModal.test.jsx`
Expected: PASS (all four cases).
- [ ] **Step 5: Run the full suite**
Run: `npm test`
Expected: `dataExport` + `ExportDataModal` green; only the two known pre-existing suites fail; no new failures.
- [ ] **Step 6: Commit**
```bash
git add frontend/src/components/ExportDataModal.jsx frontend/tests/ExportDataModal.test.jsx
git commit -m "feat(export): modal becomes X / Data / Group by pickers, subjects as columns"
```
---
### Task 4: Remove the dead assignable-axes code
**Files:**
- Modify: `frontend/src/lib/dataExport.js` (delete unused exports)
- Test: `frontend/tests/dataExport.test.js` (delete their describe blocks + imports)
**Interfaces:**
- Removes (now unused after Task 3): `EXPORT_DIMENSIONS`, `dimLabel`, `otherDimension`, `listDateMembers`, `buildStatusIndex`, `listParameterMembers`, `listDimensionMembers`, and `buildMatrix`.
- Keeps: `extractValue`, `listExportableParams`, `subjectGroupValue`, `listSubjectMembers`, `numericFieldVal`, `listSampleMembers`, `buildSampleIndex`, `buildSubjectSeriesMatrix`, `toCSV`, `csvFilename`, and the private `hasValue`, `dateKey`, `escapeCsvCell`, `xFieldLabel`.
- [ ] **Step 1: Delete the dead source exports**
In `frontend/src/lib/dataExport.js`, delete these eight exported definitions entirely (they are only used by each other and the old modal, now replaced): `EXPORT_DIMENSIONS`, `dimLabel`, `otherDimension`, `listDateMembers`, `buildStatusIndex`, `listParameterMembers`, `listDimensionMembers`, `buildMatrix`. Also update the top-of-file comment to describe the subject-series export (drop the "dimension-agnostic buildMatrix" wording). Keep `dateKey`, `hasValue`, `escapeCsvCell`, and everything listed under "Keeps" above.
- [ ] **Step 2: Delete their tests**
In `frontend/tests/dataExport.test.js`, delete the describe blocks and their now-unused import identifiers for the removed functions: the `describe('dimension primitives', …)`, `describe('subjectGroupValue', …)` KEEP (subjectGroupValue is kept — do NOT delete it), `describe('listDateMembers', …)`, `describe('buildStatusIndex', …)`, `describe('listParameterMembers / listDimensionMembers', …)`, and `describe('buildMatrix', …)` blocks. Remove `EXPORT_DIMENSIONS`, `dimLabel`, `otherDimension`, `listDateMembers`, `buildStatusIndex`, `listDimensionMembers`, `listParameterMembers`, and `buildMatrix` from the file's `import` statements, leaving the imports for kept functions (`extractValue`, `listExportableParams`, `subjectGroupValue`, `listSubjectMembers`, `numericFieldVal`, `listSampleMembers`, `buildSampleIndex`, `buildSubjectSeriesMatrix`, `toCSV`, `csvFilename`).
- [ ] **Step 3: Verify nothing references the removed names**
Run: `grep -rnE "buildMatrix|EXPORT_DIMENSIONS|listDimensionMembers|buildStatusIndex|listDateMembers|listParameterMembers|otherDimension|dimLabel" frontend/src frontend/tests`
Expected: no output (all references gone).
- [ ] **Step 4: Run the suites**
Run: `npx jest tests/dataExport.test.js tests/ExportDataModal.test.jsx`
Expected: PASS. Then `npm test` — only the two known pre-existing suites fail, no new failures.
- [ ] **Step 5: Commit**
```bash
git add frontend/src/lib/dataExport.js frontend/tests/dataExport.test.js
git commit -m "refactor(export): remove dead assignable-axes helpers and their tests"
```
---
## Self-Review Notes
- **Spec coverage:** three pickers (Task 3) · X = Date/# Days Reach/daily field with the exact semantics (Tasks 12) · subjects always columns, clustered by group, blank rows dropped (Task 2) · Group by validated/defaulted (Task 3) · CSV `Data,<metric>` + group row + X-corner (Tasks 23, reusing `toCSV`) · assignable-axes UI/machinery removed (Tasks 34). All covered.
- **Type consistency:** sample member `{ id, label, sampleValue }`, subject column `{ id, label, animalId, group }`, and matrix `{ context, corner, groupAxis, columns, rows }` are used identically across Tasks 13 and match the existing `toCSV`.
- **Out of scope (per spec):** assignable columns/pinned, subject subsetting, analyzed-only day counting, footer/repeat line.
@@ -0,0 +1,86 @@
# Subject-Series Data Export — Design
**Date:** 2026-07-19
**Status:** Approved (design)
**Area:** `frontend/src/lib/dataExport.js`, `frontend/src/components/ExportDataModal.jsx`, `frontend/tests/`
**Supersedes:** the assignable-axes UI from `2026-07-19-configurable-data-export-design.md` (that branch's `buildMatrix`/`toCSV`/subject-group work is reused; its rows/columns/pinned modal is replaced).
## Problem
The export modal currently offers assignable row/column/pinned axes. That is more than needed. The user wants the export to mirror the **cross-subject metrics plot**: pick an **X field** and a **data value**, with each **subject as a column series**, empty where a subject has no data point, plus each subject's **group** included. The one capability missing today is choosing the X (row) field — it is fixed to calendar Date, whereas the plot's X-axis offers **# Days Reach** and daily fields.
## Layout (fixed)
- **Columns = subjects**, one series per subject, ordered clustered-by-group then by name (existing `listSubjectMembers`).
- **Rows = the chosen X field's values.**
- **Cells = the chosen Data value** for that (subject, X); empty (`''`) when absent. `0` and `''` remain real values (existing `hasValue`/`extractValue`).
- **Group row** gives each subject's group value (existing group-row support).
There is no pinned dimension and no assignable columns.
## The three pickers
1. **X (rows).** Options: `Date`, `# Days Reach`, and each **active** daily-template field. Default `Date` (keeps the same date-rows × subject-columns grid as today; only the line-1 label changes to `Data`, see CSV output).
2. **Data (cells).** Options: `listExportableParams(dailyTemplate, statuses)` — total, success rate, each count category, each daily field. Default: first parameter (`Total attempts`).
3. **Group by.** Options: `— None —`, `Name`, `Subject ID`, and each active `subject_info` field. Defaults to the experiment's saved group field (`localStorage['exp-subject-group-<id>']`), validated against the available options and coerced to `__none__` when unset or stale. Always visible (subjects are always the columns).
Preview line: `N rows × M subjects`. Export disabled when there are no data rows.
## X-field semantics
Each X coordinate maps a `(subject, X-value)` pair to a single daily status, from which the Data value is read:
- **`Date`** → rows are the distinct calendar dates present (`YYYY-MM-DD`), sorted. Cell reads the subject's status on that date. (Today's behavior.)
- **`# Days Reach`** → rows are `1, 2, 3, …` up to the maximum, per subject, of the count of that subject's recorded daily statuses. A cell for `(subject, N)` reads the subject's **Nth recorded day**, counting **all** recorded statuses sorted ascending by date (Day 1 = earliest record; days without an analysis summary are still counted — this intentionally differs from the plot, which counts analyzed sessions only).
- **daily field** (e.g. `Weight (g)`) → rows are the distinct **numeric** values that field takes across the data (`numericFieldVal`: builtin via `status[field.key]`, custom via `status.custom_fields[field.fieldId]`, `parseFloat`, non-numeric → excluded), sorted ascending. A cell for `(subject, v)` reads that subject's status where the field equals `v`; on ties, first status by date wins.
Row members that are entirely blank across all subjects are dropped (matching Date's existing behavior); subject columns are always kept even when empty.
## CSV output
Structure (X = # Days Reach, Data = Success rate, Group by = Treatment):
```
Data,Success rate
Group,Control,Control,Drug
# Days Reach,Mouse-A,Mouse-B,Mouse-C
1,0.5,0.6,0.7
2,0.55,,0.72
3,0.6,0.65,
```
- **Line 1** names the Data metric so the file is self-describing: two cells, `Data` and the metric label.
- **Blank line.**
- **Group row** (`Group` + each subject's group), only when Group by ≠ None; missing group value → `—`.
- **Header row:** the X-field label in the corner, then subject headers.
- **Data rows:** the X value, then one cell per subject.
LF line endings; every cell escaped via existing `escapeCsvCell`. Filename: existing `csvFilename(title, dataLabel)`.
## Implementation notes
Reuse from the current branch: `extractValue`, `listExportableParams`, `subjectGroupValue`, `listSubjectMembers`, `escapeCsvCell`, `toCSV` (its `{ context, corner, groupAxis, columns, rows }` shape is kept), and `csvFilename`.
Replace: the general assignable-axes `buildMatrix(config, ctx)` and its members/dispatch helpers (`otherDimension`, `dimLabel`, `EXPORT_DIMENSIONS`, `listDateMembers`, `listDimensionMembers`, `listParameterMembers`, `buildStatusIndex` in its current form) — remove what the fixed layout no longer uses rather than leaving dead code.
Add:
- `numericFieldVal(status, field)` — mirror of the plot's helper (or import/share it).
- `listSampleMembers(statuses, xField, dailyTemplate)` → the ordered row members for the chosen X coordinate (`{ id, label, ... }`).
- `buildSampleIndex(statuses, xField, dailyTemplate)``Map(animalId → Map(sampleValue → status))`, first-status-per-cell wins, encoding the date/#days/field resolution above.
- `buildSubjectSeriesMatrix({ xField, dataParam, groupField }, { statuses, animals, dailyTemplate })``{ context: { label: 'Data', value: dataParam.label }, corner: <X label>, groupAxis: grouping ? 'col' : null, columns: <subjects>, rows: [{ member, values }] }`, consumable by the existing `toCSV`.
Simplify `ExportDataModal.jsx` to the three pickers over these helpers; drop the Rows/Columns/Fixed controls.
## Testing
- `listSampleMembers` / `buildSampleIndex` for each X coordinate: Date (unchanged), # Days Reach (all-records ordinal, per-subject alignment), daily field (distinct numeric values sorted, non-numeric dropped, tie first-by-date).
- `buildSubjectSeriesMatrix`: subjects as columns clustered by group; empty cells where a subject lacks a point; blank row dropped; `groupAxis` null when Group by None.
- `toCSV` reused; add/keep a snapshot for the new context line + group row.
- `ExportDataModal`: renders three pickers; default X = Date produces date rows × subject columns with the chosen metric in cells; switching X to # Days Reach changes rows to ordinals; group row present when a valid group field is chosen; stale/unset group field coerces to None.
## Out of scope (YAGNI)
- Assignable columns / pinned dimension (removed).
- Choosing a subset of subjects (all subjects always included).
- Matching the plot's analyzed-only day counting (we count all recorded days).
- A footer or repeated context line.
+59 -94
View File
@@ -2,10 +2,7 @@ import React, { useEffect, useMemo, useState } from 'react';
import { experimentsApi } from '../api/client';
import Button from './ui/Button';
import Alert from './ui/Alert';
import {
EXPORT_DIMENSIONS, dimLabel, otherDimension,
listDimensionMembers, buildMatrix, toCSV, csvFilename,
} from '../lib/dataExport';
import { listExportableParams, buildSubjectSeriesMatrix, toCSV, csvFilename } from '../lib/dataExport';
const PARAM_GROUP_LABELS = { metrics: 'Session metrics', daily: 'Daily record fields' };
@@ -28,10 +25,13 @@ export default function ExportDataModal({
const [error, setError] = useState(null);
const [statuses, setStatuses] = useState([]);
const [rowDim, setRowDim] = useState('date');
const [colDim, setColDim] = useState('subject');
const [pinnedId, setPinnedId] = useState(null);
const [groupBy, setGroupBy] = useState(groupField ?? '__none__');
const [xField, setXField] = useState('__date__');
const [dataId, setDataId] = useState(null);
const [groupBy, setGroupBy] = useState(() => {
const valid = groupField === '__none__' || groupField === '__name__' || groupField === '__id__'
|| subjectTemplate.some((f) => f.active && f.fieldId === groupField);
return valid ? groupField : '__none__';
});
useEffect(() => {
let alive = true;
@@ -44,42 +44,38 @@ export default function ExportDataModal({
return () => { alive = false; };
}, [experimentId]);
const pinnedDim = otherDimension(rowDim, colDim);
const subjectIsAxis = rowDim === 'subject' || colDim === 'subject';
const effGroup = subjectIsAxis ? groupBy : '__none__';
const ctx = useMemo(() => ({ statuses, animals, dailyTemplate }), [statuses, animals, dailyTemplate]);
const params = useMemo(() => listExportableParams(dailyTemplate ?? [], statuses), [dailyTemplate, statuses]);
const effDataId = dataId ?? params[0]?.id ?? null;
const dataParam = params.find((p) => p.id === effDataId) ?? params[0] ?? null;
// Members of the pinned dimension populate its value dropdown.
const pinnedMembers = useMemo(
() => listDimensionMembers(pinnedDim, { ...ctx, groupField: effGroup }),
[pinnedDim, ctx, effGroup],
);
const effPinnedId = pinnedId ?? pinnedMembers[0]?.id ?? null;
const pinnedMember = pinnedMembers.find((m) => m.id === effPinnedId) ?? pinnedMembers[0] ?? null;
const activeDaily = useMemo(() => (dailyTemplate ?? []).filter((f) => f.active), [dailyTemplate]);
const xOptions = useMemo(() => [
{ id: '__date__', label: 'Date' },
{ id: '__days__', label: '# Days Reach (ordinal)' },
...activeDaily.map((f) => ({ id: f.fieldId, label: f.label })),
], [activeDaily]);
const matrix = useMemo(
() => (pinnedMember
? buildMatrix({ rowDim, colDim, pinnedMember, groupField: effGroup }, ctx)
: { columns: [], rows: [], context: { label: '', value: '' }, corner: '', groupAxis: null }),
[rowDim, colDim, pinnedMember, effGroup, ctx],
() => (dataParam
? buildSubjectSeriesMatrix({ xField, dataParam, groupField: groupBy }, { statuses, animals, dailyTemplate })
: { columns: [], rows: [], context: { label: 'Data', value: '' }, corner: '', groupAxis: null }),
[xField, dataParam, groupBy, statuses, animals, dailyTemplate],
);
const hasData = matrix.rows.length > 0;
// When the user changes an axis, keep the two distinct and reset the pinned pick.
function changeRowDim(next) {
setRowDim(next);
if (colDim === next) setColDim(otherDimension(next, null) ?? EXPORT_DIMENSIONS.find((d) => d.dim !== next).dim);
setPinnedId(null);
}
function changeColDim(next) {
setColDim(next);
if (rowDim === next) setRowDim(EXPORT_DIMENSIONS.find((d) => d.dim !== next).dim);
setPinnedId(null);
}
const groupedParams = useMemo(() => {
const out = [];
for (const p of params) {
let g = out.find((x) => x.group === p.group);
if (!g) { g = { group: p.group, items: [] }; out.push(g); }
g.items.push(p);
}
return out;
}, [params]);
function handleExport() {
if (!hasData || !pinnedMember) return;
triggerDownload(csvFilename(experimentTitle, pinnedMember.label), toCSV(matrix));
if (!hasData || !dataParam) return;
triggerDownload(csvFilename(experimentTitle, dataParam.label), toCSV(matrix));
onClose();
}
@@ -92,66 +88,46 @@ export default function ExportDataModal({
);
const selectCls = 'w-full border border-gray-200 rounded px-2 py-1.5 text-sm bg-white focus:outline-none focus:ring-1 focus:ring-indigo-400';
const colOptions = EXPORT_DIMENSIONS.filter((d) => d.dim !== rowDim);
return (
<div className="space-y-4">
<p className="text-sm text-gray-500">
Choose what goes on the rows and columns. The remaining dimension is fixed to one value, shown at the top of the file.
One value per subject over an X axis each column is a subject, each row an X value. Blank where a subject has no value.
</p>
<div className="grid grid-cols-2 gap-3">
<div>
<label htmlFor="export-rows" className="block text-xs font-medium text-gray-500 mb-1">Rows</label>
<select id="export-rows" value={rowDim} onChange={(e) => changeRowDim(e.target.value)} className={selectCls}>
{EXPORT_DIMENSIONS.map((d) => <option key={d.dim} value={d.dim}>{d.label}</option>)}
</select>
</div>
<div>
<label htmlFor="export-cols" className="block text-xs font-medium text-gray-500 mb-1">Columns</label>
<select id="export-cols" value={colDim} onChange={(e) => changeColDim(e.target.value)} className={selectCls}>
{colOptions.map((d) => <option key={d.dim} value={d.dim}>{d.label}</option>)}
</select>
</div>
</div>
<div>
<label htmlFor="export-pinned" className="block text-xs font-medium text-gray-500 mb-1">
Fixed: {dimLabel(pinnedDim)}
</label>
<select
id="export-pinned"
value={effPinnedId ?? ''}
onChange={(e) => setPinnedId(e.target.value)}
className={selectCls}
>
{pinnedDim === 'parameter'
? groupParamMembers(pinnedMembers).map((g) => (
<optgroup key={g.group} label={PARAM_GROUP_LABELS[g.group] ?? g.group}>
{g.items.map((m) => <option key={m.id} value={m.id}>{m.label}</option>)}
</optgroup>
))
: pinnedMembers.map((m) => <option key={m.id} value={m.id}>{m.label}</option>)}
<label htmlFor="export-x" className="block text-xs font-medium text-gray-500 mb-1">X axis (rows)</label>
<select id="export-x" value={xField} onChange={(e) => setXField(e.target.value)} className={selectCls}>
{xOptions.map((o) => <option key={o.id} value={o.id}>{o.label}</option>)}
</select>
</div>
{subjectIsAxis && (
<div>
<label htmlFor="export-group" className="block text-xs font-medium text-gray-500 mb-1">Group by</label>
<select id="export-group" value={groupBy} onChange={(e) => setGroupBy(e.target.value)} className={selectCls}>
<option value="__none__"> None </option>
<option value="__name__">Name</option>
<option value="__id__">Subject ID</option>
{subjectTemplate.filter((f) => f.active).map((f) => (
<option key={f.fieldId} value={f.fieldId}>{f.label}</option>
))}
</select>
</div>
)}
<div>
<label htmlFor="export-data" className="block text-xs font-medium text-gray-500 mb-1">Data (cells)</label>
<select id="export-data" value={effDataId ?? ''} onChange={(e) => setDataId(e.target.value)} className={selectCls}>
{groupedParams.map((g) => (
<optgroup key={g.group} label={PARAM_GROUP_LABELS[g.group] ?? g.group}>
{g.items.map((p) => <option key={p.id} value={p.id}>{p.label}</option>)}
</optgroup>
))}
</select>
</div>
<div>
<label htmlFor="export-group" className="block text-xs font-medium text-gray-500 mb-1">Group by</label>
<select id="export-group" value={groupBy} onChange={(e) => setGroupBy(e.target.value)} className={selectCls}>
<option value="__none__"> None </option>
<option value="__name__">Name</option>
<option value="__id__">Subject ID</option>
{subjectTemplate.filter((f) => f.active).map((f) => (
<option key={f.fieldId} value={f.fieldId}>{f.label}</option>
))}
</select>
</div>
<p className="text-xs text-gray-400">
{hasData
? `${matrix.rows.length} ${matrix.corner.toLowerCase()}${matrix.rows.length !== 1 ? 's' : ''} × ${matrix.columns.length} ${dimLabel(colDim).toLowerCase()}${matrix.columns.length !== 1 ? 's' : ''}`
? `${matrix.rows.length} row${matrix.rows.length !== 1 ? 's' : ''} × ${matrix.columns.length} subject${matrix.columns.length !== 1 ? 's' : ''}`
: 'No data for this selection yet.'}
</p>
@@ -162,14 +138,3 @@ export default function ExportDataModal({
</div>
);
}
// Group parameter members by their .group for <optgroup> rendering, preserving order.
function groupParamMembers(members) {
const out = [];
for (const m of members) {
let g = out.find((x) => x.group === m.group);
if (!g) { g = { group: m.group, items: [] }; out.push(g); }
g.items.push(m);
}
return out;
}
+104 -102
View File
@@ -1,4 +1,7 @@
// Pure helpers for exporting daily-parameter data as a date×subject CSV matrix.
// Pure helpers for exporting daily-parameter data as a CSV matrix. The export is a
// subject-series matrix: subjects are always columns, the chosen X coordinate
// (Date / # Days Reach / a daily field) provides the rows, and one Data
// parameter supplies the cell values.
// No React, no I/O — mirrors the crossSubjectChart.js pure-helper pattern.
// Read a single parameter's raw value from one daily status.
@@ -54,18 +57,104 @@ export function listExportableParams(dailyTemplate, statuses) {
return params;
}
export const EXPORT_DIMENSIONS = [
{ dim: 'date', label: 'Date' },
{ dim: 'subject', label: 'Subject' },
{ dim: 'parameter', label: 'Parameter' },
];
export function dimLabel(dim) {
return EXPORT_DIMENSIONS.find((d) => d.dim === dim)?.label ?? '';
// Numeric value of a daily field on one status (mirrors the metrics plot's helper).
export function numericFieldVal(status, field) {
if (!field) return null;
const raw = field.builtin ? status?.[field.key] : status?.custom_fields?.[field.fieldId];
const n = parseFloat(raw);
return Number.isNaN(n) ? null : n;
}
export function otherDimension(rowDim, colDim) {
return EXPORT_DIMENSIONS.map((d) => d.dim).find((d) => d !== rowDim && d !== colDim) ?? null;
// Ordered row members for the chosen X coordinate.
// xField: '__date__' | '__days__' | <daily fieldId>
export function listSampleMembers(statuses, xField, dailyTemplate) {
const list = statuses ?? [];
if (xField === '__date__') {
const keys = new Set();
for (const s of list) if (s?.date) keys.add(dateKey(s.date));
return [...keys].sort().map((k) => ({ id: k, label: k, sampleValue: k }));
}
if (xField === '__days__') {
const counts = new Map();
for (const s of list) {
if (s?.animal_id == null || !s?.date) continue;
counts.set(s.animal_id, (counts.get(s.animal_id) ?? 0) + 1);
}
const max = counts.size ? Math.max(...counts.values()) : 0;
return Array.from({ length: max }, (_, i) => ({ id: String(i + 1), label: String(i + 1), sampleValue: i + 1 }));
}
const field = (dailyTemplate ?? []).find((f) => f.fieldId === xField);
const vals = new Set();
for (const s of list) {
const v = numericFieldVal(s, field);
if (v !== null) vals.add(v);
}
return [...vals].sort((a, b) => a - b).map((v) => ({ id: String(v), label: String(v), sampleValue: v }));
}
// animalId -> Map(sampleValue -> status). First status per (subject, sampleValue) wins,
// scanning each subject's statuses in ascending date order.
export function buildSampleIndex(statuses, xField, dailyTemplate) {
const bySubject = new Map();
for (const s of statuses ?? []) {
if (s?.animal_id == null || !s?.date) continue;
if (!bySubject.has(s.animal_id)) bySubject.set(s.animal_id, []);
bySubject.get(s.animal_id).push(s);
}
const field = xField !== '__date__' && xField !== '__days__'
? (dailyTemplate ?? []).find((f) => f.fieldId === xField)
: null;
const index = new Map();
for (const [animalId, subjStatuses] of bySubject) {
const ordered = [...subjStatuses].sort((a, b) => dateKey(a.date).localeCompare(dateKey(b.date)));
const m = new Map();
ordered.forEach((s, i) => {
let key;
if (xField === '__date__') key = dateKey(s.date);
else if (xField === '__days__') key = i + 1;
else { const v = numericFieldVal(s, field); if (v === null) return; key = v; }
if (!m.has(key)) m.set(key, s);
});
index.set(animalId, m);
}
return index;
}
function xFieldLabel(xField, dailyTemplate) {
if (xField === '__date__') return 'Date';
if (xField === '__days__') return '# Days Reach (ordinal)';
return (dailyTemplate ?? []).find((f) => f.fieldId === xField)?.label ?? '';
}
// Subjects as columns, X-field values as rows, one Data value per cell.
// Returns the { context, corner, groupAxis, columns, rows } shape toCSV consumes.
export function buildSubjectSeriesMatrix({ xField, dataParam, groupField }, { statuses, animals, dailyTemplate }) {
const columns = listSubjectMembers(animals, groupField);
const rowMembers = listSampleMembers(statuses, xField, dailyTemplate);
const index = buildSampleIndex(statuses, xField, dailyTemplate);
const cellValue = (sampleValue, animalId) => {
const status = index.get(animalId)?.get(sampleValue);
if (!status || !dataParam) return '';
const v = extractValue(status, dataParam);
return hasValue(v) ? v : '';
};
let rows = rowMembers.map((rm) => ({
member: rm,
values: columns.map((c) => cellValue(rm.sampleValue, c.animalId)),
}));
rows = rows.filter((r) => r.values.some((v) => v !== '')); // drop entirely-blank sample rows
const grouped = !!groupField && groupField !== '__none__';
return {
corner: xFieldLabel(xField, dailyTemplate),
context: { label: 'Data', value: dataParam?.label ?? '' },
groupAxis: grouped ? 'col' : null,
columns,
rows,
};
}
// Resolve a subject's group value for a given group field id.
@@ -79,29 +168,6 @@ export function subjectGroupValue(animal, groupField) {
return v === null || v === undefined || v === '' ? '—' : v;
}
export function listDateMembers(statuses) {
const keys = new Set();
for (const s of statuses ?? []) {
if (s?.date) keys.add(dateKey(s.date));
}
return [...keys].sort().map((k) => ({ id: k, label: k, dateKey: k }));
}
// dateKey -> Map(animalId -> status); first status per (date, animal) wins,
// scanning in date order (matches the legacy buildMatrix tie-break).
export function buildStatusIndex(statuses) {
const index = new Map();
const ordered = [...(statuses ?? [])].sort((a, b) => dateKey(a?.date).localeCompare(dateKey(b?.date)));
for (const s of ordered) {
if (!s?.date || s.animal_id === null || s.animal_id === undefined) continue;
const dk = dateKey(s.date);
if (!index.has(dk)) index.set(dk, new Map());
const byAnimal = index.get(dk);
if (!byAnimal.has(s.animal_id)) byAnimal.set(s.animal_id, s);
}
return index;
}
export function listSubjectMembers(animals, groupField) {
const list = [...(animals ?? [])];
const nameCounts = new Map();
@@ -112,7 +178,7 @@ export function listSubjectMembers(animals, groupField) {
const members = list.map((a) => {
const name = a?.animal_name ?? '';
const label = nameCounts.get(name) > 1 ? `${name} (${a?.animal_id_string ?? ''})` : name;
return { id: a.id, label, animalId: a.id, group: subjectGroupValue(a, groupField) };
return { id: a?.id, label, animalId: a?.id, group: subjectGroupValue(a, groupField) };
});
const grouped = !!groupField && groupField !== '__none__';
members.sort((x, y) => {
@@ -125,19 +191,6 @@ export function listSubjectMembers(animals, groupField) {
return members;
}
export function listParameterMembers(dailyTemplate, statuses) {
return listExportableParams(dailyTemplate, statuses).map((p) => ({
id: p.id, label: p.label, param: p, group: p.group,
}));
}
export function listDimensionMembers(dim, { statuses, animals, dailyTemplate, groupField }) {
if (dim === 'date') return listDateMembers(statuses);
if (dim === 'subject') return listSubjectMembers(animals, groupField);
if (dim === 'parameter') return listParameterMembers(dailyTemplate, statuses);
return [];
}
// A cell has a value unless it is null, undefined, or the empty string.
// (0 and false are real values.)
function hasValue(v) {
@@ -148,66 +201,13 @@ function dateKey(date) {
return String(date).slice(0, 10); // ISO datetime or date → YYYY-MM-DD
}
// Pivot statuses into a 2-D matrix per the chosen row/column dimensions.
// The leftover (pinned) dimension is fixed to pinnedMember. Every cell resolves
// a (dateKey, animalId, param) triple through the status index.
export function buildMatrix(config, ctx) {
const { rowDim, colDim, pinnedMember, groupField } = config;
const pinnedDim = otherDimension(rowDim, colDim);
const index = buildStatusIndex(ctx.statuses);
const full = { ...ctx, groupField };
const rowMembers = listDimensionMembers(rowDim, full);
const colMembers = listDimensionMembers(colDim, full);
const cellValue = (rowM, colM) => {
const byDim = { [rowDim]: rowM, [colDim]: colM, [pinnedDim]: pinnedMember };
const dk = byDim.date?.dateKey ?? null;
const animalId = byDim.subject?.animalId ?? null;
const param = byDim.parameter?.param ?? null;
if (dk === null || animalId === null || animalId === undefined || !param) return '';
const status = index.get(dk)?.get(animalId);
if (!status) return '';
const v = extractValue(status, param);
return hasValue(v) ? v : '';
};
let rows = rowMembers.map((rowM) => ({
member: rowM,
values: colMembers.map((colM) => cellValue(rowM, colM)),
}));
// Empty-member rule: keep all subjects; drop entirely-blank date/parameter rows & columns.
if (rowDim !== 'subject') rows = rows.filter((r) => r.values.some((v) => v !== ''));
let keptCols = colMembers.map((_, i) => i);
if (colDim !== 'subject') keptCols = keptCols.filter((i) => rows.some((r) => r.values[i] !== ''));
const columns = keptCols.map((i) => colMembers[i]);
rows = rows.map((r) => ({ member: r.member, values: keptCols.map((i) => r.values[i]) }));
const grouped = !!groupField && groupField !== '__none__';
const groupAxis = grouped
? (rowDim === 'subject' ? 'row' : colDim === 'subject' ? 'col' : null)
: null;
return {
rowDim,
colDim,
pinnedDim,
corner: dimLabel(rowDim),
context: { label: dimLabel(pinnedDim), value: pinnedMember?.label ?? '' },
groupAxis,
columns,
rows,
};
}
function escapeCsvCell(value) {
const s = value === null || value === undefined ? '' : String(value);
return /[",\n]/.test(s) ? `"${s.replace(/"/g, '""')}"` : s;
}
// Serialize a matrix to an RFC-4180-ish CSV string (LF line endings):
// <pinned label>,<pinned value>
// Data,<metric>
// (blank line)
// [Group,<group per subject column>] -- only when groupAxis === 'col'
// <corner>[,Group?],<column headers...>
@@ -222,6 +222,8 @@ export function toCSV(matrix) {
lines.push(groupRow.map(escapeCsvCell).join(','));
}
// toCSV is a general serializer; the subject-series builder never emits groupAxis
// === 'row' (only 'col' or null), but the branch is retained for that generality.
const header = matrix.groupAxis === 'row'
? ['Group', matrix.corner, ...matrix.columns.map((c) => c.label)]
: [matrix.corner, ...matrix.columns.map((c) => c.label)];
+1 -1
View File
@@ -417,7 +417,7 @@ export default function ExperimentDetail() {
dailyTemplate={dailyTemplate}
animals={animals}
subjectTemplate={subjectTemplate}
groupField={groupField}
groupField={localStorage.getItem(`exp-subject-group-${id}`) ?? '__none__'}
onClose={() => setShowExport(false)}
/>
</Modal>
+77 -17
View File
@@ -12,8 +12,9 @@ const animals = [
{ id: 'a2', animal_name: 'Beta', animal_id_string: 'R-002', subject_info: { grp: 'Drug' } },
];
const statuses = [
{ animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 5 } },
{ animal_id: 'a2', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 9 } },
{ animal_id: 'a1', date: '2026-07-01T00:00:00Z', analysis_summary: { total: 5, success_rate: 0.5 } },
{ animal_id: 'a1', date: '2026-07-02T00:00:00Z', analysis_summary: { total: 7, success_rate: 0.7 } },
{ animal_id: 'a2', date: '2026-07-01T00:00:00Z', analysis_summary: { total: 9, success_rate: 0.9 } },
];
const subjectTemplate = [{ fieldId: 'grp', label: 'Group', active: true }];
@@ -22,6 +23,37 @@ beforeEach(() => {
client.experimentsApi.getDailyStatuses.mockResolvedValue(statuses);
});
// Intercept the Blob handed to URL.createObjectURL so tests can read the real CSV.
// Also stub anchor.click() — jsdom treats a click on <a href="blob:..."> as an
// unimplemented navigation that throws async and can flake a later test.
function mockDownload() {
let blob = null;
const origCreate = global.URL.createObjectURL;
const origRevoke = global.URL.revokeObjectURL;
const origClick = window.HTMLAnchorElement.prototype.click;
global.URL.createObjectURL = jest.fn((b) => { blob = b; return 'blob:mock'; });
global.URL.revokeObjectURL = jest.fn();
window.HTMLAnchorElement.prototype.click = jest.fn();
return {
getBlob: () => blob,
restore: async () => {
await new Promise((r) => setTimeout(r, 0)); // flush handleExport's setTimeout revoke
global.URL.createObjectURL = origCreate;
global.URL.revokeObjectURL = origRevoke;
window.HTMLAnchorElement.prototype.click = origClick;
},
};
}
function readBlobText(blob) {
return new Promise((resolve, reject) => {
const fr = new FileReader();
fr.onload = () => resolve(fr.result);
fr.onerror = reject;
fr.readAsText(blob);
});
}
function renderModal(props = {}) {
return render(
<ExportDataModal
@@ -38,27 +70,55 @@ function renderModal(props = {}) {
}
describe('ExportDataModal', () => {
it('defaults to Rows=Date, Columns=Subject and shows the preview', async () => {
it('renders X / Data / Group by pickers; X defaults to Date', async () => {
renderModal();
expect(await screen.findByLabelText('Rows')).toHaveValue('date');
expect(screen.getByLabelText('Columns')).toHaveValue('subject');
await waitFor(() => expect(screen.getByText(/1 date.*2 subjects/i)).toBeInTheDocument());
expect(await screen.findByLabelText('X axis (rows)')).toHaveValue('__date__');
expect(screen.getByLabelText('Data (cells)')).toBeInTheDocument();
expect(screen.getByLabelText('Group by')).toHaveValue('__none__');
await waitFor(() => expect(screen.getByText(/2 rows × 2 subjects/i)).toBeInTheDocument());
});
it('hides the Group by control until Subject is an axis', async () => {
it('default export (X=Date, Data=Total attempts) has subject columns, date rows, blank where missing', async () => {
const dl = mockDownload();
renderModal();
await screen.findByLabelText('Rows');
// subject is a column by default -> Group by visible
expect(screen.getByLabelText('Group by')).toBeInTheDocument();
// move subject off both axes: Rows=Date, Columns=Parameter -> subject pinned
fireEvent.change(screen.getByLabelText('Columns'), { target: { value: 'parameter' } });
await waitFor(() => expect(screen.queryByLabelText('Group by')).not.toBeInTheDocument());
await screen.findByLabelText('X axis (rows)');
await waitFor(() => expect(screen.getByText(/2 rows/i)).toBeInTheDocument());
fireEvent.click(screen.getByText('Export CSV'));
const text = await readBlobText(dl.getBlob());
expect(text).toBe('Data,Total attempts\n\nDate,Alpha,Beta\n2026-07-01,5,9\n2026-07-02,7,');
await dl.restore();
});
it('keeps row and column dimensions distinct', async () => {
it('switching X to # Days Reach makes rows the day ordinals', async () => {
const dl = mockDownload();
renderModal();
const rows = await screen.findByLabelText('Rows');
fireEvent.change(rows, { target: { value: 'subject' } }); // collides with column=subject
await waitFor(() => expect(screen.getByLabelText('Columns')).not.toHaveValue('subject'));
fireEvent.change(await screen.findByLabelText('X axis (rows)'), { target: { value: '__days__' } });
await waitFor(() => expect(screen.getByText(/2 rows/i)).toBeInTheDocument());
fireEvent.click(screen.getByText('Export CSV'));
const text = await readBlobText(dl.getBlob());
expect(text).toBe('Data,Total attempts\n\n# Days Reach (ordinal),Alpha,Beta\n1,5,9\n2,7,');
await dl.restore();
});
it('a valid group field adds a Group row and clusters subjects', async () => {
const dl = mockDownload();
renderModal({ groupField: 'grp' });
await screen.findByLabelText('X axis (rows)');
await waitFor(() => expect(screen.getByText(/2 rows/i)).toBeInTheDocument());
fireEvent.click(screen.getByText('Export CSV'));
const text = await readBlobText(dl.getBlob());
expect(text).toMatch(/^Group,Control,Drug$/m);
await dl.restore();
});
it('a stale/unset group field coerces to None (no Group row in the export)', async () => {
const dl = mockDownload();
renderModal({ groupField: 'ghost-field' });
await screen.findByLabelText('X axis (rows)');
await waitFor(() => expect(screen.getByText(/2 rows/i)).toBeInTheDocument());
fireEvent.click(screen.getByText('Export CSV'));
const text = await readBlobText(dl.getBlob());
expect(text).not.toMatch(/^Group,/m);
await dl.restore();
});
});
+118 -128
View File
@@ -71,73 +71,6 @@ describe('listExportableParams', () => {
});
});
import { buildMatrix } from '../src/lib/dataExport';
const gAnimals = [
{ id: 'a2', animal_name: 'Beta', animal_id_string: 'R-002', subject_info: { grp: 'Drug' } },
{ id: 'a1', animal_name: 'Alpha', animal_id_string: 'R-001', subject_info: { grp: 'Control' } },
];
const gStatuses = [
{ animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 5, success_rate: 0.5 } },
{ animal_id: 'a2', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 9, success_rate: 0.9 } },
{ animal_id: 'a1', date: '2026-07-02T00:00:00.000Z', analysis_summary: { total: 7, success_rate: 0.7 } },
];
const ctx = { statuses: gStatuses, animals: gAnimals, dailyTemplate: [] };
const totalMember = { id: '__total__', label: 'Total attempts', param: { kind: 'total' }, group: 'metrics' };
describe('buildMatrix', () => {
it('default preset: rows=date, cols=subject, pinned=parameter (legacy layout)', () => {
const m = buildMatrix({ rowDim: 'date', colDim: 'subject', pinnedMember: totalMember, groupField: '__none__' }, ctx);
expect(m.corner).toBe('Date');
expect(m.context).toEqual({ label: 'Parameter', value: 'Total attempts' });
expect(m.columns.map((c) => c.label)).toEqual(['Alpha', 'Beta']);
expect(m.rows.map((r) => r.member.label)).toEqual(['2026-07-01', '2026-07-02']);
expect(m.rows[0].values).toEqual([5, 9]);
expect(m.rows[1].values).toEqual([7, '']); // Beta blank on 07-02
expect(m.groupAxis).toBeNull();
});
it('drops blank date rows but keeps all subject columns', () => {
const statuses = [
{ animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 0 } }, // 0 is a value
{ animal_id: 'a1', date: '2026-07-03T00:00:00.000Z', analysis_summary: null }, // no value
];
const m = buildMatrix({ rowDim: 'date', colDim: 'subject', pinnedMember: totalMember, groupField: '__none__' },
{ statuses, animals: gAnimals, dailyTemplate: [] });
expect(m.rows.map((r) => r.member.label)).toEqual(['2026-07-01']);
expect(m.columns.length).toBe(2);
expect(m.rows[0].values).toEqual([0, '']);
});
it('subject as columns sets groupAxis=col and clusters subjects', () => {
const m = buildMatrix({ rowDim: 'date', colDim: 'subject', pinnedMember: totalMember, groupField: 'grp' }, ctx);
expect(m.groupAxis).toBe('col');
expect(m.columns.map((c) => [c.group, c.label])).toEqual([['Control', 'Alpha'], ['Drug', 'Beta']]);
});
it('subject as rows sets groupAxis=row; pinned date; cols=parameter', () => {
const dateMember = { id: '2026-07-01', label: '2026-07-01', dateKey: '2026-07-01' };
const m = buildMatrix({ rowDim: 'subject', colDim: 'parameter', pinnedMember: dateMember, groupField: 'grp' },
{ statuses: gStatuses, animals: gAnimals, dailyTemplate: [] });
expect(m.corner).toBe('Subject');
expect(m.context).toEqual({ label: 'Date', value: '2026-07-01' });
expect(m.groupAxis).toBe('row');
expect(m.rows.map((r) => [r.member.group, r.member.label])).toEqual([['Control', 'Alpha'], ['Drug', 'Beta']]);
// columns are parameters (Total attempts, Success rate); Alpha on 07-01: total 5, rate 0.5
const totalCol = m.columns.findIndex((c) => c.label === 'Total attempts');
expect(m.rows[0].values[totalCol]).toBe(5);
});
it('drops blank parameter columns (params always dropped when empty)', () => {
// dailyTemplate adds a custom field with no data anywhere -> its column drops
const dailyTemplate = [{ fieldId: 'c-x', key: 'x', label: 'X', builtin: false, active: true }];
const dateMember = { id: '2026-07-01', label: '2026-07-01', dateKey: '2026-07-01' };
const m = buildMatrix({ rowDim: 'subject', colDim: 'parameter', pinnedMember: dateMember, groupField: '__none__' },
{ statuses: gStatuses, animals: gAnimals, dailyTemplate });
expect(m.columns.map((c) => c.label)).not.toContain('X');
});
});
import { toCSV, csvFilename } from '../src/lib/dataExport';
describe('toCSV', () => {
@@ -206,23 +139,7 @@ describe('csvFilename', () => {
});
});
import {
EXPORT_DIMENSIONS, dimLabel, otherDimension, subjectGroupValue,
listDateMembers, buildStatusIndex,
} from '../src/lib/dataExport';
describe('dimension primitives', () => {
it('exposes the three dimensions with labels', () => {
expect(EXPORT_DIMENSIONS.map((d) => d.dim)).toEqual(['date', 'subject', 'parameter']);
expect(dimLabel('date')).toBe('Date');
expect(dimLabel('subject')).toBe('Subject');
expect(dimLabel('parameter')).toBe('Parameter');
});
it('otherDimension returns the leftover', () => {
expect(otherDimension('date', 'subject')).toBe('parameter');
expect(otherDimension('parameter', 'date')).toBe('subject');
});
});
import { subjectGroupValue } from '../src/lib/dataExport';
describe('subjectGroupValue', () => {
const animal = { animal_name: 'Alpha', animal_id_string: 'R-001', subject_info: { sex: 'M', cohort: '' } };
@@ -241,37 +158,7 @@ describe('subjectGroupValue', () => {
});
});
describe('listDateMembers', () => {
it('returns sorted unique dateKeys with id/label', () => {
const statuses = [
{ date: '2026-07-02T00:00:00.000Z' },
{ date: '2026-07-01T12:00:00.000Z' },
{ date: '2026-07-02T09:00:00.000Z' },
];
expect(listDateMembers(statuses)).toEqual([
{ id: '2026-07-01', label: '2026-07-01', dateKey: '2026-07-01' },
{ id: '2026-07-02', label: '2026-07-02', dateKey: '2026-07-02' },
]);
});
it('tolerates empty input', () => {
expect(listDateMembers(undefined)).toEqual([]);
});
});
describe('buildStatusIndex', () => {
it('indexes by dateKey then animalId, first status wins', () => {
const statuses = [
{ animal_id: 'a1', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 7 } },
{ animal_id: 'a1', date: '2026-07-01T06:00:00.000Z', analysis_summary: { total: 99 } },
{ animal_id: 'a2', date: '2026-07-01T00:00:00.000Z', analysis_summary: { total: 3 } },
];
const idx = buildStatusIndex(statuses);
expect(idx.get('2026-07-01').get('a1').analysis_summary.total).toBe(7);
expect(idx.get('2026-07-01').get('a2').analysis_summary.total).toBe(3);
});
});
import { listSubjectMembers, listParameterMembers, listDimensionMembers } from '../src/lib/dataExport';
import { listSubjectMembers } from '../src/lib/dataExport';
describe('listSubjectMembers', () => {
const animals = [
@@ -300,19 +187,122 @@ describe('listSubjectMembers', () => {
});
});
describe('listParameterMembers / listDimensionMembers', () => {
const dailyTemplate = [{ fieldId: 'b-vitals', key: 'vitals', label: 'Vitals', builtin: true, active: true }];
const statuses = [{ date: '2026-07-01T00:00:00.000Z', animal_id: 'a1', analysis_summary: { total: 1 } }];
const animals = [{ id: 'a1', animal_name: 'Alpha', animal_id_string: 'R-001' }];
it('wraps params with id/label/param/group', () => {
const m = listParameterMembers(dailyTemplate, statuses);
expect(m[0]).toMatchObject({ id: '__total__', label: 'Total attempts', group: 'metrics' });
expect(m[0].param).toMatchObject({ kind: 'total' });
import { numericFieldVal, listSampleMembers, buildSampleIndex } from '../src/lib/dataExport';
const sDaily = [{ fieldId: 'c-w', key: 'w', label: 'Weight', builtin: false, active: true }];
const sStatuses = [
{ animal_id: 'a1', date: '2026-07-01T00:00:00Z', custom_fields: { 'c-w': '250' } },
{ animal_id: 'a1', date: '2026-07-03T00:00:00Z', custom_fields: { 'c-w': '260' } },
{ animal_id: 'a2', date: '2026-07-01T00:00:00Z', custom_fields: { 'c-w': '250' } },
];
describe('numericFieldVal', () => {
it('reads builtin via key and custom via fieldId, parsed to number', () => {
expect(numericFieldVal({ weight: '250' }, { builtin: true, key: 'weight' })).toBe(250);
expect(numericFieldVal({ custom_fields: { 'c-w': '12.5' } }, { builtin: false, fieldId: 'c-w' })).toBe(12.5);
});
it('dispatches by dimension', () => {
const ctx = { statuses, animals, dailyTemplate, groupField: '__none__' };
expect(listDimensionMembers('date', ctx).map((d) => d.id)).toEqual(['2026-07-01']);
expect(listDimensionMembers('subject', ctx).map((d) => d.id)).toEqual(['a1']);
expect(listDimensionMembers('parameter', ctx)[0].id).toBe('__total__');
it('returns null for non-numeric, missing, or no field', () => {
expect(numericFieldVal({ custom_fields: { 'c-w': 'abc' } }, { builtin: false, fieldId: 'c-w' })).toBeNull();
expect(numericFieldVal({}, { builtin: true, key: 'weight' })).toBeNull();
expect(numericFieldVal({ weight: 1 }, null)).toBeNull();
});
});
describe('listSampleMembers', () => {
it('date: distinct dates sorted', () => {
expect(listSampleMembers(sStatuses, '__date__', sDaily).map((m) => m.sampleValue)).toEqual(['2026-07-01', '2026-07-03']);
});
it('# days reach: 1..max records per subject', () => {
expect(listSampleMembers(sStatuses, '__days__', sDaily).map((m) => m.sampleValue)).toEqual([1, 2]);
});
it('daily field: distinct numeric values sorted ascending', () => {
expect(listSampleMembers(sStatuses, 'c-w', sDaily).map((m) => m.sampleValue)).toEqual([250, 260]);
});
});
describe('buildSampleIndex', () => {
it('date coord maps animalId -> dateKey -> status', () => {
const idx = buildSampleIndex(sStatuses, '__date__', sDaily);
expect(idx.get('a1').get('2026-07-03').date).toContain('2026-07-03');
});
it('# days reach assigns per-subject ordinals in date order', () => {
const idx = buildSampleIndex(sStatuses, '__days__', sDaily);
expect(idx.get('a1').get(1).date).toContain('2026-07-01');
expect(idx.get('a1').get(2).date).toContain('2026-07-03');
expect(idx.get('a2').get(1).date).toContain('2026-07-01');
expect(idx.get('a2').has(2)).toBe(false);
});
it('daily field maps numeric value to first status by date', () => {
const idx = buildSampleIndex(sStatuses, 'c-w', sDaily);
expect(idx.get('a1').get(250).date).toContain('2026-07-01');
expect(idx.get('a1').get(260).date).toContain('2026-07-03');
});
it('# days reach counts per-status, not per-distinct-date (same-day statuses get separate ordinals)', () => {
const sameDay = [
{ animal_id: 'z1', date: '2026-07-01T08:00:00Z', analysis_summary: { total: 1 } },
{ animal_id: 'z1', date: '2026-07-01T20:00:00Z', analysis_summary: { total: 2 } },
];
const idx = buildSampleIndex(sameDay, '__days__', []);
expect(idx.get('z1').has(1)).toBe(true);
expect(idx.get('z1').has(2)).toBe(true);
expect(idx.get('z1').has(3)).toBe(false);
});
});
import { buildSubjectSeriesMatrix } from '../src/lib/dataExport';
const ssAnimals = [
{ id: 'a1', animal_name: 'Alpha', animal_id_string: 'R-001', subject_info: { grp: 'Control' } },
{ id: 'a2', animal_name: 'Beta', animal_id_string: 'R-002', subject_info: { grp: 'Drug' } },
];
const ssStatuses = [
{ animal_id: 'a1', date: '2026-07-01T00:00:00Z', analysis_summary: { total: 5 } },
{ animal_id: 'a1', date: '2026-07-02T00:00:00Z', analysis_summary: { total: 7 } },
{ animal_id: 'a2', date: '2026-07-01T00:00:00Z', analysis_summary: { total: 9 } },
];
const totalParam = { id: '__total__', label: 'Total attempts', kind: 'total', group: 'metrics' };
describe('buildSubjectSeriesMatrix', () => {
it('date X: subjects as columns, date rows, blank where missing, context names the metric', () => {
const m = buildSubjectSeriesMatrix(
{ xField: '__date__', dataParam: totalParam, groupField: '__none__' },
{ statuses: ssStatuses, animals: ssAnimals, dailyTemplate: [] },
);
expect(m.corner).toBe('Date');
expect(m.context).toEqual({ label: 'Data', value: 'Total attempts' });
expect(m.groupAxis).toBeNull();
expect(m.columns.map((c) => c.label)).toEqual(['Alpha', 'Beta']);
expect(m.rows.map((r) => r.member.label)).toEqual(['2026-07-01', '2026-07-02']);
expect(m.rows[0].values).toEqual([5, 9]);
expect(m.rows[1].values).toEqual([7, '']);
});
it('# days reach X: rows align each subject by day ordinal', () => {
const m = buildSubjectSeriesMatrix(
{ xField: '__days__', dataParam: totalParam, groupField: '__none__' },
{ statuses: ssStatuses, animals: ssAnimals, dailyTemplate: [] },
);
expect(m.corner).toBe('# Days Reach (ordinal)');
expect(m.rows.map((r) => r.member.label)).toEqual(['1', '2']);
expect(m.rows[0].values).toEqual([5, 9]);
expect(m.rows[1].values).toEqual([7, '']);
});
it('grouping sets groupAxis col and clusters subjects', () => {
const m = buildSubjectSeriesMatrix(
{ xField: '__date__', dataParam: totalParam, groupField: 'grp' },
{ statuses: ssStatuses, animals: ssAnimals, dailyTemplate: [] },
);
expect(m.groupAxis).toBe('col');
expect(m.columns.map((c) => [c.group, c.label])).toEqual([['Control', 'Alpha'], ['Drug', 'Beta']]);
});
it('drops entirely-blank rows', () => {
const statuses = [
{ animal_id: 'a1', date: '2026-07-01T00:00:00Z', analysis_summary: { total: 5 } },
{ animal_id: 'a1', date: '2026-07-05T00:00:00Z', analysis_summary: null },
];
const m = buildSubjectSeriesMatrix(
{ xField: '__date__', dataParam: totalParam, groupField: '__none__' },
{ statuses, animals: ssAnimals, dailyTemplate: [] },
);
expect(m.rows.map((r) => r.member.label)).toEqual(['2026-07-01']);
});
});