Experiments DB Dev
800189371b
Adds make_figure (2-panel learning curves + per-phase slopes, colorblind-safe Box-B2/Box-A2 palette) -> results/figure_learning.png; analysis/writeup.md (figure + methods + results summary); and tdcs_export_all/run_export exporting every data variation into an organized export/ tree (curated_all, scenarios/9, anchors/3 with tDCS factor, phases/9 + MANIFEST). Adds tExport tests. Suite 36/36. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
145 lines
6.5 KiB
Markdown
145 lines
6.5 KiB
Markdown
# tDCS reaching study — MATLAB GLM port
|
||
|
||
This directory is a MATLAB port of the Python tDCS GLM analysis
|
||
(`analysis/tdcs_glm.py`, documented in `analysis/METHODS.md`), driven by raw
|
||
app-export matrices rather than the pre-joined `tdcs_reach_data.csv`. It
|
||
reproduces the same three models (count level, rate level, learning rate) and
|
||
the same H2 anchor check / unknown-condition classification, but replaces the
|
||
Python's population-average GEE with a `fitglme` GLMM (Poisson/Binomial,
|
||
subject random intercept) — see "GLMM vs GEE" below.
|
||
|
||
## Requirements
|
||
|
||
- MATLAB R2025b + Statistics and Machine Learning Toolbox.
|
||
- A license activated at `~/matlab/licenses/license.dat` (already in place in
|
||
this environment). If MATLAB ever reports it is unlicensed, activate with:
|
||
|
||
```
|
||
~/matlab/bin/activate_matlab.sh
|
||
```
|
||
|
||
## Layout
|
||
|
||
- `tdcs_config.m` — curated 12-animal group map, reference group, merge maps.
|
||
- `tdcs_load_data.m` — parses `raw/raw_success.csv` and `raw/raw_total.csv`
|
||
(app-export matrix format), joins them, and applies the curated group map.
|
||
- `tdcs_scenario_data.m` — builds one of the 6 scenarios (merge map + day
|
||
window), and saves the scenario table to `derived/<scenario>.csv`.
|
||
- `tdcs_models.m` — fits the count/rate/learning GLMEs, the per-animal
|
||
Welch/Mann-Whitney tests, the H2 anchor check, and (unmerged scenarios
|
||
only) the unknown-condition classification.
|
||
- `tdcs_report.m` — prints descriptives, the raw GLME fixed-effects tables,
|
||
and an interpretation section (H2 verdict, per-animal p-values,
|
||
classification, caveats); also writes `results/<scenario>.txt`.
|
||
- `tdcs_glm.m` — the entry point: `tdcs_glm(scenario)` runs
|
||
`tdcs_scenario_data` → `tdcs_models` → `tdcs_report` for one named
|
||
scenario, via an explicit `switch/case` over all 6 scenario names.
|
||
- `run_all.m` — runs all 6 scenarios in one script.
|
||
- `run_tests.m` — runs the `matlab.unittest` suite in `tests/` and exits
|
||
nonzero on any failure.
|
||
- `tests/` — `matlab.unittest` test classes (`tLoadData`, `tScenarioData`,
|
||
`tModels`) with golden values cross-checked against the Python.
|
||
- `raw/` — pre-generated app-export CSVs (input; not modified by this code).
|
||
- `derived/` — scenario tables written by `tdcs_scenario_data` (generated).
|
||
- `results/` — report text files written by `tdcs_report`/`tdcs_glm`
|
||
(generated).
|
||
|
||
## Running
|
||
|
||
Run one scenario (valid names: `unmerged_full`, `unmerged_d0_10`,
|
||
`mergeA2_full`, `mergeA2_d0_10`, `mergeB2_full`, `mergeB2_d0_10`):
|
||
|
||
```
|
||
cd analysis/matlab
|
||
~/matlab/bin/matlab -batch "addpath(pwd); tdcs_glm('mergeB2_full')"
|
||
```
|
||
|
||
This prints the full report to the console and writes it to
|
||
`results/mergeB2_full.txt`.
|
||
|
||
Run all 6 scenarios:
|
||
|
||
```
|
||
~/matlab/bin/matlab -batch "addpath(pwd); run_all"
|
||
```
|
||
|
||
Run the test suite:
|
||
|
||
```
|
||
~/matlab/bin/matlab -batch "addpath(pwd); run_tests"
|
||
```
|
||
|
||
(`run_tests` calls `error(...)` on any failure, so the `matlab -batch`
|
||
process exits with a nonzero status — suitable for CI.)
|
||
|
||
## Validation against the Python
|
||
|
||
Per-animal statistics (per-subject Welch t-test / one-sided Mann-Whitney U,
|
||
Box-B2 vs Box-A2) are pure stats independent of the GLM choice, and were
|
||
matched to the Python's golden values to 3 decimals across all 6 scenarios
|
||
(see `docs/superpowers/plans/2026-07-20-matlab-tdcs-analysis.md`, Task 3).
|
||
|
||
## GLMM vs GEE (read this before comparing numbers to the Python)
|
||
|
||
The Python reference uses **GEE** (population-average effects, exchangeable
|
||
working correlation, robust/sandwich SEs) for the count and learning-rate
|
||
models, and a cluster-robust Binomial GLM for the rate model. This MATLAB
|
||
port instead uses **GLMM** (`fitglme`, subject-specific effects via a
|
||
`(1|subject)` random intercept, Laplace/PL approximation) for all three
|
||
models, because MATLAB's Statistics and Machine Learning Toolbox has no GEE
|
||
implementation.
|
||
|
||
GLMM and GEE estimate related but distinct quantities (subject-specific vs.
|
||
population-average effects), so **exact coefficients, ratios, and p-values
|
||
will differ** between the two implementations — sometimes by a fair amount
|
||
for the small, unbalanced groups in this study. What should agree is the
|
||
**interpretation**: the direction of the H2 anchor effect, which unknown
|
||
condition classifies as A2-like/B2-like/ambiguous, and whether the learning
|
||
rates differ across groups. Treat the MATLAB numbers as a sensitivity check
|
||
on the Python's conclusions, not as a numerically-identical reproduction.
|
||
|
||
## Scope vs. the Python
|
||
|
||
The MATLAB port reproduces the Python's three core models (count, rate,
|
||
learning-rate), per-animal tests, unknown-group classification, and the H2
|
||
verdict (read off the main model's A2-vs-B2 term). The Python's two auxiliary
|
||
sensitivity sections — `report_anchor_only` (a two-anchor-only refit) and
|
||
`report_mixed` (a Bayesian MAP random-intercept model) — are intentionally
|
||
**not** ported: the anchor-only refit is closely tracked by the main-model H2
|
||
term, and MATLAB's native `fitglme` already *is* the random-intercept model.
|
||
|
||
## Phased learning analysis and power
|
||
|
||
- **Phased days × tDCS LME** — the same `success ~ day*tDCS + (1|subject)` model refit within
|
||
learning phases (0–5, 6–10, 6–13), Box-B2 vs Box-A2, with per-phase slopes and the interaction
|
||
95% CI:
|
||
```
|
||
matlab -batch "tdcs_glm('phase_mergeA2')" # or phase_unmerged / phase_mergeB2
|
||
```
|
||
The Box-B2 faster-acquisition signal sits in the early (0–5) phase; late phases converge. The
|
||
interaction p/CI use fitlme observation-level DF and are anticonservative at these small n
|
||
(see power analysis).
|
||
|
||
- **Monte-Carlo power** — power to detect the early-phase interaction, scored with the
|
||
cluster-honest per-animal test and the LME test, across subjects-per-group and effect sizes:
|
||
```
|
||
matlab -batch "run_power" # default: mergeA2, phase 0–5
|
||
```
|
||
Result: at our n (3–5/group) honest power is ~0.3–0.7 even at the observed effect; ~8/group
|
||
reaches ~80–90% if the effect is as large as observed, ~20/group if it is half that. The LME
|
||
test is anticonservative only at small n and converges to the honest test by n≈12.
|
||
|
||
## Figure and data export
|
||
|
||
- **Figure** — two-panel learning figure (curves + per-phase slopes, Box-B2 vs Box-A2),
|
||
written to `results/figure_learning.png`:
|
||
```
|
||
matlab -batch "make_figure"
|
||
```
|
||
- **Export all data variations to CSV** — organized under `export/`:
|
||
```
|
||
matlab -batch "run_export"
|
||
```
|
||
Produces `curated_all.csv`, `scenarios/<merge>_<window>.csv` (9), `anchors/<merge>_B2vsA2.csv`
|
||
(3, with the binary tDCS factor), and `phases/<merge>_phase_<lo>-<hi>.csv` (9), plus a MANIFEST.
|