diff --git a/analysis/writeup.md b/analysis/writeup.md index 29a0dbe..e03ec83 100644 --- a/analysis/writeup.md +++ b/analysis/writeup.md @@ -24,9 +24,18 @@ restricted to the common 0–13 window and, to separate acquisition from plateau refit within a *fast* (days 0–5) and *slow* (days 6–13) phase. Given the small number of animals, the subject was treated as the unit of inference: per-animal learning slopes were compared between groups (Welch *t*, Mann–Whitney) and within -groups across phases (paired *t*); the `fitlme` interaction tests, which use -observation-level degrees of freedom, are anticonservative at this sample size -and are reported only as a reference. Statistical power was estimated by +groups across phases (paired *t*). For each interaction we report three +denominator-DF treatments: (i) **residual DF** (the paper's method), (ii) +**Satterthwaite DF**, and (iii) an **honest random-slope test** from the richer +model `successes ~ day * tDCS + (day | subject)`. On a random-*intercept* model +the slope's error is estimated at the session level, so Satterthwaite DF stays +≈ residual and does *not* correct the pseudoreplication; only a random *slope* +lets between-animal slope variance enter the standard error, collapsing the +interaction DF toward the number of animals and agreeing with the per-animal +slope test. The random-slope test (and the per-animal test) are therefore the +honest inference; the residual/Satterthwaite `fitlme` interaction p-values are +anticonservative at this sample size and reported for comparison only. +Statistical power was estimated by Monte-Carlo simulation from the fitted early-phase model across a range of per-group sample sizes and effect sizes. Overall accuracy (successes/attempts) and counts were additionally modeled with binomial and Poisson GLMMs. All @@ -82,3 +91,70 @@ power for the observed effect; ≈8/group would be needed for the observed effec previously reported days × tDCS interaction, here compressed into the early acquisition phase — plausibly because the improved protocol's higher performance ceiling leaves little late-phase headroom for a benefit to accumulate. + +### Honest inference (random-slope test) + +Under observation-level DF the early-phase days × tDCS interaction is +"significant" in several scenarios, but this is anticonservative. **Satterthwaite +DF barely changes it** (e.g. `unmerge_d0_5` p = 0.015 → 0.015; `mergeA2_full` +0.009 → 0.009), confirming that on the random-intercept model the correction is +inert. The **honest random-slope test** — where the interaction DF collapses +toward the animal count — separates the real signal from the pseudoreplicated +one: + +| scenario | resid p | Satterthwaite p | **random-slope p** | verdict | +|---|---|---|---|---| +| `mergeA2_full` (unequal coverage) | 0.009 | 0.009 | **0.75** (df ≈ 10) | artifact — collapses | +| `unmerge_d0_5` (B2 vs A2, n = 3+3) | 0.015 | 0.015 | **0.13** | underpowered — n.s. | +| `right_only_d0_5` | 0.006 | 0.006 | **0.088** | n.s. | +| **`naive_a2_d0_5`** (pooled control) | 0.001 | 0.001 | **0.028** | **survives** | +| **`naive_boxa_d0_5`** (pooled control) | 0.003 | 0.003 | **0.036** | **survives** | +| **`paper_mergeNaive_d0_13`** | 0.009 | 0.009 | **0.014** | **survives** | + +The pattern is coherent: the full-range/unequal-coverage interaction is a +truncation artifact that vanishes under honest DF; the pure two-anchor early +window (n = 3 + 3) shows the effect directionally but is underpowered; and the +effect reaches **honest significance only in the pre-specified pooled-control +scenarios** (`naive_a2`, `naive_boxa`, `mergeNaive`), where the larger control +sample makes the between-animal slope variance estimable. This matches the +per-animal (cluster-honest) tests exactly and is the inference we report. The +`d0_5` matched-effort and slow-phase (`d6_*`) windows do not survive. + +## Scenario notation + +Each analysis is named `_`, optionally prefixed by a model +family. Group codes: **B2** = Box-B2 (anodal / contralateral tDCS, the +treatment); **A2** = Box-A2 (ipsilateral sham, the control); **Naive** = +untreated; **Right** = Right-Electrode, **Box-A** = the two single-animal +"unknown" conditions. + +**Groupings** (treatment = stim 1 vs control = stim 0; other groups dropped): + +| name | treatment (stim = 1) | control (stim = 0) | +|---|---|---| +| `unmerge` / `unmerged` | B2 | A2 | +| `right_only` | B2 + Right | A2 | +| `naive_a2` | B2 | A2 + Naive | +| `naive_boxa` | B2 | A2 + Naive + Box-A | +| `mergeNaive` | B2 + Right | A2 + Naive | +| `mergeA2` | B2 + Right | A2 + Box-A | +| `mergeB2` | B2 + Right + Box-A | A2 | + +**Windows** (training day, 0-indexed; day 0 = the previous study's "Day 1"): +`full` = all days; `d0_5` = 0–5 (fast/acquisition phase); `d0_10` = 0–10; +`d0_13` = 0–13 (last day Box-A2 has data — the fair equal-coverage window for the +anchors); `d6_10`, `d6_13` = 6–10 / 6–13 (slow/plateau phase). + +**Model families:** `lme_*` = linear `successes ~ day*tDCS + (1|subject)`; +`paper_*` = the paper's formula `behavior ~ stim + day + stim:day + (1|rat)` +(same model, paper's term order); `phase_*` = the LME refit within fast/slow +phases; GLMM = Poisson (count) and Binomial (rate) mixed models. + +**Interaction p-value columns:** *resid* = observation-level DF (paper's method, +anticonservative); *Satterthwaite* = Satterthwaite DF; *random-slope* / *HONEST* += random-slope model, DF ≈ animal count (the honest test); *per-animal* = +Welch/Mann–Whitney on per-subject slopes or rates (cluster-honest). + +**Previous study (`_f`, Forouzan, N = 24):** `b2_f` = Anodal (treatment), +`a2_f` = Control; validated because `b2_f`-as-stim reproduces the paper's +positive tDCS × day interaction (t(227) = 2.66, F(1) = 7.09, p = 0.008).