docs(writeup): add honest random-slope results + scenario notation

Expand Methods to describe the three interaction DF treatments (residual,
Satterthwaite, honest random-slope) and why Satterthwaite is inert on the
random-intercept model. Add a 'Honest inference' subsection: the early-phase
interaction reaches honest significance only in the pre-specified pooled-control
scenarios (naive_a2/naive_boxa/mergeNaive d0_5-d0_13, random-slope p 0.014-0.036)
while confounded/small-n windows collapse (mergeA2_full 0.75; unmerge_d0_5 0.13).
Add a 'Scenario notation' legend defining every grouping, window, model family,
p-value column, and the _f (Forouzan) labels.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Experiments DB Dev
2026-07-22 15:49:17 -04:00
parent 8a18c894dd
commit b03549982e
+79 -3
View File
@@ -24,9 +24,18 @@ restricted to the common 013 window and, to separate acquisition from plateau
refit within a *fast* (days 05) and *slow* (days 613) phase. Given the small
number of animals, the subject was treated as the unit of inference: per-animal
learning slopes were compared between groups (Welch *t*, MannWhitney) and within
groups across phases (paired *t*); the `fitlme` interaction tests, which use
observation-level degrees of freedom, are anticonservative at this sample size
and are reported only as a reference. Statistical power was estimated by
groups across phases (paired *t*). For each interaction we report three
denominator-DF treatments: (i) **residual DF** (the paper's method), (ii)
**Satterthwaite DF**, and (iii) an **honest random-slope test** from the richer
model `successes ~ day * tDCS + (day | subject)`. On a random-*intercept* model
the slope's error is estimated at the session level, so Satterthwaite DF stays
≈ residual and does *not* correct the pseudoreplication; only a random *slope*
lets between-animal slope variance enter the standard error, collapsing the
interaction DF toward the number of animals and agreeing with the per-animal
slope test. The random-slope test (and the per-animal test) are therefore the
honest inference; the residual/Satterthwaite `fitlme` interaction p-values are
anticonservative at this sample size and reported for comparison only.
Statistical power was estimated by
Monte-Carlo simulation from the fitted early-phase model across a range of
per-group sample sizes and effect sizes. Overall accuracy (successes/attempts)
and counts were additionally modeled with binomial and Poisson GLMMs. All
@@ -82,3 +91,70 @@ power for the observed effect; ≈8/group would be needed for the observed effec
previously reported days × tDCS interaction, here compressed into the early
acquisition phase — plausibly because the improved protocol's higher performance
ceiling leaves little late-phase headroom for a benefit to accumulate.
### Honest inference (random-slope test)
Under observation-level DF the early-phase days × tDCS interaction is
"significant" in several scenarios, but this is anticonservative. **Satterthwaite
DF barely changes it** (e.g. `unmerge_d0_5` p = 0.015 → 0.015; `mergeA2_full`
0.009 → 0.009), confirming that on the random-intercept model the correction is
inert. The **honest random-slope test** — where the interaction DF collapses
toward the animal count — separates the real signal from the pseudoreplicated
one:
| scenario | resid p | Satterthwaite p | **random-slope p** | verdict |
|---|---|---|---|---|
| `mergeA2_full` (unequal coverage) | 0.009 | 0.009 | **0.75** (df ≈ 10) | artifact — collapses |
| `unmerge_d0_5` (B2 vs A2, n = 3+3) | 0.015 | 0.015 | **0.13** | underpowered — n.s. |
| `right_only_d0_5` | 0.006 | 0.006 | **0.088** | n.s. |
| **`naive_a2_d0_5`** (pooled control) | 0.001 | 0.001 | **0.028** | **survives** |
| **`naive_boxa_d0_5`** (pooled control) | 0.003 | 0.003 | **0.036** | **survives** |
| **`paper_mergeNaive_d0_13`** | 0.009 | 0.009 | **0.014** | **survives** |
The pattern is coherent: the full-range/unequal-coverage interaction is a
truncation artifact that vanishes under honest DF; the pure two-anchor early
window (n = 3 + 3) shows the effect directionally but is underpowered; and the
effect reaches **honest significance only in the pre-specified pooled-control
scenarios** (`naive_a2`, `naive_boxa`, `mergeNaive`), where the larger control
sample makes the between-animal slope variance estimable. This matches the
per-animal (cluster-honest) tests exactly and is the inference we report. The
`d0_5` matched-effort and slow-phase (`d6_*`) windows do not survive.
## Scenario notation
Each analysis is named `<grouping>_<window>`, optionally prefixed by a model
family. Group codes: **B2** = Box-B2 (anodal / contralateral tDCS, the
treatment); **A2** = Box-A2 (ipsilateral sham, the control); **Naive** =
untreated; **Right** = Right-Electrode, **Box-A** = the two single-animal
"unknown" conditions.
**Groupings** (treatment = stim 1 vs control = stim 0; other groups dropped):
| name | treatment (stim = 1) | control (stim = 0) |
|---|---|---|
| `unmerge` / `unmerged` | B2 | A2 |
| `right_only` | B2 + Right | A2 |
| `naive_a2` | B2 | A2 + Naive |
| `naive_boxa` | B2 | A2 + Naive + Box-A |
| `mergeNaive` | B2 + Right | A2 + Naive |
| `mergeA2` | B2 + Right | A2 + Box-A |
| `mergeB2` | B2 + Right + Box-A | A2 |
**Windows** (training day, 0-indexed; day 0 = the previous study's "Day 1"):
`full` = all days; `d0_5` = 05 (fast/acquisition phase); `d0_10` = 010;
`d0_13` = 013 (last day Box-A2 has data — the fair equal-coverage window for the
anchors); `d6_10`, `d6_13` = 610 / 613 (slow/plateau phase).
**Model families:** `lme_*` = linear `successes ~ day*tDCS + (1|subject)`;
`paper_*` = the paper's formula `behavior ~ stim + day + stim:day + (1|rat)`
(same model, paper's term order); `phase_*` = the LME refit within fast/slow
phases; GLMM = Poisson (count) and Binomial (rate) mixed models.
**Interaction p-value columns:** *resid* = observation-level DF (paper's method,
anticonservative); *Satterthwaite* = Satterthwaite DF; *random-slope* / *HONEST*
= random-slope model, DF ≈ animal count (the honest test); *per-animal* =
Welch/MannWhitney on per-subject slopes or rates (cluster-honest).
**Previous study (`_f`, Forouzan, N = 24):** `b2_f` = Anodal (treatment),
`a2_f` = Control; validated because `b2_f`-as-stim reproduces the paper's
positive tDCS × day interaction (t(227) = 2.66, F(1) = 7.09, p = 0.008).