docs(writeup): add honest random-slope results + scenario notation
Expand Methods to describe the three interaction DF treatments (residual, Satterthwaite, honest random-slope) and why Satterthwaite is inert on the random-intercept model. Add a 'Honest inference' subsection: the early-phase interaction reaches honest significance only in the pre-specified pooled-control scenarios (naive_a2/naive_boxa/mergeNaive d0_5-d0_13, random-slope p 0.014-0.036) while confounded/small-n windows collapse (mergeA2_full 0.75; unmerge_d0_5 0.13). Add a 'Scenario notation' legend defining every grouping, window, model family, p-value column, and the _f (Forouzan) labels. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+79
-3
@@ -24,9 +24,18 @@ restricted to the common 0–13 window and, to separate acquisition from plateau
|
||||
refit within a *fast* (days 0–5) and *slow* (days 6–13) phase. Given the small
|
||||
number of animals, the subject was treated as the unit of inference: per-animal
|
||||
learning slopes were compared between groups (Welch *t*, Mann–Whitney) and within
|
||||
groups across phases (paired *t*); the `fitlme` interaction tests, which use
|
||||
observation-level degrees of freedom, are anticonservative at this sample size
|
||||
and are reported only as a reference. Statistical power was estimated by
|
||||
groups across phases (paired *t*). For each interaction we report three
|
||||
denominator-DF treatments: (i) **residual DF** (the paper's method), (ii)
|
||||
**Satterthwaite DF**, and (iii) an **honest random-slope test** from the richer
|
||||
model `successes ~ day * tDCS + (day | subject)`. On a random-*intercept* model
|
||||
the slope's error is estimated at the session level, so Satterthwaite DF stays
|
||||
≈ residual and does *not* correct the pseudoreplication; only a random *slope*
|
||||
lets between-animal slope variance enter the standard error, collapsing the
|
||||
interaction DF toward the number of animals and agreeing with the per-animal
|
||||
slope test. The random-slope test (and the per-animal test) are therefore the
|
||||
honest inference; the residual/Satterthwaite `fitlme` interaction p-values are
|
||||
anticonservative at this sample size and reported for comparison only.
|
||||
Statistical power was estimated by
|
||||
Monte-Carlo simulation from the fitted early-phase model across a range of
|
||||
per-group sample sizes and effect sizes. Overall accuracy (successes/attempts)
|
||||
and counts were additionally modeled with binomial and Poisson GLMMs. All
|
||||
@@ -82,3 +91,70 @@ power for the observed effect; ≈8/group would be needed for the observed effec
|
||||
previously reported days × tDCS interaction, here compressed into the early
|
||||
acquisition phase — plausibly because the improved protocol's higher performance
|
||||
ceiling leaves little late-phase headroom for a benefit to accumulate.
|
||||
|
||||
### Honest inference (random-slope test)
|
||||
|
||||
Under observation-level DF the early-phase days × tDCS interaction is
|
||||
"significant" in several scenarios, but this is anticonservative. **Satterthwaite
|
||||
DF barely changes it** (e.g. `unmerge_d0_5` p = 0.015 → 0.015; `mergeA2_full`
|
||||
0.009 → 0.009), confirming that on the random-intercept model the correction is
|
||||
inert. The **honest random-slope test** — where the interaction DF collapses
|
||||
toward the animal count — separates the real signal from the pseudoreplicated
|
||||
one:
|
||||
|
||||
| scenario | resid p | Satterthwaite p | **random-slope p** | verdict |
|
||||
|---|---|---|---|---|
|
||||
| `mergeA2_full` (unequal coverage) | 0.009 | 0.009 | **0.75** (df ≈ 10) | artifact — collapses |
|
||||
| `unmerge_d0_5` (B2 vs A2, n = 3+3) | 0.015 | 0.015 | **0.13** | underpowered — n.s. |
|
||||
| `right_only_d0_5` | 0.006 | 0.006 | **0.088** | n.s. |
|
||||
| **`naive_a2_d0_5`** (pooled control) | 0.001 | 0.001 | **0.028** | **survives** |
|
||||
| **`naive_boxa_d0_5`** (pooled control) | 0.003 | 0.003 | **0.036** | **survives** |
|
||||
| **`paper_mergeNaive_d0_13`** | 0.009 | 0.009 | **0.014** | **survives** |
|
||||
|
||||
The pattern is coherent: the full-range/unequal-coverage interaction is a
|
||||
truncation artifact that vanishes under honest DF; the pure two-anchor early
|
||||
window (n = 3 + 3) shows the effect directionally but is underpowered; and the
|
||||
effect reaches **honest significance only in the pre-specified pooled-control
|
||||
scenarios** (`naive_a2`, `naive_boxa`, `mergeNaive`), where the larger control
|
||||
sample makes the between-animal slope variance estimable. This matches the
|
||||
per-animal (cluster-honest) tests exactly and is the inference we report. The
|
||||
`d0_5` matched-effort and slow-phase (`d6_*`) windows do not survive.
|
||||
|
||||
## Scenario notation
|
||||
|
||||
Each analysis is named `<grouping>_<window>`, optionally prefixed by a model
|
||||
family. Group codes: **B2** = Box-B2 (anodal / contralateral tDCS, the
|
||||
treatment); **A2** = Box-A2 (ipsilateral sham, the control); **Naive** =
|
||||
untreated; **Right** = Right-Electrode, **Box-A** = the two single-animal
|
||||
"unknown" conditions.
|
||||
|
||||
**Groupings** (treatment = stim 1 vs control = stim 0; other groups dropped):
|
||||
|
||||
| name | treatment (stim = 1) | control (stim = 0) |
|
||||
|---|---|---|
|
||||
| `unmerge` / `unmerged` | B2 | A2 |
|
||||
| `right_only` | B2 + Right | A2 |
|
||||
| `naive_a2` | B2 | A2 + Naive |
|
||||
| `naive_boxa` | B2 | A2 + Naive + Box-A |
|
||||
| `mergeNaive` | B2 + Right | A2 + Naive |
|
||||
| `mergeA2` | B2 + Right | A2 + Box-A |
|
||||
| `mergeB2` | B2 + Right + Box-A | A2 |
|
||||
|
||||
**Windows** (training day, 0-indexed; day 0 = the previous study's "Day 1"):
|
||||
`full` = all days; `d0_5` = 0–5 (fast/acquisition phase); `d0_10` = 0–10;
|
||||
`d0_13` = 0–13 (last day Box-A2 has data — the fair equal-coverage window for the
|
||||
anchors); `d6_10`, `d6_13` = 6–10 / 6–13 (slow/plateau phase).
|
||||
|
||||
**Model families:** `lme_*` = linear `successes ~ day*tDCS + (1|subject)`;
|
||||
`paper_*` = the paper's formula `behavior ~ stim + day + stim:day + (1|rat)`
|
||||
(same model, paper's term order); `phase_*` = the LME refit within fast/slow
|
||||
phases; GLMM = Poisson (count) and Binomial (rate) mixed models.
|
||||
|
||||
**Interaction p-value columns:** *resid* = observation-level DF (paper's method,
|
||||
anticonservative); *Satterthwaite* = Satterthwaite DF; *random-slope* / *HONEST*
|
||||
= random-slope model, DF ≈ animal count (the honest test); *per-animal* =
|
||||
Welch/Mann–Whitney on per-subject slopes or rates (cluster-honest).
|
||||
|
||||
**Previous study (`_f`, Forouzan, N = 24):** `b2_f` = Anodal (treatment),
|
||||
`a2_f` = Control; validated because `b2_f`-as-stim reproduces the paper's
|
||||
positive tDCS × day interaction (t(227) = 2.66, F(1) = 7.09, p = 0.008).
|
||||
|
||||
Reference in New Issue
Block a user