EXP-002: The Oracle Haircut

EXP-001 established the ceiling: if you could perfectly identify patient trajectory classes, you'd recover enormous statistical power in ALS trials, especially when treatment effects are class-specific. But "perfectly identify" is doing a lot of work in that sentence.

In practice, class membership is latent. You observe noisy longitudinal data and must infer which trajectory pattern each patient follows. The standard tool for this is the Latent Class Mixed Model (LCMM) — a model that simultaneously estimates the number of classes, their trajectory shapes, and each patient's probability of belonging to each class.

The question is how much of the oracle's power advantage leaks away through this estimation step. If the haircut is small, the two-stage approach is viable for real trials. If it's catastrophic, we need a different strategy.

Data-Generating Process. Identical to EXP-001: three latent trajectory classes (slow, fast, stable-then-crash) with the same proportions and parameters. Informative dropout, random effects, 200 simulations per configuration.

Sample sizes: 100, 200, and 400 patients per arm. Three scenarios: null (no effect), uniform (25% slowing in all classes), and class-specific (50% slowing in slow progressors only).

Fig. 1 — Power curves across all five methods. The two-stage LCMM approaches (orange, red) sit between the baselines (blue) and the oracle (green).

Click each scenario below to see detailed power tables.

Finding 1: The class-specific scenario is where two-stage shines. When the drug only works for one subgroup, LCMM-Hard beats both LMM and ANCOVA at every sample size. At N=400, it reaches 95% power versus 75% for the standard methods. The subgroup signal is real and recoverable.

Finding 2: Uniform treatment effects don't need subgrouping. When the drug works for everyone, splitting the sample into classes only hurts. LMM reaches 76% power at N=100 while LCMM-Hard manages just 11%. Don't subgroup when you don't need to.

Finding 3: LCMM-Soft is slightly more conservative than LCMM-Hard. The pseudo-class draws with Rubin's rules properly propagate classification uncertainty, which means slightly wider confidence intervals and slightly lower power. The trade-off: better-calibrated inference at the cost of a few percentage points of power.

Finding 4: K always selects 4 — a potential overfitting flag. BIC selected K=4 classes in every single simulation, despite the true data-generating process having 3 classes. The model consistently overfits the number of classes. This doesn't necessarily doom the analysis — the extra class may absorb noise without corrupting the class of interest — but it warrants investigation.

Finding 5: Type I error is controlled. Under the null scenario, all five methods stay near the nominal 5% rate. LCMM-Hard shows a slight inflation at N=200 (9.5%) that should be monitored with more simulations, but LCMM-Soft remains conservative throughout.

The two-stage LCMM pipeline is viable. Not perfect — the oracle haircut is real, and the K-selection overfitting is a genuine concern — but viable. When you believe the treatment effect is concentrated in a trajectory subgroup, fitting an LCMM and testing within the estimated class recovers substantial power that standard methods leave on the table.

The practical calculus works out like this: if you'd need N=400/arm with a standard LMM to hit 75% power in a class-specific scenario, the two-stage approach gets you to 95% with the same sample size. Alternatively, you could reach 67% power with N=200/arm — half the patients, in a disease where every enrollment slot is precious.

The uniform scenario result is equally important as a guardrail. If the drug works for everyone, don't subgroup. The two-stage approach should be deployed when there's prior biological reason to expect heterogeneous treatment effects — not as a default analysis strategy.

Why does BIC always pick K=4? With a true 3-class DGP, BIC should prefer K=3. The consistent overfitting suggests either the penalty isn't strong enough for this sample size range, or the stable-then-crash class creates complexity that an extra class absorbs. ICL (Integrated Classification Likelihood) may be a better selection criterion.

Can we pre-specify K=3? If we have strong prior knowledge about the number of classes from PRO-ACT data, we can bypass BIC selection entirely. This would eliminate the overfitting concern and likely improve power.

What about joint modeling? The two-stage approach estimates classes separately from treatment testing. A joint model that does both simultaneously should be more efficient — but also more complex to implement and validate.

Builds on: EXP-001: The Cost of Linearity — established the oracle ceiling and the 4× sample size penalty.

Requested by: Board Room Session 004 — "What happens with a realistic LCMM pipeline?"

Next: Investigate K-selection alternatives (ICL, pre-specified K). Validate trajectory classes on real PRO-ACT data. Explore joint modeling as an alternative to the two-stage approach.