Assert arm beats baseline on metric. By default just a positive gap > by (pass the combined se to clear the noise floor by hand). Pass { significant: true } to demand a Welch t-test at alpha instead — the computed noise floor.
arm
baseline
metric
by
{ significant: true }
alpha
Assert
armbeatsbaselineonmetric. By default just a positive gap >by(pass the combined se to clear the noise floor by hand). Pass{ significant: true }to demand a Welch t-test atalphainstead — the computed noise floor.