Sequential inference
Continuous means use InferenceSpec(kind="asymptotic_mean") with an ordinary
fixed-allocation design. The source binds the registration before capture;
expected_decision_sample_size optionally controls pre-outcome tuning.
Conversion and retention metrics may instead use InferenceSpec(kind="always_valid", baseline_rate=...), which binds the exact Bernoulli e-process from the plan alone; see
sequential monitoring of a conversion metric.
Definitions-based experiments carry assignment weights in Experiment.allocation.
See ordinary continuous monitoring
for method assumptions, finalization, and the qualified asymptotic guarantee.
AlwaysValid
Section titled “AlwaysValid”AlwaysValidRaw likelihood evidence with a committed model, roster and reveal law.
AsymptoticMean
Section titled “AsymptoticMean”AsymptoticMeanRegistered count-clock scalar means; asymptotic, never exact e-values.
AsymptoticSequentialEvidence
Section titled “AsymptoticSequentialEvidence”AsymptoticSequentialEvidence(hypothesis, method, result)Set-exclusion evidence with registered assumptions and actual arm clocks.
rejects reads the result’s set at its own decision alpha. Family
selection instead reads result.log_e, the stopped state’s evidence,
which for a ratio law is capped by denominator stability and so is not the
exact dual of that set.
AsymptoticSequentialResult
Section titled “AsymptoticSequentialResult”AsymptoticSequentialResultPortable asymptotic evidence and confidence geometry for one stopped state (asymptotic, never exact).
log_e is the contrast’s direction-respecting plug-in mixture value,
not a finite-sample e-value. Its estimated variance does not preserve the
oracle martingale’s expectation bound. The value reads the stopped state
alone and is unchanged by freezing, replay or alpha reinversion.
A ratio law caps it by denominator stability, clearing -log(alpha)
only where the denominators are resolved. bounds invert the uncapped contrast at
decision_alpha: where that set is available, a family member rejects
when the uncapped value exceeds -log(decision_alpha). A ratio set’s
availability, unlike log_e, depends on that alpha.
SequentialRegistration
Section titled “SequentialRegistration”SequentialRegistrationImmutable version-2 specification supplied before a producer reads observations.
definitions_id binds metric windows/transformations, assignment, population and source mapping; the explicit compliance policy is carried by the compiled plan. Its equality is checked at capture and continuation; it is not a watermark.
asymptotic_family is derived, never chosen: it records that the family of
a registration with asymptotic models is selected by e-BH, so a registration
stored when those families were Bonferroni-corrected has a different identity
and cannot continue under the weaker false-discovery guarantee.
PredeclaredAdjustment
Section titled “PredeclaredAdjustment”PredeclaredAdjustmentCovariate coefficient and centre fixed from pre-period data before any outcome is read.
The asymptotic scalar-mean route captures
Y - coefficient * (X - center) as each unit’s scalar observation.
The exact Bernoulli route does not admit this transformed outcome law;
predictability alone does not supply its likelihood.
Two costs relative to the fixed-horizon fit in estimation.cuped:
the coefficient is not the contrast-optimal one (that fit reads the
in-experiment outcome), so the variance reduction is smaller on the
same data; and each arm’s adjusted mean is shifted by the same constant
coefficient * (E[X] - center), which cancels in a difference but
not in a ratio. The scalar-mean route reports a ratio, so pre-period to
experiment covariate drift is a first-order bias on its lift scale.
fit_predeclared_adjustment
Section titled “fit_predeclared_adjustment”fit_predeclared_adjustment(frame, unit, group, control, outcome, covariate, covariate_missing)Fit a sequential CUPED coefficient and covariate centre from pre-period data.
frame is one row per assigned unit observed BEFORE the experiment reads
any outcome: outcome is the metric measured in a pre-period window and
covariate its covariate measured relative to that window, the same way
the in-experiment covariate is measured relative to exposure. The rows
are reduced to the same centered moments the fixed-horizon frame path
builds, and the coefficient is estimation.cuped.fit_cuped’s inverse-n
weighted within-arm slope with the centre its count-weighted pooled
covariate mean, so sequential and fixed-horizon CUPED agree on what the
coefficient is.
The result is a declaration: pass it through InferenceSpec.adjustments
or ScalarMeanModel.adjustment and capture retains
Y - coefficient * (X - center). Costs relative to the fixed-horizon
fit: the coefficient is not contrast-optimal for the in-experiment
outcome, and coefficient * (E[X] - center) is a first-order bias on
the ratio scale the scalar-mean route reports whenever the covariate
drifts between the pre-period and the experiment.
SequentialModel
Section titled “SequentialModel”SequentialModelNo docstring.
ScalarMeanModel
Section titled “ScalarMeanModel”ScalarMeanModelCount-clock AsympCS assumptions, declared before observing outcomes.
Fixed tuning gives an asymptotic CS approximation, not finite-start
calibration or a uniform guarantee over heavy-tailed distributions. One
contract covers four laws that differ only in the per-unit vector each
arm retains (retained_dimension) and in the functional contrasted:
scalar_mean: (Y); the ratio of arm means, direct shifted contrast.adjusted_mean: (Y, X); the ratio of CUPED-adjusted arm means with the coefficient fitted from the retained within-arm cross moments.ratio_mean: (N, D); the ratio of arm ratiosE[N]/E[D].adjusted_ratio_mean: (N, D, X); the ratio of arm ratios after each component is adjusted against X with its own coefficient.
moments asserts finite 2 + delta moments for the whole retained
vector and positive_limiting_variance a positive limiting variance of
the linearised contrast; the adjusted laws additionally need a positive
within-arm covariate variance (so the coefficient is identified) and the
ratio laws a population denominator mean bounded away from zero
(positive_population_denominators). X is a pre-assignment covariate.
The three non-scalar laws are delta-method linearisations, so their sets
are asymptotic in the same sense as the scalar law plus a nuisance
plug-in that is negligible at the boundary’s rate; see
estimation.asymptotic_joint for the argument and references.
SequentialCell
Section titled “SequentialCell”SequentialCellNo docstring.
SequentialCompliancePolicy
Section titled “SequentialCompliancePolicy”SequentialCompliancePolicyExplicit design-level testing policy for Bernoulli uptake.
SequentialSnapshot
Section titled “SequentialSnapshot”SequentialSnapshotNo docstring.
snapshot_from_json
Section titled “snapshot_from_json”snapshot_from_json(payload)Reject duplicate JSON keys and malformed text before validating a current checkpoint.
Every rational in the payload is admitted by PortableRational before
conversion. JSON syntax errors and integer literals beyond the reader’s
digit limit refuse with sequential.source.invalid.
capture_sequential_snapshot
Section titled “capture_sequential_snapshot”capture_sequential_snapshot(registration, records, source_id, definitions_id, finalized, previous, reveal_cursor, append)No docstring.
declare_sequential_freeze
Section titled “declare_sequential_freeze”declare_sequential_freeze(snapshot, metrics)Stop monitoring named metrics at exactly this snapshot’s current look.
Every registered cell of a named metric — each treatment arm and each segment — that has observations on both arms keeps this look’s evidence at every later look. A cell with no observations yet has no evidence to keep and stays monitored; naming the metric at a later capture freezes it then. Only this snapshot’s own current state can be frozen, never an earlier look’s. A frozen secondary keeps its evidence, while its family’s e-BH selection is redone at every look over frozen and current evidence.
estimate_sequential
Section titled “estimate_sequential”estimate_sequential(snapshot, inference)Evaluate the complete roster with exact or asymptotic typed evidence.
sequential_definition_id
Section titled “sequential_definition_id”sequential_definition_id(metrics, design, source_mapping, transformations)Compute the pre-data metric/window/assignment/population binding.
Frame MetricSpecs belong in transformations so column bindings and missing value policies cannot change while a process continues.
PredictivePrior
Section titled “PredictivePrior”PredictivePriorProper Beta, NIG or NIW parameters; unrelated to an effect posterior.
Scalar NIG uses shape nu/2 and scale scale[0][0]/2. NIW uses
inverse-Wishart degrees nu and scale matrix scale.
JointReveal
Section titled “JointReveal”JointRevealSampling assumptions are declarations, never inferred from timestamps.