Count a unique second ticket purchase within 60 days—not opens or clicks. Verified attendance would require a separate measure.
01 / THE EXPERIMENT
From an idea to a decision
Illustrative randomized test of two follow-up approaches after a fan's first game. Choose the readout or plan an experiment before spending.
CONTROL VS. TREATMENT
What happened?
One unique, eligible first-time fan per record. Outcome: at least one second ticket order within 60 days of the first game; not verified attendance.
HYPOTHETICAL ROLLOUT ECONOMICS
Would the lift justify sending the treatment to more eligible fans?
STATISTICAL EVIDENCE
What do the results suggest?
Observed difference in repeat-order rate
Test group minus control group
The estimate and its uncertainty
● Observed difference · green band = 95% interval · dashed zero = no difference · peach marker = break-even lift
02 / RECOMMENDATION
Review the evidence
A result must be both statistically credible and worthwhile after rollout costs.
What I'd recommend next
Check assignment quality and implementation costs before any rollout.
BEFORE SENDING THE CAMPAIGN
What is worth detecting?
Size a two-group randomized experiment around the smallest meaningful change—not a hoped-for best case.
TEST SIZE & TIMELINE
A practical starting plan
CROSS-FUNCTIONAL ROLLOUT
What each team needs to know
A good experiment should improve both a business decision and the process for making the next one.
Owners & guardrails
Marketing / CRM: eligibility, messaging and holdout. Ticketing: repeat orders and inventory. Fan services: complaints and experience. Finance: true incremental contribution. Analytics: sample-size assumptions, randomization checks, and the final interpretation.
03 / WHY THIS IS DIFFERENT
Don't optimize clicks. Measure a second order.
Compare the observed improvement with a financial break-even threshold. Statistical significance is not the same thing as profitability.
Assign owners, document the holdout, guard the fan experience, and create a reusable playbook rather than a one-time marketing win.
How the statistics work, including the limits
Outcome: The unit is one eligible first-time fan with at least one second ticket purchase within 60 days of the first game. A second order does not demonstrate a physical return visit. Observe both groups for the full same-length window and deduplicate fans and purchases.
Analysis: Control = fans with a return order ÷ eligible control fans; variant = fans with a return order ÷ eligible variant fans. The two-sided p-value comes from a pooled two-proportion z-test under the hypothesis of equal conversion rates. The 95% interval for the absolute difference uses a hybrid Wilson/Newcombe approximation, which behaves better near extreme rates than a simple Wald interval.
Small samples: If an expected success or failure count falls below five, the page warns that the normal approximation is unreliable. A pre-planned exact test would be preferable. A p-value is not the probability the hypothesis is true, and even a significant result does not prove causation without valid random assignment and measurement.
Economics: Point net = future eligible fans × observed percentage-point difference × contribution per extra return order − additional rollout spending. The lower-interval case substitutes the lower 95% interval endpoint for the difference. That is sensitivity to estimation error—not a probability-weighted financial forecast.
Sample-size planning: Approximate two-sample normal design, independent Bernoulli outcomes, equal group sizes, two-sided α = 0.05, 80% or 90% power, a fixed target lift, and a finite audience availability check. It does not adjust for repeated looks, multiple variants, clustering, misclassification, or attrition.
Example SQL: turning CRM events into a clean test readout
A hypothetical data contract could assign eligible first-time buyers to one of two variants. The analysis dataset must contain exactly one assignment row per fan, plus a binary flag for at least one return order inside the agreed window. This is a schema illustration, not an actual connected database.
-- Imaginary table: one assigned fan per row.
-- returned_in_window is already deduplicated and validated.
SELECT variant,
COUNT(*) AS eligible_fans,
SUM(CASE WHEN returned_in_window THEN 1 ELSE 0 END)
AS fans_with_return_order
FROM fan_return_assignments
WHERE eligible_at_assignment = TRUE
AND first_time_fan = TRUE
GROUP BY variant
ORDER BY variant;
Before trusting the query: check that assignments are balanced, purchases are counted consistently, exclusions were defined in advance, and late-returning fans had the same opportunity to be observed.
Original AI-assisted portfolio learning demonstration. All campaigns, fan counts, purchases, budgets, and outcomes are simulated; this is not a Rockies study. Statistical approximations and business assumptions are disclosed. No personal fan data is used. Explore the companion revenue forecast ↗