ct.CAMERON TRIBBETTSTRATEGY / OPERATIONS / ANALYTICS

FAN GROWTH · TEST & LEARN · SIMULATED DATA

Can we earn a second order?

After a fan’s first game, can a useful follow-up lead to another ticket purchase within 60 days? Separate an encouraging signal from a sound business decision.

FAN EXPERIENCE × BUSINESS IMPACT

01 / THE EXPERIMENT

From an idea to a decision

Illustrative randomized test of two follow-up approaches after a fan's first game. Choose the readout or plan an experiment before spending.

CONTROL VS. TREATMENT

What happened?

One unique, eligible first-time fan per record. Outcome: at least one second ticket order within 60 days of the first game; not verified attendance.

Try:

HYPOTHETICAL ROLLOUT ECONOMICS

Would the lift justify sending the treatment to more eligible fans?

Designed for a clean experiment: randomize eligible fans once, deduplicate contacts, hold the observation window fixed, and avoid peeking at results repeatedly. The example numbers are invented.

STATISTICAL EVIDENCE

What do the results suggest?

Observed difference in repeat-order rate

+1.10 pp

Test group minus control group

Usual follow-up3.50%
New invitation4.60%
Approx. two-sided p-value—
95% interval · absolute lift—

The estimate and its uncertainty

−2 pp+2 pp

● Observed difference · green band = 95% interval · dashed zero = no difference · peach marker = break-even lift

Point-estimate contribution$11,850
Lower-interval case—
Break-even lift0.22 pp
The confidence interval reflects uncertainty in conversion rates. Translating it into contribution assumes unchanged margins, no cannibalization, and similar future audiences.

02 / RECOMMENDATION

Review the evidence

A result must be both statistically credible and worthwhile after rollout costs.

What I'd recommend next

Check assignment quality and implementation costs before any rollout.

03 / WHY THIS IS DIFFERENT

Don't optimize clicks. Measure a second order.

01 · A clear fan outcome

Count a unique second ticket purchase within 60 days—not opens or clicks. Verified attendance would require a separate measure.

02 · A business hurdle

Compare the observed improvement with a financial break-even threshold. Statistical significance is not the same thing as profitability.

03 · A repeatable process

Assign owners, document the holdout, guard the fan experience, and create a reusable playbook rather than a one-time marketing win.

How the statistics work, including the limits

Outcome: The unit is one eligible first-time fan with at least one second ticket purchase within 60 days of the first game. A second order does not demonstrate a physical return visit. Observe both groups for the full same-length window and deduplicate fans and purchases.

Analysis: Control = fans with a return order ÷ eligible control fans; variant = fans with a return order ÷ eligible variant fans. The two-sided p-value comes from a pooled two-proportion z-test under the hypothesis of equal conversion rates. The 95% interval for the absolute difference uses a hybrid Wilson/Newcombe approximation, which behaves better near extreme rates than a simple Wald interval.

Small samples: If an expected success or failure count falls below five, the page warns that the normal approximation is unreliable. A pre-planned exact test would be preferable. A p-value is not the probability the hypothesis is true, and even a significant result does not prove causation without valid random assignment and measurement.

Economics: Point net = future eligible fans × observed percentage-point difference × contribution per extra return order − additional rollout spending. The lower-interval case substitutes the lower 95% interval endpoint for the difference. That is sensitivity to estimation error—not a probability-weighted financial forecast.

Sample-size planning: Approximate two-sample normal design, independent Bernoulli outcomes, equal group sizes, two-sided α = 0.05, 80% or 90% power, a fixed target lift, and a finite audience availability check. It does not adjust for repeated looks, multiple variants, clustering, misclassification, or attrition.

Example SQL: turning CRM events into a clean test readout

A hypothetical data contract could assign eligible first-time buyers to one of two variants. The analysis dataset must contain exactly one assignment row per fan, plus a binary flag for at least one return order inside the agreed window. This is a schema illustration, not an actual connected database.

-- Imaginary table: one assigned fan per row.
-- returned_in_window is already deduplicated and validated.
SELECT variant,
       COUNT(*) AS eligible_fans,
       SUM(CASE WHEN returned_in_window THEN 1 ELSE 0 END)
           AS fans_with_return_order
FROM fan_return_assignments
WHERE eligible_at_assignment = TRUE
  AND first_time_fan = TRUE
GROUP BY variant
ORDER BY variant;

Before trusting the query: check that assignments are balanced, purchases are counted consistently, exclusions were defined in advance, and late-returning fans had the same opportunity to be observed.

Original AI-assisted portfolio learning demonstration. All campaigns, fan counts, purchases, budgets, and outcomes are simulated; this is not a Rockies study. Statistical approximations and business assumptions are disclosed. No personal fan data is used. Explore the companion revenue forecast ↗

Built on Hatchable