Jakub Linowski

JAN-2025

DEC-2024

ClearlinkJan2025

2025-july

APR2025

2025-MAY

2025 December

Have a look at 15 random slides from the course to get a taste for some of the scenarios, test design topics, heuristics (rules of thumb), as well as test design examples.

Course Slide Preview

Get a preview of a 5 leap experiment examples we'll discuss in more depth in the course. This small preview includes experiments I observed ran by: Amazon, Walmart and Netflix as well as one experiment I designed for Designlab.

See 5 Leap Experiment Examples From Leading Companies

Receive 5 example leap experiments from leading companies, including: Amazon, Walmart, Netflix and Designlab.

5 Leap Experiments (Netflix, Walmart, Amazon)

Product managers who drive strategy, development, and launch of products.

Designers who craft user experiences, interfaces, and visual design.

Marketers who drive brand awareness, customer acquisition, and growth across channels.

Using data and metrics to measure performance, run experiments, and inform decisions.

Weak A/B test design can hide great ideas beneath insignificant results. Learn 10 checks to avoid throwing away valuable results.

Change & Measure: 10 Checks For Designing Better AB Tests

Sr. Director Personalization & Experimentation

Analytics Consultant

Senior Product Designer 

Product Lead

Design Ops

Optimisation Lead

Head of eMerchandising

Founder

Director, Data Science

Director, Marketing Optimization

10 test design checks to improve the quality of your experiments: ❌ Loosely set triggers dilute your test, extending its duration and causing a loss of power.✔️ Learn to set better test triggers. ❌ Underpowered tests are likely to lead to insignificant results.✔️ Learn to estimate and set realistic Minimum Detectable Effects using historical data. ❌ Compromise and confounding often washes out effects.✔️ Learn to separate and structure tests with minimum confounding. ❌ Lack of iteration and giving up on an a/b test too quickly leads to wasted effort.✔️ Learn to design your a/b tests in ways that enables you to iterate. ❌ Lack of supporting metrics limits your ability to make correct decisions.✔️ Learn to include custom metrics in your a/b tests that reinforce your hypothesis. ❌ Lack of segmenting may hide the effect in the general sample.✔️ Learn to setup your tests with custom segments that may show effects in smaller subsets of users. ❌ Some changes may be too weak for you to see any difference.✔️ Learn to setup your tests with different intensities. ❌ Some ideas may be better than others.✔️ Learn to explore and select ideas from historical data to increase your odds of success. ❌ The effect may be too small from a single change.✔️ Learn to design compound experiments that combine multiple changes with positive probabilities. ❌ Not all experiments need to test for superiority which demands more power.✔️ Learn to design non-inferiority tests in some situations.

Design better experiments with fewer insignificant & throwaway results

Course syllabus

Tuesdays & Thursdays

Friday: A/B Test Reviews

Course schedule

You're a UI, UX or CRO Test Designer who is frustrated with too many insignificant experiment results.

You're an Experimentation &amp; Optimization Strategist responsible for leading A/B testing programs your team and your clients can trust.

You're a CRO, Growth or Ecommerce Manager responsible for increasing metrics such as revenue and sales with impactful A/B test results.

Learn directly from Jakub Linowski

Jakub Linowski

Courses

Change & Measure: 10 Checks For Designing Better AB Tests

Lightning Lessons

UI Experiments From Leading Companies

Types Of Experiments Rundown

Resources

5 Leap Experiments (Netflix, Walmart, Amazon)

Course Slide Preview

See 5 Leap Experiment Examples From Leading Companies