1# A/B Test Templates Reference
2 
3Templates for planning, documenting, and analyzing experiments.
4 
5## Contents
6- Test Plan Template
7- Results Documentation Template
8- Test Repository Entry Template
9- Quick Test Brief Template
10- Stakeholder Update Template
11- Experiment Prioritization Scorecard
12- Hypothesis Bank Template
13 
14## Test Plan Template
15 
16```markdown
17# A/B Test: [Name]
18 
19## Overview
20- **Owner**: [Name]
21- **Test ID**: [ID in testing tool]
22- **Page/Feature**: [What's being tested]
23- **Planned dates**: [Start] - [End]
24 
25## Hypothesis
26 
27Because [observation/data],
28we believe [change]
29will cause [expected outcome]
30for [audience].
31We'll know this is true when [metrics].
32 
33## Test Design
34 
35| Element | Details |
36|---------|---------|
37| Test type | A/B / A/B/n / MVT |
38| Duration | X weeks |
39| Sample size | X per variant |
40| Traffic allocation | 50/50 |
41| Tool | [Tool name] |
42| Implementation | Client-side / Server-side |
43 
44## Variants
45 
46### Control (A)
47[Screenshot]
48- Current experience
49- [Key details about current state]
50 
51### Variant (B)
52[Screenshot or mockup]
53- [Specific change #1]
54- [Specific change #2]
55- Rationale: [Why we think this will win]
56 
57## Metrics
58 
59### Primary
60- **Metric**: [metric name]
61- **Definition**: [how it's calculated]
62- **Current baseline**: [X%]
63- **Minimum detectable effect**: [X%]
64 
65### Secondary
66- [Metric 1]: [what it tells us]
67- [Metric 2]: [what it tells us]
68- [Metric 3]: [what it tells us]
69 
70### Guardrails
71- [Metric that shouldn't get worse]
72- [Another safety metric]
73 
74## Segment Analysis Plan
75- Mobile vs. desktop
76- New vs. returning visitors
77- Traffic source
78- [Other relevant segments]
79 
80## Success Criteria
81- Winner: [Primary metric improves by X% with 95% confidence]
82- Loser: [Primary metric decreases significantly]
83- Inconclusive: [What we'll do if no significant result]
84 
85## Pre-Launch Checklist
86- [ ] Hypothesis documented and reviewed
87- [ ] Primary metric defined and trackable
88- [ ] Sample size calculated
89- [ ] Test duration estimated
90- [ ] Variants implemented correctly
91- [ ] Tracking verified in all variants
92- [ ] QA completed on all variants
93- [ ] Stakeholders informed
94- [ ] Calendar hold for analysis date
95```
96 
97---
98 
99## Results Documentation Template
100 
101```markdown
102# A/B Test Results: [Name]
103 
104## Summary
105| Element | Value |
106|---------|-------|
107| Test ID | [ID] |
108| Dates | [Start] - [End] |
109| Duration | X days |
110| Result | Winner / Loser / Inconclusive |
111| Decision | [What we're doing] |
112 
113## Hypothesis (Reminder)
114[Copy from test plan]
115 
116## Results
117 
118### Sample Size
119| Variant | Target | Actual | % of target |
120|---------|--------|--------|-------------|
121| Control | X | Y | Z% |
122| Variant | X | Y | Z% |
123 
124### Primary Metric: [Metric Name]
125| Variant | Value | 95% CI | vs. Control |
126|---------|-------|--------|-------------|
127| Control | X% | [X%, Y%] | — |
128| Variant | X% | [X%, Y%] | +X% |
129 
130**Statistical significance**: p = X.XX (95% = sig / not sig)
131**Practical significance**: [Is this lift meaningful for the business?]
132 
133### Secondary Metrics
134 
135| Metric | Control | Variant | Change | Significant? |
136|--------|---------|---------|--------|--------------|
137| [Metric 1] | X | Y | +Z% | Yes/No |
138| [Metric 2] | X | Y | +Z% | Yes/No |
139 
140### Guardrail Metrics
141 
142| Metric | Control | Variant | Change | Concern? |
143|--------|---------|---------|--------|----------|
144| [Metric 1] | X | Y | +Z% | Yes/No |
145 
146### Segment Analysis
147 
148**Mobile vs. Desktop**
149| Segment | Control | Variant | Lift |
150|---------|---------|---------|------|
151| Mobile | X% | Y% | +Z% |
152| Desktop | X% | Y% | +Z% |
153 
154**New vs. Returning**
155| Segment | Control | Variant | Lift |
156|---------|---------|---------|------|
157| New | X% | Y% | +Z% |
158| Returning | X% | Y% | +Z% |
159 
160## Interpretation
161 
162### What happened?
163[Explanation of results in plain language]
164 
165### Why do we think this happened?
166[Analysis and reasoning]
167 
168### Caveats
169[Any limitations, external factors, or concerns]
170 
171## Decision
172 
173**Winner**: [Control / Variant]
174 
175**Action**: [Implement variant / Keep control / Re-test]
176 
177**Timeline**: [When changes will be implemented]
178 
179## Learnings
180 
181### What we learned
182- [Key insight 1]
183- [Key insight 2]
184 
185### What to test next
186- [Follow-up test idea 1]
187- [Follow-up test idea 2]
188 
189### Impact
190- **Projected lift**: [X% improvement in Y metric]
191- **Business impact**: [Revenue, conversions, etc.]
192```
193 
194---
195 
196## Test Repository Entry Template
197 
198For tracking all tests in a central location:
199 
200```markdown
201| Test ID | Name | Page | Dates | Primary Metric | Result | Lift | Link |
202|---------|------|------|-------|----------------|--------|------|------|
203| 001 | Hero headline test | Homepage | 1/1-1/15 | CTR | Winner | +12% | [Link] |
204| 002 | Pricing table layout | Pricing | 1/10-1/31 | Plan selection | Loser | -5% | [Link] |
205| 003 | Signup form fields | Signup | 2/1-2/14 | Completion | Inconclusive | +2% | [Link] |
206```
207 
208---
209 
210## Quick Test Brief Template
211 
212For simple tests that don't need full documentation:
213 
214```markdown
215## [Test Name]
216 
217**What**: [One sentence description]
218**Why**: [One sentence hypothesis]
219**Metric**: [Primary metric]
220**Duration**: [X weeks]
221**Result**: [TBD / Winner / Loser / Inconclusive]
222**Learnings**: [Key takeaway]
223```
224 
225---
226 
227## Stakeholder Update Template
228 
229```markdown
230## A/B Test Update: [Name]
231 
232**Status**: Running / Complete
233**Days remaining**: X (or complete)
234**Current sample**: X% of target
235 
236### Preliminary observations
237[What we're seeing - without making decisions yet]
238 
239### Next steps
240[What happens next]
241 
242### Timeline
243- [Date]: Analysis complete
244- [Date]: Decision and recommendation
245- [Date]: Implementation (if winner)
246```
247 
248---
249 
250## Experiment Prioritization Scorecard
251 
252For deciding which tests to run:
253 
254| Factor | Weight | Test A | Test B | Test C |
255|--------|--------|--------|--------|--------|
256| Potential impact | 30% | | | |
257| Confidence in hypothesis | 25% | | | |
258| Ease of implementation | 20% | | | |
259| Risk if wrong | 15% | | | |
260| Strategic alignment | 10% | | | |
261| **Total** | | | | |
262 
263Scoring: 1-5 (5 = best)
264 
265---
266 
267## Hypothesis Bank Template
268 
269For collecting test ideas:
270 
271```markdown
272| ID | Page/Area | Observation | Hypothesis | Potential Impact | Status |
273|----|-----------|-------------|------------|------------------|--------|
274| H1 | Homepage | Low scroll depth | Shorter hero will increase scroll | High | Testing |
275| H2 | Pricing | Users compare plans | Comparison table will help | Medium | Backlog |
276| H3 | Signup | Drop-off at email | Social login will increase completion | Medium | Backlog |
277```
278