A/B and Multivariate testing are key in ongoing conversion rate optimization efforts. This guide presents a cross-platform a/b test framework for email, websites or ad testing, regardless of your software.

Opportunity-Out starts by finding the best areas of opportunity in your website, and working your way into a solid A/B or multivariate test from there. This is inverse of my preferred use of the Scientific Method for Conversion Rate Optimization Testing, but is excellent in helping crack a serious case of ‘strategists block.’

We’ll cover:

• Knowing Where to run an A/B or Multivariate Test
• Knowing What to Test in an A/B or Multivariate Test
• Creating an A/B or Multivariate Test

Where to Run A/B or Multivariate Testing

Unless you’re #testing the highly explored, prepare to go back to the drawing board. Click To Tweet

A/B testing requires a large volume of traffic (or recipients) to yield statistically relevant results. With its additional variations, multivariate requires exponentially more data before it begins to yield insights.

Begin your testing efforts high in your funnel, such as homepage copy, highly trafficked landing page headlines or email subject lines.

Optimizely has a very handy A/B Test Sample Size Calculator which can help determine the number of visitors or subscribers needed for accurate test results. This calculator helps determine the sample size (visitors, email recipients) required per variation.

An A/B test is two variations.

A multivariate test is one variation per possible combination for each element. Testing a headline and call-to-action, two elements, at four versions each would be 4*4=16 variations.

Expecting a 15% lift on your 5% baseline conversion rate in a headline A/B test isn’t unrealistic, but telling your boss that 26,000 visitors will need to pass through the landing page before results are trustworthy might have you looking for a bigger impact test, or a higher trafficked opportunity for early tests.

What to Test in A/B or Multivariate Testing

If you’re #testing something with minimal impression, expect to end up without a lesson. #CRO Click To Tweet

Headline tests, subject lines, the addition of explanatory graphics or call-to-action button copy are popular starting points, as these allows highly visual test platforms for the best A/B test hypotheses.

Avoid the temptation to start in with button color, minor design modifications, and even in-copy tweaks. While these are great target areas for late-stage testing, the impact here is lower, requiring a larger investment in time for a smaller (if any) increase in conversions.

What is the most visual elements on your highly-trafficked opportunity from the step above?

• In email, this is typically the subject line (for open rates), followed by the opening header and call-to-action for click-rate.
• In a homepage, this is often a primary call to action in the navigation, ‘hero’ or banner area, followed by specific offers.
• In a landing page, this is typically opening header, form and supporting imagery.

Forming an A/B or Multivariate Test

#CRO Improvements in Succession Start With a Question. Click To Tweet

It’s called A/B testing, not A/A-ish testing. Be purposefully different in your variations by forming a hypothesis, an educated guess.

I cover this more in depth in the Scientific Method for Conversion Rate Optimization Testing, but in short:

• Take notice of a trend, untested best-practice or any element of your current status-quo.
  • We only use long-form copy to explain our product.
  • We only ask for sales, and offer no other method of engagement in our site.
  • We haven’t sent an offer to our email list in a long time, or ever!
• Ask a question.
  • Can an image be worth these thousand words?
  • Would a ‘less invested’ engagement offer get more leads?
  • Would our email list respond to a specific offer?
• Make a hypothesis.
  • An image would improve comprehension of, and comfort with, our product offering.
  • Offering a secondary offer can help engage those not yet ready to purchase.
  • Many in our email list would be interested receiving, and redeeming, an occasional offer.

It’s in the hypothesis that you’ll find your marching orders for your A/B or multivariate testing. Remember, your ‘A’ is your control, your current state. Your ‘B’ should include a test for your hypothesis.

Using this method opens the door to follow-up conversion-rate-optimization studies, such as ‘which kind of offer earns more engagement?’

Share and Swipe

I truly hope this helps you conquer your testing challenges! Pass this along to help make the internet a better converting place for all.

Designing the Best A/B and Multivariate Testing. #CRO Click To Tweet Unless you’re #testing the highly explored, prepare to go back to the drawing board. #CRO Click To Tweet If you’re #testing something with minimal impression, expect to end up without a lesson. #CRO Click To Tweet #CRO Improvements in Succession Start With a Question. Click To Tweet It’s called A/B testing, not A/A-ish testing. #CRO Click To Tweet