Flickr image: Two Loners, by Cass

A/B, or split testing, offers you a way to test out different course design ideas you have, and to see how your students understand them. It basically consists of comparing two versions of a course component, and seeing which one works best for your learner.  We started the LX Design blog to explore how or which UX methods may be useful in learner experience design, and A/B testing is a very common method. The Nielsen Norman Group provides an excellent deep dive into A/B testing on their blog, should you desire additional information. This post is written from a Q and A perspective, and questions received in the comments or via private message can be incorporated if you send them along.

How can I use A/B testing in my courses?

You can use A/B testing in a number of ways. Here are a few ideas:

• Maybe you’re thinking about two different ways to tell students how to complete an assignment. One is text, and one is a video. A/B test them and see which works better!
• Course landing pages can be a notoriously sticky area for students to get tripped up in a course. What are you saying with the first page your students see? Use A/B testing to ask them.
• Wondering what components of your course are most important to your students? Rearrange them into two or three different configurations, and see which one works best.

What tools do I need to have, in order to do A/B testing?

As with all the best questions, the answer to this one is “it depends.”

• You can do A/B testing using screenshots and a survey. For no charge, the built-in screen capture capabilities in Windows and in Mac OS allow you to capture images and ask your students about them. You could also use free third party software like Jing. Screen captures are best used for testing layouts or static items in a course, as opposed to items where you wish to discover how students use or navigate particular areas.
• Sandbox courses can prove very useful for A/B testing. Create two sandbox courses on your Learning Management system, and ask your student sample to complete concrete tasks in each. For example, “locate the syllabus,” or “check your grades for Quiz 1.” If you are doing the testing in person, have your students talk through their thought process out loud. If you’re doing the testing at a distance, have students take notes or screen record their work in each of the sandbox courses. Ask them which they found easier and why, and watch their interactions with the environment.
• Many of us may have played the “Would You Rather” game, where scenarios are presented, and players must choose which they would rather do, with an explanation of why. This can also prove to be a useful strategy when you are A/B testing learning environments. Using scenarios either in person or in a survey, and asking students which would make the learning process easier, or which they would prefer can help you form a more complete picture of student needs.

What items should I A/B test?

Getting started can be daunting if you’re not sure what questions you want to ask. Start simple and explore how your students understand:

• Module names
• Content locations
• Content titles
• Assessments (wording, structure)
• Course layout and design
  • Course landing page
  • Module arrangement
  • Color choices
  • Navigation buttons
  • Images

A/B testing doesn’t have to be an all or nothing exercise. You can test things that you think might be introducing confusion, or things that you were unsure of as you were designing your course. Maybe you received some grumbling about a certain exercise or content location, A/B testing can help you figure out how to alleviate student confusion or dissatisfaction. A few small A/B testing exercises can mean big benefits for how your course runs, and for the students taking it.

Here,  I’ve described some ways to use A/B testing. Maybe you have been using it in your work as well? Or maybe you have more questions? We’d love to hear from you in the comments.