Pilot Testing a Course

Pilot Testing with Actual Learners

In the tabletop review step, project members reviewed the course content for completeness and accuracy. Now, it’s time to put the course in front of the learners and measure how they interact with the materials. In most cases, the pilot test will be the first time actual learners experience the course.

The pilot test of the course takes place before the official course implementation. It provides the training specialists and the clients a final chance to review the course prior to its official launch.  In the production of online training this step is sometimes referred to as Validation.

The Benefits of Pilot Testing

In many ways, a course’s pilot session is similar to a software beta test. Whenever you put a piece of software or a course in front of actual learners, they’ll interact with it very differently than trained experts will. Experts and learners will be sensitive to different types of issues within the course.

Subject matter experts can make sure that the course materials are accurate and complete, but they might not catch that a learning activity’s instructions should be worded with more clarity. People who are actually trying to learn the course material provide a very rigorous test for two reasons. They’re often very willing to provide candid feedback. Additionally, at the end of the course, you can measure how well the pilot course’s learners have achieved the course’s learning objectives.

That’s why Intulogy’s representation of the ADDIE model includes a two-stage review process after course development. Both tabletop review and pilot testing provide important quality assurance checks, and there’s really no way for one process to replace the other.

Pilot Testing Methodologies

The pilot test will be conducted differently, depending on the course’s delivery format. Let’s take a look at two possible examples.

If the course is an online course, learners may sit individually at their computers and take the course. Learners may be asked to fill out a survey after they finish the course. Then, the training specialist reviews the data and perhaps conducts follow-up interviews.

However, if the course is an instructor-led course, learners and instructors may gather in a classroom while a training specialist quietly takes notes at the back of the classroom.

Issues Identified in Pilot Testing

Before the pilot test, the training specialist builds a checklist of issues. Some of these issues are standard review items, but others will be specific to the course’s content and its delivery format. Here’s a very brief list of some issues that the training specialist might measure during the course pilot:

• Measure the amount of time learners need for each module and activity
• Check learners’ engagement with the material
• Detect points where material may be too easy/too difficult
• Confirm that learners understand the instructions for activities and exercises
• Evaluate the flow and balance of the course
• Test how well learners achieve the course’s stated learning objectives by the end of the course
• Validate the course assessment tools
• Collect feedback from learners about the course
• Locate points where the course should be revised

During the course pilot, it’s important to let learners interact with the course rather than try to correct things on-the-fly. When you spot something wrong, it might be tempting to jump in and “add one thing” but that can create a cascade effect throughout the course. After the course pilot, the training specialist and the client meet and decide what revisions should occur before the course launches.