Professionals who do occasional training in their area of subject expertise often do not give much thought to evaluation. They know it should be done but wait until the last minute to whip up a participant reaction survey, commonly referred to as a "smile sheet".  At the end of the training, they beseech participants to complete the survey but then fail to provide time or motivation to do so, look at the responses received with apprehension, heave of a sign of relief when they are positive and then consider the evaluation process complete.

Research reported in professional training literature indicates that there is little, correlation between favorable participant reaction and the actual amount of learning that occurred. The "smile sheet" euphemism comes from the observation that this tool is generally supportive of the trainer. It encourages instructors to entertain rather than focus on participant learning, suggests to participants that learning is passive rather than active and comes too late in the training cycle to correct any design flaws.

As an evaluation tool, the smile sheet is familiar, inexpensive, quick and easy to create. Yet evaluation is a complex job and no single tool can measure training.  A broader and newer view of evaluation is that it must be multi-level (distributed throughout the organization), continuous throughout the training cycle, focus on business results and support the organization. The one-shot smile survey does not meet these requirements but can be combined successfully with other tools. In a rapidly changing world, evaluation tools need to be constantly refined to measure the value of training to your organization.

You should familiarize yourself with the Kirkpatrick model which uses the participant reaction survey as the first of four levels for evaluating training success: 

1. Get participant reactions (smile sheet)
2. Measure participant learning
3. Observe change of on-the-job behaviors
4. Look for organizational results

Designing An Evaluation Strategy

Begin the evaluation process in tandem with an analysis or needs assessment. When specific training objectives are identified, consult with others in your unit and organization (including the potential trainees) to evaluate whether the objectives are specific, measurable, achievable, realistic and doable within the proposed time frame.

Evaluate the instructional technique(s) to use.  How can new on-the-job behaviors be measured?  Is a pre-test necessary? How long after training should a follow-up be made? If soft skills such as customer service are involved, will the participants be open to role-playing?  Do the training facilities need to simulate the workplace reality? Explore whether training is the best solution to the problem you are trying to solve. Sometimes mentoring, coaching, counseling, or a job aid can solve the problem as well.

Select evaluation tools for all the program components:  materials, instruction methods, facilities, arrangements, equipment, media used, costs, and organizational support. An excellent program can be ravaged if offered at a time that conflicts with other organizational priorities or is held in cramped, uncomfortable facilities.

Determine what you, other decision makers and stakeholders want the evaluation process to do and how evaluation fits into your knowledge management plan. Possible evaluation goals might include:

• Provide feedback
• Compile a database of who is trained, licensed or credentialed
• Determine return on investment
• Measure costs and variables to administer the training budget
• Report relationships between training and improved job performance
• Give positive reinforcement to trainees to help them stay motivated
• Show trainers where to improve
• Build a learning community built on shared experiences
• Indoctrinate new staff
• Promote change
• Recognize accomplishments
• Test skill levels
• Identify fast-trackers
• Focus energy on specific personnel issues

During the design phase, consult with a colleague or professional trainer to review your design and assumptions for the choice of instructional technology.  Will the participants have the background information or vocabulary?  Will they be familiar with the processes related to the procedures they learn? Include a quiz with the training announcement so the participants can self-test and self-discover what they need to know. People remember what they discover for themselves.

Design an end-of-course participant survey to assess the immediate impression of the program but only ask questions that give you information you intend to use or report. Tailor a form specifically for each program.  Avoid generic forms since they send a "ho-hum" message regarding evaluation and participants will respond accordingly.

Before and after the survey use other evaluation tools: interview, test, focus group, job performance observation, or brainstorming. Interview not only the participants but the other stakeholders (trainers, supervisors, managers, co-workers, customers) after six weeks and three months to find out if there are new on-the-job behaviours. At this point, you may discover that there are environmental impediments, such as lack of equipment, supervisor support or authority, to using the newly acquired knowledge and skills.

Survey Form Tips

Explain at the beginning of the program that the evaluation is an important and integral part of the training. Participants are part of the training team and their input is vital. Transition to the survey with a brief wrap-up or brainstorming session. The group's input will verbally "jump start" individual thinking on the written form.

Ensure that you allow for time for the participants to complete the form before the training ends. "Take aways" never get returned. Provide an incentive to complete the survey such as a job aid, promotional item, or momento of the training experience.

Identify the program, trainer and date at the top of each survey. To encourage completely honest responses, the respondents should be anonymous along with the return procedure. Do not have someone standing at the door reading the responses as participants exit.

Keep your survey form simple and to one page.  Make it visually attractive. Convert reactions to numerical ratings for easier tabulation, i.e., 1 = poor, 2 = fair, 3 = good, 4 = excellent. Fewer categories are better than more since ratings convey impressions and are not exact measurements.  Ten categories afford no more accuracy than four or five.

Include several open ended questions asking participants to make comments or suggestions. What are the three most useful things you learned?

Speaking-Tips.com is one of the web's best-known resources for learning public speaking and presentation skills.