Question:
I recently read an article about the return on investment (ROI) of continuing education. It was refreshing to see some people talking about learning within the context of business, that learning can be a business driver. What are your views on the relationship between learning and business? What can be done to demonstrate that a positive relationship exists?
Answer:
One of the biggest issues facing human resources professionals today is the need to train and develop the organization’s employees. The development of a skilled and educated workforce is seen as an ongoing priority that is being woven into many organization’s strategic plans.
Let’s be clear on this point from the beginning: training and development is big business today. It has been estimated that more than $60 billion is spent annually on training for approximately 50 million employees in the United States. A confluence of events, including global competition, an infusion of information, and rapidly changing technology, has fueled the need for continuous learning in organizations. Many experts have come to believe that learning and competitiveness are now interdependent in this information age. After all, companies are facing more competitive challenges with fewer people, and those people must learn how to work harder, smarter, faster, and better.
However, it is also recognized that while the need for learning is growing, financial resources are disseminated with greater caution. Therefore, the demand for demonstrable results, impact, and some return on investment (ROI) has led many organizations to focus more heavily on the evaluation of the training programs they offer. Yet, although learning is increasingly supported as a means for facilitating planned change within organizations, few empirical studies substantiate training effectiveness.
In our consulting practice at ODC, we spend time with our clients undertaking a variety of program evaluation initiatives. We believe strongly that, “What gets measured, gets done.” So, within the context of a balanced scorecard approach, we see the ability to effectively evaluate the “people practices” aspect of the organization as very important.
In helping our clients get “ARMED” (i.e., acquiring, retaining, managing, educating, and developing employees), we make use of measurement approaches and models that demonstrate the utility of the important human resource-related activities in which the organization engages.
By far the most widely used and influential typology for evaluation of the impact of training in business and industry is Dr. Donald Kirkpatrick’s hierarchical model. Kirkpatrick uses four levels to evaluate the effectiveness of training: (1) reaction, (2) learning, (3) behavior, and (4) results.
Reaction criteria are the participants’ reactions to the program. These criteria measure impressions and feelings about the training.
Learning criteria evaluate how much has been learned in the training program. A final exam given at the end of a training program is an example of a learning criterion.
Collectively, reaction and learning criteria are called internal criteria because they refer to evaluations internal to the training program.
Behavioral criteria refer to actual changes in performance once the employee is back on the job. These criteria address such questions as, “To what extent are the desired changes in the job behaviors of the trainee realized by the learning program?”
Results criteria refer to the economic value of the training program to the company. One premise is that if the training program has any effect, the average performance of the trained group should exceed that of the untrained group. A break-even analysis can be made to determine if the economic value of the training program to the organization is greater than zero. Utility analyses are based on a careful assessment of the costs associated with developing training, training materials, training time, and production losses.
Collectively, behavioral and results criteria are called external criteria because they refer to assessments external to the training program.
A few years ago I had the pleasure of undertaking a research project with Dr. Sara Thompson, who was at that time directing continuing education programming at UWM. We pursued our work under the auspices of the local chapters of the American Society for Training and Development (ASTD) and the Society for Human Resources Management (SHRM). The focus of the research was to examine and document the training evaluation practices of organizations in southern Wisconsin.
The results of the research project suggested that organizations in southern Wisconsin were presenting employees with a varied and diverse array of learning opportunities. Significantly, though, while the organizations surveyed appeared to recognize the importance of evaluating the training they offered, in terms of Kirkpatrick’s model of evaluating training programs, the majority of organizations attended to only levels one and two of the framework.
Less than half of the organizations indicated they were evaluating at levels three and four. Many of those organizations that were evaluating these upper levels of the model were struggling to do so effectively and were not necessarily employing particularly sophisticated methods. Interestingly, in many instances, respondents identified that they were unsure about how to even begin to evaluate at levels three and four. Still other program evaluators questioned the value of such an effort when compared with the costs associated with data collection and analysis (i.e., “Is all the hard effort and work worth it?” “Does it really matter?”).
What conclusions did we draw from these results? Among other things, we felt that in order to more effectively evaluate levels three and four of the Kirkpatrick model and to offer training that has a real impact, HR and training professionals needed to forge stronger ties within the organizational hierarchy. We recommended a “cooperative management” model that involved integrating the efforts of HR and training departments, line management, and senior management.
Specifically, we identified three things that needed to occur for this suggestion to come to fruition: (1) training has to be aligned with the strategic objectives of the organization, (2) HR and training professionals need to be involved with senior management in making that link, and (3) HR and training professionals and line managers must cooperate to ensure integration between learning programming, strategic business objectives, management of on-the-job performance, and impact on the bottom line.
In the final analysis, effective learning programming is driven by a collaborative effort between senior management, as they determine the organization’s future direction and strategic objectives, HR and training departments, as they determine learning needs, training topics, delivery methods, and learning objectives, and line management, as they observe and coach on-the-job performance, that can form the foundation for effective training, and, ultimately, training evaluation.
