Question:
I saw your column on attracting top talent in the March 7 issue of Small Business Times. The eight-step model was interesting to me, especially the ones about testing. Leads me to ask this question: What do you think of on-line tests? We’re considering using one. Is this a good idea?
Answer:
Let me say at the outset that we live in a world of tests. Let’s face it. That’s just the way it is. We take tests throughout our lives, beginning in earnest during our school years.
When we learn to drive a car, we take a test to obtain a driver’s license. There are tests for admission to college and graduate school. Tests are routine in a host of performance situations like theater, music, athletics, etc. When we seek to obtain an insurance policy, we must take medical tests. We take tests when we seek professional certification. And so on. The list is seemingly endless.
When it comes to the world of work, there is also no shortage of testing. Let’s be clear on that point. Employment researchers tell us that there are currently over 2,500 tests on the market. The list of options is almost too long to sort. How does one even know where to begin?
Well, having just typed “employment test” into a search engine and received 8.1 million hits in 0.15 seconds (I do not jest), let me observe that I believe that the place to begin is with some information about what constitutes a good test.
An examination of the measurement properties of a given test is the place to start. This is the central issue. Effective tests are both reliable and valid. Sound tests are supported by statistical research that is documented in a comprehensive technical manual.
Unfortunately, there are some very dubious test vendors out there. The sad truth is that while some of these tests look like tests, are completed like tests, reported like tests, etc., they are poorly constructed (i.e., they lack adequate reliability and validity). In many ways, they are not really “tests” at all.
To be blunt, some of the tests currently on the market are fool’s gold.
Some of these tools are being driven by the dollar signs that are blinking in the eyes of the people hawking them. Be wary of any instrument that is easily accessible, where no user qualifications are required. Be suspicious when the only criterion for accessing the tool is, “Which credit card will you be paying with today?”
Frankly, I am appalled at the recent explosion of the testing enterprise in which low-quality tests are frequently used and misused by unqualified people. The desire by some to make money by hoisting spurious tools on people who should know better is disturbing to me.
So, what’s a better path to follow? As I outlined in my last column, the advice here is to implement a “multiple hurdles” strategy in which sound tools comprise each step in the process. This is consistent with the saying, “A chain is only as strong as its weakest link.” In this context, it is easy to see that the misuse of testing can undermine the entire selection process.
An effective employment selection process, however, targets bona fide occupational qualifications (e.g., knowledge, skills, abilities, behaviors) that have been documented via a thorough job analysis. Elements used to screen applicants are job-related, sound and defensible, consistent with the Uniform Guidelines on Employment Selection Procedures.
When it comes to testing, these tools should conform to the guidelines outlined in the Standards for Educational and Psychological Testing. Further, any testing vendor should be able to demonstrate, via appropriate documentation and reporting, that its instruments have “passed muster,” that they have been constructed in a manner consistent with the Principles for the Validation and Use of Personnel Selection Procedures.
If this sounds like serious business, it is meant to. Employment decision making is serious business, make no mistake about it. Hiring or promoting the wrong person (i.e., an employee who turns out to be a poor performer) can be costly, especially if this happens on an ongoing basis. In some ways, even worse situations are those in which tools are used that expose the organization to scrutiny or litigation, as in the case of deploying an employment test that is of poor quality or misusing a test (e.g., using a tool to make a hiring decision when the tool has insufficient evidence of criterion-related validity).
What can you do to avoid falling into some of these traps? Following these guidelines, originally offered by employment testing expert Robert Rose, might be helpful:
1. Make sure competent people do the testing.
If you are going to be doing the testing yourself, do yourself a favor and get some skills in this area. Various colleges and universities offer courses in tests and measures. If you are using an external vendor, make sure they are credible. Check out their credentials. Vendors without credentials should probably be bypassed.
2. Make sure outside consultants really know the organization.
Remember, one size does not fit all. The culture of your organization is different than the one across the street. Tests should fit the jobs and the organizations in which they are deployed.
3. Make sure the test reports are understandable and provide useful information.
As you read a report, if you find yourself feeling the need to credential yourself with a Ph.D., you probably need to challenge your vendor to “tone it down.” Reports laden with technical terms, jargon, etc. are less useful than reports that offer cogent insights derived from practically meaningful scales.
4. Never use tests to “cover yourself.”
If you think using a test will allow you to shift the responsibility associated with deciding whom to hire or promote, you are fooling yourself. There are no magic bullets here. No test has yet been designed that operates as a crystal ball. Good tests provide accurate information that is practically meaningful (i.e., the scores relate significantly to job success and help to separate good from not-so-good performers), that is all.
5. Never let outside consultants or the tests they use operate as decision-makers.
Outside vendors are best used as sources of information and advice. Decision making should be reserved for decision makers (i.e., hiring managers).
In conclusion, I would suggest that the adage, “let the buyer beware,” is applicable here. I would further suggest that if you have any of the concerns of the kind I touched on in this column, find yourself a capable consultant with expertise in the testing area. He or she can offer counsel, advice, training, support, etc. that can help you to avoid stepping on the many land mines that attach to this aspect of organizational practice.
