What you need to know when designing tests in-house
The use of psychometric tests is normally controlled by The Psychological Testing Centre (PTC) which is part of the British Psychological Society. They set standards for training people to use tests and will also quality assure tests which are submitted to them for use on the open market.
Psychometric tests normally take a minimum of 3 years to develop as it is essential that they can demonstrate legally defensible levels of reliability, sensitivity, validity (including face, content, criterion and construct validity) and can demonstrate that adverse impact has been considered and mitigated during the construction of the test.
Test development is a 6-stage process and will require a large sample of people to trial the test-this has to include representative samples minority groups, so it is usual for tests to have samples of hundreds and sometimes thousands of people. Use of tests which have not met these rigorous standards could potentially leave an employer vulnerable to claims of discrimination and the legal action which would result from this.
Design of psychometric tests in-house can present a number of challenges which is why they tend to be purchased from an established Test Publisher which has the resources and expertise to develop them in line with all the fairness, reliability and validity requirements. Because without being absolutely sure that testing methods are fair (not discriminating in a biased way), reliable (accurate and consistent in the way they measure) and valid (measuring relevant factors), there may be little point in using them.
What about situational judgement tests?
Psychometric publishers will normally apply the same standards to an SJT as they would to any psychometric instrument (even though they do not currently come under the remit of the PTC), because essentially this is also a test which has the ability to discriminate (even inadvertently) against protected characteristics. So it is best practice is to apply normal standards of psychometric development to SJTs, for all the same reasons outlined above.
Before designing in-house SJT’s do be aware of the standards required and the legal risks if these are not applied during test development.