First launched in 2019, the VCA Ltd Career Progression Gateway online assessments haves now been used by 8 FRS from Firefighter Recruit up to Strategic Manager (Area Manager and above) levels. We have assessed 1054 candidates in that time.
We’ve gathered a lot of data and continually developed the process to what it will be in 2022. So what have we learnt, and what might be useful for Fire & Rescue services, even those not currently using the CPG to facilitate their promotions?
1. Overall Fire & Rescue Services perform remarkably similarly
When it comes to measuring candidates’ performance on the assessment exercises there is a good level of consistency. This is useful in terms of not only knowing that the exercises are reliably doing what they are supposed to, but also to be able to make predications. For instance, we can inform a new FRS using the CPG that they can expect a roughly 50% pass rate on their assessments, with a margin of roughly 10%.
2. There are usually good reasons for changes to performance trends
When we look at the data, we might notice a slight change in how candidates have performed from one year to the next. Obviously, some variations can be expected, but more consistent or stronger trends need to be investigated, and by talking to clients we can usually get to the bottom of these. For instance, one FRS had a drop in scores of roughly 10% between Crew Manager CPG, which seemed higher in comparison with other FRS. A potential cause was discovered to be that changes to the eligibility criteria had been made, allowing a wider pool of candidates to apply, but reducing overall scores.
3. Different groups of candidates may perform differently
Early indications suggest that on-call FFs score averages will be slightly lower than whole-time. Whilst we don’t have concrete data to explain this, it does mean that expectations can be adjusted to ensure a level playing field. Another example is when the Group Manager CPG was used for applications to a Business Support role, which non-FRS staff could apply to. Whilst overall averages were similar to FRS operational candidates, there was a difference in the way leadership was demonstrated. This information allows us to not only predict outcomes, but also advise on steps which may be taken before or after the CPG to meet different needs.
4. Approaches to promotions and development differ across the FRS
Most FRS use the CPG to identify potential to go forward to a final stage interview. But around that, there are wider differences. One FRS provides development to candidates who have scored an ‘amber’ rating in any element of the CPG or interview, so that they can re-sit either one within 6 months to see if they have moved to a ‘green’- and are therefore eligible for promotion. Some FRS focus on more formal development and attainment of certain courses before the CPG, and development post-CPG is based on their feedback and line manager input.
5. FRS personnel have adapted incredibly well to on-line assessments
Feedback from both clients and candidates tells us that using on-line assessments, including a telephone interview, have been accepted well. However, some candidates would prefer the roleplay to be face-to-face. Assessment instructions are understood equally well, whether in a supported FRS environment, or remotely, i.e. from home. We are currently exploring the possibility of video role-plays to bridge the gap and make the experience more user-friendly whilst retaining the cost and convenience benefits of remote assessment.
6. Candidates are getting better at highlighting what tools they need to perform well
We’ve received an increased number of client requests and have been able to develop ways we can meet these additional requirements on an on-going basis. We anticipate this will continue, and the support options we are able to provide will also grow in tandem.
7. The Leadership Qualities are identifying performance differences
With performance on two exercises, Watch and Station Managers tend to score higher on ‘Outstanding Leadership (OL)’ and lower or ‘Organisational Effectiveness (OE)’. It evens out for Group Manager level, but OE drops again at Area Manager and above. In the case study, ‘Service Delivery (SD)’ is weaker from Crew to Station levels, and gets stronger at Group Manager and above. On the roleplay, the average score for ‘Personal Impact’ is above 70% at all levels. What’s interesting about these figures is how individual FRS compare to the averages; it’s also useful to keep an eye on trends across different years to track changes and form conclusions which can inform culture change and development practice.
8. There are some variations across FRS in how they are demonstrating the LQs
With different FRS using the same tools to measure performance across shared criteria, there is scope for comparison. What might one FRS be doing well at Crew Manager level that another FRS seems to struggle with? Which FRS is excelling in Group Manager performance in relation to its FRS counterparts, and what might the reasons be? The more insights the data and follow-up evaluation can give us, the greater the scope for identifying, and sharing, best practice for the benefit of FRS across the country.
If there is any information you would like more details on, or for a discussion on how to apply learning from the CPG to your internal processes then please do get in touch via our contact form.
