Feature - CPG in practice - March Edition

How do candidates with extra time perform?

With sufficient Career Progression Gateway projects under our belts, we have been able to turn our attention to the stats and review how candidates with additional needs and extra time have performed overall. By reviewing the data we could work out if having extra time was providing the right sort of help.

We looked at candidates from four different FRS, completing assessments from Crew to Station Manager. The data set is small, with only 1-2% of total candidates tending to request the provision of additional time.

We looked at the average score of all other candidates, and also the range of scores in the cohort, from lowest to highest.

Of the 10 candidates who had additional time across our projects last year, six scored higher than the candidate average for their specific project, 4 scored lower. This also equates to six CPG pass marks and four unsuccessful. This is consistent with candidates without additional time provision.

‘Additional time’ candidates scored from second highest in the cohort to 12th (bottom ranking). This is useful to know in terms of there being a range of scores/ performance, and not all ‘bunched’ in one area.

It’s early days in terms of having a decent number of cases to review, but so far the results are encouraging that additional time is levelling the playing field between neurotypical and neuro-diverse candidates, which still allows for a range of performance to be demonstrated. This is further evidence that the assessment is fair, reliable and valid for all candidates.

Did you know?

You can now request a ‘text to speech’ function to be added to information presented on the CPG. This service is provided for free on our system and enables candidates who require it to be able to listen to the information, with pause/ rewind and forwarding functions included.

Home based assessments- can this work?

This month we worked with a FRS client to facilitate candidates completing the CPG (both case study and roleplay exercises) from their own home.

When we began to explore the possibilities of conducting the CPG online, although manageable in theory, we had concerns over the IT set up. For in-house projects we work with a single point of contact to check the computers to be used. This time we had 15 computers in 15 locations, with 15 different setups/ configurations to worry about!

So, we developed an IT set-up process to mitigate against risks and provide the ability to assess online, whilst ensuring the same standard of service delivery.

What worked

An online IT checklist was sent to candidates ahead of the assessment to complete at home, on the computer they planned to use. Candidates answered a few, quick questions based on how a test version of the CPG performed on their computer. This provided everybody with the same user experience and guaranteed all functionality was working.

In the event that some aspect of the IT setup didn’t work (this didn’t occur for this cohort) – there was the built-in option to raise an IT help request at the time. This ensured that any potential IT issues were highlighted and resolved prior to the start of the assessment. In-house FRS administrators were reassured, knowing that they wouldn’t be faced with a remote IT issue interfering with the smooth running of the assessment itself.

Candidates were remotely invigilated whilst completing the case study exercise and their roleplay preparation time by the client via the software package of choice with the video and audio function on. The added benefit was candidates knowing they had a link to an administrator in case they needed help. For the roleplay, the invigilator logged out when the phone call started, and the assessor took over.

Candidates had the option to conduct their Role-play exercise via MS Teams internet call, mobile phone or landline. The MS Teams choice was popular, and offered flexibility to those who had poor mobile signal at home. The Role-play assessors joined an MS Teams call with the candidate ahead of time to set up the call and to ensure line quality. It was fed back that this option was well received by candidates and put them at ease for their assessment.

What we learned

Preparation and expectation setting is key. Not everyone completed the pre-IT checks. Ensuring candidates understand the importance of this stage, and potential consequences of an IT issue during their assessment, is essential.

Some improvements could be made to coordination of FRS and VCA processes during the role-play calls. This would reduce repetition of invite/ log in emails and avoid candidates being overwhelmed with duplicate information.

Is Assessing Online a Viable Option?

Whilst assessing via remote home-based assessment presents its own set of challenges, we have learned, developed and streamlined how this solution can work going forward, to increase the flexibility of assessments we can offer FRS’s.

Next feature:

Using the data to assess your organisation’s strengths and development areas in relation to the Leadership Qualities; and a review of high-volume assessments.