Presentation. Stakeholder briefing. Professional discussion. Strategic discussion. Stakeholder carousel. Group discussion.

These are just a few of the exercise formats we’ve seen appearing across Fire and Rescue Service selection processes. Some are familiar. Some are newer variations on existing approaches. The format itself isn’t really the interesting part. What matters is whether the exercise helps you assess the things you’re actually trying to assess. When designing any discussion or presentation exercise, there are a few questions worth asking.

Are you creating an opportunity for candidates to demonstrate their knowledge or their behaviours?

It’s easy to assume you’re assessing leadership capability when, in reality, you’re assessing what somebody knows about leadership.

Think about your most successful managers:

  • What is it that makes them successful?

  • When do you actually see those qualities?

  • Do those qualities tend to emerge in situations similar to the exercise you’ve designed?

It’s also worth looking in the other direction. Think about less capable managers, particularly in office/ team/ station-based leadership roles:

  • What behaviours cause problems?

  • What do you regularly see day to day?

  • What have you built into your scenarios to test for those things?

If an exercise doesn’t create the opportunity for those behaviours to appear, it’s worth questioning how much it can tell you about likely performance in role. What are you measuring, and why? Every assessment exercise should have a clear purpose.

Ask yourself:

  • What exactly am I measuring?

  • Why am I measuring it?

  • Does successful performance in the exercise link to successful performance in role?

  • How do I know?

That final question is often the most important. If a candidate performs well in a stakeholder briefing, professional discussion or presentation, what does that actually tell you about how they’ll perform as a manager? The stronger the link between assessment performance and role performance, the more confidence you can have in the exercise.

Are you creating opportunities for candidates to demonstrate a range of behaviours?

One of the risks in assessment design is believing you’ve created variety when, in reality, you’re measuring the same things repeatedly. It’s worth considering:

  • Are candidates being given opportunities to demonstrate a range of behaviours?

  • Or are you assessing the same limited range of behaviours several times using similar methods?

Different exercise titles don’t automatically mean different evidence.

What are assessors actually looking for?

A well-designed exercise still depends on assessors being clear about what they’re evaluating.

Consider the following:

  • Do assessors know how to separate behaviours by leadership concepts?

  • Do they have construct guidance?

  • Do they have indicator guidance?

  • Are they trained to look for behaviours?

  • Or are they evaluating knowledge and opinions?

That’s an important distinction. There is a significant difference between assessing what good leadership sounds like and assessing what good leadership looks like.

Is the assessment process standardised?

The more variation there is between candidates’ assessment or exercise experience, the harder it becomes to make fair comparisons.

  • Are candidates experiencing the same conditions?

  • Are they receiving the same questions?

  • Are they working through the same scenarios?

  • Are they having the same interactions?

  • Are you minimising the impact of other candidates in group exercises (so the only variable is your candidate’s performance)?

Consistency matters- it’s a foundation for fairness. This is particularly relevant in group exercises.

Consider:

  • How much influence do other candidates have on an individual’s opportunity to demonstrate behaviours?
  • What have you put in place to minimise that impact?

These aren’t always the most exciting parts of assessment design, but they’re often some of the most important.