Introduction to Signs and Samples
In today’s data-driven world, HR professionals are increasingly looking for objective data to help them make more informed choices in leadership selection and development. But with a wide range of options on the market, the first challenge is to understand different leadership assessment types.
There are two main types of assessments, which are based on the type of data they collect – signs or samples. In this section, we'll cover both types, and their applications.
What Do I Need to Do with the Assessment Data?
Before diving into different leadership assessment types, the first question you should ask is what you want to be able to do with the data. It may sound obvious, but it’s a step that many HR professionals overlook. As a result, they may end up choosing the wrong assessment for their needs.
Start by framing the questions that you need to answer. Are you trying to make promotion decisions? Do you need to assess broad skill gaps across your leadership population? Are you trying to get accurate data on skill gaps, or are you merely trying to build self-awareness in leaders?
It’s essential to answer these questions up front because it may lead you to different leadership assessment types. The danger is that if you choose the wrong one, it can be tempting to incorrectly interpret the data to suit your needs.
For example, personality insights can be very useful in driving self-awareness. However, it doesn’t tell you what someone is capable of from a behavioral perspective. Nor does it tell you anything about readiness for a leadership position.
But if that’s the only data you have, it may be tempting to use it to make a leadership selection decision. For example, you might end up choosing someone with a highly extroverted personality over a more introverted peer for a manager role, assuming they are more of a “people” person.
Once in the role, you might find that the extroverted manager struggles to listen to others and micromanages their projects. Meanwhile, the introverted person is the person that everyone on the team goes to because they listen well and give measured coaching to help them succeed.
The point is, with the wrong data, you may make poor decisions. That’s why it’s so important to state your goals up front about what you plan to do with the data.
The Difference in Leadership Assessment Types: Signs vs. Samples
One of the biggest sources of confusion when choosing between leadership assessment types is the difference between “signs” and “samples.” Signs and samples represent the different ways that data is collected, and consequently, what you can use the data for.
Signs are data points, such as personality or judgment, that can be used as indicators of future performance. In contrast, samples are actual examples of a person’s behavior. When making a prediction about a person’s performance in a role, signs require more inference than an actual sample of a person’s behavior.
A simple way to understand the difference is to think of a driving test. In most places, potential drivers are required to pass a written test to gain a learner’s permit. The test quizzes the driver on driving laws and may ask how they would react to certain situations.
If a person passes the test, you know that they have a baseline understanding of what to do and are likely to make the right choice in those situations.
Does that mean they’re ready to get on the road? Not even close.
To actually get your driver’s license, you need to pass a road test. You need to get behind the wheel, and someone needs to evaluate you while you’re driving. You need to prove that you really can perform the skill.
Back to our leadership assessment types. In this analogy, the written test for the learner’s permit is like tests on skills, values, or personality. They’re all signs of performance. Meanwhile the road test gives you a sample of performance.
In a leadership assessment, signs are usually collected through a written or multiple choice format. Leaders are asked to choose the best action to respond to a scenario. Or they are given a scenario or statement and asked to agree with it.
In contrast, simulation-based assessments – also known as assessment centers – immerse a leader in leadership challenges. They have to respond to fictional challenges in real time. Their responses are recorded as video, audio, or open-ended written responses. That way, you can truly observe their behavior.
As you might expect, samples of performance are typically more predictive (and provide more detailed insights) than signs of performance.
What Do You Learn from Signs vs. Samples?
An example of the “signs” type of leadership assessment might be a multiple choice situational judgment test. A multiple choice situational judgment test shows us that the person knows what to do in various situations. But it doesn’t tell us whether the person can actually do those things.
This is really important for leadership performance. Plenty of smart leaders understand what to do, such as in a coaching situation. But they may not be very good at doing it. And they might not realize the difference.
Personality tests are another example of signs. Based on how a person is wired, you might be able to predict what parts of a leadership role will be easy or challenging for them. For example, someone who is resistant to change may struggle to lead it. However, it doesn’t mean that they can’t lead change. It simply means it will take more energy and effort to do so.
On the flip side, assessment centers provide sample data. As part of the experience, they put leaders in an immersive situation, and see how they react. For example, someone might be given a fictional scenario in which they need to lead a change on their team. They might have to write email responses to situations, participate in a coaching role play, and create a written strategy of how they would lead that change. With that sample data, you get an accurate look at the leader’s capability to lead change.
When Should I Use Each Type of Assessment?
Knowing when to use the two different leadership assessment types is critical. Just as we started with framing your purpose, there are a few questions you should ask yourself when determining which type to use:
- How many people do you need to assess? Are you doing a small group, or trying to quickly get data across your leadership population? For high volume situations where scalability is key, signs-based assessments typically make more sense.
- Do you want to make selection decisions based on the data? Both types of leadership assessments can be used to support selection decisions (see our notes on personality testing). However, sample-based assessments provide more in-depth insights and are typically more accurate in predicting performance. If the decision is for a critical position, sample-based assessments are the better bet.
- How in-depth do you need the data to be for development? If you’re using the data to simply drive self-insights, signs-based data may be robust enough. But in some situations you may need deeper insight. For example, high-potential leaders benefit from sample-based testing that can give them highly accurate, deep insights about their skills. Similarly, if you are trying to accelerate succession readiness, sample-based assessments are the better way to go.
In short, here’s a quick summary of how to use each type of leadership assessment:
