Championing workplace inclusion: how we built our game-changing assessment

Jan 5, 2021

Employee inclusion has been all the rage in recent years and rightly so. Alongside Equality and Diversity, it’s one of the three core elements that transforms a workplace from a privileged playground to an environment where everyone can succeed.

From a cluster of feelings, attitudes and emotions to a range of experiences and perspectives, defining ‘inclusion’ was no easy feat. As such, we blended the latest science with painstaking research to find inclusion’s key aspects and compile a list of unprejudiced questions that could measure it.

The result? Our exclusion-busting inclusion scale. Here’s how we did it.

How we built our inclusion scale

1. Science. As a first step, we reviewed the scientific literature about workplace inclusion across various fields: Sociology, (Occupational and Organisational) Psychology, Law, Behavioural Science, Human Resource Management and Economics.

2. Existing scales. We then analysed existing inclusion scales and found that most were out-dated (written in the 80s and 90s), not sufficiently validated and only developed with WEIRD people (those who are White, Educated, and from Industrialised Rich and Democratic countries).

3. Distillation. Next, we distilled the key inclusion concepts into six core dimensions (later we further-distilled them to the five below) each capturing a different facet of an inclusive workplace experience.

  • A. Voice. Your manager and colleagues ask for your opinion and want to hear what you have to say. At a broader level, it's also about having the 'power' to affect change in the organisation.
  • B. Psychological safety. Not being afraid to show vulnerability, speak up and have an opinion that's different from the rest of the team. It also means being able to trust your manager and peers to have your back.
  • C. Openness. Where everyone is open to new ideas, making mistakes and learning from them as a team, while also seeking diverse perspectives.
  • D. Belonging. People care about recognising, respecting and including each other. As a result, people feel they can be themselves at work.
  • E. Fairness. Employees feel they are evaluated on their skills and experience, not on who they know, what they look like or any other part of their social identity. It's also about a fair and unbiased distribution of work, benefits, recognition and opportunities.

4. Loads of questions. Using scientific research and our six core dimensions, we wrote loads of inclusion questions: 120+ in total.

5. Less questions. Well aware no-one wants to answer 120 questions, we filtered them down by interviewing D&I professionals and asking a diverse group of real-life employees how they would answer the questions. Using this approach, we not only cut down the total number, but improved the clarity, consistency and insight of each question.

6. Pilot tests. We conducted two rounds of pilot tests with real employees based in England and in full/part-time employment, sampling a balanced mix of ethnicities, gender, sexuality and age.

7. Validation. To get all technical for a second, we then embarked on a convergent validity test. This means we tested our scale alongside the renowned Mor-Barak inclusion scale (taken by millions of people) and found a high correlation between the two scales, confirming that our scale measures the same underlying concept.

8. Analysis. To get even more technical, we then used ****statistical techniques (i.e. Cronbach Alpha) to see which questions fit nicely together. The exploratory and confirmatory factor analysis helped us to settle on our dimensions described above.

9. Elimination. Next, we eliminated questions that generated volatile responses as this indicates they measured different things for different people.

10. Super survey. Finally, we tested several scale formats and chose one that's consistent, quick-to-answer and provides a great user-experience for employees answering our inclusion survey.

🎨 Final outcome

Months later, only 22 solid, validated and, dare we say it, perfect questions made it to the final cut.

Fancy trying out the inclusion scale?

Sign up for Fair HQ private beta here [link] and we’ll notify you when we're ready to go. As we don't do half measures at Fair HQ, we’ll not only help you assess your company’s Inclusion, but also its Equality and Diversity, before building a bespoke strategy on how to improve and helping you implement it.

Dig deeper?

If you’re like us and love digging into the technical aspects of scale development and testing, here’s a list of checks we’ve done to test the reliability and validity of our assessment.

✅ Construct validity - our assessment has all the key elements of inclusion, as verified by D&I experts.

✅ High internal reliability - the individual items in the scale are consistent with each other.

✅ Convergent validity - our assessment is related to a renowned measure.

✅ Clear factor structure - the dimensions we constructed are verified by the data.

✅ Clear scale format - our questions are easy and quick to complete, but produce fine-grained data.

✅ Normally distributed - our items show a close alignment to the normal distribution.

---

Meanwhile, if you want to delve into some further research on the topic, here’s some great background reading:

Cordier, R., Milbourn, B., Martin, R., Buchanan, A., Chung, D., & Speyer, R. (2017). A systematic review evaluating the psychometric properties of measures of social inclusion. PLoS One, 12(6), e0179109.
Gehlbach, H., & Artino Jr, A. R. (2018). The survey checklist (manifesto). Academic Medicine, 93(3), 360-366.
Gehlbach, H., & Brinkworth, M. E. (2011). Measure twice, cut down error: A process for enhancing the validity of survey scales. Review of General Psychology, 15(4), 380-387.
Mor-Barak, M. E., & Cherin, D. A. (1998). A tool to expand organizational understanding of workforce diversity: Exploring a measure of inclusion-exclusion. Administration in Social Work, 22(1), 47-64.
Mor Barak, M. E. (2015). Inclusion is the key to diversity management, but what is inclusion?. Human Service Organizations: Management, Leadership & Governance, 39(2), 83-88.
Ryan, K., Gannon-Slater, N., & Culbertson, M. J. (2012). Improving survey methods with cognitive interviews in small-and medium-scale evaluations. American Journal of Evaluation, 33(3), 414-430.
Simms, L. J. (2008). Classical and modern methods of psychological scale construction. Social and Personality Psychology Compass, 2(1), 414-433.
Wright, T. A., Quick, J. C., Hannah, S. T., & Blake Hargrove, M. (2017). Best practice recommendations for scale construction in organizational research: The development and initial validation of the Character Strength Inventory (CSI).Journal of organizational Behavior, 38(5), 615-628.
GLAD YOU ENJOYED THE ARTICLE

Suggestions

Ready to boost your company through D&I?

Get notified
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.