(Photo of group of people doing a hard thing from Wikimedia user Rizimid, CC BY-SA 3.0.)
This is an extended version (more info, more sources) version of the talk I gave at EA Global San Francisco 2017. The other talk I gave, on extinction events, is here. Some more EA-focused pieces on diversity, which I’ve read but which were assembled by the indomitable Julia Wise, are:
Effective altruism means effective inclusion
Making EA groups more welcoming
EA Diversity: Unpacking Pandora’s Box
Keeping the EA Movement welcoming
How can we integrate diversity, equity, and inclusion into the animal welfare movement?
Pitfalls in diversity outreach
There are moral, social, etc. reasons to care about diversity, all of which are valuable. I’m only going to look at one aspect, which is performance outcomes. The information I’m drawing from here are primarily meta-studies and experiments in a business context.
Diversity here mostly means demographic diversity (culture, age, gender, race) as well as informational diversity – educational background, for instance. As you might imagine, each of these has different impacts on team performance, but if we treat them as facets of the same thing (“diversity”), some interesting things fall out.
(Types of diversity which, as far as I’m aware, these studies largely didn’t cover: class/wealth, sexual orientation, non-cis genders, disability, most personality traits, communication style, etc.)
Studies don’t show that diversity has an overall clear effect, positive or negative, on the performance of teams or groups of people. (1) (2) The same may also be true on an organizational level. (3)
If we look at this further, we can decompose it into two effects (one where diversity has a neutral or negative impact on performance, and one where it has a mostly positive impact): (4) (3)
This is the human tendency to have an ingroup / outgroup mindset. People like their ingroup more. It’s an “us and them” mentality and it’s often totally unconscious. When diversity interacts with this, the effects are often – though not always – negative.
Diverse teams tend to have:
- Lower feelings of group cohesion / identification with group
- Worse communication (3)
- More conflict (of productive but also non-productive varieties) (also the perception of more conflict) (5)
A silver lining: One of these ingrouping biases is the expectation that people more similar to us will also think more like us. Diversity clues us into diversity of opinions. (6) This gets us into:
— 11/9/17 – I’m much less certain about my conclusions in this section after further reading. Diversity’s effects on creativity/innovation and problem-solving/decision-making have seen mixed results in the literature. See the comments section for more details. I now think the counterbalancing positive force of diversity might mostly be as a proxy for intellectual diversity. Also, I misread a study that was linked here the first time and have removed it. The study is linked in the comments. My bad! —
Creative, intellectual work. (7) Diversity’s effects here are generally positive. Diverse teams are better at:
- Creativity (2)
- Innovation (9)
- Problem solving. Gender diversity is possibly more correlated than individual intelligence of group members. (Note: A similarly-sized replication failed to find the same results. Taymon Beal kindly brought this to my attention after the talk.) (10)
Diverse teams are more likely to discuss alternate ideas, look at data, and question their own beliefs.
This loosely maps onto the “explore / exploit” or “divergent / convergent” processes for projects. (2)
- Information processing effects benefit divergent / explore processes.
- Social categorization harms convergent / exploit processes.
If your group is just trying to get a job done and doesn’t have to think much about it, that’s when group cohesiveness and communication are most important, and diversity is less likely to help and may even harm performance. If your group has to solve problems, innovate, or analyze data, diversity will give you an edge.
How do we get less of the bad thing? Teams work together better when you can take away harmful effects from social categorization. Some things that help:
- The more balanced a team is along some axis of diversity, the less likely you are to see negative effects on performance. (12) (7) Having one woman on your ten-person research team might not do much to help and might trigger social categorization. If you have five women, you’re more likely to see benefits.
- Remote teams are less biased (w/r/t gender). Online teams will be less prone to gender bias.
- Time. Obvious diversity becomes less salient to a group’s work over time, and diverse teams end up outperforming non-diverse teams. (13) (6) Recognition of less-obvious cognitive differences (e.g. personality and educational diversity) increases over time. As we might hope, the longer a group works together, the less surface-level differences matter.
This article has some ideas on minimizing problems from language fluency, and also for making globally dispersed teams work together better.
How do we get more of the good thing? Diversity is a resource – more information and cognitive tendencies. Having diversity is a first step. How do we get more out of it?
- At least for age and educational diversity, high need for cognition. This is the drive of individual members to find information and think about things. (It’s not the same as, or especially correlated to, either IQ or openness to experience (1)).
Harvard Business Review suggests that diversity triggers people to stop and explain their thinking more. We’re biased towards liking and not analyzing things we feel more comfortable with – the “fluency heuristic.” (14) This is uncomfortable work, but if people enjoy doing it, they’re more likely to do it, and get more out of diversity.
But need for cognition is also linked with doing less social categorization at all, so maybe diverse groups with high levels of this just get along better or are more pleasant for all parties. Either way, a group of people who really enjoy analyzing and solving problems are likely to get more out of diversity.
2) A positive diversity mindset. This means that team members have an accurate understanding of potential positive effects from diversity in the context of their work. (4) If you’re working in a charity, you might think that the group you might assign to brainstorming new ways to reach donors might benefit from diversity more than the group assigned to fix your website. That’s probably true. But that’s especially true if they understand how diversity will help them in particular. You could perhaps have your team brainstorm ideas, or look up how diversity affects your particular task. (I was able to find results quickly for diversity in fundraising, diversity in research, diversity in volunteer outreach… so there are resources out there.)
Again, note that diversity’s effect size isn’t huge. It’s smaller than the effect size of support for innovation, external and internal communication, vision, task orientation, and cohesion – all these things you might correctly expect correlate with performance more than diversity (8). That said, I think a lot of people [at EA Global] want to do these creative, innovative, problem-solving things – convince other people to change lives, change the world, stop robots from destroying the earth. All of these are really important and really hard, and we need any advantage we can get.
- Work Group Diversity
- Understanding the effects of cultural diversity in teams: A meta-analysis of research on multicultural work groups
- The effects of diversity on business performance: Report of the diversity research network
- Diversity mindsets and the performance of diverse teams
- The biases that punish racially diverse teams
- Time, Teams, and Task Performance
- Role of gender in team collaboration and performance
- Team-level predictors of innovation at work: A comprehensive meta-analysis spanning three decades of research
- Why diverse teams are smarter
- Evidence of a collective intelligence factor in the performance of human groups
- When and how diversity benefits teams: The importance of team members’ need for cognition
- Diverse backgrounds and personalities can strengthen groups
- The influence of ethnic diversity on leadership, group process, and performance: an examination of learning teams
- Diverse teams feel less comfortable – and that’s why they perform better
Pingback: Rational Feed – deluks917
(crossposted from the EA Forum)
For whatever it’s worth, I looked into the 4 sources you cite in your article as improving the effectiveness of diverse teams and found the following:
1 didn’t replicate, and the replication found the opposite effect with a much larger sample size (which you link to in your article)
One is a Forbes article that cites a variety of articles, two of which I looked into and didn’t say at all what the Forbes article said they say, with the articles usually saying “we found no significant effects”
One study you cited directly found the opposite result of what you seemed to imply it does, with its results table looking like this:
And the results section of the study explicitly saying:
“whereas background diversity displayed a small negative, yet nonsignificant, relationship with innovation (.133).” (the thing that did have a positive relation was “job-related diversity” which is very much not the kind of diversity the top-level article is talking about)
The only study that you cited that did seem to cite some positive effects was one with the following results table:
Which found some effects on innovation, though overall it found very mixed effects of diversity, with its conclusion stating:
“Based on the results of a series of meta-analyses, we conclude that cultural diversity in teams can be both an asset and a liability. Whether the process losses associated with cultural diversity can be minimized and the process gains be realized will ultimately depend on the team’s ability to manage the process in an effective manner, as well as on the context within which the team operates.”
LikeLiked by 1 person
Thanks for the in-depth response!
Regarding the replication, I think its results make a decent case that the original finding might have been a statistical fluke. The replication found that individual IQ was strongly linked with group IQ, and that gender diversity wasn’t linked with performance.
That said, the replication seems to have been slightly smaller than the original, not much larger – the original used 40 groups of 3 and 152 groups of 2-5, and the replication used 26 groups of 3, and two sets of 40 groups of 3 for replicating 3 parts of the hypothesis, so the original should have been slightly larger. Am I reading this right? I’m tempted to call it ambiguous for now.
(Here’s a discussion in which the authors of both papers show up in the comment thread – the statistics quickly got over my head, but the interested and more literate may learn from it.)
The review piece you mentioned was this one, right? (I didn’t refer to anything from Forbes.) I looked into three random sources (e.g. scrolling down the page and clicking on a link without reading) and saw:
* http://www.tandfonline.com/doi/abs/10.5172/impp.2013.15.2.149 – This article was correctly described as finding a fairly strong link between gender diversity in companies and increased radical innovation. The paper also found no correlation with incremental innovation, though the article didn’t mention this. (It didn’t investigate other kinds of diversity.)
* https://www.credit-suisse.com/corporate/en/media/news/articles/media-releases/2012/07/en/42035.html – I was unable to find the full version of this study, but according to the press release, it did find effects that supported the claims from it.
* http://onlinelibrary.wiley.com/doi/10.1111/ecge.12016/full – This one found “small but significant effects” from cultural diversity.
I didn’t read these studies in detail, and they’re probably not the same ones you cross-checked. I should probably be more careful about review articles, but the bits I found seemed supportive of what the final article said. Which ones didn’t, if you have that easily available?
This article did in fact have an altogether different conclusion than what I said it did. I must have somehow misread it – this is embarrassing. I also cite it as saying something about decision-making, but maybe I was thinking of a different one? In any case, I unintentionally misrepresented the study and am going to strike it. Much obliged for catching that.
I agree that the last study doesn’t find overall positive or negative effects for diversity – I even cite it as saying such in the first section.
Now I’m wondering about the decision-making and creativity claims. If we lump the reported positive aspects into two clusters – “creativity/innovation” and “problem-solving/decision-making” – the whole section does seem to be much more ambiguous than before. (Mixed evidence for each.)
I spent a few hours looking around (with the explicit intent of not just considering things that reported what I already thought) and was hard-pressed finding other recent review articles finding results between demographic diversity and creativity/decision-making/problem solving. The Work Group Diversity study I cited describes some of the confusing results (for diversity and positive effects on performance) from the literature.
Based on that, I’m now considering that the positive aspect of diversity on performance might be mostly mediated through informational diversity, which demographic diversity sometimes relates to. (e.g. different lived experiences, women tending to be more empathetic, etc.) As summarized in this review, for instance:
“Thus, overall, most of the support for the value-in-diversity hypothesis comes from studies concentrating on functional differences (serving as a proxy for diversity in skills, information, and expertise). These differences have typically been shown to improve performance through vigorous debate that leads to creativity and improved problem solving (see also Bunderson & Sutcliffe, 2002; Carpenter, 2002; Pitcher & Smith, 2000).”
In any case, I do somewhat expect to find some upside, because if diversity’s effects on performance are overall neutral, and we have a mechanism for them to be negative, I’d also expect a mechanism for them to be positive.
I wouldn’t be at all surprised if the answer is out there, and this is important, but I’m at the limit of how much time I can spend on this right now. I’d like to do some more reading on this later (and encourage others to do the same).
Thanks for double-checking all of my double-checking!
“Based on that, I’m now considering that the positive aspect of diversity on performance might be mostly mediated through informational diversity, which demographic diversity sometimes relates to. (e.g. different lived experiences, women tending to be more empathetic, etc.)”
This is roughly my current model of the benefits of diversity, though I do actually mostly expect the net benefits of not explicitly informational diversity to be net-negative because of obvious reporting biases. Though I haven’t done a rigorous analysis (i.e. funnel plot or similar stuff) to be confident about this.
Pingback: EAG 2017 SF: Diversity in EA, and lightning talks – EARadio