Early Results of Process Quality Survey in Collaborative Networks

This summer I put out a call for people engaged in collaborative networks to take a survey about their experiences in those networks. Thank you to everyone who responded and who encouraged others to do the same. Although I may not be able to use the results in this round of research, you can still take an abbreviated version of  the survey if you like.

Below you can see some of the demographics of who responded to the survey. You will see that the answers don't always add up perfectly, that's because most of the questions did not require a response and some people chose not to respond.

Survey respondents' demographic info

You probably notice a lot of the same things I did when looking at the demographics. For example, the high number of responses from women (which is particularly noteworthy since nearly half of respondents said men and women were present in roughly equal numbers in their network), and the small number of respondents who are part of a racial minority in their network (of those 12 only 2 identified as white). Also, if you are like me, you are developing theories about why the demographics shook out as they did.

In this post, I will share some preliminary analysis of the quantitative data that was gathered regarding fairness. My next post will examine the quantitative data on collaboration. Future posts will explore open-ended responses, as well as dig more deeply into some of the interesting patterns that emerge.

Some words of caution: In general, a sample size of 30 or more is considered statistically valid. However, as you can see, we only barely have a statistically significant sample size of men and we do not have a statistically significant sample size for people who are in a racial minority in their network, nor for those in the governmental sectors. Moreover, those of us working in this field know that sometimes the term "network" is interpreted very differently by different people. I tried to filter for this by asking people entering the survey if they met certain criteria, but it's still hard to say how much "apple-to-apple" comparisons are really embedded in these results.

Lastly, because I want this blog to be readable, I am mainly avoiding detailed discussion of methodology, but please do reach out if you want more information about how I arrived at the interpretations below.

So, if you are willing to hold all these caveats and treat these results lightly, let's dig in!

Procedural Fairness 

The survey used the Process Quality Scale (PQS) developed by Darrin Hicks and colleagues to measure perceptions of how fair a process is (as opposed to how fair the outcomes of a process are, see my earlier blog post). This 15-question questionnaire deals with whether people see the process as authentic, revisable, and consistently applied. In the survey, I asked people to base their responses on their impressions about processes in their network overall, as opposed to one particular process.

The good news

In general, people find their networks to be engaged in processes that are mainly fair. At least, insofar as the PQS can measure such a thing. In 8 of the questions, the most common response was that processes are more fair than not, and in the other 7 questions, the most common response was that the processes are fair. 

I went into this analysis curious to see if a strong difference would be noted between members and staff/consultants. However, the results were virtually indistinguishable. Members seems to think their networks are a pretty fair place to be, and so do the staff and consultants. 

The interesting news

You don't have to be a novelist to know that readers don't want to hear about all the happy things, readers want the dirt! Not you, of course, but perhaps the other readers of this blog. Well, it turns out that when we break the data down, we see that the intensity with which people perceive procedural fairness varies significantly. 

I was surprised to see that the group that seems to have the strongest sense of fairness in their networks was men who are in a network that is majority-woman. The second strongest perceptions of fairness come from those who are in a racial minority in their network, although this group had a wider range of opinion than the men who were in gender minorities. 

Bubble chart showing the modal average responses to the PQS, broken down by subgroups

Women and men are the two groups that have the most pronounced differences in interpretations of the level of fairness. In 13 of the questions on the PQS, the most common response from men was that the process was fair. For women, they said the process was fair in response to only 3 questions, and for 11 others they said it was only more fair than not. To one question, the most common response from women was that things were more unfair than not (the question was about whether everyone has equal opportunity to influence decisions). 

I suspect the relatively strong feelings of fairness among racial minorities may have something to do with the fact that 75% of that group are men, and of those about half are in a network comprised largely of women. A larger sample size might reveal significantly different answers. As it is, respondents who were a racial minority were still the only group to have more than one question generate a response that things were more unfair than not (one question was about giving some more than they deserve while shortchanging others, another was about some people's merits being taken for granted while others have to justify themselves). 

As you can see, there is a lot to unpack. So stay tuned, and, if you want to see all the questions, feel free to go to the abbreviated version of the survey here. 

Interpretation

As I move forward in the research, I will be examining the other data and engaging in interviews to attempt to understand the differences in particular between men's and women's perceptions. Is it possible that women are more accustomed to collaboration than men, and thus situations that seem incredibly collaborative to men are merely acceptably collaborative to women? Could men be obliviously on the receiving end of deferential treatment? Could women be consulted less, or expected to pick up more of the procedural work? Is the fact that it was more challenging to get men to take the survey somehow related?  Most importantly, is it a problem that there is a discrepancy in the answers of men and women, and can this data help us make networks better? Drop me a line if you want to be part of the conversation!

Procedural and Distributive Fairness

In this post, I return to the topic of Fairness Heuristic Theory, and make a key distinction between two ways fairness can be experienced: in an outcome or in a process.

What's a Heuristic and Why Should I Care?

As it's used here, think of a heuristic as a rule of thumb, one that allows you make a decision without having to think through every angle. You know it's imperfect, but you use it to help you make sense of what's going on and make judgements fairly quickly. I once met a woman who would only date a man who had an iPhone. To her, Androids represented slovenliness, lack of ambition, and a willingness to inconvenience others. By focusing on iPhone users, she was able to give her attention to just a few men, one of whom she fell head over heels for and married.

In the case of Fairness Heuristic Theory (FHT), an individual's perceptions of fairness are used as a rule of thumb to guide their decisions about whether to collaborate with others. Unlike the case of the iPhone-phile above, the rule of thumb operates in the background, not usually something that is explicitly thought about. To understand why it matters for networks, we need to understand the difference between procedural and distributive fairness (also called procedural or distributive justice). 

Procedural and Distributive Fairness

When we think about whether things are fair, many of us think right away about outcomes. But research by E. Allan Lind, Kees Van den Bos and others indicates it's not usually that type of fairness that influences our decisions about whether to join with others. Rather, we look for clues about whether we can trust the process, how we can expect to be treated, and what our standing in the group will be.

This classic union cartoon from the second half of the last century demonstrates the concept of distributive (un)fairness.

Distributive fairness is about who gets what (funding, credit, blame, promotions, exposure, etc.). Procedural fairness is about how the getting gets done.

Whereas this one has to do with procedural (un)fairness

Research by D. Hicks and C. Larson indicate that a process is generally perceived as fair when it is seen as authentic, revisable, inclusive, and transparent. (Want to share how your network measures up? Take my research survey.)

Which Is More Important?

Neither type of fairness is intrinsically more important. But there is reason that our perceptions of procedural fairness generally influence willingness to join collaborations more than distributive fairness: namely, we are usually exposed to process before outcome. 

Research by Lind and colleagues has shown that we tend to form opinions about fairness very quickly, and that, once formed, those opinions are pretty stable. In other words, if our first exposure to a network is fair, then things that happen later in that network will tend to be interpreted through the understanding that the network is basically fair. Things that might otherwise be seen as unfair will either be explained away or be seen as an aberration. Conversely, if our first experience is perceived as unfair, then later experiences will be interpreted with the understanding the network is a basically unfair place. 

As a general rule, potential network members are presented first with options for participation (i.e. processes for collaborating to share knowledge, co-create strategy, experiment together, etc.). Rarely (but not never) a member is invited to a network by being presented with an outcome, such as a portion of funding or some other resource.

Because the first exposure to a network is usually a process, procedural fairness is usually a stronger force in a member's overall opinion about it. At least, if FHT holds true.

Does It Actually Matter?

I've been interviewing thought leaders in the world of social-impact networks. Some of them feel the issue of fairness is an important one that affects their networks, others feel members typically are not affected by a desire for fair treatment either in terms of process or outcome, but rather join networks for altruistic reasons.  

What do you think? Leave me a comment or take my research survey. 

Fairness Heuristic Theory & Networks

In my last post, I discussed Soft Systems Methodology, which I am using to explore the role of fairness in collaborative networks. In this post, I share the basic premise of Fairness Heuristic Theory. In my next post, I will talk about what I have been learning about how network leaders view their network culture.

Fairness Heuristic Theory

E. Allan Lind and his colleagues first proposed the Fairness Heuristic Theory (FHT) in 1992. In brief, the theory is that perceptions of fairness act as a "rule of thumb" for measuring whether one should invest in a collaborative effort. If one perceives that fairness will reign within that collaboration, one is vastly more likely to to lend one's resources and--more importantly--one's identity to the effort, according to this theory.

According to Lind, the most essential aspects of people's perception of fairness are rooted in whether one feels valued and has standing as part of the group. They render this judgment based on feelings of inclusion, ability to effectively raise concerns, and sensing respect in the way others treat them. In cases where the person feels they have standing and will be treated fairly, they will willingly become part of a whole.

Fairness Heuristic Theory in Collaborative Networks

FHT comes out of organizational science, and particularly explores how employees might become willing collaborators in the plans of their boss. Why, for example, should someone put in long hours at work rather than please themselves with more spare time? Why should they do a job very well rather than do what is easiest for them personally? Fair treatment (including a sense of standing in the group), Lind and others argue, is a significant part of the answer.

But the tension between the part and the whole is, I argue, at the heart of collaborative networks as well. A network is not a top-down structure giving marching orders to soldiers, it is a bottom-up gathering of diverse perspectives on a complex problem. Members must autonomously agree to collaborate to achieve a purpose, even while their own individual purpose is distinct.

Consider the example below, in which a network has formed to end malnutrition. Member A experiences the part/whole tension in that this organization (which focuses on childhood malnutrition) must be willing to lend its identity and resources to some efforts that are beyond the scope of its own concern: for example ending malnutrition among the geriatric population. Members B and C experience the part/whole tension differently, in that their mission encompasses many things beyond the scope of the network. In this sense, the network competes with other valuable work they might do.  In all three cases, there may be times that the good of the whole network actually runs counter to the member purpose: for example perhaps in some cases, increasing arable land decreases rather than increases hunger--putting Member B in a dilemma.

Now, when these members are asked to participate in the network, they are confronted first not with an outcome, but with the question of participating--that is, collaborating. They must decide whether, in essence, to write a check that consists of their identity and resources for a product (the end of malnutrition) that is a very long way from delivery. They could choose not to participate, or, as those of us in collaborative networks know very well, they might just to "participate" just enough to stay on the mailing list so they stay in the know, but without truly lending their identity and resources to the network's purpose. But they just might choose to participate at a very high level, which, as it happens, is the only way they can achieve the shared purpose.

Unlike an organizational setting, there is not a strong authority figure who can compel at least some level of participation. Members must decide to do it entirely themselves, making fairness even more essential.

The point of my research is to test perceptions of fairness among network stakeholders and to develop feasible recommendations to improve perceptions of fairness. Ideally, subsequent research would revisit both the desirability and feasibility of implementing them, and determine if resultant improved perceptions of fairness do indeed improve member willingness to collaborate deeply.

I will return to Fairness Heuristic Theory later, but my next post will share what I am learning already from network leaders about their network culture. This is important because, without understanding the landscape, it would be very challenging to make recommendations that are both relevant and feasible.