Crowdsourcing is Hard Part III: Next Steps, or So Now What?
Everything in your crowdsourcing project has worked beautifully. People had the right reasons to contribute, there was nothing holding them back, and now the citizen reports or contributions are pouring in.
Um, so now what?
Even initially successful crowdsourcing projects can fall down when the information comes in. There's two main areas to think about: what do you want to do with the reports, and what do you want to do with the reporters? Finally you're dealing with actual humans - what are the security implications of your project?
The Reports
Most people's response to your sincere citizen report is "who says?"
Did it happen as the person said? How do we know? "Veracity" is the relative truthiness of the information you have coming in. The good folks at Ushahidi have spent a lot of time thinking about this particular challenge, and their one-pager on the topic is excellent.
The heart of it is common sense. If there's more than one person reporting the same thing (and they actually are different people) that makes it more credible. If the source someone you already know or from an organization you trust, then it's more likely to be true. If there's video or images, then it's harder to just make up. Really, veracity is a sliding scale, and absolute truth is hard to come by. The challenge is that degree of nuance isn't really something you can convey easily to your readers, nor in a fast-moving situation does one have the time to cogitate on a particular SMS and see how the scale balances.
In a typical situation the overall situational fuzziness and your team's haste will boil the question down to a binary choice: verified or unverified?
If you think you have some degree of trust in the sender, or there is collaborating evidence of some kind, then the arrow will pivot to verified. If not, or if things feel generally sketchy, then it has to remain in the unverified camp.
So what are you going to do with your lonely, rejected, unverified reports? Put them up (in a separate category, probably) or not? To publish or not to publish - that is the question for the unverified reports. Well, you have to look at the overall game. Who's playing? What are their motivations?
In most situations even the bad actors don't care enough to try and flood your system with illegitimate reports. Remember, that takes effort, and it's a rare bird that will take a lot of time and energy to rain on someone else's parade. If you make the distinction obvious to your Dear Readers then unverified reports will not have the same weight as the trusted, verified ones. However, if you auto-publish all the junk that comes through the pipe, you're risking the sheer weight of numbers overturning the trustworthy facts on the ground. If people are given a map of red and yellow pins, and see a ton of unverified yellow ones, they will naturally assume where there's smoke there's fire. You may need to re-balance in favor of some degree of editorial control in such a situation. Staying flexible is key to crowdsourcing success.
The Reporters
Don't forget that "the crowd" is simply an agglomeration of individuals, most of whom don't care about you in the slightest. (Of course your mom and I know you're very nice, but we can’t expect submitters to know that.) Therefore, anyone who took the time and money to send you something is precious. They must be loved, tended, and cared for like delicate orchids. The first step - one too often neglected - is thanking them and letting them know that their submission was accepted. After all, without some sort of immediate response then how do they know that your system even worked? If reports disappear into a black hole, the submitters can end up more discouraged than if they hadn't participated at all.
Once a report is the system, the initial submitter will feel a sense of connection and ownership to it. Remember, the most likely incentive for people to have participated was either moral outrage or strong commitment to the process. If they're reporting a problem, they want something *done*, darn it. If their report triggers action, try and get further information on the situation back to the originator. If a submitter feels like they've had an impact, then they are much more likely to play again. These folks are your best advocates and advertisements. Encourage them to tell their friends to share their experiences as well, possibly even in the immediate feedback message.
Security
In some places, you owe it to your reporters to keep them safe. After all, any time there are human rights abuses, corruption, or election violations to report there are clearly people who would prefer that such info never come to light.
In environments where it is even a question security should be part of the conversation from the beginning. The first Q is: "Hey guys, should we, um, do this at all? Would people who contribute be at risk?”
There are situations where the deck may be completely stacked against an observer due to the Bad Guys in being in control of the telecommunications systems or with formidable hackers at their disposal. Remember, no online platform is completely secure, and some popular mapping systems have recently been found to have holes.
Why does that matter, one might ask? Well, you may not think of quick crowdsourced reports as being dangerous ways of exposing people to legal jeopardy or physical beatings, but in the right context they definitely can. Thought experiment: you are providing a service via various pipes to document problems that people have with the Powers that Be. Guess who owns the pipes?
How do your reports come in? SMS? Well, your shortcode or phone number is known, every source phone number is easily tracked (and in some places officially registered), and texts are always sent unencrypted*. Using email? Your destination email address for the service is certainly known (you did publicly advertise, right?). Given that the bad guys probably control the ISPs and therefore are able to map both source and destination addresses, your senders are exposed. (As a bonus, they can probably see message contents as well, which are almost certainly cleartext.) Breathing a sigh of relief since you're only accepting encrypted web form reports? Even if they're connecting over impenetrable SSL links, your subversive destination site address is going to be known to your adversaries, and they can assume anyone going there is not doing something as benign as looking at cute cat pictures.
The challenge we face is that a crowdsourcing project cannot be, by definition, a secret. Therefore you must always assume the bad guys know about your project. Probably before the good people of your local communities do. Sorry. Life’s tough.
As the information comes in and your responses go out, you're telling the world a story. We'll wrap up this series by thinking about what the moral of your story will be.
*if you're talking with your friends via Android phones, get TextSecure.