Two weeks ago I posted a $400 reward on crowdSpring for a book cover design for Shipping Greatness. I spent $99 to post the award, but haven't paid out the $400 yet. I've extended the "competition" as a precursor to cancelling it, since I haven't been impressed yet. You can see the creative brief that I submitted here (pdf). I learned some things about the current state of crowdsourcing in this process.
1. Volume is high, quality is low.
I received 63 entries, which is actually more like 35-40 because some designers submitted a cover and full book treatment. I rated none of them five stars, and only a couple got four stars (a "good" rating). I wasn't mean enough to give anyone no stars.
The volume of submissions is very high, the quality of submissions is very low. The level of polish was low, most of the content came from other locations, and I didn't see much creativity.
It was interesting to me how many of the designers were not native English speakers; that bodes well for crowdsourcing in one regard, because it means the world is flat and the best talent is going to be noticed.
On the flipside, communicating and iterating around design is a major skill and if the consumers are going to be English speakers (many will be) then the designers need to brush up on their language skills. Or at least proofread better with a copy of MS Word.
2. The investment designers will make is not large.
The majority of the covers were stock photos with text. One of the challenges with stock photos is that in addition to the $500 you're spending on crowdSpring, you're spending additional capital on the photo itself. It's unfortunately hard on iStockPhoto to figure out how much rights will cost, and the designers don't say. So you can't make a well informed decision.
3. Design is iterative and long-distance crowdsourcing doesn't work well.
One example of this is the first series of designs I received. Designers didn't read the supplementary materials and as a result I got a lot of pictures of boxes, hand trucks, and packaging. In their defense, I could have said "NO BOXES," which I did on day two, and that cleared things up.
4. The stupid customer doesn't know what they want!
I'm the stupid customer here and I learned two things on the day of the deadline. First, that some folks thought the cover should be light and funny-ish. Second, as one reviewer said, "one person tipping up a light bulb is not a team."
The positive part of this is that maybe I have some guidance to give to a local designer - "in addition to the creative brief, emphasize humor/snarky-ness and team-ness."
5. crowdSpring's site is good, but not Great.
The overall site latency is so poor that it becomes very hard to use. One kind focus group participant had such a hard time that he sent a screenshot of his votes because he couldn't actually submit them! (see below).
6. All in, I wasn't wowed.
If I received one design that was the clear winner or was clearly great, I'd be stoked and would forgive the latency and strangeness of the process. But I wasn't. Here are the top three covers (annoying cropped by crowdSpring so you can't see them properly!). Feel free to tell me what you think on http://facebook.com/shipgreatness
PS: Screens like this do not inspire confidence, nor does being down for ~8hrs today.