Do Startups Need Peer Review?
Do startups need a peer review system?
Two sentence backstory: at Parse.ly, interns attend lunches hosted by one of our largest funders, ff Venture Capital, during which founders of portfolio companies talk about their products. Last Thursday, Michael Nutt, co-founder and CTO of Movable Ink, discussed how he and co-founder Vivek Sharma launched this latest company after abandoning a different, failed idea “in-the-works.”
Almost all startups begin in a “research-and-development” phase that is not unlike scientific research. Coders and business strategists identify a “problem” or market niche or opportunity for disruption, work together to create a compelling product that meets the identified demand, and then test it. Despite the intimidating volume of startups entering the market every week, few attract significant funding and even fewer achieve commercial success or profitability. Even more products never get off the ground, because like Nutt and Sharma’s first project, their feasibility or odds of success are too low. In theory, the way startup culture routes creative energy into innovation should be effective, because high volatility, creative environments, when combined with market economics, should filter out low-functionality products. Those ideas that are not likely to succeed will not survive. But who or what decides which ideas or early-stage products are likely to succeed? All too often, a young company leaves a product behind that hasn’t been completely examined for potential. Even those products that seem unprofitable or unmonetizable have a clear path to the money: the possibility of massive consumer audiences. To put it bluntly, innovation is not a very efficient system. A great deal of effort is wasted on trial-and-error, which admittedly does lead to dazzling advances, sometimes. On a more regular basis, however, trial-and-error and an eye for future profitability kills a lot of interesting ideas. Conversely, the same system introduces high profit potential but low-innovation products into the marketplace. Oftentimes, those products are neither vetted nor re-innovated in order to maximize their design. Instead, it seems like the current startup scene is geared towards instant funding and near-term buyout, not long-term development. A comparison to the scientific method is instructive: if scientists got paid if and only if their research was profitable or had profit potential—that is, if grants, etc. did not exist—then low profit potential research or research with inestimable or irregularly modeled profit potential would be a non-starter. Consider what would happen if scientists started a lot of research and published it without extensive feedback and re-research or just abandoned it altogether? Fortunately, science has an additional layer of protection that startups lack: a formal peer review system. While many startups do engage in collaborative hackathons and other forms of peer feedback, the proximity to finance capital dilutes and distorts the review process. Money is a catch-22: you need it to innovate but its promise and presence block free innovation.
What would peer review for startups look like? Here’s a rough sketch of scientific peer review, thanks to Scientific American:
1. Submission of article to peer reviewed journal.
2. Group of experts review the article for accuracy, significance, and originality.
3. Experts either recommend publishing the article or revisions that the authors can make.
The peer review process is, in a perfect world, anonymous. Politics sometimes inflect the integrity of the process, and peer review can stall important research needlessly. Nevertheless, peer review has a positive aggregate effect on the quality of scientific research. Here’s an idea for startup peer review.
1. Submission of product to peer review board.
2. Group of experts review the product for functionality, significance, originality, design, etc.
3. Experts either recommend releasing the product to the public or revisions that the developers can make.
Like scientific peer review, this process would need to happen in complete anonymity and without the influence or proximity of financial instruments. What do you think? Is blinded peer review a good idea for startups?
