Hotclicks: The Science Behind Parse.ly Deathmatch
APRIL 1, 2013: FOR IMMEDIATE RELEASE
Today we released Parse.ly Deathmatch, to demonstrate a new metric for online media: the Hotclick (TM).
As Parse.ly’s Chief Hadoop Officer (CHO, fmr. CTO), I felt compelled to explain the science behind this important, game-changing metric.
Hotclicks (TM) are determined by doing a comparison between two competing news stories for a given semantic topic. A swarm of objective experiment participants (let’s call them “readers”) evaluate the content quality using the full cognitive abilities endowed in them. Millions of synapses fire, billions of sensory data points are processed, and instantaneously, a judgement is determined: HOT, or NOT.
The number of “hots” registered against a given story or publisher are then put through a complex data processing pipeline to determine its Hotclicks. We’re talking serious data science, as the following peek inside the Parse.ly codebase will show:
As explained in our introductory blog post, this takes into account all sorts of factors, including 325 elements of content compatibility, which are summarized using a variation of the Golden Ratio. An aside: isn’t Math beautiful?
Of course, because this core calculation function is stateless, we can instantly parallelize it across hundreds of thousands of Amazon Elastic Map Reduce nodes, using Hadoop technology.
We’ll now explain how we reticulated the splines of content synergy.
What’s wrong with pageviews? They don’t get at the heart of the most important aspect of content: editorial quality. Hotclicks, on the other hand, utilize the Wisdom of the Crowds and Big Data to provide much more accurate readings of content quality quality and importance.
For example, take the recent New York Times “Snow Fall” multimedia storytelling piece.
According to editor Jill Abramson, this story received more than 3.5 million pageviews.
However, when put up in a Parse.ly Deathmath against a rival story, the situation looks drastically different.
Our semantic analysis system found a semantically comparable article from FreakingNews, a reputable source of investigative photojournalism (known in insider media circles as ”photoshopping”).
EXPERIMENT VARIATION v1.08.34
A) SNOW FALL: 1,211 Hotclicks
B) FUNNY AVALANCHE PHOTOS: 300,522 Hotclicks
hotclick differential: B|+299,311
deathmatch winner: FUNNY AVALANCHE PHOTOS
We ran this experiment hundreds of times, with different demographic audiences (including, notably, one test group of people currently caught in the middle of an avalanche). Time and time again, our Hotclick metric chose the editorially superior content. If you had gone by pageviews, the 3.5M for the NYT’s hyped-up snoozer would have “outperformed” this Funny Avalanche Photos compendium by a ratio of nearly 3.5-million-to-1.
So, there you have it. The Hotclick (TM). Science. Wisdom of Crowds. Big Data. Hadoop.
Give it a try yourself over at Parse.ly Deathmatch.
Chief Hadoop Officer (CHO)
April 1, 2013