Shifting paradigms
Anand Rajaraman describes a group of students eschewing complicated movie data-mining algorithms in favour of a simple, effective alternative tactic: comparing their data with additional information from the IMDB website.
In the same post, he compares this with Google’s great leap forward in the late nineties: the inclusion of hyperlink counts and anchor text recognition in their search engine results, which helped put them firmly one step ahead of the competition.
He sums it up thus:
… if you have limited resources, add more data rather than fine-tuning the weights on your fancy machine-learning algorithm.
I would go a step further. Both of these are perfect examples of how the biggest leaps forward usually come from a paradigm shift, a fundamental change in thinking - a change which is often profoundly simpler than the alternatives, but paradoxically, much harder to see.
Tags: zen