In Praise of Small Data
Published: 05/2014
Abstract:
Abstract:
Big Data tools can give “explanations” of complex and elaborate data sets. There is the danger that we might be content with the explanations that these tools produce. It is important to bear in mind that truly understanding something often requires simplifying the initial explanation. Indeed, the well known “Occam’s Razor” states roughly that the simplest explanation is the best. This paper examines two cases, Copernicus’s Solar System Model and a “Big Data” search for Paul Revere, where the initial models were too complex. It shows that simpler models are possible, and more fruitful for further research. Some general techniques for finding simpler explanations are discussed.
Citation:
George Markowsky, In Praise of Small Data, International Workshop on Big Data Analytics for Predictive Organization and Big Transformations, Stanford, CA, USA, May 27, 2014 (in press)