In 2012, Dean Wampler claimed that Hadoop, and Big Data in general, needed Functional Programming and an alternative to Java. Today he says Scala has successfully taken over the Big Data world. This is a talk about why.
Dean started doing Hadoop consulting in 2011 and, as he puts it, “immediately starting trolling the community because […] the api’s were terrible.” A year later, he gave a talk called “Why Big Data Needs to Be Functional,” at the 2012 Northeast Scala Symposium, where he argued this point and talked about the need for languages like Scala. He made this claim:
Hadoop is the Enterprise Java Beans of our time.
Fast forward 3 years and we find that, to quote from this talk's abstract, “Scalding proved that a Scala DSL is far superior to the MapReduce Java API, and Spark is now replacing MapReduce altogether.” Dean demonstrates why this happened and argues that "Big Data is the Killer App for FP and Scala."
Further Resources
- Scala Training from ProTech
- Video & Tutorials on Scala