This post originated from an RSS feed registered with Java Buzz
by News Manager.
Original Post: Why Spark 1.6 is a big deal for big data
Feed Title: JavaWorld
Feed URL: http://www.javaworld.com/index.rss
Feed Description: JavaWorld.com: Fueling Innovation
Big data got off to a roaring start in 2016 with the release of Spark 1.6 last week. You can rely on the Spark team to deliver useful features in a point release, but Spark 1.6 goes a step further, offering a mini-milestone on the way to Spark 2.0.
The new features and improvements In Spark 1.6 will make both developers and operators very happy. Let's take a look at some of the highlights.
If you've talked to people who've used Spark in production, you'll often hear them complaining about the hand-tuning required to optimize Spark's memory management. In particular, you can spend days looking at garbage collection traces to tune the static split between execution memory (for shuffles, sorting, and shuffling) and caching for hot data memory locality.
Previous Topic
