Hakuna MapData! » 2013 » October
rss

Be map slot or not to be: that is the question!

| Posted in Monitoring |

0

A couple months ago, we got an email from Chris:

Hi!

The Hadoop cluster has been a bit slow the past few days and I noticed that the bottleneck seems to be coming from the map tasks. We have separate map and reduce task capacities and it continuously looks like the mapper slots are all taken while there’s a surplus of open reduce slots. Is there any reason that we can’t open any of the free reduce slots to map tasks?

Regards,
Chris

Hadoop Playlist at Spotify

| Posted in Tips, Troubleshooting |

0

A typical day of a data engineer at Spotify revolves around Hadoop and music. However after some time of simultaneous developing MapReduce jobs, maintaining a large cluster and listening to perfect music for every moment, something surprising might happen…!

What?

Well, after some time, a data engineer starts discovering Hadoop (and its related concepts) in the lyrics of many popular songs. How can Coldplay, Black Eyed Peas, Michael Jackson or Justin Timberlake sing about Hadoop?

Maybe it is some kind of illness? Definitely! A doctor could call it “inlusio elephans” ;)