We have a heavy machine learning algorithm to run (using a lot of memory) and we want to make sure to run a single map action per node.
setMaxConcurrencyPerNode() seems to be what we are looking for but we are still getting several (up to 8 action per node) in the same time.
Here is the script we followed:
MapReduceQueue.pause(); var j = /// .. get the MapReduceJob here j.start(); j.setMaxConcurrencyPerNode(1); MapReduceQueue.resume();
We are on