MapReduce job example


#1

Following is a MapReduce job example that you run only after code provisioning .

Define the structure of the MR job

entity type MyMRJob extends MapReduce<MyTargetType, string, string, string> type key 'MyMR' {

  // define any useful fields you want them to be for you `map` and `reduce` functions

 startJob: function(myparam1: string): MyMRJob js server

 map: ~ js server
 reduce: ~ js server
}

Define the logic for map-reduce

function map(batch, objs, job, subBatch) {
  // map logic goes here
}

function reduce(outKey, interValues, job) {
  // reduce logic goes here
}

Define action for starting the job

function startJob(myparam1) {
  var job = MyMRJob.make({
    targetType: MyTargetType,
    filter: ...,  // make sure to get only objects that you want to process
    include: ..., // list of fields to include in the objects passed to the 'map'
    ...
  });
  return MyMRJob.start(job);
}

You can also check the documentation for the base type MapReduce


Is there a way to run only map part of the mapReduce job