MapReduce with complex intermediate values failling

#1

We found out that defining a map reduce job with an intermediate values as [string] is not working as expected:

type FooMapReduce extends MapReduce<Toto, string, [string], int> type key 'FOOMR' {
  map:       ~ js server
  reduce:    ~ js server
}

function map(batch, objs, job) {
    var result={};

    _.each(objs, function(o) {
        var id=parseInt(o.id)%3;
        result[id] = result[id] || [];
        result[id].push(o.id);
    });

    return result;
}

function reduce(outKey, interValues, job) {
   
    var count = 0

    interValues.each(function(c) {
        count += c.length;
    })

    return [ count ];

}

This mapreduce job is failing on the reduce step, because this step is receiving a interValues=[null] (array with a null element).
The error is Cannot read property "length" from null (FooMapReduce_reduce.js#32
The mapreduce job is working as expected if we use a string intermediate values.
Are there any constraints about the types that are used for interValues?

1 Like

#2

Try instead with StringArry.
Your type definition will be:

type FooMapReduce extends MapReduce<Toto, string, StringArry, int> type key 'FOOMR' {

Your map() will be something like:

result[id] = result[id] || StringArry.make({value: []});
result[id].value.push(o.id)
0 Likes

#3

Great, this solution is working for that specific case ([ string ]). Does it mean that we need Boxed types?

0 Likes