I have a doubt regarding the map reduce functionality.
I saw that if I set the batchSize > 10000 , even if I have a number of elements “fetched” by my map greather than 10000, the system splits the elaboration of the map in different part. For instance:
- I did a fetch in table PointMeasurement with I filter that returns 23000 records
- I saw in splunk that the system split the computation in 3 different piece, usually 2 of 10000 element an 1 of 3000 element
Is the correct behaviour? Below a piece of my definition
include: “quantity, start,parent”,
Now , I I want to do some operation on this records (for example create a copy of each) and THEN I would delete the records I think that I cannot do this operation inside the map using the same filter that I use in JSMapReduce definition, because I will erase all the records, MAYBE before that all the records are processed: right?
In this case for the delete step is correct to define the reduce phase? If yes, the reduce phase is called only one time or 3 times, one for each “sub-map”?