Limitations on the InterValType for map reduce jobs


Is this a valid type for the InterValType in a map reduce job:

type ExampleIntermediate {

Does it need to be entity or persistable or anything, or is this fine?
I have used double without problem before, and the example has an int, are general c3 types allowed like this?


This should be fine, have you tried it? Here’s an example that works for me:

entity type ProtectionIslandEventAssignerJob extends MapReduce<ProtectionIsland, string, PIEAssignmentWeight, int>
type PIEAssignmentWeight mixes Double {  
   * The new protection island that the event might be localized to.
  protectionIsland: !ProtectionIsland
   * The probability of that the event should be localized to the island.
  value: !~

I am in the process of debugging a job that produces no output where I expect it, whereas a similar job that uses double as the intermediate type works, so I just wanted to check that there wasn’t an issue with the intermediate type being a non primitive or something else.

Does a MapReduce job need a type key at the end? The lightbulb example has AGG_CNS_MFG for example. Your example doesn’t have one.

See below.
entity type MyAggregateRollupJob mixes MapReduce<MyType1, string, AggregateRollupIntermediate, AggregateRollup> type key 'AGG_ROLLUP_JOB' {

Is there any way to see a count of how many map records got output? Or other metrics for a job so I can see what is going on?


function map(batch, objs, job) {

    var intermediaryGroup = {};
    foreach (thing in things) { 
      if(! (myDynamicKeyHere in intermediaryGroup)){
           intermediaryGroup[myDynamicKeyHere] = AggregateRollupIntermediate.make();
          intermediaryGroup[myDynamicKeyHere].Key1 += 1 ;
    return intermediaryGroup 

Do I need to .create() the type or is the definition of the dictionary wrong?

Edit: I got one record to output now, so I just need to debug more.

1 Like
closed #4