Reprocess canonical data after adding a new field


#1

I have a type Awesome, its canonical CanonicalAwesome and its transform TransformCanonicalAwesome. I have canonical data that comes in and trigger the creation of Awesome instances.
Now, I added a new field in my Awesome type, modified accordingly CanonicalAwesome and didn’t touch the transform (to use implicit transformation).

How I can force that all old instances of Awesome that were created before I update my type will have the new field value populated (the field was already present in the raw data). Is it possible or I have to do it manually?


#2

As long as the previously loaded raw (canonical) data did not change, you can call the reprocess member function on either DataLoadProcessLog or DataLoadUploadLog.

For example:

// For a specific uploaded file
DataLoadUploadLog.get('some_id').reprocess();

// For all uploaded files of a particular canonical type
DataLoadUploadLog.fetch({
    filter: Filter.eq('targetTypeName', 'CanonicalAwesome')
}).at('objs').each(function (upload) {
    upload.reprocess(true); // async true
});

Note that it is helpful for canonical transformations to create objects with predictable ids so that the canonical files can be reprocessed without creating “duplicate” records.