How 'processLog' is populated in DataLoadUploadLog?


#1

How is the processLog field populated in a DataLoadUploadLog?

c3Grid(DataLoadUploadLog.fetch({
  filter: 'targetTypeName == "CanonicalMeasurementSeries"',
  order: 'descending(meta.updated)',
  limit: 5,
  include:"processLog"
}))

I have a case where 1 entry in DataLoadUploadLog is associated with 2 DataLoadProcessLog:

  • one of them is very old (meta.updated is from January) with a status REJECTED.
  • the other one correspond to the actual date of the upload and has status SUCCESS.

Both of these entries have the same source but surprisingly the one with REJECTED status is associated with a different targetTypeName then the correct one (in the DataLoadUploadLog and the other DataLoadProcessLog).

What can explain this?


#2

We’ve seen that when canonical was first uploaded to the wrong target type (human error or wrong routing on our side).

Subsequent DataLoadUploadLogs would never succeed while these old − and wrong − DataLoadProcessLogs existed. We had to manually delete them to fix it.


#3

This is the same issue we have apparently, and to fix it we should remove the rejected entry as you suggested.

This is because processLogsStatus looks first for rejected entries then failed then success, here is how it’s populated:

processLogsStatus: string calculated
The following contains all process logs associated with the given upload log

Stored calculated: or(processLog.result == 'REJECTED') ? 'REJECTED' : or(processLog.result == 'FAILED') ? 'FAILED' : 'SUCCESS'