While running a map job that goes through devices and updates their status, we’re seeing a handful of version conflict errors when we merge the update. In the map job, we only include the id and the status, but it seems it implicitly grabs the version, too. We are actively streaming data during the map job, and our assumption is that after the job starts, the devices that it grabbed are getting updated, creating the version mismatch. What is the optimal way to prevent these version errors? Can we use a merge spec to only merge the status? Do we need to re-fetch the device before attempting to merge? Performance is already tight and it would be nice to avoid making extra gets/fetches if we don’t have to.
Using Analytics to react to streaming data instead of a MapReduce job is a good idea. This will allow the platform to merge triggers into a single analytic when relevant. Other than that, yes you should re-fetch the data before updating when you get a VersionConflict.