Any native C3 way to transform on fields except some if id object is already present

Hi,

I have a C3 object, that I want to preserve one of its fields value, if the object exists already, when new data arrives.

For illustration here is the transform:

type TransformCanonicalBillPlanning mixes BillPlanning transforms CanonicalBillPlanning {
     start :          ~ expression "start == null ? null : dateTime(start)"
     end :            ~ expression "end == null ? null : dateTime(end)"
     id :             ~ expression "billPlanningId"
     totalNumberEch : ~ expression "totalNumberEch"
     echAmount :      ~ expression "echAmount"
     adjusted  :     ~ expression "false"
}

of course, id assures unicity, so if an object say

{
  "id": 1000000009,
  "totalNumberEch": 10,
  "echAmount": 100,
  "start": "2019-09-21",
  "end": "2020-09-21",
  "adjusted": true
}

exists in database, and a new-coming update with different values:

{
  "id": 1000000009,
  "totalNumberEch": 10,
  "echAmount": 120,
  "start": "2019-09-21",
  "end": "2020-09-21",
}

I would like to preserve “adjusted” field as true. if not, then choose the default value, defined as: false.

First option is to write a transform that performs a get, before insertion. Which is logical, but seems greedy. Any C3 native (don’t know if I have to expect a possible way anyway ?)

Many thanks.
[edit]! reviving the thread :slight_smile: :slight_smile:

@sean.summers @AlexBakic @bachr

Doing fetch from a big pool of data in transformation is discouraged because it cannot be optimized.
Also, dustinguishing between the first time and the other times is not in line with eventual consistency - you would need an idempotent or commutative[1] operation instead but that would require a fetch again. The recommended way is without any dependencies between individual data loads/integrations.

(However, if your data pool is small enough and you cannot break dependencies, you may try caching it in memory, see type Cached[2].)

[1] https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type
[2] A small example of how to use Cached:

entity type HWCWprocessState mixes Cached schema name 'HWCWPS' {
    lastEvalTime: datetime post default '1970-01-01'
    eventsIgnored: int post default 0
}
----
        var state = HWCWprocessState.get(housing.site.id) ||
            HWCWprocessState.create({ id: housing.site.id });
        state.eventsIgnored = state.eventsIgnored + 1;
...
        state.upsert();
1 Like