Custom register measurements and associated consumption metrics

#1

In a hierarchy of Facility, lowest-level ones are where meters are attached that have (incrementing) registers with possibly different wraparound index values (indexMax). For unrelated reasons, we cannot use RegisterMeasurement and we use registerReadInterpolator instead to compute the difference between indices at t+1 and t (consumption), like this:

registerReadInterpolator(sum(sum(normalized.data.quantity)), 'GT', 0, indexMax)

However, this EEF does not use modulo operation and can overflow, so our current solution is to bracket it using:

E >= indexMax ? E - indexMax : (E < 0 ? E + indexMax : E)

with E being the previous expression (even though just E >= indexMax ? E - indexMax : E might suffice).

The problem is when we want to aggregate (sum) at a higher-level Facility, which in a test contains 3 lowest-level facilities, the test fails when an overflow occurs. We use sum(sum(... in the argument to registerReadInterpolator because there are other problems when there is only one sum, and it seems that registerReadInterpolator gets the sum of the 3 meters’ measurements as input. But indexMax is not the sum of these meters’ indexMaxs: it is only one of them (in one case investigated, it’s the first one).

Next I wrote a custom EEF:

type ProjectMFL mixes MetricFunctionLibrary {

  diffR: function(incr: !Timeseries, rollover: Timeseries): !Timeseries js server
  //diffR: function(incr: ![Timeseries], rollover: [Timeseries]): ![Timeseries] js server
}

and

c3Import('metadata', 'timeseries')
var log = C3.logger('Customer/ProjectMFL')

function diffR_1(ts, rollover) {
  var data = ts.data()
  var results = []
  var next = data.slice(1).concat(data.slice(-1)[0])
  try {
    if (rollover) {
      results = _.map(
        _.zip.apply(_, [data, next, rollover]),
        function(t) { return t[2] ? (t[1] - t[0]) % t[2] : t[1] - t[0] })
    } else {
      results = _.map(
        _.zip.apply(_, [data, next]),
        function(t) { return t[1] - t[0] })
    }
  } catch (e) {
    log.error(e)
  }
  return Timeseries.makeNorm(NormTimeseriesDoubleSpec.make({
    start: ts.start(),
    end: ts.end(),
    interval: ts.interval(),
    data: Double.array(results),
    unit: ts.unit()}))
}

function diffR(ts, rollover) {
  if (ts.constructor === Array) {
    if (rollover) {
      return _.map(_.zip.apply(_, [ts, rollover]),
                   function(p) { return diffR(p[0], p[1]) })
    } else {
      return _.map(ts, function(ts) { return diffR(ts) })
    }
  } else {
    return diffR_1(ts, rollover)
  }
}

hoping to deal with both lowest- and higher-level Facility (aggregation).

Now, for the simple metric/case, the following expression works:

diffR(sum(normalized.data.quantity), identity(indexMax))

and for a compound metric IndividualElectricityConsumption, the following also works:

diffR(IndividualElectricityMeasurements, IndexMax)

where for IndividualElectricityMeasurements the expression is sum(sum(normalized.data.quantity)) and for IndexMax it is just indexMax, but for the aggregate case, if we try a simple metric with:

sum(diffR(sum(normalized.data.quantity), identity(indexMax)))

we get a run-time error saying that diffR got 3 arguments corresponding to the first parameter incr. The commented-out API does not work/help either. So, again, the following does not throw engine errors:

sum(diffR(sum(sum(normalized.data.quantity)), identity(indexMax)))

but the test fails for the same reason (summing before computing the difference with a wraparound).

Similarly, a compound metric version with the expression:

sum(diffR(IndividualElectricityMeasurements, IndexMax))

gives an engine error java.lang.NoSuchMethodError: c3.love.expr.bytecode.TsFuncLib.sum(Lc3/love/timeseries/LazyTimeseries;). The same error with the following expression:

sum(IndividualElectriityConsumption)

(It is not impossible that something god messed up in all these attempts…)

How should we do what we need?

Could someone please explain the polymorphic meaning of sum in the context of non-/normalized, simple/compound metrics, sum(...) vs. sum(sum...)? We are using v7.6.1.

Is it possible to overload diffR with multiple c3typ signatures (and the same name of a single Javascript function)?

Finally, it might help if there were built-in EEFs tail (dropping the first value and, if necessary, pulling the rest forward in time) and mod (modulo) to do this more efficiently.

Thanks,
Alex

0 Likes

#2

@AlexBakic Lots of information in there, but basically the gist is (correct me if i’m over simplifying), you want to apply the registerReadInterpolator before you apply the outer sum correct?

If that is the case then I propose using:

{
  "type": "SimpleMetric",
  "id": "YourMetric_YourSourceType",
  "name": "YourMetric",
  "srcType": "your_source_type",
  "expression": "sum(measurements.map(registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, indexMax))"
}

Can you try the above?

1 Like

#3

Thanks, @rohit.sureka, the sum part works, but the reisterReadInterpolator still causes test failures due to the fact that sometimes it returns a value outside of [0, indexMax). But the following fix almost works:

path: site.denormChildFacilities.individualElectricityServicePoint.meterMappings.to
expression: sum(pointMeasurements.map(registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, indexMax) > indexMax ? registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, indexMax) - indexMax : (registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, indexMax) < 0 ? registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, indexMax) + indexMax : registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, indexMax))))

Unfortunately, after running the same test after a dozen of times, I get:

Expected [ 1944, 2226, 2501, 2031, 1171, 1675, 2566, 2002, 2364, 1763, 1304, 1553, 2019, **1338**, 2268, 1704, 1451, 2175, 1378, 0 ] to equal [ 1944, 2226, 2501, 2031, 1171, 1675, 2566, 2002, 2364, 1763, 1304, 1553, 2019, **1633**, 2268, 1704, 1451, 2175, 1378, 0 ]

I suspect it is using a wrong IndexMax in comparisons, I will try plucking over pointMeasurements, too.

Can/should we remove the second sum in the individual simple metric?

expression: registerReadInterpolator(sum(sum(normalized.data.quantity) * multiplier), 'GT', 0, indexMax)

=>

expression: registerReadInterpolator(sum(normalized.data.quantity) * multiplier, 'GT', 0, indexMax)

If not, why?

0 Likes

#4

This one works (I had tried parent.indexMax but @scott.kruyswyk got it right with pointMeasurements.indexMax):

sum(pointMeasurements.map(registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, pointMeasurements.indexMax) > pointMeasurements.indexMax ? registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, pointMeasurements.indexMax) - pointMeasurements.indexMax : (registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, pointMeasurements.indexMax) < 0 ? registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, pointMeasurements.indexMax) + pointMeasurements.indexMax : registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, pointMeasurements.indexMax))))

So it’s the indexMax that failed silently. This explains why even the simpler form works (which also does in the simpler, individual consumption case):

sum(pointMeasurements.map(registerReadInterpolator(sum(normalized.data.quantity), 'GT', 0, pointMeasurements.indexMax)))

because the documentation should be improved. For example, below is a more intuitive version of the registerReadInterpolator documentation:

Parameters:
ts: Timeseries required
input timeseries on which rolling diff needs to be applied

resetOper: string required
reset operation to be applied on the value. Should be one of PERCENT, GTE, GT.

resetThreshold: double required

rolloverMax: double required
maximum value allowed for the timeseries after which it rolls over. The following formula is used to compute the differential reading:

diff = nextValue - currentValue
if diff resetOp resetThreshold:
  nextValue = diff
else:
  nextValue = diff + rolloverMax

Returns Timeseries required
a timeseries which has values adjusted for monotonically increasing register timeseries

1 Like