Caching metrics for a whole interval vs two/more sub-intervals?

#1

Does caching a metric for a whole interval [a, b] is different from caching it for two intervals [a, c] and [c, b]?
I cache a metric for the last year, than for the year before with two different calls (1st with [now-1year, now] then [now-2year, now-1year]).
Then I noticed that when evaluate my metric over the last one year and half (i.e. [now-1.5year, now]) the metric is re-evaluated (i.e. the cached result is not used)!

Details on my setup are as follows, I’ve a bunch of SimpleMetrics that I configured the caching as follows:

{
  "id": ..
  "expression": . . .
  "cache":  {
    "intervals": ["MONTH"],
    "monthsInPast": 24
  }
}

Then I configured two CronJobs to call the CacheMetricsJob with my list of metrics and a given ids as follows:

  1. first job caches one year starting from today() - 1.year and ending today()
  2. second job caches one year starting from today() - 2.year and ending today() - 1.year

Here is snipet from job1

{
  "id": "job1",
  "action" : {
    "actionName" : "startJob",
    "moduleName" : "metricUtils",
    "typeName" : "CacheMetricsJob"
  },
  "inputs" : {
    "ids": . . .,
    "expressions": . . .,
    "startObj": {
      "type": {
        "name": "Json"
      },
      "value": {
        "referenceConfig": ["start", "hour"],
        "direction": "before",
        "number": 1,
        "unit": "YEAR"
      }
    },
    "endObj": {
      "type": {
        "name": "Json"
      },
      "value": {
        "referenceConfig": ["start", "hour"]
      }
    }
  }
}

Here is snipet from job1

{
  "id": "job2",
  "action" : {
    "actionName" : "startJob",
    "moduleName" : "metricUtils",
    "typeName" : "CacheMetricsJob"
  },
  "inputs" : {
    "ids": . . .,
    "expressions": . . .,
    "startObj": {
      "type": {
        "name": "Json"
      },
      "value": {
        "referenceConfig": ["start", "hour"],
        "direction": "before",
        "number": 2,
        "unit": "YEAR"
      }
    },
    "endObj": {
      "type": {
        "name": "Json"
      },
      "value": {
        "referenceConfig": ["start", "hour"],
        "direction": "before",
        "number": 1,
        "unit": "YEAR"
      }
    }
  }
}
0 Likes

#2

@bachr The metrics are always cached from (now + 1 MONTH) to (now - monthsInPast) irrespective of the time range the evalMetrics is called with. So even if you have two jobs calling the metric with 2 different time ranges, the cache should have been established from now - monthsInThePast.

Also, how did you deduce that the cache results are not used?

0 Likes

#3

thanks @rohit.sureka the thing is that we noticed that the call takes a very long time to return when we increase the time range (end-start) of an evalMetric to a year and half (e.g. end-start == 1.5 year).
While if we use range of less than a year the response is returned way faster!

Also, not all my metrics (the ones I provide in expressions) are actually cached. Here is the code snippet I use to check the caching:

var oid = . . . // object id
var metricIds = [ . . .]
console.log('Cahed metrics for Organization with ID', oid);
metricIds.forEach(function (id){
  var sm = SimpleMetric.get(id);
  var timeRange = sm.cacheRange(oid, 'MONTH', 'Organization');
  console.log(id, timeRange);
});

The result shows that some simple metrics (provided to CacheMetricsJob) are not cached (lines with null):

IsGrossProductionMeter_Organization C3.typesys.Obj {start: DateTime, end: DateTime}
EnergyProductionTotalBackFilledMeter_Organization null
EnergyProductionTotalRawMeter_Organization C3.typesys.Obj {start: DateTime, end: DateTime}
RealPowerBackFilledMeter_Organization null
RealPowerRawMeter_Organization C3.typesys.Obj {start: DateTime, end: DateTime}
IsInverter_Organization C3.typesys.Obj {start: DateTime, end: DateTime}
EnergyProductionTotalBackFilledInverter_Organization null
EnergyProductionTotalRawInverter_Organization C3.typesys.Obj {start: DateTime, end: DateTime}
EnergyProductionTodayBackFilledInverter_Organization null
EnergyProductionTodayRawInverter_Organization C3.typesys.Obj {start: DateTime, end: DateTime}

How can I get the reason why the job failed to cache some of these metrics?

0 Likes

#4

If I understand your explanation, I can run evalMetric with a time range of one or two months yet the metric will be cached for the whole period of time now - monthsInThePast?
Will be faster if in my job I do that (i.e. use small period of time)?

0 Likes

#5

It should be slightly faster but not that much since the engine will be expanding the range of query for caching in any case. The time saving will only be in getting the results back to the user. But you should provide as less of a range as possible to not put any memory pressure on the system

1 Like