Metric cache and invalidate cache

#1

If we enable cache on SimpleMetric, is MetricEngine able to automatically invalidate cache after reloading data?

1 Like

#2

Yes, only for expression based metrics ending on timeseries headers & tsDecl metrics. But not for action decl metrics

1 Like

#3

Is it possible to prevent the cache invalidation when new data arrives? My use case is for reporting (and not for analytics/DFEs); basically, I want to support the cacheing of metrics that use large amounts data for a fleet of assets. I would like to be able to invalidate the cache manually and/or with a Cron which I can implement if I know the API to force invalidation. This invalidation would occur less often than new data arriving that affects those metrics, which is why I dont want the default invalidation behavior suggested by Rohit/Chen.

0 Likes

#4

Unfortunately not. If a SimpleMetric is defined with a value for the field “cache”, it will be invalidated when data it depends on is changed. However, I don’t think that invalidating the cache will cause it to be recomputed immediately. I believe that will happen the first time it is used. You may want to check with Rohit on that.

0 Likes

#5

So there is no way to query the cache when new data has arrived that invalidated that cache? Thats a bummer.

We will have to implement an explicit cache (ie new type) in order to achieve the performance we want for these metric workloads.

0 Likes

#6

Is the recomputation incremental? I mean, when data arrive, say once per day, would a cached metric be kept up to date in an efficient manner?

0 Likes

#7

Currently the timeseries will be fully recomputed

0 Likes

#8

Hm. And in 7.9? This could be a good start: http://firsov.ee/incremental/incremental.pdf

0 Likes

#9

Was there a plan to allow incremental caching? I remember some discussions with Sunpower on this.

0 Likes