If we enable cache on SimpleMetric, is MetricEngine able to automatically invalidate cache after reloading data?
Yes, only for expression based metrics ending on timeseries headers & tsDecl metrics. But not for action decl metrics
Is it possible to prevent the cache invalidation when new data arrives? My use case is for reporting (and not for analytics/DFEs); basically, I want to support the cacheing of metrics that use large amounts data for a fleet of assets. I would like to be able to invalidate the cache manually and/or with a Cron which I can implement if I know the API to force invalidation. This invalidation would occur less often than new data arriving that affects those metrics, which is why I dont want the default invalidation behavior suggested by Rohit/Chen.
Unfortunately not. If a SimpleMetric is defined with a value for the field “cache”, it will be invalidated when data it depends on is changed. However, I don’t think that invalidating the cache will cause it to be recomputed immediately. I believe that will happen the first time it is used. You may want to check with Rohit on that.
So there is no way to query the cache when new data has arrived that invalidated that cache? Thats a bummer.
We will have to implement an explicit cache (ie new type) in order to achieve the performance we want for these metric workloads.
Is the recomputation incremental? I mean, when data arrive, say once per day, would a cached metric be kept up to date in an efficient manner?
Currently the timeseries will be fully recomputed
Hm. And in 7.9? This could be a good start: http://firsov.ee/incremental/incremental.pdf
Was there a plan to allow incremental caching? I remember some discussions with Sunpower on this.