Metric Performance


#1

I have two versions of the same metric. I am interested in the performance trade-offs that should be evaluated when comparing the two. Namely, what is the performance difference between referencing the normalized timeseries vs. the raw (un-normalized timeseries). I would prefer to not be beholden to normalization having been completed to determine the earliest measurement in a series, but I get OOM errors when referencing the raw timeseries in the metric.

Two metrics in question:

{
    "id": "MetricName_TypeName",
    "name": "MetricName",
    "srcType": "TypeName",
    "path": "path.to.measurementSeries",
    "expression": "earliest(normalized.data.value)"
  }

and

{
    "id": "MetricName_TypeName",
    "name": "MetricName",
    "srcType": "TypeName",
    "path": "path.to.measurementSeries",
    "expression": "earliest(data.start)"
  }

#2

The series header has a field earliest that is populated on normalization. Using that field directly is the most efficient.

{
    "id": "MetricName_TypeName",
    "name": "MetricName",
    "srcType": "TypeName",
    "path": "path.to.measurementSeries",
    "expression": "identity(earliest)"
  }

#3

@jinyanliu Your recommendation is not reliable and should NOT be used (e.g. if the series is not normalized then you won’t get any value)

Instead you should always use:

{
    "id": "MetricName_TypeName",
    "name": "MetricName",
    "srcType": "TypeName",
    "path": "path.to.measurementSeries",
    "expression": "earliest(normalized.data.value)"
  }

This guarantees that if the series is not normalized then it will normalize the series on the fly before returning the result. This is also more efficient that access the earliest data from the raw data since each time a metric is evaluated it has to load all the data in memory


#4

What would be the recommendation for the following use case: the measurement type has both a value field and a stringValue field.

The value is being normalized, but stringValue cannot be normalized since we don’t currently know of the entire enumerable list of possible values this string can be, so we cannot use @ts(valueMapping=...)

If we need to be able to access the timestamp of the earliest non-normalized measurement data, is there a performant way to do so?


#5

Since both the value and the stringField are on the same object, you can still get the earliest(normalized.data.value) since it will be the date for the string field as well


#6

Thanks, @rohit.sureka, the catch is that one measurement series will only ever have one of either stringValue or value populated. It will not have both, unless we could enumerate the string value as @scott.kruyswyk mentioned, which is not possible at this time given data constraints.


#7

In that case, it will have to be the tsDecl metric

{
    "id": "MetricName_TypeName",
    "name": "MetricName",
    "srcType": "TypeName",
    "path": "path.to.measurementSeries",
    "tsDecl": {
      "treatment": "LATEST",
      "start": "start",
      "value": "earliest(start)",
      "filter":"exists(stringField)"
    }
  }