How to filter data in a metric declaration?


#1

Hi all, I have this metric:

{
"id": "SensorInternalTemperature_FixedAsset",
"name": "SensorInternalTemperature",
"srcType": {
  "moduleName": "structure",
  "typeName": "FixedAsset"
},
"expression": "avg(avg(normalized.data.quantity))",
"path": "denormChildren.to.servicePoints.measurements.(resource.id == 'temperatureInternal' && measurementType == 'Temperature')",
"description": "Sensor Internal temperature - FixedAsset"

}

Is it possible to skip some data in the expression field? I must skip all data with data.quantity.statusCode == ‘BAD’.

Thank you.
Davide


#6

Can you ignore the data with statusCode == 'BAD' in your transform, i.e. do not create the measurement at all?
If you need to store the measurements then the safer option would be to:

  • assign them to a specific series and
  • filter out this series in your metric expression (just like how you’re doing with measurementType)

e.g. your PointMeasuremnt transform would be

type TransformToPointMeasuremnt . . . {
  parent: ~ {id: 'status!="BAD"? RegularSeriesID: BadSeriesID'}
}

your measurement series should look:

remix PointPhysicalMeasurementSeries {
  dataQuality: string
}

your transform for measurement series should create two series one with dataQuality set to BAD.
then the metric’s path would look like after adding a check on dataQuality:

{
...
"path": "denormChildren.to.servicePoints.measurements.(resource.id == 'temperatureInternal' && measurementType == 'Temperature' && dataQuality!='BAD')",
}

#7

See this answer to a related post (thanks @rohit.sureka)

Your options are

  1. If the corrected points have the same start/end as the bad datapoint, then make sure to have the bad data point to have a lower dataVersion than the right one.
  2. Handle that in the TSDecl metric (on raw points)
  3. Move the bad data to a different type all together.

#8

I think solution proposed by bachir is better for me. Thank you guys :wink: