Problem with AVG treatment in 7.6 version

#1

Hi, I have a problem with the treatment AVG in the normalization of data. I have a different output from version 7.2 and 7.6 for measurementSeries that have this kind of treatment, even if the configuration and the raw data of the measurementSeries are the same. I tried to do a refreshNormalization but nothing has changed, someone can suggest me something to fix this strange behaviour?

Many thanks,
Laura

0 Likes

#2

@lauraminicucci this seems like it will need to be investigated. Can you please work with a C3 services professional to file and ticket and provide exact steps to reproduce and the environment details?

0 Likes

#3

@rohit.sureka it seem the temperature data is normalized into a wrong scale!

c3Viz(PointPhysicalMeasurementSeries.tsEval({
  projection: 'avg(avg(normalized.data.quantity))',
  filter: 'id=="00-AsNet1.00-Group00000000.AMB_UTA3_231_04____01"',
  start: '2017-10-05',
  end: '2017-10-20',
  interval: 'DAY'
}))

The normalized data looks like this

c3Grid(NormalizedTimeseriesPersister.getId(PointPhysicalMeasurementSeries.normalizedTimeseriesKey('00-AsNet1.00-Group00000000.AMB_UTA3_231_04____01', 'quantity'))) 
 0: 0.3045754054054054
 1: 0
 2: 0
 3: 0
 4: 0.20746947916666666
 5: 0.20663697916666668
 6: 0.21895864583333333
 7: 0.2147959375
 8: 0.21621125
 9: 0.2252859375
 10: 0.2184590625
 11: 0.21779302083333332
 12: 0.2184590625
 13: 0.22137302083333332
 14: 0.2208734375
 15: 0.4388330208333333
 16: 0.21412989583333333
 17: 0.4277602083333334
 18: 0.21288104166666666
 19: 0.20846854166666665
 20: 0.2031403125
 21: 0.22678447916666666
 22: 0.24426791666666667
 23: 0.2489301041666667
 24: 0.23660854166666667
 25: 0.23669177083333334
 26: 0.25192729166666666
0 Likes

#4

@bachr what does raw data look like?

Also shouldn’t temperature data come from Observation? Or is this some custom type?

0 Likes

#5

Those are custom temperature data,
this returns nothing:

c3Grid(PointMeasurement.fetch({filter: 'parent.id=="00-AsNet1.00-Group00000000.AMB_UTA3_231_04____01" && quantity.value <= 1'}))

this returns the data

c3Grid(PointMeasurement.fetch({filter: 'parent.id=="00-AsNet1.00-Group00000000.AMB_UTA3_231_04____01" && quantity.value <= 30'}))
0 Likes

#6

I would say let’s look at the time period and see if the normalizer is doing the right thing. I also propose to just normalize again to eliminate that out of the picture.

0 Likes

#7

I tried refreshing normalization with PointPhysicalMeasurementSeries.refreshNormalization() and waiting for the norm/batch queues to drain I still have same tsEval result!

0 Likes

#8

You may want to run PointPhysicalMeasurementSeries.refreshNormalization({force:true, ids:['00-AsNet1.00-Group00000000.AMB_UTA3_231_04____01']}) and see if that helps.

1 Like

#9

@yaroslav thanks for the tip, unfortunately i’m on 7.6 and the field force does not exist. I tried by precising the id but it didn’t help, still the temperatures are scaled as shown above.

0 Likes

#10

@rohit.sureka it seems the problem is that in 7.6 AVG treatment would take average for day as total/count of quarter_hours. So as it is defined, it it is doing correct. In the other hand, treatment should be “RATE” to see same temperature until a new temperature is read.
Any idea why AVG is not behaving as it was in 7.2? (in fact it seems we were having the behavior of RATE when using the AVG treatment).

0 Likes

#11

Based on my discussion with @pavan.nandikonda this is a bug in Avg treatment in v7.6 where it counts the interval in the divisor even though data is completely missing in that interval. This has been fixed in the v7.8 version.

0 Likes

#12

thanks @rohit.sureka for your input, what would you suggest us to do?
Does switching ALL AVG series into RATE will be enough?

0 Likes