How can I view the performance characteristics of a metric?


#1

I’d like to able view the performance characteristics and for metric evaluation. Is there a API to obtain such data?


#2

To view the performance characteristics of a metric, use the evalMetrics function while passing in explain:true and evalPlanInclude options. For example, to include data dependencies, metric dependencies, and instrumentation details, construct the following input:

var result = Facility.evalMetrics({
  "ids": [
    "Facility1"
  ],
  "expressions": [
    "energySpend"
  ],
  "start": "2015-01-01T00:00:00",
  "end": "2015-12-01T00:00:00",
  "interval": "MONTH",
  "explain": true,
  "options": {
    "evalPlanInclude": "hierarchies, simpleMetrics, metricsMetadata, evalInstrumentation, tsEvalInstrumentation"
  }
});

The abbreviated output will look like:
{
“type”: {
“module”: “metric”,
“name”: “EvalMetricsResult”
},
“result”: {
“Facility1”: {
“energySpend”: {
“type”: {
“module”: “timeseries”,
“name”: “NormTimeseriesDouble”
},
“m_start”: “2015-01-01T00:00:00.000”,
“m_end”: “2015-12-01T00:00:00.000”,
“m_data”: [… ],
“m_availabilityNormalized”: true,
“m_gaps”: [… ],
“m_estimates”: [ … ],
“m_unavailable”: [ … ],
“m_interval”: “MONTH”,
“m_tzShifts”: [],
“m_unit”: {
“id”: “USD”
},
“m_timeZone”: “NONE”,
“m_sourceInfo”: {
“source”: {
“id”: “Facility1”,
“moduleName”: “facilitymgt”,
“typeName”: “Facility”
},
“expr”: “energySpend”
}
}
}
},
“evalPlan”: {
“hierarchies”: {
“services.(resource.id == ‘electricity’)”: {
“Facility1”: {
“2015-01-01T00:00:00.000Z_2015-12-01T00:00:00.000Z”: []
}
},
“services.(resource.id == ‘naturalgas’)”: {
“Facility1”: {
“2015-01-01T00:00:00.000Z_2015-12-01T00:00:00.000Z”: []
}
}
},
“simpleMetrics”: {
“Facility_Facility1_BilledNaturalGasSpend_2015-01-01T00:00:00.000_2015-12-01T00:00:00.000_MONTH”: {
“type”: {
“module”: “metric”,
“name”: “EvalPlanTimeseries”
},
“metric”: “BilledNaturalGasSpend”,
“srcId”: “Facility1”,
“data”: {
“type”: {
“module”: “timeseries”,
“name”: “NormTimeseriesDouble”
},
“m_start”: “2015-01-01T00:00:00.000”,
“m_end”: “2015-12-01T00:00:00.000”,
“m_data”: […],
“m_availabilityNormalized”: false,
“m_gapRanges”: [],
“m_estimateRanges”: [],
“m_unavailableRanges”: [
{
“type”: {
“module”: “metadata”,
“name”: “TimeRange”
},
“start”: “2015-01-01T00:00:00.000”,
“end”: “2015-12-01T00:00:00.000”
}
],
“m_interval”: “MONTH”,
“m_tzShifts”: [],
“m_unit”: {
“id”: “USD”
},
“m_timeZone”: “NONE”
},
“time”: 0.000196604,
“count”: 1
},
“Facility_Facility1_BilledElectricitySpend_2015-01-01T00:00:00.000_2015-12-01T00:00:00.000_MONTH”: {
“type”: {
“module”: “metric”,
“name”: “EvalPlanTimeseries”
},
“metric”: “BilledElectricitySpend”,
“srcId”: “Facility1”,
“data”: {
“type”: {
“module”: “timeseries”,
“name”: “NormTimeseriesDouble”
},
“m_start”: “2015-01-01T00:00:00.000”,
“m_end”: “2015-12-01T00:00:00.000”,
“m_data”: […],
“m_availabilityNormalized”: false,
“m_gapRanges”: [],
“m_estimateRanges”: [],
“m_unavailableRanges”: [
{
“type”: {
“module”: “metadata”,
“name”: “TimeRange”
},
“start”: “2015-01-01T00:00:00.000”,
“end”: “2015-12-01T00:00:00.000”
}
],
“m_interval”: “MONTH”,
“m_tzShifts”: [],
“m_unit”: {
“id”: “USD”
},
“m_timeZone”: “NONE”
},
“time”: 0.000125865,
“count”: 1
}
},
“evalInstrumentation”: " % Time Unit Self Unit CPU Unit Count Node \n---------- ------- ----- ------- ----- ------- ----- --------------------------------------------- \n========== 173,020 nano 48,590 nano 49,079 nano 1 [energySpend]\n === 44,926 nano 44,926 nano 44,588 nano 1 SM$BilledNaturalGasSpend\n ==== 76,544 nano 26,020 nano 26,176 nano 1 SM$BilledNaturalGasSpend + SM$BilledElectricitySpend\n ====== 101,933 nano 25,389 nano 25,456 nano 1 convertToUnit(“USD”, SM$BilledNaturalGasSpend + SM$BilledElectricitySpend)\n ======= 124,430 nano 22,497 nano 22,643 nano 1 energySpend = (convertToUnit(“USD”, SM$BilledNaturalGasSpend + SM$BilledElectricitySpend))\n = 5,598 nano 5,598 nano 5,631 nano 1 SM$BilledElectricitySpend\n\n\n % Time Unit Self Unit CPU Unit Count Node \n---------- ------- ----- ------- ----- ------- ----- --------------------------------------------- \n ====== 349,374 nano 349,374 nano 348,865 nano 1 TS : SM$BilledNaturalGasSpend\n === 152,908 nano 152,908 nano 152,523 nano 1 TS : SM$BilledElectricitySpend\n ========= 567,908 nano 65,626 nano 66,482 nano 1 TS : SM$BilledNaturalGasSpend + SM$BilledElectricitySpend\n========== 619,885 nano 27,529 nano 27,687 nano 1 TS : energySpend=(convertToUnit(“USD”, SM$BilledNaturalGasSpend + SM$BilledElectricitySpend))\n ========= 592,356 nano 24,448 nano 24,462 nano 1 TS : convertToUnit(“USD”, SM$BilledNaturalGasSpend + SM$BilledElectricitySpend)\n",
“time”: 0.00177369,
“ioTime”: 0.002252586,
“metricsMetadata”: [
{
“type”: {
“module”: “metric”,
“name”: “EvalPlanMetricsMetadata”
},
“simpleMetric”: {
“srcType”: {
“moduleName”: “structure”,
“typeName”: “FixedAsset”
},
“path”: “services.(resource.id == ‘electricity’)”,
“id”: “BilledElectricitySpend_FixedAsset”,
“name”: “BilledElectricitySpend”,
“expression”: “treatAsUnit(‘USD’, sum(sum(normalized.data.usage))+sum(sum(normalized.data.fees))+sum(sum(normalized.data.tax))+sum(sum(normalized.data.adjustment)))”
}
},
{
“type”: {
“module”: “metric”,
“name”: “EvalPlanMetricsMetadata”
},
“simpleMetric”: {
“srcType”: {
“moduleName”: “structure”,
“typeName”: “FixedAsset”
},
“path”: “services.(resource.id == ‘naturalgas’)”,
“id”: “BilledNaturalGasSpend_FixedAsset”,
“name”: “BilledNaturalGasSpend”,
“expression”: “treatAsUnit(‘USD’, sum(sum(normalized.data.usage))+sum(sum(normalized.data.fees))+sum(sum(normalized.data.tax))+sum(sum(normalized.data.adjustment)))”
}
}
]
},
“errors”: {}
}


#3

Hey Scott,

We are running the below query to view the evaluation path of a metric:

var myVar = ServicePoint.rollupMetricWithCustomerId(
{"explain": true, 
   "options": {"evalPlanInclude":  "hierarchies, simpleMetrics, metricsMetadata, evalInstrumentation, tsEvalInstrumentation"},
 "interval":"MONTH",
"start":"2018-01-17T00:00:00.000Z",
"end":"2018-05-17T00:00:00.000Z",
"include":"data, dates, missing, estimates",
"ids":["955789289_0001"],
"expressions":["MeteredElectricityConsumption_ByCustomer"],
"rollupFunc":"SUM",
"bindings":{"customer_id":{"type":{"module":"metadata","name":"String"},"value":"customer-id"}}})

myVar.evalPlan -> doesn’t return anything

Anything amiss in here? or something changed in v7.7?


#4

@amit.tomar have a look to the implementation of rollupMetricWithCustomerId. This function seems to be doing:

  1. an eval metric i.e. ServicePoint.evalMetric
  2. then takes the result (that contains an evalPlan) aggregate it by customer id
  3. and dynamically create a new Timeseries to be returned

In the new Timeseries, the field evalPlan is not set.


#5

@bachr is correct. Assuming that rollupMetricWithCustomerId has the same output as rollupMetric/rollupMetric, the response is a single timeseries. The evalPlan is not available and is only available on EvalMetricsResult

You could look at the evalPlan using the evalMetrics API to get a sense metric performance. Unfortunately the rollupMetrics API does not return evalPlan results today.