@venkata.paruchuri & @ColumbusL
Thanks for the information. The use case is to determine whether certain actions taken on the cluster are impacting other higher priority items / jobs.
For instance, we receive near real time interval data in a timely manner from the source system, but we find that we are quite behind on data loads from that source system (expectation is 1 hour lag, but reality is more like 5 to 7 hour lag). This impacts some of our use cases for near real time customer data presentment.
So the goal is to better understand where the cluster’s resources are being used and make a determination as to what we can prioritize to strike a balance between timely data loads and processing of jobs on the cluster.