Invalidation Queues Questions


#1
  1. Where can one find actions, outside of c3ShowType(), related to the following queues: AnalyticsQueue ,HierDenormQueue, MetricDepsQueue, PopulateAclQueue, and CascadeQueue?
  2. How can the processes behind these queues be stopped if they are not useful or necessary for a customer’s use case(s)? How could one govern resource allocation across different tenants and tags?

For 2, @bachr shared an idea to use a CronJob to enable and disable the JMS queue at regular intervals.
The scenario here is that a 0.5GB file is ingested daily, but another data set comes in every 15m. The idea is to pause the 0.5GB file processing temporarily to process the 15m-interval data set when it comes in.

@yaroslav


#2
  1. If you are asking about actions that are currently running in a particular queue, they can be reviewed via c3QReport(%queue%) in the Console.
    The actions that could in theory put entries into a certain queue are not easy to enumerate programmatically. Refer to the “in-depth” documentation section on Invalidation Queues for a bit more context on what each queue is supposed to do, and if there are specific questions ask here in Community. (@trothwein FYI - there may be opportunity to enhance our documentation.)

  2. In general, actions on each queue are executed in FIFO order, which means that it is possible for one tenant to “hog” the queue for extended periods of time. Due to that, it is not recommended to commit to a tight SLA (like “15 minutes”) in a multi-tenant environment. It is simply not possible to guarantee.
    The proposed approach of pausing the queue may help - needs to be tested in a real-life setting.