Cleaning measurements


#1

I noticed that removeAll keep returning non 0 results:

> while(PointMeasurement.removeAll()>0) {console.log('removing...')}
removing...
removing...
> while(PointMeasurement.removeAll()>0) {console.log('removing...')}
removing...
removing...
....

Is there another way to remove measurements or any other Cassandra type?

Also, Is c3Count reliable when dealing with measurements and Cassandra types?


#2

We usually do this via a MapReduce job on the header type:

var map = function(batch, objs, job) { 
  objs.each(function(pms) { 
    Measurement.removeAll("parent.id == '" + pms.id + "'");
  }); 
}

var spec = JSMapReduceSpec.make({
	targetType: PhysicalMeasurementSeries,
	include: "id",
	filter: "meta.sourceSystem == 'foo'", // modify filter as appropriate
	limit: -1,
	batchSize: 100,
	map: map
});

var mrj = JS.mapReduce(spec);
mrj.status();

#3

Yeah I think this is the safest option, thanks yaro. But It seems that even this approach is not 100% reliable, I still see some PointMeasuremt

// after running the cleaning job finished
> c3Count(PointMeasurement)
6734