How to speedup a very slow importData

#1

I’m trying to move measurements data from a cluster (7.2) into another one (7.6).
I’ve exported the data with the following command:

var header = 'comments,dataVersion,isEstimated,lastEditor,lastModification,name,outlierCode,parent,quantity.value,quantity.unit,start,statusCode,unitCode,version';
var spec = ExportDataSpec.make({
  contentName: 's3://...',
  progressCallbackPeriod: 30,
  contentType: 'text/csv',
  contentEncoding: 'gzip',
  csvHeader: header
});
PointMeasurement.exportData(spec);

Then I copied the file into the destination cluster, and started the import with:

var async = true;
PointMeasurement.importData({url: 's3://...'}, async);

The CSV file contains only 100k rows, I’ve been monitoring this import it’s been running for a long time yet not finished. When looking to the age of the ClusterAction I see it’s 01:19:15.832 yet it imported less then the file’s 1/5

> c3Count(PointMeasurement)
17000

Any suggestions on how to speedup the import?

0 Likes

#2

In importData if its a cassandra type could you try

PointMeasurement.importData({url: 's3://...', createOnly: true, directPath: true }, async);

That could speed things up

1 Like

#3

thanks @garrynigel for the tip, does those options have effect on non-cassandra types?

0 Likes