Batch Export Data

#1

I’m attempting to use the Export API to batch export data to S3. I have created the below BatchExportSpec for the job. This creates an entry in the Export type and adds the job to the BatchQueue. The problem is the Export job is stuck in the running state and never moves from Pending in the BatchQueue.

Any ideas on why the job never gets past the pending state?

var exportSpec = BatchExportSpec.make({
targetType: "Currency",
fileUrlOrEncodedPathPrefix : "currency",
numFiles: 1,
contentType: "application/json"
});

var exportResult = Export.startExport(exportSpec);
0 Likes

#2

I have no idea… but i am curious why you’re exporting all the Currency data to S3?

0 Likes

#3

purely for testing purposes :slight_smile:

0 Likes

#4

Check for errors with c3QErrs(BatchQueue), or the state of other queues with c3Grid(InvalidationQueue.countAll()) may be there is no resources for your job.

0 Likes

#5

No errors and there is nothing else running on the cluster.

0 Likes

#6

Can you check if anything was actually produced?

  1. you can get the urls with job.files(true), then
  2. to read content you can for instance do S3File.make({url: job.files(true)[0]}).readString()
0 Likes

#7

job.files does not return anything but we tested in another environment and it works.

0 Likes