SourceFile takes a long time to load in c3Grid


#1

When trying to run c3Grid(SourceType.fetch(…)) – its suddenly taking a long time to load. What’s going on?


#2

In certain versions of c3 Platform, if you have a processed a source file that returns many errors (e.g. > many thousands) – the errors array in (one of) the SourceFile objects can be very large.

The fetch takes longer to return, and the rendering in c3Grid can also take a long time.

Two things I figured out:

  1. Add an annotation to the SourceFile type to avoid this problem in the future:

@dataLoad(numErrorsToAbort=500)

This will cause file processing to stop after N errors… so this would become the max error array size in any instance of SourceFile type

  1. Clean up any/all files that have many data processing errors, and re-sync them:
SourceFile.get(<id>).remove()
SourceFile.syncFile(<file>)

This will reset the file to initial status and c3Grid(SourceFile.fetch()) should speed up again.


#3

I added FailedSource for errors in SourceFile and that has a lot of information, when you fetch do not include errors that should speed up the process. Fixing this in 7.9 where we store the firstError and add a method to return failedSources for a SourceFile. Watch out for SourcesStatus in 7.9