CSV parsing errors when loading unknown columns

We have errors when uploading CSV canonical file with additional columns that are not declared in the canonical:

errorMsg: Invalid value path 'substation_code<*>' at 16 for type CanonicalWindFarmStaticReferencec3.love.exceptions.C3RuntimeException: Invalid value path 'substation_code<*>' at 16 for type CanonicalWindFarmStaticReference at c3.love.typesys.ValuePath.formatError(ValuePath.java:278) at c3.love.typesys.ValuePath.parse(ValuePath.java:270) at c3.love.typesys.ValuePath.parse(ValuePath.java:78) at c3.love.typesys.obj.pipe.ser.CsvObjValueIterator.<init>(CsvObjValueIterator.java:100) at c3.love.typesys.obj.pipe.ser.CsvSerDeser.iterator(CsvSerDeser.java:111) at

We would have expected the parsing to simply ignore additional columns and to integrate the CSV file since the declared fields were all present in the CSV.
With this kind of errors we are obliged to declared all additional columns that we won’t actually use, and any additional columns would require a new code modification.

Is there a way to avoid that?

Starting at least from version 7.9, you can use the field skipExtraFields available in the SourceCollection type to indicate you want to ignore fields not declared in the canonical.

Thanks @Hugues that is indeed what I was looking for.
We are on 7.8.10, this field is marked as beta but is seems to be doing the job.

@NabilKoroghli @Hugues Yes SourceCollection is the metadata for the process of DI in 7.9 and we are hoping to drive the entire process through the metadata present there. A request was made to back-port a few of these changes, hence its available in 7.8.