JSON Data Integration

A few questions about data integration using JSON format.

  1. When building a canonical, do the key/val pairs in a json file map directly to the header/data when using csv?
    • I built a very simple canonical and JSON file:

type CanonicalTest mixes Canonical {
SerialNumber: string

Latitude: double

Longitude: double
}

and

{
“SerialNumber” : “abc”,
“Latitude” : 1.0,
“Longitude” : 1.0
}

When I send the file, I get the following error:

errorSummary: Unexpected token VALUE_STRING when expecting FIELD_NAME or END_OBJECT in JSON document at 2:26.

Am I missing something?

  1. How are nested JSON objects handled in the canonical?

  2. Are there any examples/documentation for using JSON format for data integration?

JSON format used to be differently handled in DI before 7.9. But going forward
File.make().readObjs({serType:CanonicalTest}).collect() should be the test if your file is parseable.

  1. Yes while building the json the fields in json map should match the type definition
  2. Nested json objects will be parsed looking at the type definition and matching the value types of the fields.
  3. Apart from suggesting that fields should match the type which is kind of implicit, other options for parsing files of various content types are available in the documentation for
    FileObjsOperSpec

The above issue you’re facing seems to be a bug and seems to have been fixed, depends on the version your using

If its 7.8, there is a workaround where you set the jmsDisabled flag on the FileSourceCollection to true.

1 Like

Thank you, that helps a lot! I’m currently on 7.8. How do I set that flag? I’m trying to do that through the console but it doesn’t look like there’s a function or direct access to the flag.

in 7.8

FileSourceCollection fsc = FileSourceCollection.make(<id>).get();
fsc.jmsDisabled = true;
fsc.upsert();

This was introduced as a work around in 7.8. Note you don’t have to do this in 7.9.

1 Like

Working now! Thanks again!