Sync - JSON Mappings and JS Processors

Once payloads are saved in the system, each record is sent through the mapper individually. The system allows you to map fields with a JSON configuration file for each entity we map. Each record also can be manipulated by a pre-processor and a post-processor to make more customizations as necessary. The Javascript processors have full access to our JS toolkit.

JSON Mappings

Mappings are created/edit in the Scripts section of the operations portal
Mappings must have the name mapping-[mappingCode] with type of JSON
The mappingCode to use for a payload is defined when sending the payloads to Kodaris in the API endpoint. (See Sending Payloads to Kodaris)
Mappings can be made for the following sync entities in Kodaris
product, payment, company, companyAddress, order, orderItem, orderDiscount, orderNote, productPrice, sku, unitConversion, manufacturer, pricingType, icss, basicTaxRate, erpSetting, orderShipment, productCrossReference, customer, wareHouse, etc
Use the syncClass to tell Kodaris which processor to use from the list above.

Field	Type	Default	Example	Description
mappingName	String		CSD ICSP Product Mapping	Nice name developers can see on the mapping grids
syncClass	String		product	Which Kodaris sync processor to use when processing data.
lookupFieldName	String		code, extra1, etc	The Kodaris field to use to lookup an existing record in Kodaris from the incoming payload. The incoming record should have this field as well or use the preprocessor to set it. This is used as a unique identifier so we know whether to add a new record or update an existing one.
lookupSettingName	String		uniqueid, erpcode, etc	If the above is blank, lookup can be done via settings (custom fields). This would be a custom field name being saved in the integration.
alternateLookupFieldName	String		someotherfield	If the initial lookups based on lookupFieldName and lookupSettingName fail, try looking up the record by this field name.
alternateLookupSettingName	String		someothersetting	If the initial lookups based on lookupFieldName and lookupSettingName fail and alternateLookupFieldName is blank, try looking up the record by this field
lookupParentFieldName	String		externalOrderNumber	More complicated integrations need to look up a parent entity to relate them in the system. In this case, the order line item sync uses the externalOrderNumber to store the header order number to look up in the Kodaris system to relate the line items to the header order information. See individual table syncs for more information on when to use this.
lookupParentSettingName	String		erpordernumber	Same as above where a custom setting on the line item could be used as a way to lookup the parent relation
isSourceDeletedFieldName	String	isSourceDeleted	Deleted	If this field exists in the incoming record and the value of the field is true, 1, yes, or y, the record will be treated as a Delete coming from the ERP and will also delete in Kodaris.
deleteLookupFieldName	String		code, extra1, etc	The field in Kodaris to match to the field in the record to identify which row to delete in Kodaris.
deleteLookupSettingName	String		rowpointer	The same as above, but can be looked up by custom setting. Sometimes the code field or normal fields do not come in on deletes, but we do have a rowpointer on the record from the ERP which can be stored in the custom settings for delete lookups.
deleteLookupParentFieldName	String		erpordernumber	Will use this to look up the parent record for certain entities like order items -> order. If it is not set, it will use the normal parentLookup settings from above.
deleteLookupParentSettingName	String		rowpointer	Same idea as above except using a custom setting value.
sourceModifiedTimeFieldName	String	sourceModifiedTime		Name of the field in the payload that tells the system the last time the ERP modified the record. This is important to make sure we do things in proper order. You will most likely want to set this in your payload to the value in the default column here if possible.
deleteUserIdFieldName	String	deleteUserId	deletedBy	Optional string on deletes identifying the user that deleted the item in the ERP. This will be logged in Kodaris when doing deletes as well.
deleteEntryNumberFieldName	String	deleteEntryNumberFieldName	deleteID	Optional delete ID in the ERP which will be logged in Kodaris as well when doing deletes.
usePlatformLogic	boolean	true		For certain syncs, the system will treat Creates differently than Updates as some fields will only be synced from the ERP during creates and not updates. For example, product descriptions come from the ERP only once and then are updated within Kodaris. If this is false, the ERP will overwrite this every time. It's usually best to keep this value as is.
allowOutOfOrderSyncs	boolean	false		Works together with the source modified setting above to determine if the synced record is older than the current one so old data does not overwrite new data. Some syncs may allow this, but normally this is left as false.
settingsFieldNames	Array		[“field1”,”field2”]	Fields in the payload record that should be stored as custom settings in Kodaris. If these are also fields mapped to actual fields in Kodaris, in the mapping be sure to also add keepFromField attribute to the copy operation otherwise this will not happen.
textValueSettings	Array		[“field1”,”field2”]	Same as settingsFieldNames except will save the value as a text value instead of normal medium value. This should only be used for large values.
skipFieldNames	Array		[“field1”,”field2”]	A list of fields to ignore in the payload. By default, the system will try to map any field named in the payload to an existing same name in Kodaris even if it is not directly mapped. This is a way to skip any field you just want to completely ignore. It is also a fast way to stop saving data without altering the mappings themselves.
mappingFields	Array		Mapping Fields - See examples	A list of mapping fields and operations telling the system how to map the payload fields to Kodaris fields.
patch	boolean	true		Only update the fields that are sent in instead of assuming any not sent in should be treated as NULL in Kodaris. Normally this would be true.
skipAllRecords	boolean	false		Used to ignore all records without retrying which can be used to temporarily turn sync processing off.
ignoreMissingFields	boolean	false		Ignore missing fields even if the field is added as a mapping field. The default is false, but usually this is set to true to avoid errors.
autoAddSettings	boolean	false		Automatically adds all unmapped fields in the record as settings without the need to declare them in the settingsFieldNames list. This is a good option when using only Kodaris fields without the need to map many fields.
dbSync	boolean	true		Used to signal that a sync should be persisted into the db
s3Sync	boolean	false		Used to signal that a sync should be persisted into s3
documentDbSync	boolean	false		Used to signal that a sync should be persisted into the document db tables
dynamicDbTableName	String	null		Will use the mappingCode if this is not set. Normally this would be blank.
dynamicDbUseIdentifier	boolean	true		Normally this should be left as is.
update	boolean	true		Signals that the sync should be allowed to update records. This is usually true.
create	boolean	true		Signals that the sync should be allowed to create records. This is usually true.
saveLogFile	boolean	false		Adds fileLogs list of strings object to js context to add to while running records and then if this is on will save the entire log file for viewing later.
settingsCaseInsensitive	boolean	false		When creating or updating settings, should the check for the setting code be case-insensitive? Since some ERPs like CSD are case-insensitive, some settings may be present with mixed case like "Securefl" whereas in the incoming payload they will be all lower-case like "securefl". Without this flag set to true it causes duplicate key exceptions.
propagateProductCodeChanges	boolean	false		If true and the sync processor detects that the product code is changing, this signals it to change the product code in all the dependent tables such as sku, product cross reference, product price, etc.

Working with Mapping Fields

The mappingFields list allows you to map a field from the name in your incoming record to a field name in Kodaris. It also allows you to perform operations when copying it into the Kodaris field. Optionally, you can use pre-processors to manipulate the data and create the exact field names to match Kodaris in Javascript. Both ways will work.

Mapping Field Attributes

fromField	String		custpo	The field name in the incoming record. Note: JS can add/remove fields from the incoming record which can also be used here.
fromValue	String			When doing a normal copy operations, if the field is null, will use this value instead when mapping.
targetField	String		purchaseOrder	The Kodaris field to copy the record value into
operationType	String	copy	copyNumeric	copy, copyNumeric, concatenate, formateDate, concatenateAndFormatDate, uuid, createSetting
operationPayload	Object			Data telling the system what to do based on the operationType

Mapping Field Operation Payload

copy	keepFromField (boolean)	true	The keepFromField tells the system to copy the field but also does not remove it from the record in case it is needed for custom settings. Without this, it will not be added as a setting.
copyNumeric	keepFromField (boolean)	false	Same as above
concatenate	prefix (String)	AB-	Any hardcoded prefix
	fieldNames (array)	[“field1”,”field2”]	A list of fields in the record you want to concatenate the values of.
	suffix (String)	-CD	Any hardcoded suffix
	delimiter (String)	,	This can be any string or a blank string. I will be used between the field values when concatenating.
formatDate	incomingDateFormat	mm/dd/yy	The date format of the incoming field value
	saveAsDateFormat	yyyy-MM-dd	The date format to save in Kodaris, defaults to the format to the left with optional time as well.
concatenateAndFormatDate			Combines both formatDate and concatenate into one operation
uuid			Creates a UUID in the Kodaris field
createSetting			Creates a blank setting with the name as the targetName in Kodaris. This can then be used in JS post processing script to update as needed.

Example Mapping Script for CSD ICSP

mapping-icsp.json

{
   "mappingName": "Infor CSD - ICSP Product Mapping",
   "syncClass" : "product",
   "deleteLookupSettingName": "rowpointer",
   "lookupFieldName": "code",
   "alternateLookupSettingName": "rowpointer",
   "propagateProductCodeChanges": true,
   "patch": true,
   "isSourceDeletedFieldName": "Deleted",
   "preProcessingScript": "pre-icsp",
   "postProcessingScript": "post-icsp",
   "ignoreMissingFields" : true,
   "textValueSettings": [
       "descrip3"
   ],
   "settingsFieldNames": [
       "rowpointer",
       "descrip3",
       "lifocat",
       "kittype",
       "vendcoregrcfl",
       "implyqty",
       "corecharge",
       "webpage",
       "warrlength",
       "bodtransferty",
       "warrtype",
       "icspecrecno",
       "usesuppunits",
       "exponinvfl",
       "webpageext",
       "randommixfl",
       "edicd",
       "custgraceper",
       "kitrollty",
       "tariffcd",
       "reqbundleidfl",
       "nospecrecno",
       "slgroup",
       "unspsc",
       "enterdt",
       "impliedcoreprod",
       "cubes",
       "certifiedtype",
       "prodcat",
       "prodtier",
       "vendgraceper",
       "notesfl",
       "tiedcompprt",
       "memomixfl",
       "msdsfl",
       "autoupcd",
       "kitnsreqfl",
       "custcoregrcfl",
       "pbseqno",
       "esbactioncode",
       "seqno",
       "brandcode",
       "lookupnm",
       "taxweight",
       "dirtycoreprod",
       "oespecrecno"
   ],
   "mappingFields": [
       {"fromField": "prod", "targetField": "code"},
       {
           "targetField": "name",
           "operationType": "concatenate",
           "operationPayload": {
               "delimiter": " ",
               "fieldNames": ["descrip_1", "descrip_2"]
           }
       },
       {
           "targetField": "shortDescription",
           "operationType": "concatenate",
           "operationPayload": {
               "delimiter": " ",
               "fieldNames": ["descrip_1", "descrip_2"]
           }
       },
       {
           "targetField": "description",
           "operationType": "concatenate",
           "operationPayload": {
               "delimiter": " ",
               "fieldNames": ["descrip_1", "descrip_2"]
           }
       },
       {"fromField": "cono",               "targetField": "cono"},
       {"fromField": "height",             "targetField": "height"},
       {"fromField": "weight",             "targetField": "weight"},
       {"fromField": "width",              "targetField": "width"},
       {"fromField": "length",             "targetField": "length"},
       {"fromField": "priceonty",          "targetField": "priceOnType"},
       {"fromField": "prodtype",           "targetField": "productType"},
       {"fromField": "unitsell",           "targetField": "unitSell"},
       {"fromField": "termspct",           "targetField": "termsPercentage"},
       {"fromField": "unitstock",          "targetField": "unitStock"},
       {"fromField": "bolclass",           "targetField": "bolClass"},
       {"fromField": "statustype",         "targetField": "statusType"},
       {"fromField": "unitcnt",            "targetField": "unitContainer"},
       {"fromField": "unitconvfl",         "targetField": "unitconversion"},
       {"fromField": "msdssheetno",        "targetField": "msdsSheet"},
       {"fromField": "termsdiscfl",        "targetField": "hasTermsDiscount"},
       {"fromField": "mfgprod",            "targetField": "manufacturerProductCode"},
       {"fromField": "sellmult",           "targetField": "intervalOrderQuantity"},
       {"operationType": "createSetting",  "targetField": "trendCompiledSettings"},
       {
           "fromField": "DCTransDate",
           "targetField": "sourceModifiedTime",
           "operationType": "formatDate",
           "operationPayload": {
               "incomingDateFormat": "yyyy-MM-dd'T'HH:mm:ss.SSSz"
           }
       }
   ]  
}

Mapping Javascript Processors

Each mapping can have a pre and post processor written in Javascript ECMAScript 5.
The scripts run for each record sent through the mapper
The normal Kodaris JS toolkit can be used in the scripts (see developer docs)
Scripts are defined using the preProcessingScript and postProcessingScript fields in the mapping. These are not required.
The script must be saved under the Scripts section of Kodaris with type Javascript.
The scripts must be saved with the following naming convention in Kodaris:
mappingscript-[name].js

Pre-Processor

This script is normally used to manipulate the incoming record before it gets sent to processing by Kodaris. It can also be used to do other things as it is just a normal javascript file in Kodaris. The following are added in scope for the pre-processor to use.

mappingSyncUtils	Helper methods for use in sync JS, but may be deprecated at some point as the normal server side API can be used for almost everything needed.
scriptServiceUtils	Used to run API methods. See developer docs.
record	JS object holding the name and value of the incoming record that was synced from the ERP. This record can be manipulated as needed before sending it to the processor.
mapping	Object containing the mapping script information (the JSON defined above)
fileLog	Optionally contains this if defined in the JSON mapping above.

Example Pre-Processing Script

mappingscript-pre-icsp.js

// Compile a list of fields to be saved into a single json setting
var trendCompiledSettingsList = record.keySet().toArray();
var trendCompiledSettingsMap = {};
for (var s=0, sLen=trendCompiledSettingsList.length; s<sLen; s++) {
   var key = trendCompiledSettingsList[s];
   var value = record[trendCompiledSettingsList[s]];
   if (value == null || typeof value == 'undefined' || value === '') continue;
   trendCompiledSettingsMap[key] = value;
}
record['compiledSettings'] = trendCompiledSettingsMap;


// Handle isActive
var statusTypeUpperCase = isNotEmpty(record["statustype"], true) ? record["statustype"].toUpperCase() : '';
var activeStatusTypes = ['A', 'L'];
var inactiveStatusTypes = ['I', 'S'];


if (activeStatusTypes.indexOf(statusTypeUpperCase) !== -1) record['active'] = true;
else if (inactiveStatusTypes.indexOf(statusTypeUpperCase) !== -1) record['active'] = false;


// Set display = false if product is inactive
if (record['active'] === false) {
   record['display'] = false;
}


// Uppercase to match uppercase unit conversions as a standard
if (record['unitstock']) {
   record['unitstock'] = record['unitstock'].toUpperCase();
}
if (record['unitsell']) {
   record['unitsell'] = record['unitsell'].toUpperCase();
}


// Remove the user1 through user10 fields
record.remove('user1');
record.remove('user2');
record.remove('user3');
record.remove('user4');
record.remove('user5');
record.remove('user6');
record.remove('user7');
record.remove('user8');
record.remove('user9');
record.remove('user10');


// --- Helper Functions
function isNotEmpty(str, trim) {
   return !isEmpty(str, trim);
}


function isEmpty(str, trim) {
   if (trim != null && trim === true) {
       return str == null || str.trim().length === 0;
   } else {
       return str == null || str.length === 0;
   }
}

// Compile a list of fields to be saved into a single json setting
var trendCompiledSettingsList = record.keySet().toArray();
var trendCompiledSettingsMap = {};
for (var s=0, sLen=trendCompiledSettingsList.length; s<sLen; s++) {
   var key = trendCompiledSettingsList[s];
   var value = record[trendCompiledSettingsList[s]];
   if (value == null || typeof value == 'undefined' || value === '') continue;
   trendCompiledSettingsMap[key] = value;
}
record['compiledSettings'] = trendCompiledSettingsMap;


// Handle isActive
var statusTypeUpperCase = isNotEmpty(record["statustype"], true) ? record["statustype"].toUpperCase() : '';
var activeStatusTypes = ['A', 'L'];
var inactiveStatusTypes = ['I', 'S'];


if (activeStatusTypes.indexOf(statusTypeUpperCase) !== -1) record['active'] = true;
else if (inactiveStatusTypes.indexOf(statusTypeUpperCase) !== -1) record['active'] = false;


// Set display = false if product is inactive
if (record['active'] === false) {
   record['display'] = false;
}


// Uppercase to match uppercase unit conversions as a standard
if (record['unitstock']) {
   record['unitstock'] = record['unitstock'].toUpperCase();
}
if (record['unitsell']) {
   record['unitsell'] = record['unitsell'].toUpperCase();
}


// Remove the user1 through user10 fields
record.remove('user1');
record.remove('user2');
record.remove('user3');
record.remove('user4');
record.remove('user5');
record.remove('user6');
record.remove('user7');
record.remove('user8');
record.remove('user9');
record.remove('user10');


// --- Helper Functions
function isNotEmpty(str, trim) {
   return !isEmpty(str, trim);
}


function isEmpty(str, trim) {
   if (trim != null && trim === true) {
       return str == null || str.trim().length === 0;
   } else {
       return str == null || str.length === 0;
   }
}

Post-Processor

This script runs after Kodaris saves the mapped record in the system. It is used to do further work on the record as needed. It can also be used to do other things as it is just a normal javascript file in Kodaris. The following are added in scope for the post-processor to use.

mappingSyncUtils	Helper methods for use in sync JS, but may be deprecated at some point as the normal server side API can be used for almost everything needed.
scriptServiceUtils	Used to run API methods. See developer docs.
record	JS object holding the name and value of the incoming record that was synced from the ERP. This record can be manipulated as needed before sending it to the processor.
mapping	Object containing the mapping script information (the JSON defined above)
originalEntity	A JS object representing the Kodaris object before it was updated by the processor.
entity	The current representation of the Kodaris object after the mapping processor saved it.

Example Post-Processing Script

mappingscript-post-icsp.js

2520_product_documentation_developers_sync_json_mappings_and_js_processors_mapping_icsp_post_icsp.jpg

if (entity) {


   // create seo code from code if null
   if (entity.seoCode == null) {
       entity.seoCode = entity.code;
   }


   // update seo code to remove non url friendly chars
   entity.seoCode = entity.seoCode.replaceAll('/', '_');
}


// Create the compiled settings list
// Save the trendCompiledSettings text value setting
fixSettingAndUpdate('trendcompiledsettings', 'trendCompiledSettings',
   JSON.stringify(record['compiledSettings']), true);


// Used for creating a setting by exact code
function fixSettingAndUpdate(badCode, correctCode, val, isTextValue) {
   if (entity.getProductSettings().get(correctCode) == null) {
       // create the correct setting code
       entity.getProductSettings().get(badCode).setCode(correctCode);
       entity.getProductSettings().put(correctCode, entity.getProductSettings().get(badCode));
   }
   entity.getProductSettings().remove(badCode);


   if (isTextValue) {
       entity.getProductSettings().get(correctCode).setValueType(mappingSyncUtils.getSettingValueType('textValue'));
       entity.getProductSettings().get(correctCode).setTextValue(val);
   } else {
       entity.getProductSettings().get(correctCode).setMediumValue(val);
   }
}