pre save page hook in celigo

pre save page

The pre save page hook is invoked on a page of records before the page is sent to subsequent steps in your flow. This hook can be used to add, update or delete records. This hook is a great place to execute logic on batches of records at the same time. For example:

• Filter or group records in batches
• Apply calculations in batches
• Perform complex data transformations in batches
• Validate data and return errors for records

Function stub

 /*
* preSavePageFunction stub:
*
* The name of the function can be changed to anything you like.
*
* The function will be passed one 'options' argument that has the following fields:
*   'data' - an array of records representing one page of data. A record can be an object {} or array [] depending on the data source.
*   'files' - file exports only. files[i] contains source file metadata for data[i]. i.e. files[i].fileMeta.fileName.
*   'errors' - an array of errors where each error has the structure {code: '', message: '', source: '', retryDataKey: ''}.
*   'retryData' - a dictionary object containing the retry data for all errors: {retryDataKey: { data: <record>, stage: '', traceKey: ''}}.
*   '_exportId' - the _exportId currently running.
*   '_connectionId' - the _connectionId currently running.
*   '_flowId' - the _flowId currently running.
*   '_integrationId' - the _integrationId currently running.
*   'pageIndex' - 0 based. context is the batch export currently running.
*   'lastExportDateTime' - delta exports only.
*   'currentExportDateTime' - delta exports only.
*   'settings' - all custom settings in scope for the export currently running.
*   'testMode' - Boolean flag that executes script only on test mode and preview/send actions.
*
* The function needs to return an object that has the following fields:
*   'data' - your modified data.
*   'errors' - your modified errors.
*   'abort' - instruct the batch export currently running to stop generating new pages of data.
*   'newErrorsAndRetryData' - return brand new errors linked to retry data: [{retryData: <record>, errors: [<error>]}].
* Throwing an exception will signal a fatal error and stop the flow.
*/
function preSavePage (options) {
  // sample code that simply passes on what has been exported
  return {
    data: options.data,
    errors: options.errors,
    abort: false,
    newErrorsAndRetryData: []
  }
}

Example

This preSavePage hook function replaces a potentially large sub-object with an error message if its size exceeds 500,000 characters (approximately 1 MB) in each data record.

function swapHugeSubObjectWithInlineErrorMsg (options) {
  options.data.forEach(function(d) {
    let size = JSON.stringify(d.potentiallyBigSubObject).length
    if (size > 500000) {
      d.potentiallyBigSubObject = {
        errorMessage: "potentiallyBigSubObject  1MB"
      }
    }
  })
  return {
    data: options.data,
    errors: options.errors,
    abort: false
  }
}