Configuring Map/Reduce Interruptions Handling

Map/reduce scripts are susceptible to interruptions, whether from application server disruptions or uncaught errors. To manage these interruptions effectively, the system provides two configuration options: retryCount and exitOnError. Let’s delve into each option to understand how they can fine-tune your script’s response to interruptions.

retryCount

The retryCount option impacts the map and reduces stages exclusively. It allows you to specify how many times the script should attempt to restart the map or reduce the function for key/value pairs left in an uncertain state following an interruption. Here’s how it works:

Application Server Restart: In the event of a server restart, the script can’t pinpoint the exact key/value pair being processed. However, it can identify pairs flagged for processing but not yet completed. In such cases, the script retries processing for all flagged pairs.

Uncaught Error: When an uncaught error occurs, the script can identify the specific key/value pair affected. It retries the map or reduces the function for that pair.

You can set retryCount from 0 to 3, indicating the number of retries permitted for each affected pair. If not explicitly set, the behaviour varies based on the interruption type:

Server Restart: All affected key/value pairs are restarted.

Uncaught Error: Processing for the affected pair doesn’t restart.

exitOnError

The exitOnError option comes into play when an uncaught error is encountered during the map or reduce stage. It determines whether the script exits the stage immediately after:

An uncaught error occurs.

All retries permitted by retryCount have been exhausted.

Here are the available settings:

false: The script continues processing in the current stage (default behaviour if not specified).

true: The script exits the stage and proceeds directly to the summarize stage.

exitOnError doesn’t affect the system’s behaviour after an application server restart; it’s solely for handling uncaught errors.

Code:

To utilize retryCount or exitOnError, include them in the return statement within a config block in your script, as demonstrated below:

// Add additional code.
...

return {
    config:{
        retryCount: 3,
        exitOnError: true
    },
    getInputData: myGetInputData,
    map: myMap,
    reduce: myReduce,
    summarize: mySummarize
};

...
// Add additional code.