The SFTP GET operation in Boomi is used to retrieve files from a remote SFTP server. One of the most commonly used actions is “Get and Delete”, which not only downloads the file but also removes it from the remote directory after successful retrieval. This is useful for workflows where files should not remain on the server after processing, ensuring no duplication or reprocessing.
Understanding the Configuration Options
The screenshot provided shows the Edit SFTP Configuration page in Boomi. Let’s break down the key sections:
1. SSH Options
- Use Private Key Authentication:
- If checked, Boomi will use an SSH private key for authentication instead of a password. This is recommended for secure connections.
- Known Host Entry:
- Allows you to specify the host key fingerprint for additional security validation.
- Use Max Key Exchange size of 1024:
- This option enforces a maximum key exchange size of 1024 bits for SSH. It’s typically used for compatibility with older servers.
2. Get Options
This section controls how files are retrieved from the SFTP server.
- Use Default Get Options:
- If enabled, Boomi uses the default settings for file retrieval. Leaving it unchecked allows customization.
- SFTP Action:
- Here, the selected action is “Get and Delete”.
- Get: Downloads the file from the remote directory.
- Get and Delete: Downloads the file and then deletes it from the remote directory after successful transfer.
- This is ideal for one-time processing scenarios.
- Remote Directory:
- Specifies the folder on the SFTP server where files are located. In this example:
/outgoing/ntchg
- File Filter:
- Allows you to define patterns (e.g.,
*.csv) to retrieve only specific files. - Maximum Files to Read:
- Controls how many files Boomi will fetch in one execution.
0means no limit (fetch all matching files).- Force Move:
- If enabled, Boomi will move files even if errors occur during processing.
- Data Processing Options:
- Clicking Edit allows you to configure additional processing rules for the retrieved files.
How “Get and Delete” Works
- Boomi connects to the SFTP server using the configured credentials.
- It navigates to the specified Remote Directory.
- It retrieves files matching the File Filter.
- After successful download, Boomi deletes the files from the remote directory.
- If any error occurs during download, the deletion will not happen (unless Force Move is enabled).
Best Practices
- Test in a Non-Production Environment: Always verify the configuration before enabling deletion in production.
- Enable Logging: Keep detailed logs to track which files were retrieved and deleted.
- Use File Filters: Avoid fetching unnecessary files by using specific patterns.
- Backup Before Delete: If possible, maintain a backup of files before deletion for audit purposes.
Common Use Cases
- EDI Transactions: Fetching and clearing processed interchange files.
- Data Pipelines: Moving files from an external system to Boomi for transformation and integration.
- Scheduled Jobs: Automating file cleanup after processing.
