Retrieving Files and Importing Data from a Remote SFTP Server
  • 21 Mar 2024
  • 3 minute read
  • Dark
    Light
  • PDF

Retrieving Files and Importing Data from a Remote SFTP Server

  • Dark
    Light
  • PDF

Article Summary

Slate supports retrieving files from a remote SFTP server.  However, this practice is generally discouraged, because the import process fails if the remote SFTP server becomes unavailable (such as with a network connectivity or maintenance issue). It is generally recommended that files be delivered to the Technolutions SFTP servers, where we can provide high availability. These best practices aside, Slate does support the pulling of files from a remote SFTP server if this is useful to your business process.

The recommended procedure to retrieve from a remote server is to use the Secure File Transfer Protocol (SFTP) with certificate-based authentication to provide the highest level of security.

Standard FTP is not an encrypted transfer protocol, which means that all data is sent in clear text. Therefore, FTP is not recommended.

Outbound connections are initiated from the following IP addresses:

  • 34.197.57.115

  • 34.197.58.165

These IP addresses must be allowed at the remote SFTP server.

Certificate-Based Authentication (Recommended)

Create a Remote Service Account User

The service account you create will store an SSH Private Key that will be used by any source formats configured to import files from a remote server for the User ID for this account.

Note: The Security Administrator exclusive permission (not inherited by any roles, even the Administrator role) is required to create or edit user accounts.

  1. Select Database on the top navigation bar and select User Permissions.

  2. Click New User.

    • If you do not already have a key pair to use, you can use an SSH key generator, like this one.
      Note: The corresponding public key needs to be installed on the remote server for this User ID.

    • Since this is a service account, enter a name that clearly states this as a service account, rather than providing the name of a person who is already a Slate user.

  3. Enter the email address of a real user or distribution group able to receive email notifications.

  4. Change the User Type to Service Account (Remote).

  5. Provide a User ID (this should match the User ID created in the remote system).

  6. Enter the SSH Private Key.

  7. Be sure to select Enable account for access.

  8. Click Save.

Tip

The SSH Private Key should be an RSA key of at least 2048 bits and be saved without a passphrase. In addition, the private key should include the wrapper comments as shown below.

Note: The private key in this example has been shortened for illustrative purposes. If using PuTTYgen, use the  Conversions > Export OpenSSH Key  to format the newly generated Private Key correctly. You will want to save the OpenSSH Key  without a passphrase  by ignoring PuTTYgen's warnings.

Password-Based Authentication

Schedule Imports using a Source Format

  1. Select Database on the top navigation bar.

  2. Under Import, select Source Formats.

  3. Select New Source Format.

  4. Set the Status to Active.

  5. Enter a Name.

  6. Set the Remap as of Date to the current date.

  7. Select Save.

  8. Select the Source Format created above, or select another existing Source Format.

  9. Select Edit.

  10. Select the Import Automation tab.

  11. Enter the import path on the remote server as well as the file Mask of the files you will be retrieving.

  12. Enter the Server information in the Import Remote Server field.

If using certificate-based authentication, provide:

  • protocol

  • username

  • hostname

For example: sftp://remoteuser@ft.technolutions.net

Be sure to set the username as the User ID of the service account you created in the previous steps. The hostname should be the remote server address.

If using password authentication, provide the protocol, username, password and hostname (for example, sftp://remoteuser:[email protected]).

Tip

If a port number must be specified, the port is appended to the the Server information in the Import Remote Server field (for example, sftp://remoteuser:[email protected]:22).

Path / Mask

The settings on the Import Automation Tab are optional and will not apply to every Source Format. The Import Path/Mask field tells Slate that you want to pull in files automatically. In this field, include the file name pattern that Slate should use to locate the file to be imported using this Source Format. We recommend appending the file name with a date, rather than using the exact same file name each time. Slate retains files in a Processed directory in the /incoming folder for a period of time, so creating a file with a date-appended name prevents the overwriting or deleting of existing files (and subsequent data loss) with each import.

An example value for the Import Path/Mask field would be: SIS_to_Slate_*.txt. With this value, Slate would automatically import any file in the /incoming directory that contained that file name pattern, including: SIS_to_Slate_09032013.txt.


Was this article helpful?