Status

Select Active to make the source format available for use in Upload Dataset.

Name

Give the source format a descriptive internal name.

Format

Enter the imported file format for display purposes only. For example, Excel, CSV, or Fixed Width.

File formatting is defined on the Format Definition tab in the XML field.

Type

One-Time / Differential

Retains existing source files alongside new source. Most imports use this type.

Cumulative / Replaceable

Replaces any existing source files with a new source file each time. For example: the standard Checklist Import (Financial Aid), which includes every record every time the file is imported.

Remap As Of Date

For now, enter the current date.

This date can be changed to limit the data values available to the source format for remapping.

If, for example, a file is imported with no header row, and first row data become the headers, enter a remap-as-of date after this erroneous file was imported.

Remap Active

Leave this as Inactive until you have configured remap settings (step five).

The remap-as-of date must be populated to activate this setting.

Scope

Leave this setting set to Person/Dataset Record unless the source format imports Slate users.

Dataset

Select a dataset into which the source format should import data.

Unsafe

If a source format is Unsafe, it can update person-scoped data on student records with an active application.

Select Unsafe only if you trust the source to contain accurate, up-to-date information. An import into Slate from your student information system is an example of a source format that you might set as Unsafe.

Purchased lists of student information are examples of imports that should have Safe source formats.

Hide

Determines whether the import will create source interactions on the affected record. You'll usually want to Create source interactions to display activity history on a record's timeline.

Daily or frequent data updates/imports can contribute to the creation of many rows of data interactions on records. If the source format will be used for a daily feed from an SIS or other application, select Hide source interactions. 

Disable Update Queue

Allow records to enter update queue upon import (allow rules to fire)

Records imported or updated from this source format enter an update queue.

This queue runs deferred rules and adds records to queues for those data exports that are configured to only include records updated since the query last ran.

Prevent records from entering update queue upon import (disables rules from firing)

Select this option if the source format is a backfeed that sets irrelevant information in the context of query update queues and rules.

This prevents a feedback loop wherein each updated record is exported from Slate into an external system, which then exports data for each of those updated records back to Slate.

If the update queue were enabled, those records in that import would be included in the export again the next time it ran, even if the import was the only update on those records.

Update Only

This tells Slate how to handle records in the incoming dataset that do not match records within Slate.

Allow record creation

Slate will create a new record for any records in the incoming dataset that do not match records in Slate.

Update only

Slate will not create new records, and the dataset will only update records on which it matches.  Note that when Update Only is selected for an application data import, an application will not be created even if the person record already exists.

Dedupe Records

By default, Slate will create a new record for every row when Allow record creation is selected and an existing match cannot be found.  By selecting Dedupe records, Slate will evaluate the source file and dedupe records in the source based on an exact match of First Name, Last Name, Email, and Birthdate.

Notification and Notification Email

To receive email notification if an imported file associated with the Source Format fails, select Failures only and enter the email address(es) for notification. Select Successes and Failures to be notified of both successful and failed imported files for the Source Format.

Read Permission and Upload/Build Permission

Permission to the source format can be limited to specific permissions. 

XML

Use XPath syntax to tell Slate how to parse the data source. For example, you can define whether the file is delimited, if there is a header row, and what file format is used.

See the Common XML Examples article for specific examples that can be copied and pasted into this field.

SQL

For most data imports, the SQL text area should be left blank. Consider creating a community forum post to determine if your case requires custom SQL.

Source Index

Leave this set to the default Use modern cloud storage unless you are working with an existing source format that requires the legacy source.index table.

PGP Private Key

Paste the private key from the PGP software.

PGP Password

Each PGP private key is typically encrypted with a password.

Import Path/Mask

Include the file name pattern that Slate should look for to know that the file should be imported using this Source Format.

Import Remote Server

Leave this field blank unless the data for this source format will be delivered via an API call from Slate or an external SFTP server.

API Calls

Include the full URL, including https:// and any necessary query string parameters.

Remote SFTP Servers

The syntax (including support for remote service account users) is the same as when configuring a remote server for data exports.

HTTP Headers

Add a list of headers in the form, for example: VALUE

Import Order

(Optional, if import path/mask exists) Specify an import order.

For example, if you have an application file and a supplement file delivered to the SFTP directory together, and you want the application file to be processed prior to the supplement, enter 1 in the application's source format and 2 in the supplement's.

Import Frequency

Remote SFTP

By default, Slate sweeps the SFTP directories for new files that match the specified import path/mask every 15 minutes.

Decrease frequency by entering the number of minutes between file sweeps, for example, 1440 minutes for a once-daily import.

The import frequency setting is limited to source formats picking up from remote SFTP servers and web services—not the Technolutions SFTP. 

Web Service

Slate's remote import automation polls the end point every 24 hours by default.

Adjust the import frequency up to hourly by setting the frequency value to 60 minutes; any value less than 60 will poll the endpoint hourly.

Web Callback URL

Enter a webhook URL to send a Web Service HTTP POST to an external system. This POST includes a JSON Payload and confirms that the import was successful in Slate. The external system receiving this confirmation can use it to mark its records as exported and updated in Slate.

Below is an example of a POST. For any records not updated or exported, a process can be implemented within your institution that prompts a member of your team to review and determine the root cause.

  {
    "status": "success",
    "source": "d13a8c60-d6da-2844-4704-0d3d78eed25f",
    "summary": "Common App Prospects",
    "database": "XYZ",
    "started" : "2022-06-28T06:12:22.345Z",
    "stopped" : "2022-06-28T06:12:23.577Z"
   }