Importing, Exporting, & Integrations Overview
  • 23 Apr 2024
  • 7 minute read
  • Dark
    Light
  • PDF

Importing, Exporting, & Integrations Overview

  • Dark
    Light
  • PDF

Article Summary

Slate has integrated successfully with a wide variety of software applications, including student information systems (SIS) such as Banner, PeopleSoft, and Colleague, as well as with business intelligence solutions and enterprise content management systems. In this article, you'll find a review of the methods and best practices for integrating external systems with Slate, along with links to relevant articles that provide more detail about each step.

Overview Demonstration of Data Integrations with Alexander

This video provides an overview of data integrations in Slate and a discussion of the tools used to share data between Slate and other systems.

Video Topics

Topic

Timestamp

Data Exports

0:30

Scheduled Export Options (SFTP, etc)

01:57

Enable SFTP Access for Users

3:22

SIS Integration Discussion

4:22

Real-Time Web Services

9:10

Direct SQL Access & Materialized Views

10:38

Importing Data into Slate (Upload Dataset)

12:30

Integrating Data from Forms or Events

15:10

Leveraging the Slate Test Environment 

16:25

Deduplication & Record Matching

17:20

Consolidate Records

19:05

Address Validation & Standardization

20:30

Export Formatting Example - Phone Numbers

21:20

Before We Begin: Data & Process Audit

As you embark on a new integration, whether a one-time migration of historical data into Slate or a bi-directional feed with a new external system, you can avoid serious headaches by engaging in a data and process audit. Much like the one we ask clients to perform at the earliest stages of an implementation, taking stock of your data, who's using it, and why will answer ahead of time a lot of the questions you might otherwise be asking when you're elbow-deep in the integration, when their cost in time and resources is much greater.

While going through the audit, it may be helpful to pose the following questions about stored data points to key stakeholders:

  • Is this data point accurate?

  • Is this data point necessary for us to do our job?

  • Who uses this piece of information? In what capacity? 

  • Is this value something that can change? Or is it purely historical?

  • Why do we collect information in this manner?

💡 Tip

Your peer institutions may be your best resource for information specific to your SIS or external system. We encourage reaching out through the Slate Community Forum.

With the big picture sketched out, we can turn our thoughts to what makes a successful integration.

Integration Considerations

Ultimately, the goal of any integration is to design a reliable, stable, supportable, and sustainable process for your institution. What does that mean exactly?

  • Reliability: The integration process can be more reliable when more of it lives in the Slate infrastructure, so using the Technolutions SFTP servers is typically preferred.

  • Stability: The integration process should run without constant human intervention. In other words, you should be able to “set it and forget it.” It should not need to change with any frequency. 

  • Supportability and sustainability: The integration process should minimize the use of custom SQL, and value translations should be stored centrally in Slate via prompt exports or translation codes. This enables those closest to the process (like operations staff) to manage the periodic changes which may be necessary because of new terms, programs, or majors.

Student Success Considerations

Slate can operate as the central repository for data generated in various systems across campus.  Integration with campus systems (such as SIS, LMS, and Financial Aid System) is a critical step to import the data points into Slate, and set the frequency of uploads (such as weekly, nightly, and twice a day).  Structuring a sustainable data integration ensures that the most up-to-date information is available to your users when they need it. 

Data points that may not change often, such as Class Standing or Expected Year of Graduation, may be better suited for a data exchange that happens less frequently.  Conversely, data points such as updates to class registration during the drop/add period should be included in exchanges with similar frequency (for example, overnight).

A majority of institutions will use SFTP processes to transfer files, however, RESTful Web Services are also available.  Prior to set up, this may be a good time to have a strategic conversation with a member from the IT staff in order to best align this process.

Ultimately, the goal is to design a process that is reliable, stable, supportable, and sustainable for your institution.

  • Reliable - The integration can be more reliable when it lives within the Slate infrastructure, and is exported to our SFTP servers.

  • Stable - The integration should run without constant human intervention - in other words, you should be able to “set it and forget it.” It should not need to change with any frequency and code and value translations should not live within the query itself.

  • Supportable - The integration should minimize the use of custom SQL wherever possible.

  • Sustainable - The integration should host the value translation inside Slate, but outside of the query, enabling the people closest to the process (for example, student success staff) to manage the periodic changes that may be necessary because of new data objects.

Tip

For information specific to your SIS or external system, peer institutions may be your best source of information. We recommend reaching out through the Slate Community Forum.

Best Practices

While Slate provides incredible flexibility in how data is manipulated, we strongly encourage sticking to these best practices to ensure the integration goals can be met.

  • Use batched files: While Slate provides the ability to use web service calls, if it's possible to send or receive data in batches, it will always be more efficient.

  • Send files overnight (2:00 AM-4:00 AM Eastern time): Slate databases have limited resources and are optimized for the speed and efficiency of the user interface. Schedule data exports and imports overnight when users are not actively interacting with the database.

  • Use the Technolutions SFTP server: Send data into Slate using the /incoming folder and export data into the /outgoing folder. Using the Technolutions server allows us to provide better performance.

Pitfalls to Avoid

Your institution can choose to deviate from the best practices when there is a business need. However, some processes are explicitly discouraged as part of data integrations. The items in this list are against the intention of these features and cannot be supported by Technolutions staff:

  • Don't automate the use of "Force Process Pickup" or "Force Process Import": These tools are provided to simplify the initial setup of integrations. When testing, it's much faster to make minor tweaks and force the processes. However, these are not intended to be used consistently, and doing so can have negative performance impacts.

  • Don't use direct SQL access as part of an integration: The direct SQL interfaces are provided on an "as is" basis and should be used for ad-hoc reporting only. Because SQL queries built and run outside of Slate cannot be tracked or managed, even minor changes to database schemas may break some of those scripts. Direct SQL access for an organization may be discontinued at any time at the sole discretion of Technolutions if it impacts database or server performance. For additional information, refer to the article on Direct SQL Access.

 

Example: Integrating with an SIS

A common type of integration is between Slate and a campus SIS. Typically, applicant and financial aid data is exchanged via a transfer of flat files on a scheduled basis through an SFTP server, where the institution can dictate the specifications for the data exchange and where the value and code translations (for example, those for country codes, major codes, or term codes) happen in Slate.

The data points from Slate to the campus system typically include student biographic and demographic data, as well as key application components that are necessary outside of admissions (for example, entry term, admission plan, or admission decision). A Slate ID is also sent along with a placeholder for the Campus or Institutional ID.

A return feed is then provided, and the Slate ID is included with the matched or newly created record's Institutional ID. Subsequent data feeds from Slate into the external system, then include that identifier for direct matching.

 

Does Slate have an API?

There is no "preset" Slate API. Instead, Slate allows you to customize your import and export configurations specifically for your business process or integration goals:

  • On import, Source Formats define the incoming data format and map the incoming values into Slate fields. You often do not need to modify the files or web service calls from external systems.

  • Queries enable you to choose the relevant data fields and define the export format on export. You can essentially build an API based on the requirements of your external system.

In this way, you can import your data from various sources, and your exports can conform to the formats your external systems need. Watch our Data Integrations Overview Demonstration for more information.

 

Next Steps

After reviewing the overall goals, best practices, and pitfalls to avoid, you're ready to begin building your integration. 

  1. The Importing Data Overview article outlines how data imports work in Slate and describes each import method.

  2. The Upload Dataset article describes how data is mapped and imported once it arrives.

  3. The Exporting Data Overview article describes using the Query tool to format and schedule data exports.


Was this article helpful?