The FTP API method of communicating with the server is passive and asynchronous, where two processes run independently of one another. A very specific protocol is used requiring two files. The Data file contains commands that must be written in the proper format in order to be processed. The Signal File is a blank file that indicates the Data File is ready to be processed.
Data File
The first file you must create and send is the data (.dat) file. It is a flat ASCII file comprising a header section and a body section of one or more lines. The file contains the instructions to be processed, such as subscription commands. The data file can be edited and viewed in a normal text editor.
In terms of naming conventions, you can use almost any file name as long as the extension is ".dat" to indicate it is a Data File. Do not use spaces or special characters in the file name. Characters to avoid include:
% * ~ \ / | [space]
File names like ‘data0001.dat’, ‘test.dat’, and ‘subscription_data_oct_2002.dat’ are all valid. However, in order to streamline troubleshooting should an error occur, we recommend using a combination of your realm name, the current date, and the type of commands in the file (if all of one type). For example:
- myrealm_201201_subscriptions.dat
- myrealm_20120501_campaign.dat
Header
The header of the data file consists of authentication information and options to control default behavior or perform tasks prior to importing contact information. The various commands that may appear in the header include (with no spaces between command and value):
|
Name |
Description |
Command |
|
Realm |
Authentication is based on realm and password. The realm name is determined when your account is created and can be seen in the Welcome message you receive with your platform login credentials. The realm must be defined first. |
realm=[realm_name] |
|
Password |
The password, also known as an authentication code, must be defined after the realm. To generate the password, login to the platform and then go to CUSTOMER CENTER > API MANAGEMENT > SETUP API. If you do not see a password in bold, red text, click Generate Password. Changing the code invalidates any files using the old code. If the authentication code already exists and you regenerate it, make sure to notify everyone using any of the APIs. |
password=[password] |
|
API Client* |
Client authentication provides another level of user access and security. This is optional instead of using the Default API Client. API Client is available to better manage clients' access and usage of APIs. Each realm has a default API Client which requires only the Realm and Password above. When using one of the four possible API Clients, the API Client Name and Client Authentication must also be included in the header. To define an API Client, login to the platform and then go to CUSTOMER CENTER > API MANAGEMENT > SETUP API. You must generate the Authorization Code, then generate the API Password If using an API client, define it after the password. **Added in v15.02** |
api_client=[client_name] |
|
API Client Authorization |
Changing the API Client Authorization code invalidates any files using the old code. If the authorization code already exists and you regenerate it, make sure to notify everyone using the API Client in any of the APIs. This is required when using any API Client other than Default. |
client_auth=[client_auth] |
|
Confirmation Email |
You can specify who will receive a confirmation email after the data file is processed. This message details the results of the process, such as any errors that occurred. To specify more than one email address, separate each using commas, semi-colons, or spaces. This field must not exceed 255 characters (not including the confirmation_email= characters). |
confirmation_email= [email_address] |
|
Verbose |
If you have specified a confirmation email address, you can set verbose to true to see additional results, such as the number of distinct subscribe or unsubscribe events. By default, this flag is set to false. |
verbose=[0,false,1,true]
DEFAULT==false |
|
Description |
An optional description can be defined to help you recognize the purpose of a file. |
description=[text_string] |
|
Default List |
Instead of setting the list for each separate command, you can specify a default list id in the header to use for the entire process. Specifying a list id per contact will override this setting for that contact. The benefit is the list will be cached to decrease processing time. To obtain list ids, you can use the WebServices API or the web platform directly. For customers with child realms, the default list must belong to the parent realm (the realm specified during authentication), rather than a child realm. |
list_id=[list_id] |
|
Opt Outs |
Control the list level opt out behavior by modifying the opt out flag. By default, an unsubscribe command removes a contact from a list and generates an opt out record. This opt out record prevents the contact from re-subscribing to the specific list without a forced override. To prevent the opt out record from being created, set the flag to false. |
optout=[0,false,1,true]
DEFAULT==true |
|
Global Opt Outs |
To control opt out behavior on the realm level, you must use a combination of the opt out flag and the list_id flag. If set properly, when an unsubscribe command removes a contact from a list, a Global opt out record is also generated, removing the contact from all lists. This Global opt out record prevents the contact from re-subscribing to any list in the realm without a forced override. To generate a Global opt out, set the opt out flag to true, and then in the header, or in each Unsubscribe command, use list_id=0. |
optout=[0,false,1,true] list_id=0
DEFAULT==false |
|
Universal Opt Outs* |
The universal opt out is only available for Broadcaster users. To control opt out behavior on the broadcaster level, you must use a combination of the universal opt out flag and the list_id flag. If set properly, when an unsubscribe command removes a contact from a list a Universal opt out record is also generated, removing the contact from all lists in every realm on the system. This Universal opt out record prevents the contact from re-subscribing to any list in any realm. To generate a Universal opt out, set the flag to true, and then in the header, or in each Unsubscribe command, use list_id=0. By default, this flag is set to false. * NOTE: Universal Opt Out is only available on Broadcasters and must be enabled in the configuration file. Please contact Support if you would like to use this command. |
universal= [0,false,1,true] list_id=0
DEFAULT==false |
|
Force Optout |
If a Contact or Subscription specified in an Unsubscribe command does not exist, the command is ignored. However, if you want to create an opt out record for the email address, set the Force Optout flag to true. An unsubscribe request for an email address that is not subscribed to the list will create an Opt Out record. To generate a global Forced opt out, set the flag to true, and then in the header, or in each Unsubscribe command, use list_id=0. By default, this flag is set to false. |
force_optout= [0,false,1,true]
DEFAULT==false |
|
Force Subscribe |
The platform supports a sophisticated opt-out protection system that makes it difficult to override the opt-out wishes of contacts. However, there are cases where you may want to override such requests. By default, this flag is set to false. To override opt out rules, you can force subscriptions by setting the Force Subscribe flag to true. When an opt out record is found for a contact, the opt out record will be removed and the subscription created. |
force_sub= [0,false,1,true]
DEFAULT==false |
|
Drop Subscription |
If you need to drop all contacts from a list, define one or more list ids for the Drop Subscriptions command. All subscriptions for the defined lists will be deleted. Use commas to separate the list_ids. While this command will remove subscription records, no Contact data or tracking data will be deleted. |
drop_subscriptions= [list_ids] |
|
Allow Duplicates |
You may want to allow contacts to have multiple records using the same email address. This requires forcing "duplicate" Contact records, where the same email address can be related to more than one contact, as long as the first name is different. By default, this flag is set to false. In order to force duplicates, identify a unique first name and set this flag to true. Alternately, you can use the 'duplicate' field on a per contact basis. |
allow_duplicates= [0,false,1,true]
DEFAULT==false |
|
Field Positions |
You can specify the field positions for your subscribe commands in the header rather than on each line in the body. This allows you to define the data fields that you will be importing and the order for each contact, like the column headers of a table. The benefit is smaller file size and the header information is cached to cut down on processing time. In the commands below, the data can be defined with no additional labels, but a caret is still required: data=^values Field positions may be comma, tab, or pipe delimited and can be quoted if necessary. header=email,first,last,custom_gender,format header=email first last format header=email|first|last|custom_gender|format header="email","first","last" If you use quotes to separate your fields, and need to include double quotes in your imported values, use the backslash escape character: \" |
header= [field_names] |
|
Campaign Name |
If the List properties are set to Sticky Campaigns, you may define a Campaign Name in the header and all campaigns sent using the Send command will be associated to and reported in a single campaign. If the Campaign Name is also defined in a specific Send command, it will override the header definition for that single command. |
campaign_name= [campaign_name] |
|
Relational Tables |
Relational Tables tag is only required for Importing or Deleting data for Relational Data tables. By default, this flag is set to false (0). Set Relational to 1 for the commands relationalsave and relationaldelete. |
relational=[0,1] DEFAULT==0 |
|
Relational Bulk Import |
Relational Bulk Import tag is only required for Bulk Importing via FTP CSV batch files. By default, this flag is set to false (0). Set isrelational to 1 for CSV Relational data imports. **New in v13.04 |
isrelational=[0,1] DEFAULT==0 |
|
Relational Contact |
When working with Relational Contacts batch process, the Is Relational Contact flag must be set. If set to 1, the import processor will complete import of the contacts in the file before processing any other Relational Bulk Imports. **New in v15.15 |
isrelationalsubscriber=[0,1] DEFAULT==0 |
Body
The body of the Data File includes commands interpreted by the server daemon process. All commands must be contained on a single line, and each new command must appear on a new line. The caret (^) character is used to represent new virtual lines.
The format for each command line follows URL-like encoding standards. Commands can take several arguments, depending on the type of command.
For example, to add a contact to a particular list, the command could be:
cmd=subscribe&list_id=5&data=email^mail@domain.com
In the following example, several subscribe command lines include different data fields to import for each contact:
cmd=sub&list_id=5&data=email,zip^jane@domainA.com,98122 cmd=sub&list_id=5&data=email^mary@domainB.com cmd=sub&list_id=5&data=email,custom_age^steve@domainC.com,35
If the data fields to include are already defined in the header, the field names can be removed from the data argument leaving a command as follows:
header=email,custom_age
cmd=sub&list_id=5&data=^steve@domainC.com,35
A single-line command can handle multiple contacts. In the example below, the single-line subscribe command is displayed on several lines. In your Data File, all data for a single command must be contained on a single line.
cmd=sub&list_id=5&data=email,custom_age,first,zip ^jane@domainA.com,25,Jane,98122 ^mary@domainB.com,25,Mary,98075 ^steve@domainC.com,35,Steve,99011
We recommend using batches of 100 or fewer single-line commands. The data is parsed based upon the presence of the caret (^), which is an in-memory method. Having thousands of records on one line may cause out-of-memory exceptions. Another option is to use the Field Positions command in the header to identify the fields to import and then use one command per line.
Data Fields
The Data argument requires additional fields to identify individual contacts or email recipients. For example, with a Subscribe command, the data argument represents a Contact to import and subscribe to a list. Each data argument requires fields to identify a contact, such as email and first. All date fields require a 2 digit month, 2 digit date, and four digit year, such as 03/27/2019, 2019-03-27.
The Contact Data fields include:
|
|
|
|
|
|
|
|
|
|
|
|
|
Four Contact Data fields are used for special values:
|
duplicate |
To force duplicates, a unique first name must be identified, and the duplicate field must be set to 1. An alternative is to use the allow_duplicates flag in the header instead. |
|
format_number |
To specify the format of the email messages your contact will receive, you can set the format_number in the Data fields instead of as an argument. 1 = plain text (DEFAULT) 2 = HTML 99 = Multi-part MIME |
|
format_text |
To specify the format of the email messages your contact will receive, you can set the format_text in the Data fields instead of as an argument. plain = plain text (DEFAULT) HTML = HTML mime = Multi-part MIME multi = Multi-part MIME |
|
Contact's unique identification string. Only available on some systems. This field is required when enabled in the realm settings. |
Custom Fields
Custom fields can be specified by appending the string “custom_” to the field name. For example, if your realm has a custom field named “age”, specify it as a data field using “custom_age”. If you are attempting to import custom data and the data remains blank in the database, check that the custom field has been properly identified in the data file preceded by “custom_”. Please note that custom fields are case sensitive and long names or names with spaces are prone to error.
Relational Data Fields
Relational Data fields are specified by preceding the field name with the name of the table in which the field exists. For example, if your realm has a Table named “purchase” and a field within the table is named “itemSKU”, specify it as a relational data field using “purchase.itemSKU”.
Signal File
The second required file is a signal (.sig) file, an empty file to indicate that your data file is ready for processing. As soon as it the signal file is detected, the processing daemon starts working on your data file.
The important part of a signal file is its name. The contents of the file are not used, and the file should be empty. The file name must exactly match the name of the data file, outside of the file extension. If the signal file name is even slightly different, the data file will not be picked up to process.
After you have created and uploaded a data file, create a file with the same name and the extension ".sig" to indicate it is a signal file. Upload the signal file to the server to indicate that your files are ready to be processed.
myrealm_date_import.dat -----> myrealm_date_import.sig