Learn about the fields used to create a Flat file connection with ThoughtSpot DataFlow.
Here is a list of the fields for a Flat file connection in ThoughtSpot DataFlow. You need specific information to establish a seamless and secure connection.
Connection properties
- Connection name
- Name your connection.
- Mandatory field.
- Example:
FilesConnection - Connection type
- Choose the Files connection type.
- Mandatory field.
- Example:
Files - File location
- Specify the base location of the file on the server.
- Mandatory field.
- Example:
/home/admin/dataflow/data - Files on remote location
- Specify If the files on remote server.
- Mandatory field.
- Default:
No - Other notes:
These are under Advanced configuration - Protocol
- Select the required remote server connection
- Mandatory field.
For remote location files only. - Example:
FTP - Valid Values:
FTP, SFTP - Default:
FTP - Authentication type
- Specify the authentication type for SFTP Protocol
- Mandatory field.
For SFTP protocol only - Example:
Password - Valid Values:
Key, Password - Default:
Password - Host
- Specify the Hostname or the IP address of the remote server
- Mandatory field.
For remote location files only. - Example:
www.example.com - Port
- Specify the Port to connect the remote server
- Mandatory field.
For remote location files only. - Example:
1234 - User
- Specify the user to connect to remote server. This user must have data access privileges.
- Mandatory field.
For remote location files only. - Example:
userdi - Password
- Specify the password.
- Optional field.
For remote location files only, when using password authentication. - Example:
pswrd234%! - Key file
- Specify the key file and its fully qualified path.
- Mandatory field.
For remote location files only, when using key authentication. - Example:
/app/key/sftp.key - Passphrase for key file
- Specify the passphrase for the key file.
- Optional field.
For remote location files only, when using key authentication. - Example:
!hello
Sync properties
- File name
- Specify name of the file.
- Mandatory field.
- Example:
CUSTOMER.csv - Valid Values:
Any string literal - Default:
The name of the file - Other notes:
To specify wildcard, use '*' character in the file name - Column delimiter
- Specify the column delimiter character.
- Mandatory field.
- Example:
, - Valid Values:
Any printable ASCII character or decimal value for ASCII character - Default:
The delimiter specified in sync - Enable archive on success
- Specify if system is to archive file after successful sync.
- Optional field.
- Example:
No - Valid Values:
Yes, No - Default:
No - Delete file on success
- Specify if system to delete file after successful sync.
- Optional field.
- Example:
No - Valid Values:
Yes, No - Default:
No - Compression
- Specify if file uses compression, and compression type.
- Optional field.
- Example:
gzip - Valid Values:
None, gzip - Default:
None - Skip header rows
- Specify the number of header rows to skip when loading data.
- Optional field.
- Example:
5 - Valid Values:
Any numeric value - Default:
0 - Enclosing character
- Specify if the text columns in the source data needs to be enclosed in quotes.
- Optional field.
- Example:
Single - Valid Values:
Single, Double, Empty - Default:
Empty - Escape character
- Specify the escape character if using a text qualifier in the source data.
- Optional field.
- Example:
\\ - Valid Values:
Any ASCII character - Default:
Empty - Null value
- Specifies the string literal indicates the null value for a column. During the data load, the column value matching this string will be loaded as null in the target.
- Optional field.
- Example:
NULL - Valid Values:
Any string literal - Default:
NULL - Date style
- Specifies how to interpret the date format
- Optional field.
- Example:
YMD - Valid Values:
YMD
,MDY
,DMY
,DMONY
,MONDY
,Y2MD
,MDY2
,DMY2
,DMONY2
, andMONDY2
- Default:
YMD
- Other notes:
MDY
: 2-digit month, 2-digit day, 4-digit yearDMY
: 2-digit month, 2-digit day, 4-digit yearDMONY
: 2-digit day, 3-character month name, 4-digit yearMONDY
: 3-character month name, 2-digit day, 4-digit yearY2MD
: 2-digit year, 2-digit month, 2-digit dayMDY2
: 2-digit month, 2-digit day, 2-digit yearDMY2
: 2-digit day, 2-digit month, 2-digit yearDMONY2
: 2-digit day, 3-character month name, 2-digit yearMONDY2
: 3-character month name, 2-digit day, 2-digit year - Date delimiter
- Specifies the separator used in the date format.
- Optional field.
- Example:
- - Valid Values:
Any printable ASCII character - Default:
- - Time style
- Specifies the format of the time portion in the data.
- Optional field.
- Example:
24HOUR - Valid Values:
12 Hour, 24 Hour - Default:
24HOUR - Time delimiter
- Specifies the character used as separate the time components.
- Optional field.
- Example:
: - Valid Values:
Any printable ASCII character - Default:
: - Statistics
- Specify if the file source count needs to be displayed in Monitor statistics.
- Optional field.
- Example:
ON - Valid Values:
OFF - Default:
ON - Skip trailer rows
- Skip trailer row while processing the data
- Optional field.
- Example:
5 - Valid Values:
Any numeric value - Default:
0 - Reader encoder
- Specify the encoding format of the source system.
- Optional field.
- Example:
UFT-8 - Valid Values:
ISO-8859-1, UTF-16, UTF-16BE, WINDOWS-1252 - Default:
UFT-8 - TS load options
- Specifies the parameters passed with the
tsload
command, in addition to the commands already included by the application. The format for these parameters is:--<param_1_name> <optional_param_1_value>
--<param_2_name> <optional_param_2_value>
- Optional field.
- Example:
--max_ignored_rows 0
- Valid Values:
--user "dbuser"
--password "$DIWD"
--target_database "ditest"
--target_schema "falcon_schema"
- Default:
--max_ignored_rows 0
- Reference:
tsload flag reference - Boolean representation
- Specifies the representation of data in the boolean field
- Optional field.
- Example:
true_false - Valid Values:
true_false, T_F, 1_0, T_NULL - Default:
true_false