Learn about the fields used to create a Flat file connection with ThoughtSpot DataFlow.

Here is a list of the fields for a Flat file connection in ThoughtSpot DataFlow. You need specific information to establish a seamless and secure connection.

Connection properties

Connection name
Name your connection.
Mandatory field.
Example:
FilesConnection
Connection type
Choose the Files connection type.
Mandatory field.
Example:
Files
File location
Specify the base location of the file on the server.
Mandatory field.
Example:
/home/admin/dataflow/data
Files on remote location
Specify If the files on remote server.
Mandatory field.
Default:
No
Other notes:
These are under Advanced configuration
Protocol
Select the required remote server connection
Mandatory field.
For remote location files only.
Example:
FTP
Valid Values:
FTP, SFTP
Default:
FTP
Authentication type
Specify the authentication type for SFTP Protocol
Mandatory field.
For SFTP protocol only
Example:
Password
Valid Values:
Key, Password
Default:
Password
Host
Specify the Hostname or the IP address of the remote server
Mandatory field.
For remote location files only.
Example:
www.example.com
Port
Specify the Port to connect the remote server
Mandatory field.
For remote location files only.
Example:
1234
User
Specify the user to connect to remote server. This user must have data access privileges.
Mandatory field.
For remote location files only.
Example:
userdi
Password
Specify the password.
Optional field.
For remote location files only, when using password authentication.
Example:
pswrd234%!
Key file
Specify the key file and its fully qualified path.
Mandatory field.
For remote location files only, when using key authentication.
Example:
/app/key/sftp.key
Passphrase for key file
Specify the passphrase for the key file.
Optional field.
For remote location files only, when using key authentication.
Example:
!hello

Sync properties

File name
Specify name of the file.
Mandatory field.
Example:
CUSTOMER.csv
Valid Values:
Any string literal
Default:
The name of the file
Other notes:
To specify wildcard, use '*' character in the file name
Column delimiter
Specify the column delimiter character.
Mandatory field.
Example:
,
Valid Values:
Any printable ASCII character or decimal value for ASCII character
Default:
The delimiter specified in sync
Enable archive on success
Specify if system is to archive file after successful sync.
Optional field.
Example:
No
Valid Values:
Yes, No
Default:
No
Delete file on success
Specify if system to delete file after successful sync.
Optional field.
Example:
No
Valid Values:
Yes, No
Default:
No
Compression
Specify if file uses compression, and compression type.
Optional field.
Example:
gzip
Valid Values:
None, gzip
Default:
None
Skip header rows
Specify the number of header rows to skip when loading data.
Optional field.
Example:
5
Valid Values:
Any numeric value
Default:
0
Enclosing character
Specify if the text columns in the source data needs to be enclosed in quotes.
Optional field.
Example:
Single
Valid Values:
Single, Double, Empty
Default:
Empty
Escape character
Specify the escape character if using a text qualifier in the source data.
Optional field.
Example:
\\
Valid Values:
Any ASCII character
Default:
Empty
Null value
Specifies the string literal indicates the null value for a column. During the data load, the column value matching this string will be loaded as null in the target.
Optional field.
Example:
NULL
Valid Values:
Any string literal
Default:
NULL
Date style
Specifies how to interpret the date format
Optional field.
Example:
YMD
Valid Values:
YMD, MDY, DMY, DMONY, MONDY, Y2MD, MDY2, DMY2, DMONY2, and MONDY2
Default:
YMD
Other notes:
MDY: 2-digit month, 2-digit day, 4-digit year
DMY: 2-digit month, 2-digit day, 4-digit year
DMONY: 2-digit day, 3-character month name, 4-digit year
MONDY: 3-character month name, 2-digit day, 4-digit year
Y2MD: 2-digit year, 2-digit month, 2-digit day
MDY2: 2-digit month, 2-digit day, 2-digit year
DMY2: 2-digit day, 2-digit month, 2-digit year
DMONY2: 2-digit day, 3-character month name, 2-digit year
MONDY2: 3-character month name, 2-digit day, 2-digit year
Date delimiter
Specifies the separator used in the date format.
Optional field.
Example:
-
Valid Values:
Any printable ASCII character
Default:
-
Time style
Specifies the format of the time portion in the data.
Optional field.
Example:
24HOUR
Valid Values:
12 Hour, 24 Hour
Default:
24HOUR
Time delimiter
Specifies the character used as separate the time components.
Optional field.
Example:
:
Valid Values:
Any printable ASCII character
Default:
:
Statistics
Specify if the file source count needs to be displayed in Monitor statistics.
Optional field.
Example:
ON
Valid Values:
OFF
Default:
ON
Skip trailer rows
Skip trailer row while processing the data
Optional field.
Example:
5
Valid Values:
Any numeric value
Default:
0
Reader encoder
Specify the encoding format of the source system.
Optional field.
Example:
UFT-8
Valid Values:
ISO-8859-1, UTF-16, UTF-16BE, WINDOWS-1252
Default:
UFT-8
TS load options
Specifies the parameters passed with the tsload command, in addition to the commands already included by the application. The format for these parameters is:
--<param_1_name> <optional_param_1_value>
--<param_2_name> <optional_param_2_value>
Optional field.
Example:
--max_ignored_rows 0
Valid Values:
--user "dbuser"
--password "$DIWD"
--target_database "ditest"
--target_schema "falcon_schema"
Default:
--max_ignored_rows 0
Reference:
tsload flag reference
Boolean representation
Specifies the representation of data in the boolean field
Optional field.
Example:
true_false
Valid Values:
true_false, T_F, 1_0, T_NULL
Default:
true_false

Dataflow tips