Learn about the fields used to create an Amazon S3 connection with ThoughtSpot DataFlow.
Here is a list of the fields for an Amazon S3 connection in ThoughtSpot DataFlow. You need specific information to establish a seamless and secure connection.
Connection properties
- Connection name
- Name your connection.
- Mandatory field.
- Example:
AmazonS3Connection - Connection type
- Choose the Amazon S3 connection type.
- Mandatory field.
- Example:
Amnazon S3 - Amazon S3 URL
- Specify the Amazon S3 hostname link.
- Mandatory field.
- Example:
https://example1-amazonaws.com - Bucket
- Specify the bucket.
An Amazon S3 bucket name is globally unique, and the namespace is shared by all AWS accounts. - Mandatory field.
- Example:
bucket - Access key
- Specify the access key ID generated when creating AWS security credentials.
- Mandatory field.
- Example:
access key - Secret key
- Specify the secrete access key generated when creating AWS security credentials.
- Mandatory field.
- Example:
ABCDEFGH245HIJK
Sync properties
- File name
- Specify the name of the file.
- Mandatory field.
- Example:
PRODUCT.csv - Valid Values:
Any string literal - Default:
File name for sync. - Other notes:
To specify wildcard pattern, use*
. - Enable archive on success
- Specify if must archive file after operation succeeds.
- Optional field.
- Example:
No - Valid Values:
Yes, No - Default:
No - Delete file on success
- Specify if must delete file after operation succeeds.
- Optional field.
- Example:
No - Valid Values:
Yes, No - Default:
No - Column delimiter
- Specify the column delimiter character.
- Mandatory field.
- Example:
, - Valid Values:
Any printable ASCII character or decimal value for ASCII character - Default:
1 - Skip header rows
- Skip the specified number of header rows when loading data.
- Optional field.
- Example:
5 - Valid Values:
Any numeric value - Default:
0 - Compression
- Specify if the file has compression, and type of compresssion.
- Mandatory field.
- Example:
gzip - Valid Values:
None, gzip - Default:
None - Row delimiter
- Specify the end of the row character in the extracted data.
- Optional field.
- Example:
\\n - Valid Values:
Any printable ASCII character - Default:
\\n (new line character) - Enclosing character
- Specify if text columns in the source data use enclosing quotation marks, and their type.
- Optional field.
- Example:
Single - Valid Values:
Single, Double, Empty - Default:
None - Escape character
- Specify the escape character if using a text qualifier in the source data.
- Optional field.
- Example:
\\ - Valid Values:
Any ASCII character - Default:
None - Null value
- Specifies the string literal indicates the null value for a column. During the data load, the column value matching this string will be loaded as null in the target.
- Optional field.
- Example:
NULL - Valid Values:
Any string literal - Default:
NULL - Date style
- Specifies how to interpret the date format.
- Optional field.
- Example:
YMD - Valid Values:
YMD
,MDY
,DMY
,DMONY
,MONDY
,Y2MD
,MDY2
,DMY2
,DMONY2
, andMONDY2
- Default:
YMD
- Other notes:
MDY
: 2-digit month, 2-digit day, 4-digit yearDMY
: 2-digit month, 2-digit day, 4-digit yearDMONY
: 2-digit day, 3-character month name, 4-digit yearMONDY
: 3-character month name, 2-digit day, 4-digit yearY2MD
: 2-digit year, 2-digit month, 2-digit dayMDY2
: 2-digit month, 2-digit day, 2-digit yearDMY2
: 2-digit day, 2-digit month, 2-digit yearDMONY2
: 2-digit day, 3-character month name, 2-digit yearMONDY2
: 3-character month name, 2-digit day, 2-digit year - Date delimiter
- Specifies the separator used in the date format.
- Optional field.
- Example:
- - Valid Values:
Any printable ASCII character - Default:
- - Time style
- Specifies the format of the time portion in the data.
- Optional field.
- Example:
24 hour - Valid Values:
12 Hour, 24 Hour - Default:
24 Hour - Time delimiter
- Specifies the character used as separate the time components.
- Optional field.
- Example:
: - Valid Values:
Any printable ASCII character - Default:
: - Skip trailer rows
- Skip the number of trailer rows specified while loading the data.
- Optional field.
- Example:
5 - Valid Values:
Any numeric value - Default:
0 - TS load options
- Specifies the parameters passed with the
tsload
command, in addition to the commands already included by the application. The format for these parameters is:--<param_1_name> <optional_param_1_value>
--<param_2_name> <optional_param_2_value>
- Optional field.
- Example:
--max_ignored_rows 0 - Valid Values:
--null_value ""
--escape_character ""
--max_ignored_rows 0
- Default:
--max_ignored_rows 0
- Reference:
tsload flag reference - Boolean representation
- Specifies the representation of data in the boolean field
- Optional field.
- Example:
true_false - Valid Values:
true_false, T_F, 1_0, T_NULL - Default:
true_false