Learn about Syncers

Syncers allow CS Tools to interact with popular data storage formats without having to know the details of how to do so. Think of it as a way to plug and play different places to store your data.

We'll define for CS Tools which data storage we're using, and then pass that along with the rest of our command.

Formats that CS Tools currently supports

CSV
SQLite
ThoughtSpot Falcon
Snowflake
Google BigQuery
Google Sheets

With Archiver, we can supply a Syncer by passing the --syncer option.

Along with the --dry-run option, this will allow to inspect the Answers and Liveboards that would be marked for deletion, all without ever affecting the platform we're targeting.

Set up the CSV Syncer

Syncer Syntax

csv :// /filepath/to/your/configuration.toml

It's broken into 3 parts.. the protocol, a separator, and the definition.

The syntax for defining a Syncer configuration (called a "definition file") is like a local URL which defines all of the information a Syncer needs to be able to operate properly. All Syncers can take their arguments either in the form of a definition file, or directly on the command line.

To keep things simple, let's define a configuration for the CSV Syncer. This will allow us to export data that tools produce directly to a directory on your local machine. You can find all the configuration settings for syncers in their section of the documentation.

The CSV syncer doesn't take many arguments, all that's required is the directory location to save the data to.

Command FormatFile Format

_{Find the copy button to the right of the code block.}

--syncer csv://directory=.

_{Find the copy button to the right of the code block.}

definition.toml

[configuration]
directory = "."

This will save one or more CSVs in the current direction. The . means the current working directory from where you are running the CS Tools command.

Specifying the Syncer

We've created a way for CS Tools to export data to CSV, but how do we use it? Remember, Archiver has an option called --syncer protocol://DEFINITION.toml.

All Syncers in CS Tools take their arguments a standard format, called a protocol. Additionally, each Syncer can be customized using their own set of parameters. For example, a CSV Syncer and a Snowflake Syncer require different setup information to be used.

Whenever you need to specify a Syncer, you'll do so by providing the protocol name followed by the location of your syncer definition file or its arguments inline, separated by the three characters ://.

You can find the full details for each Syncer definition, as well as their protocol name in our documentation.

Tip

Since Syncer definitions are just files, they can be referenced across multiple configurations or even stored in a centrally located directory and be leveraged by multiple users interacting with CS Tools.

Try it out

Now that we've learned everything about how to use our CSV Syncer, let's try the command from the diagram above.

Windows Mac, Linux, ThoughtSpot cluster

_{Find the copy button to the right of the code block.}

cs_tools tools archiver identify `
--syncer csv://directory=. `
--dry-run

_{Find the copy button to the right of the code block.}

cs_tools tools archiver identify \
--syncer csv://directory=. \
--dry-run

If we look at the contents of the file that were dumped to our local directory, we'll find details about all the content that will be tagged and identified as inactive.

content_type	guid	name	created_at	modified_at	author	operation
answer	3d13b8e4-..	ThoughtSpo..	3 months ago	3 months ago	abc@thoughtspot.com	identify
answer	dc659672-..	Accounts P..	3 months ago	3 months ago	def@thoughtspot.com	identify
answer	7c4916bc-..	Customers ..	3 months ago	3 months ago	ghi@thoughtspot.com	identify
answer	cc623165-..	Cohort Ans..	3 months ago	3 months ago	jkl@thoughtspot.com	identify
...	...	...	...	...	...	...

Define a governance process

Now that we've got the tools to be able to maintain a clean ThoughtSpot platform as it grows, it's an excellent idea to regularly visit the state of metadata in our cluster.

In the final section, we'll explore ways to schedule the tools on both Windows and Unix platforms.