Getting Started

What is CS Tools?

CS Tools is a suite of tools written in Python which connects to your ThoughtSpot cluster through the V2.0 REST APIs.

They are designed to complement ThoughtSpot native functionality with advanced automation and management capabilities.

Important

While CS Tools is maintained by members of the ThoughtSpot team, they are TOTALLY FREE!

ThoughtSpot Support Team will be unable to help you resolve any issues.

Instead, Feature Requests and Support are handled {> on Github Discussions <}

~cs~tools tools --help

Install

Follow the steps below to get CS Tools installed on your platform.

_{CS Tools requires at least python 3.9 to install!}

Windows Mac, Linux ThoughtSpot cluster Serverless

Open up Windows Terminal or Powershell.

_{Find the copy button to the right of the code block.}

powershell -ExecutionPolicy ByPass -c "IRM https://thoughtspot.github.io/cs_tools/install.py | python - --reinstall"

^{* changing the execution policy allows running a script from the internet}

To open Powershell

Press the Win key, type Powershell, then hit Enter

Open up a new Terminal window.

_{Find the copy button to the right of the code block.}

curl -LsSf https://thoughtspot.github.io/cs_tools/install.py | python - --reinstall

command not found: python3

If you see this error in your terminal, try using python instead of python3 above.

Proceed with caution!

CS Tools can run pretty much anywhere! We strongly recommend against running this on your production ThoughtSpot software cluster.

_{Find the copy button to the right of the code block.}

curl -LsSf https://thoughtspot.github.io/cs_tools/install.py | python - --reinstall

command not found: python3

If you see this error in your terminal, try using python instead of python3 above.

If you want to run CS Tools from a serverless environment, skip the install script and instead install the python package directly.

GitHub Actions GitLab CI/CD Pipelines Azure Pipelines Docker Python Notebook

actions-workflow.yaml

name: Extract Metadata with CS Tools.

on:
  workflow_dispatch:
    inputs:
      cs_tools_version:
        description: "The CS Tools version to target for a manual run."
        required: false
        type: string

  schedule:
    # Runs every day at 5:20 AM UTC
    - cron: "20 5 * * *"

jobs:
  extract_data_from_thoughtspot:
    runs-on: ubuntu-latest

    env:
      # CS TOOLS IS COMMAND LINE LIBRARY WRAPPING TS APIS
      # https://thoughtspot.github.io/cs_tools/
      CS_TOOLS_VERSION: ${{ github.event_name == 'workflow_dispatch' && inputs.cs_tools_version || 'v1.6.0' }}
      CS_TOOLS_THOUGHTSPOT__URL: ${{ secrets.THOUGHTSPOT_URL }}
      CS_TOOLS_THOUGHTSPOT__USERNAME: ${{ secrets.THOUGHTSPOT_USERNAME }}
      CS_TOOLS_THOUGHTSPOT__SECRET_KEY: ${{ secrets.THOUGHTSPOT_SECRET_KEY }}

      # COMMON PARAMETERS FOR THE SNOWFLAKE SYNCER
      # https://thoughtspot.github.io/cs_tools/syncer/snowflake/
      DECLARATIVE_SYNCER_SYNTAX:
        "\
        account_name=${{ secrets.SNOWFLAKE_ACCOUNT }}\
        &username=${{ secrets.SNOWFLAKE_USERNAME }}\
        &secret=${{ secrets.SNOWFLAKE_PASSWORD }}\
        &warehouse=${{ secrets.SNOWFLAKE_WAREHOUSE }}\
        &role=${{ secrets.SNOWFLAKE_ROLE }}\
        &database=${{ secrets.SNOWFLAKE_DATABASE }}\
        &schema=${{ secrets.SNOWFLAKE_SCHEMA }}\
        &authentication=basic\
        "

    steps:
      # SETUP PYTHON.
      - name: Set up Python
        uses: actions/setup-python@v5

      # UPDATE PIP.
      - name: Ensure pip is up to date.
        run: python -m pip install --upgrade pip

      # INSTALL A SPECIFIC VERSION OF cs_tools.
      - name: Install a pinned version of CS Tools
        run: python -m pip install "cs_tools[cli] @ https://github.com/thoughtspot/cs_tools/archive/v${{ env.CS_TOOLS_VERSION }}.zip"

      # ENSURE SYNCER DEPENDENCIES ARE INSTALLED.
      #   found in: https://github.com/thoughtspot/cs_tools/blob/master/sync/<dialect>/MANIFEST.json
      - name: Install a pinned version of CS Tools
        run: python -m pip install "snowflake-sqlalchemy >= 1.6.1"

      # RUNS THE searchable metadata COMMAND.
      # https://thoughtspot.github.io/cs_tools/tools/searchable
      #
      # THE CLI OPTION  --config ENV:  TELLS CS TOOLS TO PULL THE INFORMATION FROM ENVIRONMENT VARIABLES.
      - name: Refresh Metadata from ThoughtSpot
        run: |
          cs_tools tools
          searchable metadata
          --syncer 'snowflake://${{ env.DECLARATIVE_SYNCER_SYNTAX }}&load_strategy=TRUNCATE'
          --config ENV:

.gitlab-ci.yml

variables:
  # CS TOOLS IS COMMAND LINE LIBRARY WRAPPING TS APIS
  # https://thoughtspot.github.io/cs_tools/
  CS_TOOLS_VERSION: "v1.6.0"
  CS_TOOLS_THOUGHTSPOT__URL: ${THOUGHTSPOT_URL}
  CS_TOOLS_THOUGHTSPOT__USERNAME: ${THOUGHTSPOT_USERNAME}
  CS_TOOLS_THOUGHTSPOT__SECRET_KEY: ${THOUGHTSPOT_SECRET_KEY}

  # COMMON PARAMETERS FOR THE SNOWFLAKE SYNCER
  # https://thoughtspot.github.io/cs_tools/syncer/snowflake/
  DECLARATIVE_SYNCER_SYNTAX:
    "\
    account_name=${SNOWFLAKE_ACCOUNT}\
    &username=${SNOWFLAKE_USERNAME}\
    &secret=${SNOWFLAKE_PASSWORD}\
    &warehouse=${SNOWFLAKE_WAREHOUSE}\
    &role=${SNOWFLAKE_ROLE}\
    &database=${SNOWFLAKE_DATABASE}\
    &schema=${SNOWFLAKE_SCHEMA}\
    &authentication=basic\
    "

# WORKFLOW CAN BE TRIGGERED MANUALLY OR BY SCHEDULE
workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
    - if: $CI_PIPELINE_SOURCE == "web"

extract_data_from_thoughtspot:
  image: python:3.12-slim
  script:
    # UPDATE PIP.
    - python -m pip install --upgrade pip

    # INSTALL A SPECIFIC VERSION OF cs_tools.
    - python -m pip install "cs_tools[cli] @ https://github.com/thoughtspot/cs_tools/archive/v${CS_TOOLS_VERSION}.zip"

    # ENSURE SYNCER DEPENDENCIES ARE INSTALLED.
    #   found in: https://github.com/thoughtspot/cs_tools/blob/master/sync/<dialect>/MANIFEST.json
    - python -m pip install "snowflake-sqlalchemy >= 1.6.1"

    # RUNS THE searchable metadata COMMAND.
    # https://thoughtspot.github.io/cs_tools/tools/searchable
    #
    # THE CLI OPTION  --config ENV:  TELLS CS TOOLS TO PULL THE INFORMATION FROM ENVIRONMENT VARIABLES.
    - >-
    cs_tools tools
    searchable metadata
    --syncer "snowflake://${DECLARATIVE_SYNCER_SYNTAX}&load_strategy=UPSERT"
    --config ENV:

# RUNS EVERY DAY AT 5:20 AM UTC
.schedule:
  cron: "20 5 * * *"

azure-pipelines.yml

variables:

  # CS TOOLS IS COMMAND LINE LIBRARY WRAPPING TS APIS
  # https://thoughtspot.github.io/cs_tools/
  CS_TOOLS_VERSION: '1.6.0'
  CS_TOOLS_THOUGHTSPOT__URL: $(THOUGHTSPOT_URL)
  CS_TOOLS_THOUGHTSPOT__USERNAME: $(THOUGHTSPOT_USERNAME)
  CS_TOOLS_THOUGHTSPOT__SECRET_KEY: $(THOUGHTSPOT_SECRET_KEY)

  # NEEDED TO TELL CS TOOLS ABOUT THE ENVIRONMENT (Azure doesn't set this by default~)
  CI: true

  # COMMON PARAMETERS FOR THE SNOWFLAKE SYNCER
  # https://thoughtspot.github.io/cs_tools/syncer/snowflake/
  DECLARATIVE_SYNCER_SYNTAX: >-
    account_name=$(SNOWFLAKE_ACCOUNT)
    &username=$(SNOWFLAKE_USERNAME)
    &secret=$(SNOWFLAKE_PASSWORD)
    &warehouse=$(SNOWFLAKE_WAREHOUSE)
    &role=$(SNOWFLAKE_ROLE)
    &database=$(SNOWFLAKE_DATABASE)
    &schema=$(SNOWFLAKE_SCHEMA)
    &authentication=basic

schedules:
# Runs every day at 5:20 AM UTC
- cron: '20 5 * * *'
  displayName: Daily metadata sync
  branches:
    include:
    - main
  always: true

# DEFINE MANUAL TRIGGER CAPABILITY
trigger: none # DISABLE CONTINUOUS INTEGRATION TRIGGER
pr: none      # DISABLE PULL REQUEST TRIGGER

# ALLOW MANUAL TRIGGER FROM AZURE DEVOPS UI
resources:
  repositories:
  - repository: self

pool:
  vmImage: 'ubuntu-latest'

jobs:
- job: extract_data_from_thoughtspot
  displayName: 'Extract Data from ThoughtSpot'
  steps:
  - task: UsePythonVersion@0
    inputs:
      versionSpec: '3.12'
      addToPath: true

  - script: |
      # UPDATE PIP
      python -m pip install --upgrade pip

      # INSTALL A SPECIFIC VERSION OF cs_tools
      python -m pip install "cs_tools[cli] @ https://github.com/thoughtspot/cs_tools/archive/v$(CS_TOOLS_VERSION).zip"

      # ENSURE SYNCER DEPENDENCIES ARE INSTALLED
      python -m pip install "snowflake-sqlalchemy >= 1.6.1"

      # RUNS THE searchable metadata COMMAND.
      # https://thoughtspot.github.io/cs_tools/tools/searchable
      #
      # THE CLI OPTION  --config ENV:  TELLS CS TOOLS TO PULL THE INFORMATION FROM ENVIRONMENT VARIABLES.
      cs_tools tools searchable metadata --syncer "snowflake://$(DECLARATIVE_SYNCER_SYNTAX)&load_strategy=UPSERT" --config ENV:
    displayName: 'Extract Metadata with CS Tools.'

Dockerfile

# CS TOOLS IS COMMAND LINE LIBRARY WRAPPING TS APIS
# https://thoughtspot.github.io/cs_tools/
docker build --build-arg CS_TOOLS_VERSION=v1.6.0 -t cs-tools-image:1.0.0 .

# COMMON PARAMETERS FOR THE SNOWFLAKE SYNCER
# https://thoughtspot.github.io/cs_tools/syncer/snowflake/
docker run `
  -e CS_TOOLS_THOUGHTSPOT__URL="" `
  -e CS_TOOLS_THOUGHTSPOT_USERNAME="" `
  -e CS_TOOLS_THOUGHTSPOT_SECRET_KEY="" `
  -e DECLARATIVE_SYNCER_SYNTAX="" `
  cs-tools-image:1.0.0

# BASE DISTRIBUTION MUST INCLUDE PYTHON + PIP.
FROM python:3.12-slim

# METADATA
LABEL version="1.0.0"
LABEL description="ThoughtSpot CS Tools container"
LABEL maintainer="https://github.com/thoughtspot/cs_tools/discussions"

# CS TOOLS IS COMMAND LINE LIBRARY WRAPPING TS APIS
# https://thoughtspot.github.io/cs_tools/
ARG CS_TOOLS_VERSION="v1.6.0"
ENV CS_TOOLS_VERSION=${CS_TOOLS_VERSION}

# AVOID PROMPTS FROM APT
ENV DEBIAN_FRONTEND=noninteractive

# INSTALL GIT (NEEDED FOR PIP INSTALL FROM GITHUB)
RUN apt-get update && apt-get install -y \
    git \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# UPDATE PIP.
RUN python3 -m pip install --upgrade pip

# INSTALL A SPECIFIC VERSION OF cs_tools.
RUN python3 -m pip install "cs_tools[cli] @ https://github.com/thoughtspot/cs_tools/archive/${CS_TOOLS_VERSION}.zip"

# SET THE WORKING DIRECTORY
WORKDIR /app

# RUNS THE searchable metadata COMMAND.
# https://thoughtspot.github.io/cs_tools/tools/searchable
#
# THE CLI OPTION  --config ENV:  TELLS CS TOOLS TO PULL THE INFORMATION FROM ENVIRONMENT VARIABLES.
CMD ["sh", "-c", "cs_tools tools searchable metadata --syncer \"snowflake://${DECLARATIVE_SYNCER_SYNTAX}&load_strategy=UPSERT\" --config ENV:"]

Untitled.ipynb

### Install CS Tools

Change the `v1.6.0` for any version or branch you want, but the latest version has all the bells as whistles.

!pip install "cs_tools[cli] @ https://github.com/thoughtspot/cs_tools/archive/v1.6.0.zip"

from cs_tools.programmatic import CSToolInfo
import os

### While [CS Tools](https://thoughtspot.github.io/cs_tools/) is a CLI, we can still use it programmatically.

The simplest way to specify your config is to set environment variables.

os.environ["CS_TOOLS_THOUGHTSPOT__URL"] = ...
os.environ["CS_TOOLS_THOUGHTSPOT__USERNAME"] = ...
os.environ["CS_TOOLS_THOUGHTSPOT__SECRET_KEY"] = ...
# os.environ["CS_TOOLS_THOUGHTSPOT__PASSWORD"] = ...
# os.environ["CS_TOOLS_THOUGHTSPOT__TOKEN"] = ...

### Common Parameters for the [Snowflake Syncer](https://thoughtspot.github.io/cs_tools/syncer/snowflake/)

__Load Strategies__
- `UPSERT` _update or insert incoming rows_
- `TRUNCATE` _first delete all rows, then insert incoming rows_
- `APPEND` _only insert, without regarding primary key_

UPSERT_SYNCER_CONF = {
    "account_name": ...,
    "username": ...,
    "secret": ...,
    "warehouse": ...,
    "role": ...,
    "database": ...,
    "schema": ...,
    "authentication": "basic",
    "load_strategy": "UPSERT",
}

syncer_def = "&".join(f"{k}={v}" for k, v in UPSERT_SYNCER_CONF.items())

### Runs the [`searchable metadata` command](https://thoughtspot.github.io/cs_tools/generated/cli/reference.html#metadata_1)

The option `--config ENV:` tells CS Tools to fetch config information from the environment variables we set earlier.

searchable = CSToolInfo.fetch_builtin(name="searchable")
searchable.invoke("metadata", "--syncer", f"snowflake://{syncer_def}", "--config", "ENV:")

Configure

In order to use the APIs, ThoughtSpot enforces authenticated sessions. All of the standard ThoughtSpot security controls apply.

CS Tools offers support for..

Basic Auth Trusted Auth Bearer Token

This is your standard combination of username and password.

Basic Authentication

Your password is not held in cleartext.

This is a global password which allows you to log in as any user you choose. You can find the Secret Key in the Developer tab under Security Settings.

Trusted Authentication

Only Administrators can see the Trusted Authentication secret key.

This is a user-local password placement with a designated lifetime. Call the API with your password (or secret key) to receieve a bearer token.

Bearer Token Authentication

This token will expire after the validitiy_time_in_sec.

Type the command cs_tools config create --help and press Enter .

~cs~tools config create --help ^{any option marked with a red asterisk ( * ) is required.}

cs_tools config create --config dogfood --url https://anonymous.thoughtspot.cloud --username anonymous

Successful config output

~cs~tools config check --anonymous --config ENV: