Custom self-hosted deployment

Architecture overview 

​Basic architecture​ 

Tines is a Rails application that uses a PostgreSQL database for its persistent data storage and Redis for its cached data storage and message queuing.

It features two core application containers: tines-app and tines-sidekiq. tines-app runs the Rails server for web application hosting/receiving webhooks. tines-sidekiq runs background worker processes. Both containers read and write to and from the PostgreSQL database and Redis cache - they don't communicate directly, instead, they swap data and messages through PostgreSQL/Redis.

Both the tines-app and tines-sidekiq containers use the same tines-app Docker image with different entry points (start-tines-app and start-tines-sidekiq respectively).

The tines-app and tines-sidekiq containers both make requests to external services to run your automation stories.

The tines-sidekiq container may be optionally configured to periodically load new Action templates from Tines (at template-data.tines.com) and to send basic version and feature usage information back to Tines (at hq.tines.io).

A tines-nginx container is usually deployed in front of the tines-app container to terminate SSL. Alternatively, any load balancer, load balancer service, or reverse proxy can fill this role.

Serving the web application​ 

A user's browser connects via HTTPS to the Tines web application via the tines-nginx container or an LB/reverse proxy, which terminates the SSL connection and connects to the tines-app container over HTTP.

Running Actions on tines-sidekiq​ 

The Tines product features 7 different types of automation "Actions", 3 of which cause Tines to connect to external services: IMAP, Email (SMTP), HTTP Request.

When a user "runs" an Action via the web application, a task is enqueued in the Redis cluster by the tines-app container. The tines-sidekiq container reads from this queue, reads related data from the PostgresSQL database and performs the Action, which may result in a HTTP, SMTP or IMAP request to an external server.

The data gathered from these requests is stored in an "event" in the PostgresSQL database.

Actions can also run as a result of an upstream Action running successfully, or on a schedule. When this happens, a task is enqueued in the Redis cluster by the tines-sidekiq container for each Action that will run.

Testing Actions on tines-app​ 

When "test" mode is invoked via the web application, the tines-app container directly executes the Action (rather than enqueuing it via the Redis instance for later execution by the tines-sidekiq container).

When this happens, the tines-app container may itself make a HTTP, SMTP or IMAP request to an external server.

Receiving webhooks​ 

An external service may post webhooks to the tines-app container through the same tines-nginx container or LB/reverse proxy used to host the web application.

When a webhook is received, the data is stored in an event in the PostgresSQL database. Further downstream Actions may be executed in turn by the tines-app container enqueuing tasks in the Redis cluster.

Connecting to a SQL database​ 

Tines also provides a separate container, called sql-over-http, that allows the HTTP Request Action in the product to communicate with a SQL database (currently only PostgreSQL databases are supported).

An HTTP Request Action can be configured to invoke the sql-over-http service via a HTTP request, which will, in turn, connect to the target database and execute the given SQL statement.

To connect to Snowflake databases, we recommend using the Snowflake SQL API.

Installation 

Sample Docker Compose File​ 

This simple Docker Compose file can be used as a reference for how the different containers need to be configured on your orchestration system of choice. In this example, both PostgreSQL and Redis are run in containers. You may wish to use PostgreSQL and Redis deployed via other means for this. This example also uses nginx to terminate SSL, and you may wish to use an external load balancing service.

version: "2.2"
services:
  db:
    container_name: postgres
    image: postgres:11.9-alpine
    restart: always
    ports:
      - 5432:5432
    environment:
      POSTGRES_PASSWORD: your-database-password
      POSTGRES_USER: tines
      PGDATA: /var/lib/postgresql/data
    volumes:
      - db-data:/var/lib/postgresql/data
    networks:
      - tines-net

  redis:
    container_name: redis
    image: redis:6.0.9-alpine
    restart: always
    command: redis-server --appendonly yes
    ports:
      - 6379:6379
    volumes:
      - redis-data:/data
    networks:
      - tines-net

  nginx:
    container_name: nginx
    image: tines/tines-nginx:latest
    restart: always
    ports:
      - "80:80"
      - "443:443"
    env_file: .env
    volumes:
      - ./tines.crt:/ssl/tines.crt
      - ./tines.key:/ssl/tines.key
    depends_on:
      - tines-app
    networks:
      - tines-net

  tines-app:
    image: tines/tines-app:latest
    env_file: .env
    restart: always
    command: start-tines-app
    depends_on:
      - db
      - redis
    networks:
      - tines-net

  tines-sidekiq:
    container_name: tines-sidekiq
    image: tines/tines-app:latest
    restart: always
    command: start-tines-sidekiq
    env_file: .env
    depends_on:
      - tines-app
      - db
      - redis
    networks:
      - tines-net

networks:
  tines-net:

volumes:
  db-data:
  redis-data:

Sample .env file​ 

Tines gets the configuration it requires to start from environment variables. The following describes the minimum set of environment variables you must provide:

#############################
# Required: Tenant Configuration #
#############################

# A human friendly identifier for this instance of Tines, e.g. your company name:
TENANT_NAME=company_name
# Company name and stack name (eg. tines_prod). This is used to identify your tenant's telemetry data,
# if you have enabled that feature.
TELEMETRY_ID=company_name_prod

# This will be the first user to be created and get invited to this Tines instance:
SEED_EMAIL=alice@example.com
SEED_FIRST_NAME=Alice
SEED_LAST_NAME=Smith

# The domain that you ultimately want your Tines installation to be accessible at in your browser.
# This should match the SSL certificate installed into your tines-nginx container or load balancer.
DOMAIN=tines.example.com
# This should match the port that you use to access the Tines UI.
# Unless you have chosen a custom port, you should use 443 as typical for HTTPS.
PORT=443

# This should be set to a random 128 character string to ensure security for your installation.
# Changing this value may force users to log in again.
# You can generate a value for this by running: openssl rand -hex 64
APP_SECRET_TOKEN=__SET_YOUR_SECRET_TOKEN__

#############################
# Required: Email Configuration #
#############################
# Outgoing email settings. This must be configured correctly in order for the invite email
# to be sent to the first user.
#
# To use Gmail or Google Apps, put your Google Apps domain or gmail.com
# as the SMTP_DOMAIN and your Gmail username and password as the SMTP_USER_NAME and SMTP_PASSWORD.
#
# If you have trouble with port 587 on Gmail, you can also try setting
# SMTP_AUTHENTICATION to login and the SMTP_PORT to 465.

SMTP_DOMAIN=mail.example.com
SMTP_USER_NAME=tines_smtp
SMTP_PASSWORD=...
SMTP_SERVER=smtp.example.com
SMTP_PORT=465
SMTP_AUTHENTICATION=login
SMTP_ENABLE_STARTTLS_AUTO=true

# This address will be the sender for all emails from this Tines instance:
EMAIL_FROM_ADDRESS=Example Support <support@example.com>

############################
# Required: Database connections
############################

# Your PostgreSQL database server settings:
DATABASE_HOST=your-database-host.example.com
DATABASE_NAME=tines
DATABASE_POOL=20
DATABASE_USERNAME=tines
# This password needs to match the value in your docker-compose.yml file.
# A value that contains punctuation other than underscores and dashes may cause errors.
# You can generate a value for this by running: openssl rand -hex 32
DATABASE_PASSWORD=__SET_YOUR_DATABASE_PASSWORD__
DATABASE_PORT=5432

# Your Redis server settings:
REDIS_URL=redis://your-redis-host.example.com:6379/1

########################
# Optional feature configuration #
########################

# Enables a periodic job to update public template date from template-data.tines.com
SYNC_TEMPLATES=true

########################
# Core configuration
#
# These values should not be changed.
########################

# Ensure system logs are included in Docker container logs.
RAILS_LOG_TO_STDOUT=true

# Configure Rails environment. This should always be set to 'production'.
RAILS_ENV=production

# Force all requests to use SSL.
FORCE_SSL=true

# Set the installation's timezone.
TIMEZONE=UTC

# Set worker count
SIDEKIQ_CONCURRENCY=6

Preparing the database​ 

Once you have the .env file / container environment variables prepared correctly, the tines-app and tines-sidekiq containers won't complete start-up until you run the prepare-database command using the tines-app container image via your orchestration system of choice, e.g.:

docker run --env-file .env tines-app prepare-database

Running tines-app and tines-sidekiq​ 

With the correct environment variables in place and with the database prepared, the tines-app and tines-sidekiq containers should be run using the tines-app image with the commands start-tines-app and start-tines-sidekiq via your orchestration system of choice, e.g.:

docker run --env-file .env tines-app start-tines-app
docker run --env-file .env tines-app start-tines-sidekiq
Was this helpful?