How to use the Reflect Agent in your own infrastructure

Reflect’s self-hosted agent is a secure, scalable system that connects directly to your databases and communicates with the Reflect API without sharing your credentials.

Considerations

Reflect provides a cluster of agents to all users. When you set up a connection in the Reflect application, it uses our cluster. Your database credentials are securely stored in this cluster, and you must allow access to your database directly from our cluster’s external IP address.

If you do not want to share your database credentials with Reflect or expose your databases to the Internet, you have the option of running the agent as a self-hosted instance or replicated cluster. The agent communicates with the Reflect API using WebSockets; it does not require any inbound firewall changes.

When you run your own agent, you are responsible for maintaining it in your infrastructure, including any hardware or network requirements.

Downloads

The Reflect Agent is available for Debian Linux, for Ubuntu Linux, and as a Docker image.

Debian

• Debian sid
• Debian stretch
• Debian jessie
• Debian wheezy

Ubuntu

• Ubuntu zesty
• Ubuntu xenial
• Ubuntu trusty

Docker

# docker run -v /your/agent/storage:/agent --name reflect-agent reflect/agent:latest

Other

If the agent isn’t available for your operating system, let us know. We’ll do our best to support it as soon as possible.

Setup

The Reflect Agent is ready to be used as soon as you install it. You can optionally configure clustering or encryption if needed.

Configuration file

When you start the agent, you can specify the path to a configuration file. By default, the agent uses the following paths:

• If you are root, the path is /etc/reflect/reflect-agent.json.
• For any other user, the agent uses the following ordered strategy:
  1. If the environment variable $XDG_CONFIG_HOME is defined, the path is $XDG_CONFIG_HOME/reflect/reflect-agent.json.
  2. If $HOME is defined, the path is $HOME/.config/reflect/reflect-agent.json.
  3. Otherwise, the path is /etc/reflect/reflect-agent.json.

Most options available in the Reflect Agent command-line interface can be set in the configuration file. Here is an example of a mostly complete configuration file:

{
  "data_dir": "/var/lib/reflect/agent-1",
  "key_path": "/etc/reflect/reflect-agent.key",
  "read_only": false,
  "cluster": {
    "member_name": "reflect-agent-1",
    "client_bind_addresses": ["0.0.0.0:9703"],
    "peer_bind_addresses": ["0.0.0.0:9704"],
    "join_urls": ["agent://10.10.10.10:9701"]
  },
  "storage": {
    "backend": "s3",
    "backends": {
      "s3": {"bucket": "my-bucket"}
    }
  }
}

Clustering

Agent nodes can optionally federate with each other, maintaining state and sharing credential data. Agents use the Raft consensus algorithm to coordinate activity. When planning for a Reflect Agent cluster, you should first understand the requirements of Raft, in particular how it manages quorum, the number of members required for distributed consensus:

The optimal number of cluster members is 3, 5, or 7, depending on your availability requirements. It is not recommended to run an even number of members, nor is it recommended to have more than 7 members in the same cluster.

Cluster membership must be established when first starting an agent node. Once a node has started, it cannot be moved to a different cluster. If a node is removed from a cluster, it cannot be started again.

The following options are available in the cluster section of the configuration file, assuming the public IP address of the interface of the default gateway is 10.10.10.10:

Example

We are going to set up a two-node cluster with the following configuration:

We’ll start with alpha.

When you install the Reflect Agent using a package, it will automatically set up a single-node cluster. Let’s first disable the default instance:

# systemctl stop reflect-agent.service
# systemctl disable reflect-agent.service
Removed /etc/systemd/system/multi-user.target.wants/reflect-agent.service.

The agent uses templated systemd units to make it easy to run multiple agents on the same machine. For example, the unit reflect-agent@my-config.service will be configured by the file /etc/reflect/reflect-agent-my-config.json.

Create a new file, /etc/reflect/reflect-agent-alpha.json, with the following content:

{
  "data_dir": "/var/lib/reflect/agent-alpha",
  "cluster": {
    "member_name": "alpha"
  }
}

Create the data directory:

# mkdir /var/lib/reflect/agent-alpha
# chown reflect:reflect /var/lib/reflect/agent-alpha
# chmod 0750 /var/lib/reflect/agent-alpha

Enable and start the service:

# systemctl enable reflect-agent@alpha.service
Created symlink /etc/systemd/system/multi-user.target.wants/reflect-agent@alpha.service → /lib/systemd/system/reflect-agent@.service.
# systemctl start reflect-agent@alpha.service
# reflect-agent -c /etc/reflect/reflect-agent-alpha.json cluster members
+---+-------+--------------------------+--------+
|   | NAME  |        ADDRESSES         | LEADER |
+---+-------+--------------------------+--------+
| * | alpha | agent://10.10.10.10:9701 | Yes    |
+---+-------+--------------------------+--------+

Now let’s configure bravo. The process is essentially the same, but we’ll need to specify the URL to alpha in the configuration file /etc/reflect/reflect-agent-bravo.json:

{
  "data_dir": "/var/lib/reflect/agent-bravo",
  "cluster": {
    "member_name": "bravo",
    "join_urls": [
      "agent://10.10.10.10:9701"
    ]
  }
}

Remember, the join URLs only apply when the node is started for the very first time. They will be ignored subsequently, so if alpha is eventually removed from the cluster, bravo will continue to work without a problem (assuming it retains quorum).

Disable the default instance and set up bravo’s data directory. Then enable and start the new node:

# systemctl enable reflect-agent@bravo.service
Created symlink /etc/systemd/system/multi-user.target.wants/reflect-agent@bravo.service → /lib/systemd/system/reflect-agent@.service.
# systemctl start reflect-agent@bravo.service
# reflect-agent -c /etc/reflect/reflect-agent-bravo.json cluster members
+---+-------+--------------------------+--------+
|   | NAME  |        ADDRESSES         | LEADER |
+---+-------+--------------------------+--------+
|   | alpha | agent://10.10.10.10:9701 | Yes    |
| * | bravo | agent://10.10.10.11:9701 | No     |
+---+-------+--------------------------+--------+

Additional cluster nodes can be added in the same way.

Encryption

By default, the agent does not encrypt your credential information. (But communication between your agent and the Reflect API is always automatically encrypted using TLS.) If needed, you can enable connection credential encryption by generating a secret key and specifying the path to it in your configuration file.

The agent uses AES-256 in CBC mode with a SHA-256 HMAC.

To enable encryption, you must generate a 512-bit (64-byte) random key. For example, using dd:

# dd if=/dev/urandom of=/etc/reflect/reflect-agent.key bs=1 count=64
64+0 records in
64+0 records out
64 bytes copied, 0.000233683 s, 274 kB/s
# chown reflect /etc/reflect/reflect-agent.key
# chmod 0400 /etc/reflect/reflect-agent.key

Then set the key_path configuration option to the path to the generated key. All connections added or edited after setting the key path will be encrypted.

All members of a cluster must use the same secret key.

Login

You must authenticate to Reflect before using a self-hosted agent. This is a one-time operation: once a cluster is associated to a Reflect account, it cannot be deassociated.

# reflect-agent login
Authenticating with Reflect account credentials
E-mail address: you@example.com
Password: 
OK

You can verify the authentication attempt by listing connections. When you initially log in, your agent will request metadata about all connections in your account. Note that they are not available for use on the agent until configured.

# reflect-agent connections list
+--------------------------+--------------------------+------------+----------+----+
|           NAME           |           SLUG           |   DRIVER   | ON AGENT | UP |
+--------------------------+--------------------------+------------+----------+----+
| My Very Fancy Connection | my-very-fancy-connection | PostgreSQL | No       |    |
| Sample Connection #1     | sample-connection-1      | PostgreSQL | No       |    |
+--------------------------+--------------------------+------------+----------+----+

Backup and restore

You may want to periodically back up the data stored in your agent:

# reflect-agent backup --path /your/backups/reflect-agent-$( date -I ).backup

Only connection data is backed up, not metadata about cluster nodes. Backups are an effective way to transfer your data to a new cluster.

Only a new agent instance can be restored from a backup. If an agent has already been started or joined to a cluster, it will ignore any requests to restore data.

The first time you start an agent instance, specify the path to the backup file:

# reflect-agent serve --restore-path /your/backups/reflect-agent-2017-05-22.backup

Connection management

When using a self-hosted agent, you do not configure connections using the Reflect application. You must manage them using the reflect-agent command.

Add a connection

# reflect-agent connections add postgres \
  --name 'My Very Fancy Connection' \
  --host db.internal.example.com \
  --port 5432 \
  --user example \
  --password s3cr3t \
  --database example \
  --ssl require
Added connection "My Very Fancy Connection".

When you add a new connection, the agent will attempt to contact the data source and verify it is working. If successful, the driver and connection name are sent to the Reflect API. All other driver options are only stored on the agent.

See the list of available drivers and driver options for more information.

List connections

# reflect-agent connections list
+--------------------------+--------------------------+------------+----------+-----+
|           NAME           |           SLUG           |   DRIVER   | ON AGENT | UP  |
+--------------------------+--------------------------+------------+----------+-----+
| My Very Fancy Connection | my-very-fancy-connection | PostgreSQL | Yes      | Yes |
| Sample Connection #1     | sample-connection-1      | PostgreSQL | No       |     |
+--------------------------+--------------------------+------------+----------+-----+

Edit a connection

You can change the name or individual driver options of a given connection:

# reflect-agent connections edit my-very-fancy-connection name 'My Somewhat OK Connection'
Saved connection "My Somewhat OK Connection"
# reflect-agent connections edit my-very-fancy-connection options \
  --user readonly_example \
  --password readonly_s3cr3t
Saved connection "My Somewhat OK Connection"
# reflect-agent connections list
+---------------------------+--------------------------+------------+----------+-----+
|           NAME            |           SLUG           |   DRIVER   | ON AGENT | UP  |
+---------------------------+--------------------------+------------+----------+-----+
| My Somewhat OK Connection | my-very-fancy-connection | PostgreSQL | Yes      | Yes |
| Sample Connection #1      | sample-connection-1      | PostgreSQL | No       |     |
+---------------------------+--------------------------+------------+----------+-----+

Delete a connection

# reflect-agent connections delete my-very-fancy-connection
Deleted connection "My Somewhat OK Connection"

Drivers

• Amazon Redshift
• PostgreSQL
• MySQL and derivatives, including MariaDB, Percona, and Google CloudSQL
• Microsoft SQL Server
• SQLite
• CSV
• HTTP

Redshift

This driver supports all versions of Amazon Redshift.

SSL modes

In most cases, you should use require or verify-ca with Redshift.

PostgreSQL

This driver supports PostgreSQL version 9.0 and higher.

SSL modes

MySQL

This driver supports MySQL version 4.1 and higher, MariaDB, Percona Server for MySQL, and Google CloudSQL.

Microsoft SQL Server

This driver supports Microsoft SQL Server 2008 and higher.

SSL modes

SQLite

This driver supports SQLite version 3.0 and higher.

CSV

HTTP

Contact us for more information and an API specification.

Cluster management

The setup section describes the process for configuring a cluster.

List members

# reflect-agent cluster members
+---+---------------------+--------------------------+--------+
|   |        NAME         |        ADDRESSES         | LEADER |
+---+---------------------+--------------------------+--------+
|   | reflect-agent-tno-1 | agent://10.10.10.10:9701 | Yes    |
|   | reflect-agent-tno-2 | agent://10.10.10.11:9701 | No     |
| * | reflect-agent-tno-3 | agent://10.10.10.12:9701 | No     |
+---+---------------------+--------------------------+--------+

Remove a member

# reflect-agent cluster remove-member reflect-agent-tno-2
+---+---------------------+--------------------------+--------+
|   |        NAME         |        ADDRESSES         | LEADER |
+---+---------------------+--------------------------+--------+
|   | reflect-agent-tno-1 | agent://10.10.10.10:9701 | Yes    |
| * | reflect-agent-tno-3 | agent://10.10.10.12:9701 | No     |
+---+---------------------+--------------------------+--------+