The new self-hosted Reflect Agent
You can now provide data to Reflect’s reporting and visualization API without sharing your database connection info
Crucial to the Reflect platform is the ability of our API to securely connect to a variety of sources—without requiring ETL or a complicated ingestion API—using a process that we call the Reflect Agent.
Reflect offers a hosted cluster of Agents for all users, paid and free tier. Previously, using Reflect meant using only our hosted Agents. But now that’s changing, as you can now run your own self-hosted Reflect Agent on your own infrastructure and your own terms.
Supported data sources
The Reflect Agent currently enables you to securely connect to four databases:
Why run a self-hosted Reflect Agent?
The advantage of using a self-hosted Reflect Agent is that it enables you to securely share your data with the Reflect API without needing to provide your database connection information with Reflect. Self-hosted agents store all database information encrypted and in an embedded database. If you have security requirements that prevent you from sharing database credentials with a third party (even read-only credentials) then you should explore the Reflect Agent as a solution.
How the Reflect Agent works
The Reflect Agent acts as a mediary between two things:
- Your data sources (we currently support Amazon Redshift, PostgreSQL, and MySQL, with more on the way soon)
- The Reflect API, which provides our public-facing data reporting and visualization interface (essentially our core product)
When you run your own Reflect Agent, our API makes requests only to your agent (or cluster of agents), not directly to your databases.
Installing and running
Check out our documentation
In this blog post we’ll provide just a very basic overview of how the Reflect Agent works and how you can install and use it. For a full treatment, see our Agent doc.
The Reflect Agent can be installed and run in two ways:
We currently offer the Reflect Agent for the following Linux distributions:
- Debian sid/unstable
- Debian stretch
- Debian jessie
- Debian wheezy
- Ubuntu yakkety
- Ubuntu xenial
- Ubuntu trusty
If you want to run the Reflect Agent as a Docker container, you can simply pull the image from Docker Hub and start it:
$ docker pull reflect/agent:latest $ docker run \ -v /opt/reflect/agent:/agent # Specify a storage volume --name=reflect-agent # Provide a name for the container --net=host \ # Run the container in "host" mode (required) reflect/agent:latest # The Reflect Agent Docker image
The clustered Reflect Agent
Running the Reflect Agent on a single server should be enough to handle most development and staging workloads, but for production workloads we recommend running multiple agents as a cluster.
When running in a cluster, Reflect agents rely on an implementation of the Raft consensus protocol to coordinate cluster state across nodes. This enables a cluster of agents to provide higher fault tolerance than running a single agent.
You can run Dockerized agents in a cluster as well.
Conclusion: same Reflect platform, now with more security options
Whether you’re using a self-hosted Reflect Agent (or cluster of agents) or using the agents that Reflect provides, you can always connect to your databases securely. But if you need to go an extra step and use Reflect without sharing your database connection information or exposing your databases to the public internet, running a self-hosted Reflect Agent is an option that you should absolutely bear in mind.