InfluxDB

InfluxDB is an open-source time series database which is especially designed for the storage and retrieval of temporal events and timestamped metrics. This is typically used for operational monitoring and real-time analytics concerning databases, digital systems and IoT sensors. InfluxDB can be installed on Windows, Mac and Linux systems and includes a GUI for administration and data visualisation.

InfluxDB has its own query language called Flux which is optimised for ETL, monitoring and alerting tasks. To get data into the database Influx provide a configurable, server-based, data collection agent called Telegraf. This is a plugin based-system which can collect metrics and events from a range of databases and messaging systems including Redis, Postgres, Apache Kafka and MQTT. Alternatively Influx also provide client libraries for programmatically interacting with the database via it's built-in RestAPI using Arduino, C#, Go, Java, JavaScript, Kotlin, Node.js, PHP, Python, R, Ruby, Scala, and Swift.

Personal Experience

I've used InfluxDB both as a standalone time series database and as part of an integrated real-time monitoring system for IoT sensors using the 'TIG stack' which combines Telegraf and InfluxDB with the Grafana dashboard for data visualisation. For convenience I usually deploy these components using Docker containers. As an alternative to Telegraf I often use the InfluxDB connectors in Node-Red to prototype workflows involving Websockets or MQTT.

Tools & Software Integrations

• Docker images for rapid deployment:
  • InfluxDB Docker Image - Official InfluxDB image.
  • Telegraf Docker Image - Official Telegraf image.
• Grafana - A user configurable, open-source, and web-based dashboard for interactive data visualisation. Grafana provides a built-in connector for InfluxDB along with plugins for a wide-range of different charts and visualisations. Combined with Telegraf and InfluxDB, Grafana forms a key component of the 'TIG stack' for monitoring time series data and events. The connection to InfluxDB can be set up in Grafana using the following instructions: Use Grafana with InfluxDB OSS
• Flux VS Code Extension - An extension for Microsoft's VS Code IDE which provides Flux syntax highlighting, autocompletion, and a direct integration with InfluxDB that lets you run Flux scripts natively and show results in VS Code.

Recommended Plugins

• HTTP Input Plugin - Collects metrics from one or more HTTP(S) endpoints. This can be used to access services via a Rest API such the New York Citibike API as demonstrated here: Using the HTTP input plugin with Citi Bike data
• Kafka Consumer Input Plugin - Reads from Kafka and create metrics in InfluxDB.
• MQTT Consumer Input Plugin - Reads from the specified MQTT topics and create metrics in InfluxDB.
• OpenWeatherMap Input Plugin - Useful plugin to collect current weather and forecast data from OpenWeatherMap. Use of this plugin requires an APPID as api key

Resources

• InfluxDB Documentation - Official documentation with guides on installation, writing, querying, processing, visualising data and setting up monitoring and alerts.
• Telegraf Documentation - Official documentation with installation, quick start and details of plugin configuration.

Notes and Troubleshooting

N/A