Data -> Telegraf -> InfluxDB -> Grafana

This is quite a common setup - Telegraf understands an enormous number of data input formats, and can output to a large number as well. InfluxDB is a good choice for time series data (ie: values which are recorded regularly, or at specific points in time, rather than being indexed against some other data value, which is better done with a standard RDBMS).

There are plenty of guides out on the Internet telling you how simple it is to get these applications working with each other, for example:

https://larsbergqvist.wordpress.com/2017/03/02/influxdb-and-grafana-for-sensor-time-series/ discusses InfluxDB and Grafana.

https://www.howtoforge.com/tutorial/how-to-install-tig-stack-telegraf-influxdb-and-grafana-on-ubuntu-1804/ deals with all three, including Telegraf.

https://www.vultr.com/docs/install-influxdb-on-debian-jessie-with-telegraf covers Telegraf and InfluxDB (and right at the end suggests you might like to use Grafana as well).

https://angristan.xyz/monitoring-telegraf-influxdb-grafana/ also deals with all three.

There are more.

So, given such a large number of guides on how to set things up, the Internet clearly needs another one. Here it is.

Background

These notes start from Debian version 9 "Stretch", and assume that you are happy to use the Influx and Grafana DEB repositories to install packages from.

You can't use the standard Debian repositories, because they contain a very old version of InfluxDB (1.0.2, release date October 2016) and don't contain Telegraf or Grafana at all.

TL;DR

If you just want to get things installed and running to the point where you can play with Grafana, run the following shell script and then skip to the section below headed "Grafana".

teleinfluxgrafana.sh
#!/bin/bash

aptitude install apt-transport-https

echo 'deb https://repos.influxdata.com/debian stretch stable' >/etc/apt/sources.list.d/influx.list
echo 'deb https://packages.grafana.com/oss/deb stable main' >/etc/apt/sources.list.d/grafana.list

wget -O- https://repos.influxdata.com/influxdb.key | apt-key add -
wget -O- https://packages.grafana.com/gpg.key | apt-key add -

aptitude update

aptitude install telegraf influxdb grafana

sed -i 's/# database = "telegraf"/database = "telegraf"/' /etc/telegraf/telegraf.conf

/etc/init.d/influxdb start
/etc/init.d/telegraf restart

influx -execute 'show databases'

/etc/init.d/grafana-server start

If you want a more hands-on feel for what those commands do, read on…

Repository setup

Add the following to /etc/apt/sources.list.d

influx.list
deb https://repos.influxdata.com/debian stretch stable
grafana.list
deb https://packages.grafana.com/oss/deb stable main

Note that:

  • the Influx repository provides both InfluxDB and Telegraf.
  • you will need the apt-transport-https package installed before you can use these repositories, and that's not installed by default

Add the associated package signing keys:

# wget -O- https://repos.influxdata.com/influxdb.key | apt-key add -
# wget -O- https://packages.grafana.com/gpg.key | apt-key add -

Fetch the repository contents with apt-get update or aptitude update as you prefer.

I got 1855 new packages available after doing this.

Installing

# aptitude install telegraf influxdb grafana
The following NEW packages will be installed:
  grafana influxdb telegraf 
0 packages upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/122 MB of archives. After unpacking 333 MB will be used.
Selecting previously unselected package grafana.
(Reading database ... 34869 files and directories currently installed.)
Preparing to unpack .../grafana_6.1.3_amd64.deb ...
Unpacking grafana (6.1.3) ...
Selecting previously unselected package influxdb.
Preparing to unpack .../influxdb_1.7.4-1_amd64.deb ...
Unpacking influxdb (1.7.4-1) ...
Selecting previously unselected package telegraf.
Preparing to unpack .../telegraf_1.10.2-1_amd64.deb ...
Unpacking telegraf (1.10.2-1) ...
Setting up telegraf (1.10.2-1) ...
telegraf process is not running [ FAILED ]
Starting the process telegraf [ OK ]
telegraf process was started [ OK ]
Setting up influxdb (1.7.4-1) ...
Processing triggers for man-db (2.7.6.1-2) ...
Setting up grafana (6.1.3) ...
Adding system user `grafana' (UID 111) ...
Adding new user `grafana' (UID 111) with group `grafana' ...
Not creating home directory `/usr/share/grafana'.
### NOT starting grafana-server by default on bootup, please execute
 sudo update-rc.d grafana-server defaults 95 10
### In order to start grafana-server, execute
 sudo service grafana-server start

I have no idea why Grafana doesn't start itself, or set itself to start when you reboot the machine, and instead insists that you do these things manually.

Configuring

One of the best things about these applications, which is mentioned in most documentation, is how little configuration needs to be done to get them working.

InfluxDB needs no configuration whatsoever, as Telegraf will create its own database and start putting values into it automatically. These are then available to Grafana for display.

Telegraf has a very large configuration file, because it can understand so many input and output formats. The majority of lines are commented-out, and the file also contains copious comments explaining what the various options mean - the file is its own documentation.

The Telegraf configuration file /etc/telegraf/telegraf.conf contains 5245 lines, of which 29 (0.55%) are not comments:

[global_tags]
[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = ""
  debug = false
  quiet = false
  logfile = ""
  hostname = ""
  omit_hostname = false
[[outputs.influxdb]]
[[inputs.cpu]]
  percpu = true
  totalcpu = true
  collect_cpu_time = false
  report_active = false
[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "overlay", "aufs", "squashfs"]
[[inputs.diskio]]
[[inputs.kernel]]
[[inputs.mem]]
[[inputs.processes]]
[[inputs.swap]]
[[inputs.system]]

Assuming you don't want to add any extra data sources (for example, MQTT) as yet, just configure the InfluxDB output section for now by uncommenting the line database = "telegraf":

[[outputs.influxdb]]
  database = "telegraf"

That's it.

Start and check services

# /etc/init.d/influxdb start
Starting influxdb...
influxdb process was started [ OK ]
# /etc/init.d/telegraf restart
telegraf process was stopped [ OK ]
Starting the process telegraf [ OK ]
telegraf process was started [ OK ]
# influx
Connected to http://localhost:8086 version 1.7.4
InfluxDB shell version: 1.7.4
Enter an InfluxQL query
> show databases
name: databases
name
----
telegraf
_internal
> use telegraf
Using database telegraf
> show measurements
name: measurements
name
----
cpu
disk
diskio
kernel
mem
processes
swap
system
> quit

The existence of the telegraf database, and the measurements contained within it, show that both Telegraf and InfluxDB are working and talking to each other.

Grafana

Grafana needs no editing of its configuration files - you just need to start the service and then poke it with a web browser.

# /etc/init.d/grafana-server start
[ ok ] Starting Grafana Server:.
# netstat -lptn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:8088          0.0.0.0:*               LISTEN      29500/influxd
tcp6       0      0 :::8086                 :::*                    LISTEN      29500/influxd
tcp6       0      0 :::3000                 :::*                    LISTEN      29621/grafana-server

You should find several other network services listening on a standard machine - I've only listed the ones of interest for this article.

You can then point a web browser at http://ser.ver.host.name:3000 or http://ip.add.re.ss:3000 and proceed…

  1. Log in with the username admin and password admin
  2. Change the password when prompted on the next screen
  3. You should then find a screen confirming that you've installed Grafana (!) and inviting you to add a data source:(I do wonder whether there's a way of getting this screen to tell you that you haven't yet installed Grafana…)
  4. Click on the orange square to add a data source, and select InfluxDB from the list of possible sources
  5. You can leave the Name as InfluxDB or change it, as you wish
  6. Note that the URL shown (http://localhost:8086) looks correct, and you may be tempted to leave it as it is because it does not need changing - but beware - this is not a default setting, it is simply an example of what you might like to type in in order to get things to work. I have ranted about this in another article
  7. You actually have to type http://localhost:8086 over the top of where it says http://localhost:8086 otherwise you will later get an obscure Network Error message about a Bad Gateway (which, for accessing localhost, would be quite a feat)
  8. Enter the Database name as telegraf and then click on Save & Test at the bottom
  9. You should see two green boxes appear (one of them only briefly): Data source is working and Datasource updated
  10. You can then click on the four squares in a square arrangement towards the top of the left-hand column of icons, and this should take you back to the Home Dashboard, where you are prompted to create a New Dashboard:
  11. Click on New dashboard
  12. You get a New Panel containing three items: Add Query, Choose Visualization and Convert to row
  13. Click on Add Query
  14. You get a grid with a Panel Title and No data points
  15. Underneath this (if you're using Firefox) or on top of it (if you're using Chrome) is a bar containing Queries to, and default is selected.
  16. To the right are three buttons: Add Query, Query Inspector and a Question Mark:
  17. If this is all you see, and there is no query form shown below, you need to reduce the width of your browser window. Don't ask me why, just try it.

I first discovered this after spending a few days trying (and failing) to get the query form to show on my desktop PC, by using Firefox on my Android phone (which has a 720 x 1280 screen). I found that in landscape mode, the form was invisible, but in portrait mode, it became visible.

Returning to my PC, if I make the browser window 635 pixels wide or less, I can see the query form. It's not very usable at that size, but it's visible. If the browser window is more than 635 pixels wide (the height appears to make no difference), then the query form just disappears and there's no way of interacting with it.

Update

The above problem is present in Grafana version 6.0.0 and above (tested up to 6.1.3).

Reverting to version 5.4.3 resolves the problem. It's possible (I haven't yet checked) that dashboard JSON definitions can be exported from 5.4.3 and used in 6.1.3.


Go up
Return to main index.