Installing Prometheus Agents via Saltstack

So now you’ve got your Prometheus server up-and-running and you want to monitor everything on the network. Understandable. You’ve installed a couple of agents on some sample targets to get a feel for how that goes. Time to write a saltstack state file to get the agent, node_exporter, installed on everything.

Saltstack Server Preparation

We need to have a version of the node_exporter file for every hardware that we want to monitor. This post explained how to download those. Download all of the requisite node_exporters onto your saltstack server and save in the files area.

Copy your working service unit file from /lib/systemd/system/node_exporter.service on one of your sample targets to your saltstack server.

Now, use the following state file to push the exporter out to each or your Raspberries:

node_exporter:
   user.present:
     - fullname: Prometheus Agent
     - shell: /bin/false
     - password: 53cr3t
     - hash_password: True

 install_exporter_file:
   file.managed:
     - name: /home/node_exporter/node_exporter
     {%   if grains['cpuarch'] == 'armv61' %}
     - source: salt://files/node_exporter-1.0.1.linux-armv6/node_exporter}
     {% elif grains['cpuarch'] == 'armv71' %}
     - source: salt://files/node_exporter-1.0.1.linux-armv7/node_exporter}
     {% endif %}
     - user: node_exporter
     - group: node_exporter
     - mode: 500
 /lib/systemd/system/node_exporter.service:
   file.managed:
     - source: salt://files/node_exporter.service
     - mode: 700

 run_prom_agent:
   service.running:
       - name: node_exporter.service
       - enable: true

What Does It DO?

The first paragraph creates the node_exporter user.

The next paragraph checks the type of CPU on the target and copies the proper executable, based on the cpuarch grain, to the /opt directory on the target. I’ve got a mixture of RPi Zeroes, 3’s, and 4’s, so I’ve made the armv61 and armv71 architectures available for installation. The long pathname is based on what was created when I untarred the distribution files in /srv/salt/files

Finally, the last paragraph enables the service and make sure it’s running.