Prometheus (node_exporter)

Configuring Your Prometheus (node_exporter) Server for Mutual TLS

How to use TLS, client authentication, and CA certificates in Prometheus (node_exporter)

Create a private key and request a certificate for your Prometheus (node_exporter) server

Before you can teach your server to speak TLS, you will need a certificate issued by a trusted certificate authority (CA). If your organization already runs its own CA and you have a private key and certificate for your Prometheus (node_exporter) server, along with your CA's root certificate, you can skip to the next step.

To request a certificate from your CA using the step CLI, bootstrap your CA with step ca bootstrap and run the following command (sub the server name for the actual name / DNS name of your Prometheus (node_exporter) server).

$ step ca certificate "myserver.internal.net" server.crt server.key

Your certificate and private key will be saved in server.crt and server.key respectively.

Request a copy of your CA root certificate, which will be used to make sure each application can trust certificates presented by other applications.

$ step ca root ca.crt

Your certificate will be saved in ca.crt.

Configure Prometheus (node_exporter) to authenticate itself with its TLS certificate

We now want to instruct our Prometheus (node_exporter) server to identify itself using the certificate issued in the last step and to force clients to connect over TLS.

Copy the server.crt and server.key files to a node_exporter configuration directory. You may need to make a directory for this, eg. /etc/node_exporter.

$ sudo cp server.crt /etc/node_exporter/server.crt $ sudo cp server.key /etc/node_exporter/server.key

Make sure these files are owned and readable only by the user that node_exporter runs as.

Now create a file called /etc/node_exporter/web-config.yml and configure your tls_server_config block to use the server certificate and key:

tls_server_config:
  # This is the server certificate for your `node_exporter` server.
  cert_file: "/etc/node_exporter/server.crt"
  key_file: "/etc/node_exporter/server.key"

  ...

Configure Prometheus (node_exporter) to require clients to authenticate with a certificate issued by your CA

To tell Prometheus (node_exporter) to use mutual TLS and not just one-way TLS, we must instruct it to require client authentication to ensure clients present a certificate from our CA when they connect.

Copy the ca.crt file to a node_exporter configuration directory. You may need to make a directory for this, eg. /etc/node_exporter.

$ sudo cp ca.crt /etc/node_exporter/root_ca.crt

Make sure these files are owned and readable only by the user that node_exporter runs as.

Now modify /etc/node_exporter/web-config.yml to require client authentication (in your tls_server_config block):

tls_server_config:
  ...

  # RequireAndVerifyClientCert is the most secure option; clients
  # must present a valid client certificate signed by your CA.
  client_auth_type: "RequireAndVerifyClientCert"

  # This is the CA the client certificate must be signed by.
  client_ca_file: "/etc/node_exporter/root_ca.crt"

  ...

That's it! Prometheus (node_exporter) should now be able to receive TLS connections from clients who authenticate themselves using a certificate issued by your trusted CA.

Automate certificate renewal

By default, step-ca issues certificates with a 24 hour expiration. Short-lived certificates have many benefits but also require that you renew your certificates each day before they expire. How you renew certificates is often dependent on how you deploy your application. See the step-ca certificate lifecycle management docs for more information.

All documentation content from the Hello mTLS project is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0).

Creative Commons License

Connect to Your Prometheus (node_exporter) Server from a Client