Integrating Docker Flow Monitor with Docker Flow Proxy

Having a port opened (other than 80 and 443) is, often, not a good idea. If for no other reason, at least it's not user-friendly to remember a different port for each service. In general service might need to be accessible on its own subdomain, it might need SSL certificate, it might require some URL rewriting, it might need a basic authentication, and so on and so forth. I won't go into details since you probably already know all that and you are probably already using some proxy in your organization.

We'll integrate Docker Flow Monitor (http://monitor.dockerflow.com/) with Docker Flow Proxy (DFP) (http://proxy.dockerflow.com/). If you haven't used DFP before, please visit the official documentation (http://proxy.dockerflow.com/) for tutorials, setup, and configuration.

Before we apply the knowledge about new ways to configure Prometheus, we need to run the proxy:

docker network create -d overlay proxy
    
docker stack deploy \
    -c stacks/docker-flow-proxy.yml \
    proxy

We created the proxy network and deployed the docker-flow-proxy.yml stack. We won't go into details how Docker Flow Proxy works. The essence is that it will configure itself with each service that has specific labels. For any deeper explanation, please visit Docker Flow Proxy Stack Tutorial (http://proxy.dockerflow.com/swarm-mode-stack/) or any other tutorial available.

With the proxy up and running, we should redeploy our monitor.

We'll replace the current monitor stack with a new one with in order to achieve this. The major difference is that this time we'll define startup arguments as well as the labels that will allow the proxy to reconfigure itself to enable access to monitor. You'll also notice that we will not expose port 9090. It'll be accessible through the proxy on port 80, so there's no reason to open any other port.

The stack is as follows:

  monitor: 
    image: vfarcic/docker-flow-monitor:${TAG:-latest} 
    environment: 
      - GLOBAL_SCRAPE_INTERVAL=10s 
      - ARG_WEB_ROUTE-PREFIX=/monitor 
      - ARG_WEB_EXTERNAL-URL=http://${DOMAIN:-localhost}/monitor 
    networks: 
      - proxy 
      - monitor 
    deploy: 
      labels: 
        - com.df.notify=true 
        - com.df.distribute=true 
        - com.df.servicePath=/monitor 
        - com.df.serviceDomain=${DOMAIN:-localhost} 
        - com.df.port=9090 
 
 
  swarm-listener: 
    image: vfarcic/docker-flow-swarm-listener 
    networks: 
      - monitor 
    volumes: 
      - /var/run/docker.sock:/var/run/docker.sock 
    environment: 
      - DF_NOTIFY_CREATE_SERVICE_URL=http://monitor:8080\
/v1/docker-flow-monitor/reconfigure 
      - DF_NOTIFY_REMOVE_SERVICE_URL=http://monitor:8080\
/v1/docker-flow-monitor/remove 
    deploy: 
      placement: 
        constraints: [node.role == manager] 
 
networks: 
  monitor: 
    external: true 
  proxy: 
    external: true 

This time we added a few additional environment variables. They will be used instead Prometheus' default startup arguments.

We are specifying the route prefix (ARG_WEB_ROUTE-PREFIX) as well as the full external URL (ARG_WEB_EXTERNAL-URL).

Please visit ARG variables section of the documentation (http://monitor.dockerflow.com/config/#arg-variables) for more information about environment variables that can be used as startup arguments.

We also used the com.df.* service labels that will tell the proxy how to reconfigure itself so that Prometheus is available through the path /monitor.

The second service is Docker Flow Swarm Listener (http://swarmlistener.dockerflow.com/) that will listen to Swarm events and send reconfigure and remove requests to the monitor. You'll see its usage later on. For now, just remember that we deployed it alongside the monitor service.

Let us deploy the new version of the monitor stack:

docker stack rm monitor 
 
DOMAIN=$(docker-machine ip swarm-1) \  
    docker stack deploy \  
    -c stacks/docker-flow-monitor-proxy.yml \
    monitor 

In the "real-world" situation, you should use your domain (for example monitor.acme.com) and would not need ARG_WEB_ROUTE-PREFIX and com.df.servicePath set to /monitor. However, since we do not have a domain for this exercise, we used the IP of swarm-1 node instead.

Please execute, docker stack ps monitor to check the status of the stack. Once it's up-and-running, we can confirm that the monitor is indeed integrated with the proxy:

open "http://$(docker-machine ip swarm-1)/monitor/flags"

By opening the flags screen, not only that we confirmed that the integration with Docker Flow Proxy worked but also that the arguments we specified as environment variables are properly propagated. You can observe that through the values of the web.external-url and web.route-prefix flags:

Please note that we did not specify the port of the monitor service. As soon as the service was created, swarm-listener detected it and sent a request to the proxy to reconfigure itself. The information the proxy needs was obtained through the labels (for example com.df.servicePath).

There was a hidden reason behind the integration of the two. Apart from the need to have a proxy, I wanted to show you an existing implementation of the logic we are exploring. There was no need for a manual configuration of the proxy, nor we had to define the data proxy needs anywhere but inside the service definition itself. The monitor service contains all the information, and any other part of the system can fetch it. Everything related to the service is in a single location. By everything, I mean everything that we need for now. Later on, we'll extend the definition of this and many other services.