docs: better explain the architecture and more clearly separate docker setup from lndmon setup

ZZiigguurraatt · ZZiigguurraatt · commit 2d12e2ae64e1 · 2025-12-18T13:11:13.000-05:00
diff --git a/Docker_Usage.md b/Docker_Usage.md
@@ -0,0 +1,94 @@
+# Docker Compose Setup Instructions for Collecting Data From `lnd` _AND_ `lndmon`
+
+# Table of Contents
+1. [Setup](#setup)
+   1. [Requirements](#requirements)
+   2. [Lnd](#lnd)
+   3. [Lndmon](#basic-lndmon-and-docker-compose-setup)
+2. [Usage](#usage)
+    1. [Option 1: Nginx Proxy Usage](#option-1-nginx-proxy-usage-requires-domain-name)
+    2. [Option 2: Local Usage](#option-2-local-usage)
+    3. [Customizing Grafana Metrics](#customizing-grafana-metrics)
+
+## Setup
+
+### Requirements
+* docker >= 18.09.6
+* docker-compose >= 1.24
+
+### Lnd
+
+- You need to have an `lnd` node built with the `monitoring` tag up and running
+  with ports exposed and reachable.
+- Run lnd with
+   * `prometheus.enable=true`
+   * `prometheus.listen=<IP>:8989`
+   * `rpclisten=<IP>`
+   * `tlsextraip=<IP>`
+      - Make sure you have the node's reachable IP defined in the cert.
+      - You may need to delete the existing `tls.cert` and `tls.key` in your lnd
+        data directory and restart lnd to regenerate the cert.
+- Start `lnd` *before* `lndmon`.
+
+
+Note: See README.md for some additional consideration about the options that
+are set above.
+
+
+### Basic `lndmon` and Docker Compose Setup
+
+- Clone this repo
+```
+git clone https://github.com/lightninglabs/lndmon
+cd lndmon/
+```
+- In the `.env` file in this repo
+   * Fill in the `TLS_CERT_PATH` and `MACAROON_PATH` variables relative to the
+     host filesystem to allow `lndmon` to connect to your `lnd` node.
+      - By default, `.lnd` lives in your home directory.
+   * Fill in the `LND_HOST` variable to match your lnd node's IP and port.
+   * Ensure the other lnd variables are also up-to-date.
+   * If you wish to run `lndmon` connecting to an `lnd` node on testnet or simnet:
+      - modify the `LND_NETWORK` variable to match your desired network.
+      - make sure the `MACAROON_PATH` matches the desired network as well.
+- Edit `prometheus.yml` `lnd` `targets` section to match your node's IP.
+
+
+
+## Usage
+### Option 1: Nginx Proxy Usage (requires domain name)
+If you want to enable the built-in nginx proxy feature in order to access your Prometheus and Grafana dashboards remotely, these are the steps:
+1. In the `lndmon` repository, edit the `.env` file and fill in the email, FQDN, and (optionally) timezone fields.
+2. Ensure ports 80 and 443 on your machine are exposed to the internet.
+3. (Optional) Basic auth setup for your Prometheus dashboard:
+   - Install `apache2-utils` package.
+   - Run `htpasswd -c nginx/etc/.htpasswd <YOUR_USERNAME>` and follow the prompts to enter and confirm your desired password.
+   - In `lndmon/nginx/etc/service.conf`, uncomment the lines indicated in the file to enable basic auth.
+4. If you want to use your own TLS certs:
+   - Uncomment the lines beginning with `SSL_`  in `.env` and fill in the paths to your cert files.
+   - Uncomment the lines beginning with `- SSL_` in `docker-compose.nginx.yml`.
+5. Start everything up
+   - `docker-compose -f docker-compose.yml -f docker-compose.nginx.yml up`
+      - Run this command within the `lndmon` repository directory
+      - Unless you opted to use your own certs above, this will result in the automatic generation of TLS certificates through Let's Encrypt if they haven't been generated already, or their renewal if the current certs have expired. The certs will automatically renew when they expire.
+6. Grafana is located at `https://<YOUR_DOMAIN>/grafana/`
+7. Prometheus's expression browser is located at `https://<YOUR_DOMAIN>/prometheus/graph`.
+
+### Option 2: Local Usage
+1. Start everything up
+   - `docker-compose up`
+     * Run this command within the `lndmon` repository directory
+     * If you get the error "transport: Error while dialing dial tcp 172.17.0.1:10009: i/o timeout", your docker interface may not have the default IP. Make sure your docker interface's IP matches the IP for `LND_HOST` in `.env` and the lnd target's IP in `prometheus.yml`.
+2. Access Grafana dashboard: 
+   - Get Grafana's IP:
+      - `docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' lndmon_grafana_1`
+   - Grafana's dashboard is located at `http://<GRAFANA_IP>:3000/`.
+      - The default password for the admin user is also admin (you can change it after the first login).
+3. Access Prometheus expression browser:
+   - Get Prometheus's IP:
+      -`docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' lndmon_prometheus_1`
+   - Prometheus's expression browser is located at `http://<PROMETHEUS_IP>:9090/graph`.
+
+
+### Customizing Grafana Metrics
+`lndmon`'s Grafana instance comes with a set of basic dashboards. Add additional dashboards by clicking the `+' sign on the left.
diff --git a/INSTALL.md b/INSTALL.md
diff --git a/README.md b/README.md
@@ -1,38 +1,117 @@
 # lndmon
 
-#### A drop-in monitoring solution for your lnd node using Prometheus and Grafana.
+#### A drop-in monitoring solution for your lnd node that integrates with Prometheus and Grafana.
 
 <img src="dashboard.png">
 
 ## What is this?
 
-`lndmon` is a drop-in, dockerized monitoring/metric collection solution for your
-individual lnd nodes connected to bitcoin. With this system, you'll be able to
-closely monitor the health and status of your `lnd` node.
+`lndmon` is a drop-in, monitoring/metric collection solution for your
+individual lnd nodes connected to bitcoin which allows you to closely monitor
+the health and status of your `lnd` node.
 
-There are three primary components of the `lndmon` system:
+There are four primary components of this monitoring system:
 
 1. [lnd](https://github.com/lightningnetwork/lnd) built with the `monitoring`
-   tag, which enables lnd to export metrics about its gRPC performance and
-   usage. These metrics provide insights such as how many bytes lnd is
-   transmitting over gRPC, whether any calls are taking a long time to complete,
-   and other related statistics.
-
-2. `lndmon`: while lnd provides some information, `lndmon` by far does the heavy
-   lifting with regards to metrics. With `lndmon`'s data, you can track routing
-   fees over time, track how the channel graph evolves, and have a highly
-   configurable "crystal ball" to forecast and de-escalate potential issues as
-   the network changes over time. There is also a strong set of metrics for
-   users who want to keep track of their own node and channels, or just explore
-   and create their own lightning data visualizations.
-
-3. Last but not least, `lndmon` uses [Grafana](https://grafana.com/) as its
+     tag, which enables lnd to export metrics about its gRPC performance and
+     usage.
+   - These metrics provide insights such as how many bytes lnd is transmitting
+     over gRPC, whether any calls are taking a long time to complete, and other
+     related statistics.
+   - `lnd` build notes:
+      * If you are using a release build of `lnd` or `litd`, this was enabled
+        during build and no action is required.
+      * If you are building from source, make sure to include the `monitoring`
+        tag (i.e. `make && make install tags=monitoring`)
+   - Default `lnd` config options:
+      * `prometheus.enable=false`
+         - Need to change to `true` to enable.
+      * `prometheus.listen=127.0.0.1:8989`
+         - You may want to change to `0.0.0.0:8989` to listen on all interfaces.
+      * `prometheus.perfhistograms=false`
+         - Enabling this uses more RAM and disk space.
+   - Note: This data can be obtained from `lnd` without `lndmon` being
+     installed.
+
+2. `lndmon`: while lnd provides some information about gRPC performance
+   statistics directly via its own prometheus exporter, `lndmon` by far does
+   the heavy lifting with regards to metrics.
+   - With `lndmon`'s data, you can track routing fees over time, track how the
+     channel graph evolves, and have a highly configurable "crystal ball" to
+     forecast and de-escalate potential issues as the network changes over time.
+   -  There is also a strong set of metrics for users who want to keep track of
+      their own node and channels, or just explore and create their own
+      lightning data visualizations.
+   - See [Options](#options) below for `lndmon` config options.
+   - Note: This data can be obtained without lnd being built with the
+     `monitoring` tag and without the `prometheus.enable` config option enabled,
+     but it does require `lnd` to be listening for gRPC connections.
+
+3. Prometheus is a time series database that scrapes data from `lnd` and
+  `lndmon` and stores it for later analysis.
+
+4. `lndmon` uses [Grafana](https://grafana.com/) as its
    primary dashboard to display all its collected metrics. Grafana is highly
    configurable and can create beautiful and detailed graphs organized by
    category (i.e., chain-related graphs, fee-related graphs, etc). Users have
    the option of making their Grafana dashboards remotely accessible over TLS
    with passwords to ensure their data is kept private.
 
+
+# Architectural Layout
+```mermaid
+---
+config:
+    flowchart:
+      subGraphTitleMargin:
+        bottom: 30
+---
+flowchart
+	subgraph s1["**LND**"]
+		n15@{ shape: "stadium", label: "gRPC Server \n(TCP/10009)" }
+		n20@{ shape: "stadium", label: "Prometheus Exporter \n (TCP/8989) " }
+	end
+	s1
+
+	n15
+	subgraph s2["**LNDmon**
+				(aggregate, cache)"]
+
+		n17@{ shape: "stadium", label: "Prometheus Exporter \n (TCP/9092)" }
+		n16@{ shape: "stadium", label: "gRPC Client \n (TCP/9090)" }
+	end
+	n16@{ shape: "stadium", label: "gRPC Client<br>(TCP/9090)" }
+	n21@{ shape: "fr-rect", label: "gRPC Performance Stats" }
+	n20@{ shape: "stadium", label: "Prometheus Exporter<br>(TCP/8989)" } --- n21
+	subgraph s3["**Prometheus**<br/>(time series DB)"]
+		n23@{ shape: "stadium", label: "HTTP Server" }
+		n22@{ shape: "stadium", label: "Prometheus Scraper" }
+	end
+	n21 --- n22
+	n7@{ shape: "fr-rect", label: "gRPC Call Data" }
+	n17@{ shape: "stadium", label: "Prometheus Exporter<br>(TCP/9092)" } --- n7
+	n7 --- n22
+	subgraph s4["**Grafana**<br/>(dashboard generation)"]
+		n25@{ shape: "stadium", label: "HTTP Server" }
+		n24@{ shape: "stadium", label: "HTTP Client" }
+	end
+	subgraph s5["**Firefox**<br/>(dashboard display)"]
+		n26@{ shape: "stadium", label: "HTTP Client" }
+	end
+	n23@{ shape: "stadium", label: "HTTP Server<br>(TCP/9090)" } --- n24
+	n25@{ shape: "stadium", label: "HTTP Server<br>(TCP/3000)" } --- n26
+	n15 --- n16@{ shape: "stadium", label: "gRPC Client" }
+	s1
+	n21
+	n22 --- n23
+	n24 --- n25
+	n16 --- n17
+```
+
+
+
+
+
 ## Why would I want to use this?
 
 Monitoring can provide crucial insights into the health of large-scale
@@ -46,9 +125,63 @@ ecosystem (i.e., "wow, the day [Lightning
 App](https://github.com/lightninglabs/lightning-app) was released coincides with
 the addition of 3000 channels to the network!").
 
-## How do I install this?
 
-Head over to [`INSTALL.md`](https://github.com/lightninglabs/lndmon/blob/master/INSTALL.md). It
-also includes instructions to set up, access, and password-protect the dashboard
-that comes with Prometheus, called the Prometheus expression browser, for those
-interested in using it.
+## Build
+
+```
+git clone https://github.com/lightninglabs/lndmon
+cd lndmon/cmd/lndmon
+go build
+```
+
+## Options
+
+```
+$ lndmon -h
+Usage:
+  lndmon [OPTIONS]
+
+Application Options:
+      --primarynode=                                                 Public key of the primary node in a primary-gateway setup
+      --disablegraph                                                 Do not collect graph metrics
+      --disablehtlc                                                  Do not collect HTLCs metrics
+      --disablepayments                                              Do not collect payments metrics
+
+prometheus:
+      --prometheus.listenaddr=                                       the interface we should listen on for prometheus (default:
+                                                                     localhost:9092)
+      --prometheus.logdir=                                           Directory to log output (default: /home/user/.lndmon/logs)
+      --prometheus.maxlogfiles=                                      Maximum log files to keep (0 for no rotation) (default: 3)
+      --prometheus.maxlogfilesize=                                   Maximum log file size in MB (default: 10)
+
+lnd:
+      --lnd.host=                                                    lnd instance rpc address (default: localhost:10009)
+      --lnd.network=[regtest|testnet|testnet4|mainnet|simnet|signet] network to run on (default: mainnet)
+      --lnd.macaroondir=                                             Path to lnd macaroons (default: /home/user/.lnd)
+      --lnd.macaroonname=                                            The name of our macaroon in macaroon dir to use. (default:
+                                                                     readonly.macaroon)
+      --lnd.rpctimeout=                                              The timeout for rpc calls to lnd. Valid time units are {s,
+                                                                     m, h}. (default: 30s)
+      --lnd.tlspath=                                                 Path to lnd tls certificate
+
+Help Options:
+  -h, --help                                                         Show this help message
+```
+
+
+
+
+
+
+
+
+
+
+
+## How do I use this?
+
+Head over to [`Docker_Usage.md`](https://github.com/lightninglabs/lndmon/blob/master/Docker_Usage.md)
+for information on how to setup a dockerized version. It also includes
+instructions to set up, access, and password-protect the dashboard that comes
+with Prometheus, called the "Prometheus Expression Browser", for those interested
+in using it.
