Skip to content

Commit

Permalink
Include an option to host monitor endpoint without authentication
Browse files Browse the repository at this point in the history
  • Loading branch information
dormant-user committed Oct 10, 2024
1 parent bfdb141 commit 1926318
Show file tree
Hide file tree
Showing 12 changed files with 97 additions and 54 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ pyninja start
- **MONITOR_USERNAME** - Username to authenticate the monitoring page.
- **MONITOR_PASSWORD** - Password to authenticate the monitoring page.
- **MONITOR_SESSION** - Session timeout for the monitoring page.
- **NO_AUTH** - Boolean flag to host monitoring page without authentication.
- **MAX_CONNECTIONS** - Maximum number of monitoring sessions allowed in parallel.
- **PROCESSES** - List of process names to include in the monitor page.
- **SERVICES** - List of service names to include in the monitor page.
Expand Down
1 change: 1 addition & 0 deletions docs/README.html
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ <h2>Environment Variables<a class="headerlink" href="#environment-variables" tit
<li><p><strong>MONITOR_USERNAME</strong> - Username to authenticate the monitoring page.</p></li>
<li><p><strong>MONITOR_PASSWORD</strong> - Password to authenticate the monitoring page.</p></li>
<li><p><strong>MONITOR_SESSION</strong> - Session timeout for the monitoring page.</p></li>
<li><p><strong>NO_AUTH</strong> - Boolean flag to host monitoring page without authentication.</p></li>
<li><p><strong>MAX_CONNECTIONS</strong> - Maximum number of monitoring sessions allowed in parallel.</p></li>
<li><p><strong>PROCESSES</strong> - List of process names to include in the monitor page.</p></li>
<li><p><strong>SERVICES</strong> - List of service names to include in the monitor page.</p></li>
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ pyninja start
- **MONITOR_USERNAME** - Username to authenticate the monitoring page.
- **MONITOR_PASSWORD** - Password to authenticate the monitoring page.
- **MONITOR_SESSION** - Session timeout for the monitoring page.
- **NO_AUTH** - Boolean flag to host monitoring page without authentication.
- **MAX_CONNECTIONS** - Maximum number of monitoring sessions allowed in parallel.
- **PROCESSES** - List of process names to include in the monitor page.
- **SERVICES** - List of service names to include in the monitor page.
Expand Down
1 change: 1 addition & 0 deletions docs/_sources/README.md.txt
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ pyninja start
- **MONITOR_USERNAME** - Username to authenticate the monitoring page.
- **MONITOR_PASSWORD** - Password to authenticate the monitoring page.
- **MONITOR_SESSION** - Session timeout for the monitoring page.
- **NO_AUTH** - Boolean flag to host monitoring page without authentication.
- **MAX_CONNECTIONS** - Maximum number of monitoring sessions allowed in parallel.
- **PROCESSES** - List of process names to include in the monitor page.
- **SERVICES** - List of service names to include in the monitor page.
Expand Down
4 changes: 4 additions & 0 deletions docs/genindex.html
Original file line number Diff line number Diff line change
Expand Up @@ -382,6 +382,8 @@ <h2 id="K">K</h2>
<h2 id="L">L</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="index.html#pyninja.monitor.resources.landing_page">landing_page() (in module pyninja.monitor.resources)</a>
</li>
<li><a href="index.html#pyninja.executors.auth.level_1">level_1() (in module pyninja.executors.auth)</a>
</li>
<li><a href="index.html#pyninja.executors.auth.level_2">level_2() (in module pyninja.executors.auth)</a>
Expand Down Expand Up @@ -502,6 +504,8 @@ <h2 id="N">N</h2>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="index.html#pyninja.modules.models.EnvConfig.ninja_port">ninja_port (pyninja.modules.models.EnvConfig attribute)</a>
</li>
<li><a href="index.html#pyninja.modules.models.EnvConfig.no_auth">no_auth (pyninja.modules.models.EnvConfig attribute)</a>
</li>
</ul></td>
</tr></table>
Expand Down
19 changes: 19 additions & 0 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -1686,6 +1686,11 @@ <h2>Models<a class="headerlink" href="#models" title="Permalink to this heading"
<span class="sig-name descname"><span class="pre">max_connections</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><a class="headerlink" href="#pyninja.modules.models.EnvConfig.max_connections" title="Permalink to this definition"></a></dt>
<dd></dd></dl>

<dl class="py attribute">
<dt class="sig sig-object py" id="pyninja.modules.models.EnvConfig.no_auth">
<span class="sig-name descname"><span class="pre">no_auth</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><a class="headerlink" href="#pyninja.modules.models.EnvConfig.no_auth" title="Permalink to this definition"></a></dt>
<dd></dd></dl>

<dl class="py attribute">
<dt class="sig sig-object py" id="pyninja.modules.models.EnvConfig.processes">
<span class="sig-name descname"><span class="pre">processes</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></em><a class="headerlink" href="#pyninja.modules.models.EnvConfig.processes" title="Permalink to this definition"></a></dt>
Expand Down Expand Up @@ -2155,6 +2160,20 @@ <h1>PyNinja - Monitor<a class="headerlink" href="#pyninja-monitor" title="Permal
</section>
<section id="module-pyninja.monitor.resources">
<span id="resources"></span><h2>Resources<a class="headerlink" href="#module-pyninja.monitor.resources" title="Permalink to this heading"></a></h2>
<dl class="py function">
<dt class="sig sig-object py" id="pyninja.monitor.resources.landing_page">
<span class="sig-prename descclassname"><span class="pre">pyninja.monitor.resources.</span></span><span class="sig-name descname"><span class="pre">landing_page</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Any</span><span class="p"><span class="pre">]</span></span></span></span><a class="headerlink" href="#pyninja.monitor.resources.landing_page" title="Permalink to this definition"></a></dt>
<dd><p>Returns the landing page context for monitor endpoint.</p>
<dl class="field-list simple">
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Returns a key-value pair to be inserted into the Jinja template.</p>
</dd>
<dt class="field-even">Return type<span class="colon">:</span></dt>
<dd class="field-even"><p>Dict[str, Any]</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="pyninja.monitor.resources.map_docker_stats">
<span class="sig-prename descclassname"><span class="pre">pyninja.monitor.resources.</span></span><span class="sig-name descname"><span class="pre">map_docker_stats</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">json_data</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></span></span><a class="headerlink" href="#pyninja.monitor.resources.map_docker_stats" title="Permalink to this definition"></a></dt>
Expand Down
Binary file modified docs/objects.inv
Binary file not shown.
2 changes: 1 addition & 1 deletion docs/searchindex.js

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions pyninja/modules/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,7 @@ class EnvConfig(BaseSettings):
monitor_password: str | None = None
monitor_session: PositiveInt = 3_600
max_connections: PositiveInt = 3
no_auth: bool = False
processes: List[str] = []
services: List[str] = []
gpu_lib: FilePath = get_library(GPULib)
Expand Down
4 changes: 4 additions & 0 deletions pyninja/monitor/authenticator.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,10 @@ async def validate_session(host: str, cookie_string: str, log: bool = True) -> N
SessionError:
Raises a SessionError with summary.
"""
if models.env.no_auth:
if log:
LOGGER.info("No auth set! Bypassing auth filters!")
return True
try:
decoded_payload = base64.b64decode(cookie_string)
decoded_str = decoded_payload.decode("ascii")
Expand Down
53 changes: 51 additions & 2 deletions pyninja/monitor/resources.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,66 @@
import json
import logging
import os
import platform
import shutil
import subprocess
from typing import Dict, List
import time
from datetime import timedelta
from typing import Any, Dict, List

import psutil

from pyninja.features import operations
from pyninja.executors import squire
from pyninja.features import cpu, disks, gpu, operations
from pyninja.modules import models

LOGGER = logging.getLogger("uvicorn.default")


def landing_page() -> Dict[str, Any]:
"""Returns the landing page context for monitor endpoint.
Returns:
Dict[str, Any]:
Returns a key-value pair to be inserted into the Jinja template.
"""
uname = platform.uname()
sys_info_basic = {
"System": uname.system,
"Architecture": uname.machine,
"Node": uname.node,
"CPU Cores": psutil.cpu_count(logical=True),
"Uptime": squire.format_timedelta(
timedelta(seconds=time.time() - psutil.boot_time())
),
}
if gpu_names := gpu.get_names():
LOGGER.info(gpu_names)
sys_info_basic["GPU"] = ", ".join(
[gpu_info.get("model") for gpu_info in gpu_names]
)
if processor_name := cpu.get_name():
LOGGER.info("Processor: %s", processor_name)
sys_info_basic["CPU"] = processor_name
sys_info_mem_storage = {
"Memory": squire.size_converter(psutil.virtual_memory().total),
"Disk": squire.size_converter(shutil.disk_usage("/").total),
}
if swap := psutil.swap_memory().total:
sys_info_mem_storage["Swap"] = squire.size_converter(swap)
sys_info_network = {
"Private IP address": squire.private_ip_address(),
"Public IP address": squire.public_ip_address(),
}
return dict(
logout="/logout",
sys_info_basic=dict(sorted(sys_info_basic.items())),
sys_info_mem_storage=dict(sorted(sys_info_mem_storage.items())),
sys_info_network=sys_info_network,
sys_info_disks=disks.get_all_disks(),
)


def map_docker_stats(json_data: Dict[str, str]) -> Dict[str, str]:
"""Map the JSON data to a dictionary.
Expand Down
64 changes: 13 additions & 51 deletions pyninja/monitor/routes.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,15 @@
import asyncio
import logging
import platform
import shutil
import time
from datetime import timedelta
from http import HTTPStatus

import psutil
from fastapi import Cookie, Depends, Request
from fastapi.responses import HTMLResponse, JSONResponse
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from fastapi.websockets import WebSocket, WebSocketDisconnect

from pyninja import monitor, version
from pyninja.executors import squire
from pyninja.features import cpu, disks, gpu
from pyninja.modules import exceptions, models

LOGGER = logging.getLogger("uvicorn.default")
Expand Down Expand Up @@ -59,7 +54,7 @@ async def logout_endpoint(request: Request) -> HTMLResponse:
except exceptions.SessionError as error:
response = await monitor.authenticator.session_error(request, error)
else:
models.ws_session.client_auth.pop(request.client.host)
models.ws_session.client_auth.pop(request.client.host, None)
response = monitor.config.templates.TemplateResponse(
name="logout.html",
context={
Expand Down Expand Up @@ -114,7 +109,7 @@ async def monitor_endpoint(request: Request, session_token: str = Cookie(None)):
"Maximum parallel connections limit reached. Dropping %s", first_key
)
models.ws_session.client_auth.pop(first_key, None)
if session_token:
if session_token or models.env.no_auth:
try:
await monitor.authenticator.validate_session(
request.client.host, session_token
Expand All @@ -124,47 +119,11 @@ async def monitor_endpoint(request: Request, session_token: str = Cookie(None)):
return await monitor.config.clear_session(
await monitor.authenticator.session_error(request, error)
)
else:
uname = platform.uname()
sys_info_basic = {
"System": uname.system,
"Architecture": uname.machine,
"Node": uname.node,
"CPU Cores": psutil.cpu_count(logical=True),
"Uptime": squire.format_timedelta(
timedelta(seconds=time.time() - psutil.boot_time())
),
}
if gpu_names := gpu.get_names():
LOGGER.info(gpu_names)
sys_info_basic["GPU"] = ", ".join(
[gpu_info.get("model") for gpu_info in gpu_names]
)
if processor_name := cpu.get_name():
LOGGER.info("Processor: %s", processor_name)
sys_info_basic["CPU"] = processor_name
sys_info_mem_storage = {
"Memory": squire.size_converter(psutil.virtual_memory().total),
"Disk": squire.size_converter(shutil.disk_usage("/").total),
}
if swap := psutil.swap_memory().total:
sys_info_mem_storage["Swap"] = squire.size_converter(swap)
sys_info_network = {
"Private IP address": squire.private_ip_address(),
"Public IP address": squire.public_ip_address(),
}
ctx = dict(
request=request,
logout="/logout",
sys_info_basic=dict(sorted(sys_info_basic.items())),
sys_info_mem_storage=dict(sorted(sys_info_mem_storage.items())),
sys_info_network=sys_info_network,
sys_info_disks=disks.get_all_disks(),
version=version.__version__,
)
return monitor.config.templates.TemplateResponse(
name="main.html", context=ctx
)
ctx = monitor.resources.landing_page()
ctx["request"] = request
ctx["version"] = version.__version__
LOGGER.info("Rendering initial context for monitoring page!")
return monitor.config.templates.TemplateResponse(name="main.html", context=ctx)
else:
return monitor.config.templates.TemplateResponse(
name="index.html",
Expand Down Expand Up @@ -194,9 +153,12 @@ async def websocket_endpoint(websocket: WebSocket, session_token: str = Cookie(N
await websocket.send_text(error.__str__())
await websocket.close()
return
session_timestamp = models.ws_session.client_auth.get(websocket.client.host).get(
"timestamp"
)
if models.env.no_auth:
session_timestamp = time.time()
else:
session_timestamp = models.ws_session.client_auth.get(
websocket.client.host
).get("timestamp")
# Base task with a placeholder asyncio sleep to start the task loop
task = asyncio.create_task(asyncio.sleep(0.1))
# Store disk usage information (during startup) to avoid repeated calls
Expand Down

0 comments on commit 1926318

Please sign in to comment.