diff --git a/README.md b/README.md index 023d540..ca599b7 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,8 @@ pyninja start - **WORKERS** - Number of workers for the uvicorn server. - **REMOTE_EXECUTION** - Boolean flag to enable remote execution. - **API_SECRET** - Secret access key for running commands on server remotely. +- **DATABASE** - FilePath to store the auth database that handles the authentication errors. +- **RATE_LIMIT** - List of dictionaries with `max_requests` and `seconds` to apply as rate limit. - **APIKEY** - API Key for authentication. ⚠️ Enabling remote execution can be extremely risky and can be a major security threat. So use **caution** and set the **API_SECRET** to a strong value. diff --git a/docs/README.html b/docs/README.html index 9acd69c..24d5343 100644 --- a/docs/README.html +++ b/docs/README.html @@ -95,6 +95,8 @@

Environment Variables

WORKERS - Number of workers for the uvicorn server.

  • REMOTE_EXECUTION - Boolean flag to enable remote execution.

  • API_SECRET - Secret access key for running commands on server remotely.

    • +

  • DATABASE - FilePath to store the auth database that handles the authentication errors.

    • +

  • RATE_LIMIT - List of dictionaries with max_requests and seconds to apply as rate limit.

  • APIKEY - API Key for authentication.

    • ⚠️ Enabling remote execution can be extremely risky and can be a major security threat. So use caution and set the API_SECRET to a strong value.

    diff --git a/docs/README.md b/docs/README.md index 023d540..ca599b7 100644 --- a/docs/README.md +++ b/docs/README.md @@ -58,6 +58,8 @@ pyninja start - **WORKERS** - Number of workers for the uvicorn server. - **REMOTE_EXECUTION** - Boolean flag to enable remote execution. - **API_SECRET** - Secret access key for running commands on server remotely. +- **DATABASE** - FilePath to store the auth database that handles the authentication errors. +- **RATE_LIMIT** - List of dictionaries with `max_requests` and `seconds` to apply as rate limit. - **APIKEY** - API Key for authentication. ⚠️ Enabling remote execution can be extremely risky and can be a major security threat. So use **caution** and set the **API_SECRET** to a strong value. diff --git a/docs/_sources/README.md.txt b/docs/_sources/README.md.txt index 023d540..ca599b7 100644 --- a/docs/_sources/README.md.txt +++ b/docs/_sources/README.md.txt @@ -58,6 +58,8 @@ pyninja start - **WORKERS** - Number of workers for the uvicorn server. - **REMOTE_EXECUTION** - Boolean flag to enable remote execution. - **API_SECRET** - Secret access key for running commands on server remotely. +- **DATABASE** - FilePath to store the auth database that handles the authentication errors. +- **RATE_LIMIT** - List of dictionaries with `max_requests` and `seconds` to apply as rate limit. - **APIKEY** - API Key for authentication. ⚠️ Enabling remote execution can be extremely risky and can be a major security threat. So use **caution** and set the **API_SECRET** to a strong value. diff --git a/docs/index.html b/docs/index.html index 9042246..8cd0965 100644 --- a/docs/index.html +++ b/docs/index.html @@ -70,7 +70,17 @@

    Welcome to PyNinja’s documentation!
    Keyword Arguments:
    -

    env_file – Filepath for the .env file.

    +
      +

    • variables. (- env_file - Env filepath to load the environment) –

      • +

    • server. (- workers - Number of workers for the uvicorn) –

      • +

    • server. –

      • +

    • server. –

      • +

    • execution. (- remote_execution - Boolean flag to enable remote) –

      • +

    • remotely. (- api_secret - Secret access key for running commands on server) –

      • +

    • errors. (- database - FilePath to store the auth database that handles the authentication) –

      • +

    • limit. (- rate_limit - List of dictionaries with max_requests and seconds to apply as rate) –

      • +

    • authentication. (- apikey - API Key for) –

      • +
    @@ -180,7 +190,10 @@

    Welcome to PyNinja’s documentation!
    -async pyninja.routers.process_status(request: Request, process_name: str, apikey: HTTPAuthorizationCredentials = Depends(HTTPBearer))
    +async pyninja.routers.process_status(request: Request, process_name: str, cpu_interval: Union[int, float] = 1, apikey: HTTPAuthorizationCredentials = Depends(HTTPBearer))

    API function to monitor a process.

    Args:

    -

    process_name: Name of the process to check status.

    +

    request: Reference to the FastAPI request object. +process_name: Name of the process to check status. +cpu_interval: Interval in seconds to get the CPU usage. +apikey: API Key to authenticate the request.

    Raises:

    @@ -210,7 +226,9 @@

    Welcome to PyNinja’s documentation!

    Process

    -pyninja.process.get_process_status(process_name: str) → Generator[Dict[str, int]]
    +pyninja.process.get_process_status(process_name: str, cpu_interval: int) → List[Dict[str, int | float | str | bool]]

    Get process information by name.

    Parameters:
    -

    process_name – Name of the process.

    +
      +

    • process_name – Name of the process.

      • +

    • cpu_interval – CPU interval to get the CPU performance.

      • +
    -
    Yields:
    -

    Generator[Dict[str, int]] – Yields the process metrics as a dictionary of key-value pairs.

    +
    Returns:
    +

    Returns a list of performance report for each process hosting the given process name.

    +
    +
    Return type:
    +

    List[Dict[str, int | float | str | bool]]

    -pyninja.process.get_performance(process: Process) → Dict[str, int | float]
    -

    Checks performance by monitoring CPU utilization, number of threads and open files.

    +pyninja.process.get_performance(process: Process, cpu_interval: int) → Dict[str, int | float | str | bool] +

    Checks process performance by monitoring CPU utilization, number of threads and open files.

    Parameters:
    -

    process – Process object.

    +
      +

    • process – Process object.

      • +

    • cpu_interval – CPU interval to get the CPU performance.

      • +
    Returns:

    Returns the process metrics as key-value pairs.

    Return type:
    -

    Dict[str, int]

    +

    Dict[str, int | float | str | bool]

    @@ -289,13 +316,13 @@

    Monitors
    pyninja.service.get_service_status(service_name: str) → ServiceStatus
    -

    Get service status.

    +

    Get service status by name.

    Parameters:
    -

    service_name (str) – Name of the service.

    +

    service_name – Name of the service.

    Returns:
    -

    Returns an instance of the ServiceStatus.

    +

    Returns an instance of the ServiceStatus object.

    Return type:

    ServiceStatus

    @@ -413,7 +440,7 @@

    Models
    -timeout: Union[int, float]
    +timeout: int

    diff --git a/docs/searchindex.js b/docs/searchindex.js index 8795b6f..adb0537 100644 --- a/docs/searchindex.js +++ b/docs/searchindex.js @@ -1 +1 @@ -Search.setIndex({"docnames": ["README", "index"], "filenames": ["README.md", "index.rst"], "titles": ["PyNinja", "Welcome to PyNinja\u2019s documentation!"], "terms": {"lightweight": 0, "o": [0, 1], "agnost": 0, "servic": 0, "monitor": 0, "api": [0, 1], "platform": 0, "support": 0, "deploy": 0, "recommend": 0, "instal": 0, "python": 0, "3": 0, "10": [0, 1], "11": 0, "us": [0, 1], "dedic": 0, "virtual": 0, "m": 0, "pip": 0, "initi": 0, "id": 0, "import": 0, "__name__": 0, "__main__": 0, "start": [0, 1], "cli": 0, "help": 0, "usag": 0, "instruct": 0, "sourc": 0, "from": [0, 1], "an": [0, 1], "env": [0, 1], "file": [0, 1], "By": 0, "default": 0, "look": 0, "current": 0, "work": 0, "directori": 0, "ninja": 0, "_": 0, "host": [0, 1], "hostnam": 0, "server": [0, 1], "port": 0, "number": [0, 1], "worker": [0, 1], "uvicorn": [0, 1], "remot": 0, "execut": 0, "boolean": 0, "flag": 0, "enabl": 0, "secret": [0, 1], "access": 0, "kei": [0, 1], "run": [0, 1], "command": [0, 1], "apikei": [0, 1], "authent": 0, "can": 0, "extrem": 0, "riski": 0, "major": 0, "secur": [0, 1], "threat": 0, "so": 0, "caution": 0, "set": [0, 1], "strong": [0, 1], "valu": [0, 1], "log": 0, "ini": 0, "configur": [0, 1], "custom": [0, 1], "just": 0, "place": 0, "refer": [0, 1], "sampl": 0, "exampl": 0, "docstr": 0, "format": 0, "googl": 0, "style": 0, "convent": 0, "pep": 0, "8": 0, "isort": 0, "requir": [0, 1], "gitvers": 0, "revers": 0, "f": 0, "release_not": 0, "rst": 0, "t": 0, "pre": 0, "commit": 0, "ensur": 0, "pytest": 0, "gener": [0, 1], "valid": [0, 1], "hyperlink": 0, "all": [0, 1], "markdown": 0, "includ": 0, "wiki": 0, "page": [0, 1], "sphinx": 0, "5": 0, "1": [0, 1], "recommonmark": 0, "http": 0, "org": 0, "project": 0, "thevickypedia": 0, "github": 0, "io": 0, "vignesh": 0, "rao": 0, "under": 0, "mit": 0, "kick": 1, "off": 1, "environ": 1, "variabl": 1, "code": 1, "standard": 1, "releas": 1, "note": 1, "lint": 1, "pypi": 1, "packag": 1, "runbook": 1, "licens": 1, "copyright": 1, "kwarg": 1, "none": 1, "starter": 1, "function": 1, "which": 1, "trigger": 1, "keyword": 1, "argument": 1, "env_fil": 1, "filepath": 1, "auth": 1, "epoch": 1, "async": 1, "forbidden": 1, "request": 1, "i": 1, "part": 1, "list": 1, "paramet": 1, "fastapi": 1, "object": 1, "rais": 1, "apirespons": 1, "403": 1, "If": 1, "address": 1, "level_1": 1, "httpauthorizationcredenti": 1, "httpbearer": 1, "take": 1, "author": 1, "header": 1, "token": 1, "basic": 1, "rout": 1, "401": 1, "invalid": 1, "level_2": 1, "str": 1, "addition": 1, "addit": 1, "critic": 1, "increment": 1, "attempt": 1, "int": 1, "block": 1, "time": 1, "base": 1, "fail": 1, "return": 1, "appropri": 1, "minut": 1, "type": 1, "handle_auth_error": 1, "handl": 1, "error": 1, "filebrows": 1, "The": 1, "incom": 1, "run_command": 1, "payload": 1, "depend": 1, "option": 1, "machin": 1, "arg": 1, "receiv": 1, "bodi": 1, "httpstatu": 1, "statu": 1, "detail": 1, "respons": 1, "process_statu": 1, "process_nam": 1, "name": 1, "check": 1, "service_statu": 1, "service_nam": 1, "doc": 1, "redirectrespons": 1, "redirect": 1, "user": 1, "get_all_rout": 1, "apirout": 1, "get": 1, "ad": 1, "get_process_statu": 1, "dict": 1, "inform": 1, "yield": 1, "metric": 1, "dictionari": 1, "pair": 1, "get_perform": 1, "float": 1, "perform": 1, "cpu": 1, "util": 1, "thread": 1, "open": 1, "get_service_statu": 1, "servicestatu": 1, "instanc": 1, "get_record": 1, "particular": 1, "until": 1, "when": 1, "should": 1, "put_record": 1, "block_until": 1, "insert": 1, "remove_record": 1, "delet": 1, "record": 1, "relat": 1, "class": 1, "rate_limit": 1, "rp": 1, "implement": 1, "init": 1, "call": 1, "exce": 1, "rate": 1, "limit": 1, "given": 1, "identifi": 1, "429": 1, "too": 1, "mani": 1, "status_cod": 1, "ani": 1, "httpexcept": 1, "wrap": 1, "unsupportedo": 1, "unsupport": 1, "basemodel": 1, "input": 1, "data": 1, "timeout": 1, "union": 1, "load": 1, "descript": 1, "session": 1, "store": 1, "auth_count": 1, "forbid": 1, "info": 1, "allowed_origin": 1, "max_request": 1, "second": 1, "envconfig": 1, "ninja_host": 1, "ninja_port": 1, "remote_execut": 1, "bool": 1, "api_secret": 1, "classmethod": 1, "parse_api_secret": 1, "pars": 1, "complex": 1, "from_env_fil": 1, "path": 1, "creat": 1, "config": 1, "extra": 1, "ignor": 1, "hide_input_in_error": 1, "true": 1, "complexity_check": 1, "verifi": 1, "strength": 1, "A": 1, "consid": 1, "least": 1, "ha": 1, "32": 1, "charact": 1, "digit": 1, "symbol": 1, "uppercas": 1, "letter": 1, "lowercas": 1, "assertionerror": 1, "abov": 1, "condit": 1, "match": 1, "datastor": 1, "connect": 1, "sqlite3": 1, "create_t": 1, "table_nam": 1, "column": 1, "tupl": 1, "process_command": 1, "string": 1, "result": 1, "stdout": 1, "stderr": 1, "env_load": 1, "filenam": 1, "pathlik": 1, "filetyp": 1, "where": 1, "var": 1, "have": 1, "index": 1, "modul": 1, "search": 1}, "objects": {"pyninja": [[1, 0, 0, "-", "auth"], [1, 0, 0, "-", "database"], [1, 0, 0, "-", "exceptions"], [1, 0, 0, "-", "main"], [1, 0, 0, "-", "models"], [1, 0, 0, "-", "process"], [1, 0, 0, "-", "rate_limit"], [1, 0, 0, "-", "routers"], [1, 0, 0, "-", "service"], [1, 0, 0, "-", "squire"]], "pyninja.auth": [[1, 1, 1, "", "EPOCH"], [1, 1, 1, "", "forbidden"], [1, 1, 1, "", "handle_auth_error"], [1, 1, 1, "", "incrementer"], [1, 1, 1, "", "level_1"], [1, 1, 1, "", "level_2"]], "pyninja.database": [[1, 1, 1, "", "get_record"], [1, 1, 1, "", "put_record"], [1, 1, 1, "", "remove_record"]], "pyninja.exceptions": [[1, 2, 1, "", "APIResponse"], [1, 2, 1, "", "UnSupportedOS"]], "pyninja.main": [[1, 1, 1, "", "start"]], "pyninja.models": [[1, 3, 1, "", "Database"], [1, 3, 1, "", "EnvConfig"], [1, 3, 1, "", "Payload"], [1, 3, 1, "", "RateLimit"], [1, 3, 1, "", "ServiceStatus"], [1, 3, 1, "", "Session"], [1, 1, 1, "", "complexity_checker"]], "pyninja.models.Database": [[1, 4, 1, "", "create_table"]], "pyninja.models.EnvConfig": [[1, 3, 1, "", "Config"], [1, 5, 1, "", "api_secret"], [1, 5, 1, "", "apikey"], [1, 5, 1, "", "database"], [1, 4, 1, "", "from_env_file"], [1, 5, 1, "", "ninja_host"], [1, 5, 1, "", "ninja_port"], [1, 4, 1, "", "parse_api_secret"], [1, 5, 1, "", "rate_limit"], [1, 5, 1, "", "remote_execution"], [1, 5, 1, "", "workers"]], "pyninja.models.EnvConfig.Config": [[1, 5, 1, "", "extra"], [1, 5, 1, "", "hide_input_in_errors"]], "pyninja.models.Payload": [[1, 5, 1, "", "command"], [1, 5, 1, "", "timeout"]], "pyninja.models.RateLimit": [[1, 5, 1, "", "max_requests"], [1, 5, 1, "", "seconds"]], "pyninja.models.ServiceStatus": [[1, 5, 1, "", "description"], [1, 5, 1, "", "status_code"]], "pyninja.models.Session": [[1, 5, 1, "", "allowed_origins"], [1, 5, 1, "", "auth_counter"], [1, 5, 1, "", "forbid"], [1, 5, 1, "", "info"], [1, 5, 1, "", "rps"]], "pyninja.process": [[1, 1, 1, "", "get_performance"], [1, 1, 1, "", "get_process_status"]], "pyninja.rate_limit": [[1, 3, 1, "", "RateLimiter"]], "pyninja.rate_limit.RateLimiter": [[1, 4, 1, "", "init"]], "pyninja.routers": [[1, 1, 1, "", "docs"], [1, 1, 1, "", "get_all_routes"], [1, 1, 1, "", "process_status"], [1, 1, 1, "", "run_command"], [1, 1, 1, "", "service_status"]], "pyninja.service": [[1, 1, 1, "", "get_service_status"]], "pyninja.squire": [[1, 1, 1, "", "env_loader"], [1, 1, 1, "", "process_command"]]}, "objtypes": {"0": "py:module", "1": "py:function", "2": "py:exception", "3": "py:class", "4": "py:method", "5": "py:attribute"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "function", "Python function"], "2": ["py", "exception", "Python exception"], "3": ["py", "class", "Python class"], "4": ["py", "method", "Python method"], "5": ["py", "attribute", "Python attribute"]}, "titleterms": {"pyninja": [0, 1], "kick": 0, "off": 0, "environ": 0, "variabl": 0, "code": 0, "standard": 0, "releas": 0, "note": 0, "lint": 0, "pypi": 0, "packag": 0, "runbook": 0, "licens": 0, "copyright": 0, "welcom": 1, "": 1, "document": 1, "content": 1, "main": 1, "authent": 1, "router": 1, "monitor": 1, "process": 1, "servic": 1, "databas": 1, "ratelimit": 1, "except": 1, "model": 1, "squir": 1, "indic": 1, "tabl": 1}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 6, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx": 56}}) \ No newline at end of file +Search.setIndex({"docnames": ["README", "index"], "filenames": ["README.md", "index.rst"], "titles": ["PyNinja", "Welcome to PyNinja\u2019s documentation!"], "terms": {"lightweight": 0, "o": [0, 1], "agnost": 0, "servic": 0, "monitor": 0, "api": [0, 1], "platform": 0, "support": 0, "deploy": 0, "recommend": 0, "instal": 0, "python": 0, "3": 0, "10": [0, 1], "11": 0, "us": [0, 1], "dedic": 0, "virtual": 0, "m": 0, "pip": 0, "initi": 0, "id": 0, "import": 0, "__name__": 0, "__main__": 0, "start": [0, 1], "cli": 0, "help": 0, "usag": [0, 1], "instruct": 0, "sourc": 0, "from": [0, 1], "an": [0, 1], "env": [0, 1], "file": [0, 1], "By": 0, "default": 0, "look": 0, "current": 0, "work": 0, "directori": 0, "ninja": 0, "_": 0, "host": [0, 1], "hostnam": 0, "server": [0, 1], "port": 0, "number": [0, 1], "worker": [0, 1], "uvicorn": [0, 1], "remot": [0, 1], "execut": [0, 1], "boolean": [0, 1], "flag": [0, 1], "enabl": [0, 1], "secret": [0, 1], "access": [0, 1], "kei": [0, 1], "run": [0, 1], "command": [0, 1], "databas": 0, "filepath": [0, 1], "store": [0, 1], "auth": [0, 1], "handl": [0, 1], "authent": 0, "error": [0, 1], "rate": [0, 1], "limit": [0, 1], "list": [0, 1], "dictionari": [0, 1], "max_request": [0, 1], "second": [0, 1], "appli": [0, 1], "apikei": [0, 1], "can": 0, "extrem": 0, "riski": 0, "major": 0, "secur": [0, 1], "threat": 0, "so": 0, "caution": 0, "set": [0, 1], "strong": [0, 1], "valu": [0, 1], "log": 0, "ini": 0, "configur": [0, 1], "custom": [0, 1], "just": 0, "place": 0, "refer": [0, 1], "sampl": 0, "exampl": 0, "docstr": 0, "format": 0, "googl": 0, "style": 0, "convent": 0, "pep": 0, "8": 0, "isort": 0, "requir": [0, 1], "gitvers": 0, "revers": 0, "f": 0, "release_not": 0, "rst": 0, "t": 0, "pre": 0, "commit": 0, "ensur": 0, "pytest": 0, "gener": 0, "valid": [0, 1], "hyperlink": 0, "all": [0, 1], "markdown": 0, "includ": 0, "wiki": 0, "page": [0, 1], "sphinx": 0, "5": 0, "1": [0, 1], "recommonmark": 0, "http": 0, "org": 0, "project": 0, "thevickypedia": 0, "github": 0, "io": 0, "vignesh": 0, "rao": 0, "under": 0, "mit": 0, "kick": 1, "off": 1, "environ": 1, "variabl": 1, "code": 1, "standard": 1, "releas": 1, "note": 1, "lint": 1, "pypi": 1, "packag": 1, "runbook": 1, "licens": 1, "copyright": 1, "kwarg": 1, "none": 1, "starter": 1, "function": 1, "which": 1, "trigger": 1, "keyword": 1, "argument": 1, "env_fil": 1, "load": 1, "remote_execut": 1, "api_secret": 1, "rate_limit": 1, "epoch": 1, "async": 1, "forbidden": 1, "request": 1, "i": 1, "part": 1, "paramet": 1, "fastapi": 1, "object": 1, "rais": 1, "apirespons": 1, "403": 1, "If": 1, "address": 1, "level_1": 1, "httpauthorizationcredenti": 1, "httpbearer": 1, "take": 1, "author": 1, "header": 1, "token": 1, "basic": 1, "rout": 1, "401": 1, "invalid": 1, "level_2": 1, "str": 1, "addition": 1, "addit": 1, "critic": 1, "increment": 1, "attempt": 1, "int": 1, "block": 1, "time": 1, "base": 1, "fail": 1, "return": 1, "appropri": 1, "minut": 1, "type": 1, "handle_auth_error": 1, "filebrows": 1, "The": 1, "incom": 1, "run_command": 1, "payload": 1, "depend": 1, "option": 1, "machin": 1, "arg": 1, "receiv": 1, "bodi": 1, "httpstatu": 1, "statu": 1, "detail": 1, "respons": 1, "process_statu": 1, "process_nam": 1, "cpu_interv": 1, "union": 1, "float": 1, "name": 1, "check": 1, "interv": 1, "get": 1, "cpu": 1, "service_statu": 1, "service_nam": 1, "doc": 1, "redirectrespons": 1, "redirect": 1, "user": 1, "get_all_rout": 1, "apirout": 1, "ad": 1, "get_process_statu": 1, "dict": 1, "bool": 1, "inform": 1, "perform": 1, "report": 1, "each": 1, "given": 1, "get_perform": 1, "util": 1, "thread": 1, "open": 1, "metric": 1, "pair": 1, "get_service_statu": 1, "servicestatu": 1, "instanc": 1, "get_record": 1, "particular": 1, "until": 1, "when": 1, "should": 1, "put_record": 1, "block_until": 1, "insert": 1, "remove_record": 1, "delet": 1, "record": 1, "relat": 1, "class": 1, "rp": 1, "implement": 1, "init": 1, "call": 1, "exce": 1, "identifi": 1, "429": 1, "too": 1, "mani": 1, "status_cod": 1, "ani": 1, "httpexcept": 1, "wrap": 1, "unsupportedo": 1, "unsupport": 1, "basemodel": 1, "input": 1, "data": 1, "timeout": 1, "descript": 1, "session": 1, "auth_count": 1, "forbid": 1, "info": 1, "allowed_origin": 1, "envconfig": 1, "ninja_host": 1, "ninja_port": 1, "classmethod": 1, "parse_api_secret": 1, "pars": 1, "complex": 1, "from_env_fil": 1, "path": 1, "creat": 1, "config": 1, "extra": 1, "ignor": 1, "hide_input_in_error": 1, "true": 1, "complexity_check": 1, "verifi": 1, "strength": 1, "A": 1, "consid": 1, "least": 1, "ha": 1, "32": 1, "charact": 1, "digit": 1, "symbol": 1, "uppercas": 1, "letter": 1, "lowercas": 1, "assertionerror": 1, "abov": 1, "condit": 1, "match": 1, "datastor": 1, "connect": 1, "sqlite3": 1, "create_t": 1, "table_nam": 1, "column": 1, "tupl": 1, "process_command": 1, "string": 1, "result": 1, "stdout": 1, "stderr": 1, "env_load": 1, "filenam": 1, "pathlik": 1, "filetyp": 1, "where": 1, "var": 1, "have": 1, "index": 1, "modul": 1, "search": 1}, "objects": {"pyninja": [[1, 0, 0, "-", "auth"], [1, 0, 0, "-", "database"], [1, 0, 0, "-", "exceptions"], [1, 0, 0, "-", "main"], [1, 0, 0, "-", "models"], [1, 0, 0, "-", "process"], [1, 0, 0, "-", "rate_limit"], [1, 0, 0, "-", "routers"], [1, 0, 0, "-", "service"], [1, 0, 0, "-", "squire"]], "pyninja.auth": [[1, 1, 1, "", "EPOCH"], [1, 1, 1, "", "forbidden"], [1, 1, 1, "", "handle_auth_error"], [1, 1, 1, "", "incrementer"], [1, 1, 1, "", "level_1"], [1, 1, 1, "", "level_2"]], "pyninja.database": [[1, 1, 1, "", "get_record"], [1, 1, 1, "", "put_record"], [1, 1, 1, "", "remove_record"]], "pyninja.exceptions": [[1, 2, 1, "", "APIResponse"], [1, 2, 1, "", "UnSupportedOS"]], "pyninja.main": [[1, 1, 1, "", "start"]], "pyninja.models": [[1, 3, 1, "", "Database"], [1, 3, 1, "", "EnvConfig"], [1, 3, 1, "", "Payload"], [1, 3, 1, "", "RateLimit"], [1, 3, 1, "", "ServiceStatus"], [1, 3, 1, "", "Session"], [1, 1, 1, "", "complexity_checker"]], "pyninja.models.Database": [[1, 4, 1, "", "create_table"]], "pyninja.models.EnvConfig": [[1, 3, 1, "", "Config"], [1, 5, 1, "", "api_secret"], [1, 5, 1, "", "apikey"], [1, 5, 1, "", "database"], [1, 4, 1, "", "from_env_file"], [1, 5, 1, "", "ninja_host"], [1, 5, 1, "", "ninja_port"], [1, 4, 1, "", "parse_api_secret"], [1, 5, 1, "", "rate_limit"], [1, 5, 1, "", "remote_execution"], [1, 5, 1, "", "workers"]], "pyninja.models.EnvConfig.Config": [[1, 5, 1, "", "extra"], [1, 5, 1, "", "hide_input_in_errors"]], "pyninja.models.Payload": [[1, 5, 1, "", "command"], [1, 5, 1, "", "timeout"]], "pyninja.models.RateLimit": [[1, 5, 1, "", "max_requests"], [1, 5, 1, "", "seconds"]], "pyninja.models.ServiceStatus": [[1, 5, 1, "", "description"], [1, 5, 1, "", "status_code"]], "pyninja.models.Session": [[1, 5, 1, "", "allowed_origins"], [1, 5, 1, "", "auth_counter"], [1, 5, 1, "", "forbid"], [1, 5, 1, "", "info"], [1, 5, 1, "", "rps"]], "pyninja.process": [[1, 1, 1, "", "get_performance"], [1, 1, 1, "", "get_process_status"]], "pyninja.rate_limit": [[1, 3, 1, "", "RateLimiter"]], "pyninja.rate_limit.RateLimiter": [[1, 4, 1, "", "init"]], "pyninja.routers": [[1, 1, 1, "", "docs"], [1, 1, 1, "", "get_all_routes"], [1, 1, 1, "", "process_status"], [1, 1, 1, "", "run_command"], [1, 1, 1, "", "service_status"]], "pyninja.service": [[1, 1, 1, "", "get_service_status"]], "pyninja.squire": [[1, 1, 1, "", "env_loader"], [1, 1, 1, "", "process_command"]]}, "objtypes": {"0": "py:module", "1": "py:function", "2": "py:exception", "3": "py:class", "4": "py:method", "5": "py:attribute"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "function", "Python function"], "2": ["py", "exception", "Python exception"], "3": ["py", "class", "Python class"], "4": ["py", "method", "Python method"], "5": ["py", "attribute", "Python attribute"]}, "titleterms": {"pyninja": [0, 1], "kick": 0, "off": 0, "environ": 0, "variabl": 0, "code": 0, "standard": 0, "releas": 0, "note": 0, "lint": 0, "pypi": 0, "packag": 0, "runbook": 0, "licens": 0, "copyright": 0, "welcom": 1, "": 1, "document": 1, "content": 1, "main": 1, "authent": 1, "router": 1, "monitor": 1, "process": 1, "servic": 1, "databas": 1, "ratelimit": 1, "except": 1, "model": 1, "squir": 1, "indic": 1, "tabl": 1}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 6, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx": 56}}) \ No newline at end of file diff --git a/pyninja/auth.py b/pyninja/auth.py index 61640c8..287dc45 100644 --- a/pyninja/auth.py +++ b/pyninja/auth.py @@ -9,7 +9,7 @@ from pyninja import database, exceptions, models -LOGGER = logging.getLogger("uvicorn.error") +LOGGER = logging.getLogger("uvicorn.default") EPOCH = lambda: int(time.time()) # noqa: E731 SECURITY = HTTPBearer() diff --git a/pyninja/main.py b/pyninja/main.py index b998488..7f5cd71 100644 --- a/pyninja/main.py +++ b/pyninja/main.py @@ -7,14 +7,22 @@ import pyninja from pyninja import models, routers, squire -LOGGER = logging.getLogger("uvicorn.error") +LOGGER = logging.getLogger("uvicorn.default") def start(**kwargs) -> None: """Starter function for the API, which uses uvicorn server as trigger. Keyword Args: - env_file: Filepath for the ``.env`` file. + - env_file - Env filepath to load the environment variables. + - ninja_host - Hostname for the API server. + - ninja_port - Port number for the API server. + - workers - Number of workers for the uvicorn server. + - remote_execution - Boolean flag to enable remote execution. + - api_secret - Secret access key for running commands on server remotely. + - database - FilePath to store the auth database that handles the authentication errors. + - rate_limit - List of dictionaries with `max_requests` and `seconds` to apply as rate limit. + - apikey - API Key for authentication. """ if env_file := kwargs.get("env_file"): models.env = squire.env_loader(env_file) diff --git a/pyninja/models.py b/pyninja/models.py index 71947e7..be040dc 100644 --- a/pyninja/models.py +++ b/pyninja/models.py @@ -4,14 +4,7 @@ import sqlite3 from typing import Dict, List, Set, Tuple -from pydantic import ( - BaseModel, - Field, - FilePath, - PositiveFloat, - PositiveInt, - field_validator, -) +from pydantic import BaseModel, Field, FilePath, PositiveInt, field_validator from pydantic_settings import BaseSettings @@ -60,7 +53,7 @@ class Payload(BaseModel): """ command: str - timeout: PositiveInt | PositiveFloat = 3 + timeout: PositiveInt = 3 class ServiceStatus(BaseModel): diff --git a/pyninja/process.py b/pyninja/process.py index 6a9671d..eb88c25 100644 --- a/pyninja/process.py +++ b/pyninja/process.py @@ -1,54 +1,76 @@ import logging -from collections.abc import Generator -from typing import Dict +import os +from concurrent.futures import ThreadPoolExecutor, as_completed +from typing import Dict, List import psutil +from pydantic import PositiveInt -LOGGER = logging.getLogger("uvicorn.error") +LOGGER = logging.getLogger("uvicorn.default") -def get_process_status(process_name: str) -> Generator[Dict[str, int]]: +def get_process_status( + process_name: str, cpu_interval: PositiveInt +) -> List[Dict[str, int | float | str | bool]]: """Get process information by name. Args: process_name: Name of the process. + cpu_interval: CPU interval to get the CPU performance. - Yields: - Generator[Dict[str, int]]: - Yields the process metrics as a dictionary of key-value pairs. + Returns: + List[Dict[str, int | float | str | bool]]: + Returns a list of performance report for each process hosting the given process name. """ - # todo: implement concurrency - for proc in psutil.process_iter(["pid", "name"]): - if proc.name().lower() == process_name.lower(): - process = psutil.Process(proc.pid) - process._name = process_name - try: - perf_report = get_performance(process) - LOGGER.info({f"{process_name} [{process.pid}]": perf_report}) - perf_report["pname"] = process_name - perf_report["zombie"] = False - yield perf_report - except psutil.ZombieProcess as warn: - LOGGER.warning(warn) - yield {"zombie": True, "process_name": process_name} - - -def get_performance(process: psutil.Process) -> Dict[str, int | float]: - """Checks performance by monitoring CPU utilization, number of threads and open files. + result = [] + futures = {} + executor = ThreadPoolExecutor(max_workers=os.cpu_count()) + with executor: + for proc in psutil.process_iter(["pid", "name"]): + if proc.name().lower() == process_name.lower(): + future = executor.submit( + get_performance, process=proc, cpu_interval=cpu_interval + ) + futures[future] = proc.name() + for future in as_completed(futures): + if future.exception(): + LOGGER.error( + "Thread processing for '%s' received an exception: %s", + futures[future], + future.exception(), + ) + else: + result.append(future.result()) + return result + + +def get_performance( + process: psutil.Process, cpu_interval: PositiveInt +) -> Dict[str, int | float | str | bool]: + """Checks process performance by monitoring CPU utilization, number of threads and open files. Args: process: Process object. + cpu_interval: CPU interval to get the CPU performance. Returns: - Dict[str, int]: + Dict[str, int | float | str | bool]: Returns the process metrics as key-value pairs. """ - cpu = process.cpu_percent(interval=0.5) - threads = process.num_threads() - open_files = len(process.open_files()) - return { - "cpu": cpu, - "threads": threads, - "open_files": open_files, - "pid": process.pid.real, - } + try: + cpu = process.cpu_percent(interval=cpu_interval) + threads = process.num_threads() + open_files = len(process.open_files()) + perf_report = { + "pid": process.pid.real, + "pname": process.name(), + "cpu": cpu, + "threads": threads, + "open_files": open_files, + "zombie": False, + } + LOGGER.info({f"{process.name()} [{process.pid}]": perf_report}) + except psutil.ZombieProcess as warn: + LOGGER.warning(warn) + perf_report = {"zombie": True, "process_name": process.name()} + return perf_report diff --git a/pyninja/routers.py b/pyninja/routers.py index 1709e17..3a44676 100644 --- a/pyninja/routers.py +++ b/pyninja/routers.py @@ -7,10 +7,11 @@ from fastapi.responses import RedirectResponse from fastapi.routing import APIRoute from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer +from pydantic import PositiveFloat, PositiveInt from pyninja import auth, exceptions, models, process, rate_limit, service, squire -LOGGER = logging.getLogger("uvicorn.error") +LOGGER = logging.getLogger("uvicorn.default") security = HTTPBearer() @@ -24,7 +25,10 @@ async def run_command( **Args:** + request: Reference to the FastAPI request object. payload: Payload received as request body. + apikey: API Key to authenticate the request. + token: API secret to authenticate the request. **Raises:** @@ -32,7 +36,9 @@ async def run_command( Raises the HTTPStatus object with a status code and detail as response. """ await auth.level_2(request, apikey, token) - LOGGER.info("Requested command: '%s'", payload.command) + LOGGER.info( + "Requested command: '%s' with timeout: %ds", payload.command, payload.timeout + ) try: response = squire.process_command(payload.command, payload.timeout) except subprocess.TimeoutExpired as warn: @@ -46,13 +52,17 @@ async def run_command( async def process_status( request: Request, process_name: str, + cpu_interval: PositiveInt | PositiveFloat = 1, apikey: HTTPAuthorizationCredentials = Depends(security), ): """**API function to monitor a process.** **Args:** + request: Reference to the FastAPI request object. process_name: Name of the process to check status. + cpu_interval: Interval in seconds to get the CPU usage. + apikey: API Key to authenticate the request. **Raises:** @@ -60,7 +70,7 @@ async def process_status( Raises the HTTPStatus object with a status code and detail as response. """ await auth.level_1(request, apikey) - if response := list(process.get_process_status(process_name)): + if response := process.get_process_status(process_name, cpu_interval): raise exceptions.APIResponse(status_code=HTTPStatus.OK.real, detail=response) LOGGER.error("%s: 404 - No such process", process_name) raise exceptions.APIResponse( @@ -77,7 +87,9 @@ async def service_status( **Args:** + request: Reference to the FastAPI request object. service_name: Name of the service to check status. + apikey: API Key to authenticate the request. **Raises:** @@ -114,12 +126,12 @@ def get_all_routes() -> List[APIRoute]: List[APIRoute]: Returns the routes as a list of APIRoute objects. """ - APIRoute(path="/", endpoint=docs, methods=["GET"], include_in_schema=False) dependencies = [ Depends(dependency=rate_limit.RateLimiter(each_rate_limit).init) for each_rate_limit in models.env.rate_limit ] routes = [ + APIRoute(path="/", endpoint=docs, methods=["GET"], include_in_schema=False), APIRoute( path="/service-status", endpoint=service_status, diff --git a/pyninja/service.py b/pyninja/service.py index a955bc4..d8b498b 100644 --- a/pyninja/service.py +++ b/pyninja/service.py @@ -14,14 +14,14 @@ def get_service_status(service_name: str) -> models.ServiceStatus: - """Get service status. + """Get service status by name. Args: - service_name (str): Name of the service. + service_name: Name of the service. Returns: ServiceStatus: - Returns an instance of the ServiceStatus. + Returns an instance of the ServiceStatus object. """ running = models.ServiceStatus( status_code=HTTPStatus.OK.real, diff --git a/pyninja/squire.py b/pyninja/squire.py index efc274d..43ee10f 100644 --- a/pyninja/squire.py +++ b/pyninja/squire.py @@ -10,7 +10,7 @@ from pyninja.models import EnvConfig -LOGGER = logging.getLogger("uvicorn.error") +LOGGER = logging.getLogger("uvicorn.default") def process_command(