Fix max recursion depth error during Sphinx hot reload

[Copilot]

During Sphinx hot reloads (e.g., with esbonio), configuration objects can contain circular references, causing get_safe_config to infinitely recurse when serializing to TOML.

Changes

• Track visited objects in traversal path: Added visited: set[int] parameter to get_safe_config to detect circular references
• Smart cleanup: Use try-finally blocks to remove objects from visited after processing, allowing the same object to be referenced from different paths (A→C, B→C) while preventing true cycles (A→B→C→A)
• Graceful handling: Log warning and filter out circular references instead of crashing

def get_safe_config(obj: Any, path: str = "", outpath: Path | None = None, visited: set[int] | None = None) -> Any:
    if visited is None:
        visited = set()
    
    if isinstance(obj, (dict, list, tuple, set)):
        obj_id = id(obj)
        if obj_id in visited:
            log_warning(LOGGER, f"Circular reference detected at '{path}'", ...)
            return None
        visited.add(obj_id)
        try:
            # process object...
        finally:
            visited.discard(obj_id)  # Allow same object from different paths

Test Coverage

Added test_no_recursion_error_on_rebuild to verify rebuild scenarios don't cause recursion errors.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[BUG] Max depth reached error when hot reloading via sphinx (e.g. with esbonio)</issue_title>
<issue_description>There currently seems to be a bug where sphinx errors due to this extension as it reaches the max depth recursion.
Error:

ERROR: sphinx-build failed
Traceback (most recent call last):
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/sphinx/events.py", line 404, in emit
    results.append(listener.handler(self.app, *args))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/needs_config_writer/main.py", line 19, in write
    write_needscfg_file(app, app.config)
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/needs_config_writer/builder.py", line 307, in write_needscfg_file
    safe_value = get_safe_config(value, f"needs.{config_name}", outpath)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/needs_config_writer/builder.py", line 192, in get_safe_config
    safe_value = get_safe_config(item, item_path, outpath)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/needs_config_writer/builder.py", line 183, in get_safe_config
    safe_value = get_safe_config(value, item_path, outpath)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/needs_config_writer/builder.py", line 183, in get_safe_config
    safe_value = get_safe_config(value, item_path, outpath)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/needs_config_writer/builder.py", line 192, in get_safe_config
    safe_value = get_safe_config(item, item_path, outpath)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/needs_config_writer/builder.py", line 183, in get_safe_config
    safe_value = get_safe_config(value, item_path, outpath)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/needs_config_writer/builder.py", line 183, in get_safe_config
    safe_value = get_safe_config(value, item_path, outpath)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...

This continues until:

  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/needs_config_writer/builder.py", line 183, in get_safe_config
    safe_value = get_safe_config(value, item_path, outpath)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/needs_config_writer/builder.py", line 95, in get_safe_config
    if matches_path_pattern(path, pattern):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/needs_config_writer/utils.py", line 76, in matches_path_pattern
    return re.fullmatch(pattern_regex, config_path) is not None
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/cache/bazel/05454f223d7eceee02200e9d517999d9/external/rules_python++python+python_3_12_x86_64-unknown-linux-gnu/lib/python3.12/re/__init__.py", line 172, in fullmatch
    return _compile(pattern, flags).fullmatch(string)
           ^^^^^^^^^^^^^^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/vscode/.vscode-server/extensions/swyddfa.esbonio-1.0.0/bundled/libs/esbonio/sphinx_agent/handlers/__init__.py", line 168, in build_sphinx_app
    self.app.build()
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/sphinx/application.py", line 426, in build
    self.builder.build_update()
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 375, in build_update
    self.build(
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 403, in build
    updated_docnames = set(self.read())
                           ^^^^^^^^^^^
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 508, in read
    self.events.emit('env-before-read-docs', self.env, docnames)
  File "/workspaces/docs-as-code/.venv_docs/lib/python3.12/site-packages/sphinx/events.py", line 415, in emit
    raise ExtensionError(
sphinx.errors.ExtensionError: Handler <function write at 0x74767523e200> for event 'env-before-read-docs' threw an exception (exception: maximum recursion depth exceeded)

Steps to reproduce:

• Cloning repo: https://github.com/ecl...

• Fixes [BUG] Max depth reached error when hot reloading via sphinx (e.g. with esbonio) #27

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.