Plugin registry improvements

[jlumpe]

• Increased minimum Python version to 3.11 (matching the main Snakemake package).
• Changes to PluginRegsitryBase:
  • Improved docstrings.
  • Moved abstract methods to the top to make it clearer what needs to be implemented.
  • Fixed type annotation of __new__() - previous annotation allowed for returning an instance of a different registry class as long as the type parameter was consistent. This leads to problems with type checking tools. See caveat below.
  • register_plugin() method: Fixed issue where name conflicts were checked using the full module name with prefix instead of the registry name. Changed argument name to be a bit more clear.
  • get_plugin() method: Fix error message to properly display expected package name using get_plugin_package_name(). Also corrected return type annotation.
• Added tests of registry using example registry class.
• Add tests/ to Pixi format task

Note: fix to type of annotation of PluginRegistryBase.__new__() requires typing.Self, which requires Python 3.11. I think a type annotation is required here due to returning cls._instance, tools will otherwise infer an unknown type. Since Snakemake itself now requires 3.11, I don't think there should be an issue with increasing the minimum version of this package to match.

Summary by CodeRabbit

• Bug Fixes

  • Enhanced error reporting for missing or invalid plugins with clearer diagnostic messages.
  • Improved plugin validation to detect and report attribute mismatches and type errors.

• Tests

  • Added comprehensive test coverage for plugin discovery, loading, and validation scenarios.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR refactors PluginRegistryBase to enforce a true singleton pattern via __new__, introduces three new abstract members (module_prefix, load_plugin, expected_attributes) that concrete registries must implement, and shifts plugin discovery to a module-driven approach with explicit attribute validation and improved error handling.

Changes

Cohort / File(s)	Summary
Core Registry Refactoring src/snakemake_interface_common/plugin_registry/__init__.py	Added Self import for type annotations; introduced _instance singleton storage; changed __new__ to enforce singleton pattern; added three abstract members: module_prefix property, load_plugin(name, module), and expected_attributes() methods with docstrings; updated get_plugin() to raise InvalidPluginException with computed package name on missing plugin; refined get_plugin_package_name() documentation; modified register_plugin() to accept module parameter and use provided name directly; updated collect_plugins() to derive plugin names by removing module prefix and replacing underscores; enhanced validate_plugin() to accept module and check against expected attributes; expanded register_cli_args() docstring.
Test Infrastructure tests/__init__.py	Added single-line comment noting test directory is treated as package to enable relative imports.
Example Plugin Definition tests/example_plugin.py	Added ExamplePlugin dataclass with properties name, cli_prefix, settings_cls; added ExamplePluginRegistry implementing PluginRegistryBase[ExamplePlugin] with new() classmethod for non-singleton test instances, module_prefix property ("snakemake_example_plugin_"), load_plugin() method reading ExampleSettings and example_string, and expected_attributes() describing required/optional attributes.
Test Plugin Packages tests/plugins/valid/snakemake_example_plugin_valid_1/__init__.py tests/plugins/valid/snakemake_example_plugin_valid_2/__init__.py tests/plugins/invalid-class/snakemake_example_plugin_invalid_class/__init__.py tests/plugins/invalid-object/snakemake_example_plugin_invalid_object/__init__.py tests/plugins/missing-attr/snakemake_example_plugin_missing_attr/__init__.py	Added test plugin packages covering valid cases (with ExampleSettings class and example_string attribute), invalid class (ExampleSettings with no base), invalid object (example_string as int instead of string), and missing required attribute scenarios.
Registry Test Suite tests/test_registry.py	Added comprehensive test module with five test functions: test_basic (singleton and registry attributes), test_discovery (valid plugin discovery and instantiation), test_missing_attr (error handling for missing attributes), test_invalid_object (error handling for incorrect attribute types), test_invalid_class (error handling for invalid class inheritance).

Sequence Diagram

sequenceDiagram
    participant User as User Code
    participant Registry as PluginRegistry
    participant Discoverer as Plugin Discoverer
    participant Module as Plugin Module
    participant Validator as Validator
    participant Plugin as Plugin Instance

    User->>Registry: collect_plugins()
    activate Registry
    Registry->>Discoverer: scan for modules matching module_prefix
    activate Discoverer
    Discoverer->>Module: import module
    activate Module
    Module-->>Discoverer: return module
    deactivate Module
    Discoverer-->>Registry: yield module
    deactivate Discoverer
    
    loop For each discovered module
        Registry->>Validator: validate_plugin(name, module)
        activate Validator
        Validator->>Registry: expected_attributes()
        activate Registry
        Registry-->>Validator: attribute requirements
        deactivate Registry
        Validator->>Module: check attributes exist & types match
        activate Module
        Module-->>Validator: attribute values
        deactivate Module
        alt Validation succeeds
            Validator-->>Registry: valid
        else Validation fails
            Validator-->>Registry: InvalidPluginException
        end
        deactivate Validator
        
        alt Validation passed
            Registry->>Registry: register_plugin(name, module)
            Registry->>Registry: load_plugin(name, module)
            activate Registry
            Registry->>Plugin: construct plugin from module
            activate Plugin
            Plugin-->>Registry: plugin instance
            deactivate Plugin
            Registry->>Registry: store plugin
            deactivate Registry
        end
    end
    
    User->>Registry: get_plugin(plugin_name)
    Registry-->>User: return TPlugin or InvalidPluginException
    deactivate Registry

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

• Singleton pattern changes: Verify __new__ correctly prevents reinitialization and __init__ guard logic is sound
• New abstract contract: Ensure all three new abstract members (module_prefix, load_plugin, expected_attributes) are properly documented and will force concrete implementations
• Plugin discovery flow: Trace the module-driven discovery logic in collect_plugins() with name derivation (prefix removal + underscore-to-dash conversion)
• Error handling: Review InvalidPluginException raising with computed package name in get_plugin() and validation error messages
• Test fixture completeness: Verify example plugin packages cover all validation paths (valid, missing-attr, invalid-class, invalid-object)
• Attribute validation logic: Confirm validate_plugin() correctly matches module attributes against expected_attributes() requirements (optional vs required, type checking)

Pre-merge checks and finishing touches

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.
Title check	❓ Inconclusive	The title 'Plugin registry improvements' is vague and generic, using non-descriptive language that doesn't convey the specific changes made to the codebase.	Consider a more specific title that highlights the main change, such as 'Fix PluginRegistryBase singleton pattern and type annotations' or 'Refactor plugin registry with improved type safety and tests'.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

tests/example_plugin.py (1)

49-58: Consider defensive handling of module.__file__.

Line 56 assumes module.__file__ is not None, which could raise a TypeError if the module is a built-in or namespace package. While test fixtures are file-based, adding a defensive check or assertion would improve robustness.

Consider adding a guard:

     def load_plugin(self, name: str, module: ModuleType) -> ExamplePlugin:
         settings_cls = getattr(module, "ExampleSettings", None)
         string_attr = module.example_string
 
+        if module.__file__ is None:
+            raise ValueError(f"Plugin module {name} has no __file__ attribute")
+
         return ExamplePlugin(
             _name=name,
             _settings_cls=settings_cls,
             file=Path(module.__file__),
             string_attr=string_attr,
         )

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1b3449f and c205193.

⛔ Files ignored due to path filters (1)

• pyproject.toml is excluded by !pyproject.toml

📒 Files selected for processing (9)

• src/snakemake_interface_common/plugin_registry/__init__.py (5 hunks)
• tests/__init__.py (1 hunks)
• tests/example_plugin.py (1 hunks)
• tests/plugins/invalid-class/snakemake_example_plugin_invalid_class/__init__.py (1 hunks)
• tests/plugins/invalid-object/snakemake_example_plugin_invalid_object/__init__.py (1 hunks)
• tests/plugins/missing-attr/snakemake_example_plugin_missing_attr/__init__.py (1 hunks)
• tests/plugins/valid/snakemake_example_plugin_valid_1/__init__.py (1 hunks)
• tests/plugins/valid/snakemake_example_plugin_valid_2/__init__.py (1 hunks)
• tests/test_registry.py (1 hunks)

🧰 Additional context used

📓 Path-based instructions (1)

**/*.py

⚙️ CodeRabbit configuration file

**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.

Files:

• tests/plugins/missing-attr/snakemake_example_plugin_missing_attr/__init__.py
• tests/plugins/invalid-object/snakemake_example_plugin_invalid_object/__init__.py
• tests/plugins/valid/snakemake_example_plugin_valid_2/__init__.py
• tests/plugins/valid/snakemake_example_plugin_valid_1/__init__.py
• tests/__init__.py
• tests/example_plugin.py
• src/snakemake_interface_common/plugin_registry/__init__.py
• tests/plugins/invalid-class/snakemake_example_plugin_invalid_class/__init__.py
• tests/test_registry.py

🧬 Code graph analysis (5)

tests/plugins/valid/snakemake_example_plugin_valid_1/__init__.py (2)

src/snakemake_interface_common/plugin_registry/plugin.py (1)

• SettingsBase (41-50)

tests/plugins/invalid-class/snakemake_example_plugin_invalid_class/__init__.py (1)

• ExampleSettings (6-7)

tests/example_plugin.py (3)

src/snakemake_interface_common/plugin_registry/__init__.py (1)

• PluginRegistryBase (23-206)

src/snakemake_interface_common/plugin_registry/plugin.py (2)

• PluginBase (82-360)
• SettingsBase (41-50)

src/snakemake_interface_common/plugin_registry/attribute_types.py (3)

• AttributeType (17-31)
• AttributeMode (6-8)
• AttributeKind (11-13)

src/snakemake_interface_common/plugin_registry/__init__.py (3)

src/snakemake_interface_common/plugin_registry/plugin.py (2)

• name (85-85)
• register_cli_args (134-198)

src/snakemake_interface_common/plugin_registry/attribute_types.py (1)

• AttributeType (17-31)

src/snakemake_interface_common/exceptions.py (1)

• InvalidPluginException (77-79)

tests/plugins/invalid-class/snakemake_example_plugin_invalid_class/__init__.py (1)

tests/plugins/valid/snakemake_example_plugin_valid_1/__init__.py (1)

• ExampleSettings (9-10)

tests/test_registry.py (4)

src/snakemake_interface_common/plugin_registry/plugin.py (1)

• SettingsBase (41-50)

src/snakemake_interface_common/exceptions.py (1)

• InvalidPluginException (77-79)

tests/example_plugin.py (5)

• ExamplePlugin (18-34)
• ExamplePluginRegistry (37-72)
• new (39-43)
• name (25-26)
• settings_cls (33-34)

src/snakemake_interface_common/plugin_registry/__init__.py (4)

• get_plugin_type (113-125)
• is_installed (79-81)
• get_plugin (83-98)
• get_registered_plugins (75-77)

🔇 Additional comments (23)

tests/__init__.py (1)

1-1: LGTM!

Clear comment explaining the purpose of this __init__.py file.

src/snakemake_interface_common/plugin_registry/__init__.py (7)

23-42: Excellent documentation.

The docstring clearly explains the singleton pattern, naming conventions, discovery mechanism, and includes a concrete example. The note about editable installs is a helpful caveat.

---

44-56: LGTM!

The singleton pattern using __new__ with an initialization guard in __init__ is well-implemented. Each subclass will maintain its own _instance class attribute due to Python's attribute assignment semantics.

---

58-72: LGTM!

Good organization moving abstract methods to the top. The contract for subclasses is clear and well-documented.

---

83-98: LGTM!

Good improvement to the error message using the pip-installable package name, making it actionable for users.

---

127-139: LGTM!

The plugin discovery logic correctly filters by prefix, transforms underscores to dashes in plugin names, and delegates to register_plugin for validation and registration.

---

141-206: LGTM!

The register_plugin and validate_plugin methods provide thorough validation with clear error messages. The handling of optional vs required attributes and class vs object types is correct.

---

11-11: Minimum Python version is already properly set to 3.11+.

Verification confirms that snakemake-interface-common requires Python >=3.11 in its package metadata (both PyPI and Bioconda). The Self import from typing on line 11 is therefore appropriately supported and no additional configuration changes are needed.

tests/plugins/valid/snakemake_example_plugin_valid_2/__init__.py (1)

1-3: LGTM!

Good test fixture for validating plugin discovery with only required attributes (no optional settings class).

tests/plugins/missing-attr/snakemake_example_plugin_missing_attr/__init__.py (1)

1-1: LGTM!

Appropriate negative test fixture for validating error handling when required plugin attributes are missing.

tests/plugins/invalid-class/snakemake_example_plugin_invalid_class/__init__.py (1)

1-7: LGTM!

Appropriate negative test fixture for validating error handling when a plugin's class attribute doesn't inherit from the expected base class.

tests/plugins/invalid-object/snakemake_example_plugin_invalid_object/__init__.py (1)

1-3: LGTM! Test fixture correctly implements invalid scenario.

This test fixture appropriately provides an integer value for example_string to enable validation of type-checking error handling in the registry.

tests/plugins/valid/snakemake_example_plugin_valid_1/__init__.py (2)

1-6: LGTM! Valid test fixture structure.

The module correctly defines the required example_string attribute with the expected string type.

---

9-10: LGTM! Minimal settings class is appropriate for testing.

The empty ExampleSettings class correctly inherits from SettingsBase and is sufficient to test the optional settings class attribute validation. The TODO can remain as this fixture validates that minimal settings classes are properly recognized.

tests/test_registry.py (5)

1-14: LGTM! Clean test setup.

Imports and test directory setup are correct.

---

16-28: LGTM! Comprehensive basic functionality tests.

The test correctly validates registry type inference, singleton behavior, and plugin-not-found error handling.

---

30-59: LGTM! Thorough plugin discovery and validation tests.

The test comprehensively validates plugin discovery, instantiation, and attribute checking for both plugins with and without optional settings classes.

---

61-72: LGTM! Proper validation of missing attribute handling.

The test correctly verifies that the registry raises InvalidPluginException with an appropriate error message when a required attribute is missing.

---

74-98: LGTM! Comprehensive type validation tests.

Both test_invalid_object and test_invalid_class properly validate that the registry rejects plugins with incorrect attribute types and generates informative error messages.

tests/example_plugin.py (4)

1-16: LGTM! Clean import structure.

All necessary imports are present and correctly organized.

---

17-35: LGTM! ExamplePlugin correctly implements PluginBase.

The dataclass structure and property implementations appropriately fulfill the PluginBase contract for testing purposes.

---

38-48: LGTM! Test registry setup is correct.

The new() classmethod appropriately bypasses singleton behavior for isolated testing, and module_prefix correctly identifies test plugins.

---

60-72: LGTM! Attribute specifications are correct.

The expected_attributes() implementation properly defines the validation rules for test plugins, specifying ExampleSettings as an optional class attribute and example_string as a required object attribute.