Missing config_full.yml and unregistered full predictor in swe-bench example

[Jerryguan777]

Version

Latest develop branch (commit: bc97aff)

Which installation method(s) does this occur on?

Source

Describe the bug.

When running the SWE-bench evaluation with the full predictor, two issues were encountered:

1. Missing configuration file: The config_full.yml file is missing from the expected location examples/evaluation_and_profiling/swe_bench/configs/config_full.yml, causing a file not found error.

2. Predictor not registered: After adding the missing config file, the full predictor is not registered in the PredictorRegistry, causing a KeyError: "predictor 'full' is not registered." error.

Expected behavior:

• The config_full.yml file should exist in the examples/evaluation_and_profiling/swe_bench/configs/ directory
• The full predictor should be automatically registered when the workflow is loaded, similar to how gold predictors are registered

Actual behavior:

• Error 1: Error: Invalid value for '--config_file': File 'examples/evaluation_and_profiling/swe_bench/configs/config_full.yml' does not exist.
• Error 2: KeyError: "predictor 'full' is not registered."

Minimum reproducible example

# Step 1: Try to run evaluation
uv pip install -e examples/evaluation_and_profiling/swe_bench

nat eval --config_file examples/evaluation_and_profiling/swe_bench/configs/config_full.yml

# Error 1: File not found
# Error: Invalid value for '--config_file': File 'examples/evaluation_and_profiling/swe_bench/configs/config_full.yml' does not exist.

# Step 2: After manually creating the config file, run again
nat eval --config_file examples/evaluation_and_profiling/swe_bench/configs/config_full.yml

# Error 2: Predictor not registered
# KeyError: "predictor 'full' is not registered."

Relevant log output

Click here to see error details

(NeMo-Agent-Toolkit) jerry@ubuntu:~/ai/NeMo-Agent-Toolkit$ nat eval --config_file examples/evaluation_and_profiling/swe_bench/configs/config_full.yml

Usage: nat eval [OPTIONS] COMMAND [ARGS]...

Try 'nat eval --help' for help.
Error: Invalid value for '--config_file': File 'examples/evaluation_and_profiling/swe_bench/configs/config_full.yml' does not exist.
.....

KeyError: "predictor 'full' is not registered."

Other/Misc.

I would like to submit a PR to fix this issue:

1. Add missing config file: Create config_full.yml in the correct location examples/evaluation_and_profiling/swe_bench/configs/config_full.yml

2. Register full predictors: Update examples/evaluation_and_profiling/swe_bench/src/nat_swe_bench/predictors/register.py to import all predictor classes:

config_full.yml

register.py

Code of Conduct

• I agree to follow the NeMo Agent toolkit Code of Conduct
• I have searched the open bugs and have found no duplicates for this bug report