Tests / Build Scripts: Configure PHPStan levels 1-6 [Exploratory]

[justlevine]

Trac ticket: https://core.trac.wordpress.org/ticket/61175

This PR adds a PHPStan configuration along with error baselines through Level 6.

The hope is the limited scope will make this easier to review/merge, with actual code remediations occuring in future (or even parallel) PRs.

Usage

Run PHPStan

composer run analyse

# Generate a report
# https://phpstan.org/user-guide/output-format
composer run analyse -- --error-format=checkstyle

Create a local phpstan.neon for remediating errors

1. Copy phpstan.neon.dist to phpstan.neon.
2. Comment out the unnecessary baselines, change the parameters.level to the desired level, and filter out the errors by adding them to the parameters.ignoreErrors list.
3. Run PHPStan as usual: composer run analyse.

Remediating errors using the error baselines

While parameters.ignoreErrors is used to filter out "unsupported" errors, error baselines are used to suppress preexisting tech debt.

This allows for the PR to be merged as is to enforce the rules on new code, while allowing contributors to remediate the existing errors at their own pace. This is in the spirit of 'Avoid unnecessary refactoring'.

To remediate a baselined error, remove the error from the tests/phpstan/baseline/level-{%N} file and run PHPStan again.

Triaging errors and regenerating baselines

If an error is found to be a false positive (or otherwise not worth fixing), it should be added to the parameters.ignoreErrors list in the phpstan.neon.dist file. When this happens, the baseline file suppressing the error will cause PHPStan to fail.

To avoid manual remediation, the baseline files can be regenerated by following the following steps:

1. Identify the baseline level that contains the error. (Search for the error in tests/phpstan/baseline ).

2. Change the parameters.level in phpstan.neon to the level identified in step 1.

3. Comment out the includes for that level and all levels above it.

4. Run the following command:

   # Replace {%N} with the level identified in step 1
   vendor/bin/phpstan analyse --generate-baseline tests/phpstan/baseline/level-{%N}.php 

Prior Art

This is based off the work done in #853 with some notable differences:

• Latest trunk and PHPStan (1.12.7) - as of writing.

• No changes to WordPress core codebase.

  This is to limit the scope of the PR and hopefuly prevent it from going stale.

• The phpstan.neon.dist file wraps the tests/phpstan/base.neon config that holds the codebase configuration (what to scan/skip, etc), with the top-level file holding the "rules".

  This makes it easier for contributors to remediate errors by creating a local phpstan.neon file.

• Removed many of the parameters.ignoreErrors in favor of error baselines (one per each PHPStan level).

  See Remediating errors using the error baselines section above.

• The tests/phpstan/bootstrap.php file has been reorganized to mirror the order that the constants are defined in the WP lifecycle.

  This will hopefully make it easier to maintain in the future.

• Added inline annotations to the various configs.

Additional Notes

• PHPStan Level 6 is the highest level that can be baselined without making changes to core code. While, I'd personally recommend baselining at level 8 (with ignoreErrors) - Level 9+ is primarily about mixed types - that can be handled incrementally in a future PR that contains the necessary code remediations.

Next Steps

• Add a GH Workflow to run PHPStan on PRs.
• Triage the baselines for good candidates for ignoreErrors.

AI Disclosure

tests/phpstan/generate-baseline.sh was generated and reviewed as described here. It is a tool for generating these level-by-level baselines, and not meant for core.

---

This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

[github-actions]

Hi @justlevine! 👋

Thank you for your contribution to WordPress! 💖

It looks like this is your first pull request to wordpress-develop. Here are a few things to be aware of that may help you out!

No one monitors this repository for new pull requests. Pull requests must be attached to a Trac ticket to be considered for inclusion in WordPress Core. To attach a pull request to a Trac ticket, please include the ticket's full URL in your pull request description.

Pull requests are never merged on GitHub. The WordPress codebase continues to be managed through the SVN repository that this GitHub repository mirrors. Please feel free to open pull requests to work on any contribution you are making.

More information about how GitHub pull requests can be used to contribute to WordPress can be found in the Core Handbook.

Please include automated tests. Including tests in your pull request is one way to help your patch be considered faster. To learn about WordPress' test suites, visit the Automated Testing page in the handbook.

If you have not had a chance, please review the Contribute with Code page in the WordPress Core Handbook.

The Developer Hub also documents the various coding standards that are followed:

• PHP Coding Standards
• CSS Coding Standards
• HTML Coding Standards
• JavaScript Coding Standards
• Accessibility Coding Standards
• Inline Documentation Standards

Thank you,
The WordPress Project

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props justlevine, johnbillion, westonruter, jorbin, sirlouen.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[github-actions]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

• The Plugin and Theme Directories cannot be accessed within Playground.
• All changes will be lost when closing a tab with a Playground instance.
• All changes will be lost when refreshing the page.
• A fresh instance is created each time the link below is clicked.
• Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
  it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

[github-actions]

A commit was made that fixes the Trac ticket referenced in the description of this pull request.

SVN changeset: 61699
GitHub commit: 8a82e67

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.

[westonruter]

Re-opening because only level 0 was committed.

[westonruter]

@justlevine How were these generated? It seems level-2.php omits the errors from level 1? And level-3.php omits the errors from level 2, and so on? When I run composer phpstan -- --level=3 --generate-baseline=tests/phpstan/baseline/level-3.php it includes errors from levels 1-3.

[justlevine]

Yeah you need to do it level by level. I've been doing it manually but hold on ai should be able to bash it for us.

[westonruter]

We should make a script that does that for us and register it as a new composer command. That will be super useful going forward.

[westonruter]

Oh, that's what you did? 😅

[justlevine]

Yup 😇

written by zai-coding-plan/glm-5, reviewed by openai/gpt-5.2-codex (both in opencode), tested by github-copilot/gpt-5-mini (Vscode)
"lightly reviewed" (aka skimmed for nothing crazy) by myself, and manually tested in wsl2 ubuntu24.04

Keep in mind that it iterates levels, so doing npm run test/phpstan/generate-baselines.sh --all will take a few minutes, since there's no caches. More in the update doc. NOT MEANT FOR CORE MERGE just a fun tool

[westonruter]

We should have something merged into core for for future reuse.

Also, I think writing it in PHP would be more compatible across systems when it is used.

[westonruter]

Couldn't the MAX_LEVEL be read from the PHPStan config?

[justlevine]

Yup, cool idea. feel free to throw some tokens at it if you want or maybe i will next time

[westonruter]

What does this mean? Internal to the branch? How is this not for core?

[justlevine]

Core only has a baseline.php. This whole level-by-level triage is specific to these exploration/remediation branches to make it easier to see the impacts of each level and triage.

[justlevine]

IOW once PHPStan level 1 is enforced by core, it doesnt matter if a new error we want to baseline is level 1 or level 0, it's just core "Tech debt" that we're baselining.

[westonruter]

I was thinking we'd commit those baselines to core, even up to level 8. Then contributors who want to work on level 8 can do so via their local phpstan.neon. Maybe that doesn't make sense.

But let's say we do bump to level 6 now, it would be great to have level-specific baselines to help us as we start to fix the issues incrementally until we reach a point where there are no errors left in a level-specific baseline.

[justlevine]

But let's say we do bump to level 6 now,

in that case sure, though I'd recommend a real review of the script first. I feel the likelihood of that getting approved is very low though.

[westonruter]

Getting what approved? Level 6? I don't think it will be difficult to do that, if we have the baseline. It means new code will begin to have level 6 issues reported, and we can start fixing the issues and removing them from the baseline(s) concurrently.

[justlevine]

I hope you're right, but I'd assume there's some chicken-egg work at foot based on the discussions and feedback from folks before merge.

Commiters (assumedly) want to know the impact of the particular level/ruleset before we enforce the requirements on new code. And there's also the level-specific prepwork that needs to be done so that every new PR isn't generating "noise" (more of the same error types that need to be baselined).

So yeah I do assume for new-enforcement on new code we'll need to bump level by level, so we can point and say e.g. "these are the new violations that are getting checked, these are (currently) ignored etc" and that the codebase is in a place where we can enforce it and those baselines remain stable.

we can start fixing the issues and removing them from the baseline(s) concurrently.

So this was the scenario I was originally trying to address in my reply. Levels are just shorthand convenience groupings for getting things into a codebase. Once inside the codebase tho, it doesn't matter for remediation if the violation is from a level 1 rule or a level 9, its tech debt that needs to be remediated, so having to maintain multiple lists that nerd to be non-natively generated feels like undesired friction for core.

I do see value in tools like https://github.com/staabm/phpstan-baseline-analysis to help us pull out and do rule-based triage of the baselines, but that too doesn't seem like something that should be merged into core.

(maybe I have the wrong POV: but I see a difference between committing an evergreen tool/flow versus one thats only a comparative short term triage tool. Like I wouldn't commit a script that e.g. generates all the PHPUnit tests that aren't compatible with v11.5+, or one that generates a report of any non php8.0 compatible code, since one those triage tasks are done the script no longer has value)