Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix QueryPlanAnalyser with dynamic offset and limit params #390

Conversation

keulinho
Copy link
Contributor

The QueryPlanAnalyser does not strip trailing limits and offsets.
If limits and offsets are set via parameters it would escape the limits and offsets as strings.
This resulted in incorrect SQL being analysed.

The QueryPlanAnalyser does not strip trailing limits and offsets.
If limits and offsets are set via parameters it would escape the limits and offsets as strings.
This resulted in incorrect SQL being analysed.
@keulinho
Copy link
Contributor Author

I'm not sure about the changes to the cache files here 🤔 i did a composer record and committed all changes it produced, not sure if that is the right approach 🤔

@@ -306,7 +306,7 @@ private function replaceParameters(string $queryString, array $parameters): stri
foreach ($parameters as $placeholderKey => $parameter) {
$value = $parameter->simulatedValue;

if (\is_string($value)) {
if (\is_string($value) && !is_numeric($value)) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if this is the right approach here, as it would not escape numeric values at all now 🤔
And at least one test expects that numeric values in the WHERE clause are escaped

Another approach would be to strip the limit and offset of queries here too

Copy link
Owner

@staabm staabm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx for the contribution.

I am wondering, as I don't see a testfailure/error, when reverting your fix but just leave the test-changes from this PR.

ohh it works now, I see.

not sure if that is the right approach

thats perfect, thx.

@@ -19,7 +19,12 @@ public function noindexDbal(Connection $conn): void

public function noindexPreparedDbal(Connection $conn, string $email): void
{
$conn->executeQuery('SELECT * FROM ada WHERE email = ?', [$email]);
$conn->executeQuery('SELECT * FROM ada WHERE email = ? LIMIT 5 OFFSET 2', [$email]);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please don't change existing tests.. just add new ones

@staabm
Copy link
Owner

staabm commented May 27, 2022

I guess the underlying problem is the same as that regular prepared statements do not work properly yet, see #385

which stems from that we don't pass the resolved query string thru simulateQuery, which would take care of stripping limits etc.. I need to think about a possible fix

@staabm staabm merged commit cde6423 into staabm:main May 27, 2022
@staabm
Copy link
Owner

staabm commented May 27, 2022

I guess we are still missing some cases, but its already better then before.

Thank you

@staabm
Copy link
Owner

staabm commented May 27, 2022

@keulinho
Copy link
Contributor Author

Thx for your help! ✌️

@keulinho keulinho deleted the fix/query-plan-analyzer-with-offset-and-limit-params branch May 27, 2022 13:05
@staabm
Copy link
Owner

staabm commented May 27, 2022

@keulinho btw, looks like shopware is using phpstan-dba? do you have any experience to share? feel free to open issues as you find some

@keulinho
Copy link
Contributor Author

Yep we started using this extension last week for shopware 6, because we like the idea of static analysing your DB queries.

But as shopware 6 comes with it own Data Abstraction Layer most of the queries are quite abstract and can not be analysed statically (at least not out of the box) there are not so many queries that can be analysed.

We mostly use plain SQL queries in tests, migrations of performance critical operations, and it is super cool to have this kind of analysis for the latter two cases.

Currently most issues identified were passing unnecessary params when executing an query, but i'm quite curious where this thing is heading too, looks really promising.

@staabm
Copy link
Owner

staabm commented May 27, 2022

But as shopware 6 comes with it own Data Abstraction Layer most of the queries are quite abstract and can not be analysed statically (at least not out of the box) there are not so many queries that can be analysed.

if you find examples of this thing, please open a issue. when it e.g. is similar to doctrine dba, it should be easy to re-use existing phpstan-dba functionality for your custom api, see e.g. https://github.com/staabm/phpstan-dba/blob/main/docs/rules.md

Currently most issues identified were passing unnecessary params when executing an query, but i'm quite curious where this thing is heading too, looks really promising.

I really like this kind of finds, as it often times means you can further simplify methods after you realized parameters are unnecessary

@keulinho
Copy link
Contributor Author

if you find examples of this thing, please open a issue. when it e.g. is similar to doctrine dba, it should be easy to re-use existing phpstan-dba functionality for your custom api, see e.g. https://github.com/staabm/phpstan-dba/blob/main/docs/rules.md

I was not aware that this is possible, looks cool for frameworks that work on SQL basis, but with shopware 6 we tried to build an abstraction around SQL, e.g. you define the data you want to fetch with a Criteria-object where you describe which filters should be used and which associations should be loaded etc. And the framework then genereates the SQL for that.

@staabm
Copy link
Owner

staabm commented May 27, 2022

with a Criteria-object where you describe which filters should be used and which associations should be loaded etc. And the framework then genereates the SQL for that.

I create phpstan extensions for kind of things in the past, which in turn allows to analyze such dynamic built queries even on phpstan-dba, see e.g. https://github.com/staabm/zf-select-strip/blob/a62603941cb2a2aeba6ba121318b93fb93c57430/tests/phpstan/data/zf-select.php#L129-L140

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants