Add `#[YieldReady]` to allow extensions to tell when they're ready for yielding #3999

nicolas-grekas · 2024-02-12T17:24:36Z

The current opt-in based on "use_yield" doesn't work, because there is no decider to enable the flag:
the flag can be turned on only when all extensions are compatible, and this is fragile, because extensions currently don't have a way to signal that they are ready for yielding.

This PR takes the following approach instead:

Between each "yield", the output buffer is checked and emptied.
Extensions need to label their node implementations with #[YieldReady]. If they don't, a deprecation is raised.

On Twig 4, we could then just remove this output buffer check and possibly throw when a node is not "YieldReady".

(Classes that extend AbstractExpression are implicitly considered "YieldReady".)

Best reviewed ignoring whitespaces.

src/Node/BlockNode.php

src/Template.php

src/Node/YieldReady.php

stof

What is the performance impact of the runtime check combined with yield ?

src/Compiler.php

src/NodeVisitor/YieldNotReadyNodeVisitor.php

stof · 2024-02-12T18:17:47Z

Note that an effect of this change is that support for Fiber-based runtimes will become possible only as of Twig 4, even if all extensions are yield ready, as there is no way anymore to opt-in for the yield-only mode that unlocks that usage.

nicolas-grekas · 2024-02-13T08:48:33Z

What is the performance impact of the runtime check combined with yield ?

I did a micro-benchmark comparing the time it takes to call ob_get_length() vs the time it takes to call an empty function c() {}.
I complete 10M iterations in 180ms for the ob function vs 210ms for the empty one.
This should be fine.

Note that an effect of this change is that support for Fiber-based runtimes will become possible only as of Twig 4, even if all extensions are yield ready, as there is no way anymore to opt-in for the yield-only mode that unlocks that usage.

I improved this by adding the "use_yield" flag back. When it's turned on, the node visitor will throw if any nodes are not #[YieldReady], and the Template will skip using any ob functions so that things are at full speed.

nicolas-grekas · 2024-02-13T09:37:52Z

PR ready (failures are false positives)

src/Environment.php

src/Node/CaptureNode.php

src/Node/YieldReady.php

src/NodeVisitor/YieldNotReadyNodeVisitor.php

nicolas-grekas · 2024-02-13T11:40:27Z

All comments addressed.

src/Environment.php

fabpot · 2024-02-14T18:41:19Z

src/Node/Node.php

@@ -82,7 +84,7 @@ public function __toString()
    public function compile(Compiler $compiler)
    {
        foreach ($this->nodes as $node) {
-            $node->compile($compiler);
+            $compiler->subcompile($node);


Should be reverted

We'd better not: this line allows the compiler to accurately tell which node is leaking via "echo". If I remove it, the error/deprecation message will report about the wrong node (the wrapping one).
Can't we keep this line? Why if not? Calling subcompile instead of compile looks OK to me also, semantically speaking.

note that $compiler->subcompile($node); is already calling $node->compile($compiler);

fabpot · 2024-02-15T06:18:46Z

src/Template.php

-        if ($this->env->isDebug()) {
-            ob_start();
-        } else {
-            ob_start(function () { return ''; });


You've removed this "feature". I'm fine as we won't be able to replicate it with yield, but that should be noted in the CHANGELOG.

I've read #1962 and #3058 and I think this is fine now: both issues are handled by using yield instead of output buffering. I'd keep things as is.

I updated the changelog to tell about what this PR does, but about this specific behavior, I can't figure out what to write. It's too complex to describe to really be a "feature".

Maybe CoreExtension should keep that logic in its captureOutput. The issue of not leaking output buffers in case of fatal errors still exists when this BC layer is used.

…r yielding

fabpot · 2024-02-15T09:45:20Z

Thank you @nicolas-grekas.

This PR was merged into the 5.4 branch. Discussion ---------- [TwigBridge] Fix compat with Twig v3.9 | Q | A | ------------- | --- | Branch? | 5.4 | Bug fix? | no | New feature? | no | Deprecations? | no | Issues | - | License | MIT This PR relies on twigphp/Twig#3999 Commits ------- f563a4c [TwigBridge] Fix compat with Twig v3.9

This PR was merged into the 5.4 branch. Discussion ---------- [TwigBridge] Fix compat with Twig v3.9 | Q | A | ------------- | --- | Branch? | 5.4 | Bug fix? | no | New feature? | no | Deprecations? | no | Issues | - | License | MIT This PR relies on twigphp/Twig#3999 Commits ------- f563a4c5a8 [TwigBridge] Fix compat with Twig v3.9

…colas-grekas) This PR was merged into the 3.x branch. Discussion ---------- Swap BC layer for yield-ready and reclaim perf loss Follows #3999 Fix #4146 Fix #4103 When `use_yield` is set to false (the default), this PR reverts the implementation of the `render()` method to use a wrapping output buffer instead of hooking between each steps of generators. In this mode, the behavior of the yield method is not "pure": it triggers a mix of yield and echo. But this is fine for render and display methods. When `use_yield` is set to `true`, we skip that wrapping output buffer. This makes twig compatible with fibers (and this also makes compilation fail if a non-YieldReady extension is found.) That makes the name of the option not ideal, but BC rulez FTW. Commits ------- 5d1a19a Swap BC layer for yield-ready and reclaim perf loss

fabpot · 2024-08-19T18:09:02Z

Performance regression fixed in #4216

This PR was merged into the 3.x branch. Discussion ---------- Add some tests Closes #3691 The bug described in #3691 has been fixed via #3999. This PR adds tests to prove it works now and to avoid future regressions. Commits ------- 85c2ba5 Add some tests

nicolas-grekas force-pushed the yield-layer branch 2 times, most recently from 047ec50 to 1e48731 Compare February 12, 2024 17:30

nicolas-grekas mentioned this pull request Feb 12, 2024

[TwigBridge] Fix compat with Twig v3.9 symfony/symfony#53913

Merged

nicolas-grekas force-pushed the yield-layer branch 3 times, most recently from ab91e9a to 55129da Compare February 12, 2024 17:45

nicolas-grekas commented Feb 12, 2024

View reviewed changes

src/Node/BlockNode.php Show resolved Hide resolved

src/Template.php Show resolved Hide resolved

stof reviewed Feb 12, 2024

View reviewed changes

src/Node/YieldReady.php Outdated Show resolved Hide resolved

stof reviewed Feb 12, 2024

View reviewed changes

src/Compiler.php Show resolved Hide resolved

nicolas-grekas force-pushed the yield-layer branch from 55129da to 3d3caec Compare February 12, 2024 17:56

jdreesen reviewed Feb 12, 2024

View reviewed changes

src/NodeVisitor/YieldNotReadyNodeVisitor.php Outdated Show resolved Hide resolved

nicolas-grekas force-pushed the yield-layer branch 6 times, most recently from 95c2801 to 61c7afe Compare February 13, 2024 08:40

nicolas-grekas changed the title ~~Replace "use_yield" by #[YieldReady] on nodes~~ Add #[YieldReady] to allow extensions to tell when they're ready for yielding Feb 13, 2024

nicolas-grekas force-pushed the yield-layer branch 5 times, most recently from d03553d to 285f863 Compare February 13, 2024 09:36

stof reviewed Feb 13, 2024

View reviewed changes

nicolas-grekas force-pushed the yield-layer branch 2 times, most recently from 0fc9739 to 03b884c Compare February 13, 2024 11:39

nicolas-grekas mentioned this pull request Feb 13, 2024

Implement YieldingTemplate::display*Block() #3992

Closed

stof reviewed Feb 13, 2024

View reviewed changes

src/Environment.php Outdated Show resolved Hide resolved

nicolas-grekas force-pushed the yield-layer branch from 03b884c to 01f339c Compare February 13, 2024 13:22

stof approved these changes Feb 14, 2024

View reviewed changes

nicolas-grekas force-pushed the yield-layer branch from 01f339c to 998de12 Compare February 14, 2024 12:45

fabpot requested changes Feb 15, 2024

View reviewed changes

Add #[YieldReady] to allow extensions to tell when they're ready fo…

e707774

…r yielding

nicolas-grekas force-pushed the yield-layer branch from 998de12 to e707774 Compare February 15, 2024 09:27

fabpot merged commit 0990f81 into twigphp:3.x Feb 15, 2024
16 of 67 checks passed

nicolas-grekas deleted the yield-layer branch February 15, 2024 11:25

This was referenced Apr 22, 2024

Incompatibility with twig 3.9 nelmio/NelmioSecurityBundle#343

Closed

Fix compatibility with twig 3.9 and yielding nelmio/NelmioSecurityBundle#344

Merged

barryvdh mentioned this pull request Apr 24, 2024

YieldReady rcrowe/TwigBridge#443

Merged

Chi-teck mentioned this pull request May 13, 2024

Custom tag compiler no longer works #4090

Closed

vbrouet mentioned this pull request May 30, 2024

Performance regression since v3.9.0 #4103

Closed

nicolas-grekas mentioned this pull request Aug 19, 2024

Swap BC layer for yield-ready and reclaim perf loss #4216

Merged

This was referenced Sep 24, 2024

Add some tests #4337

Merged

Different extends behavior when passing an array to extends #3691

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `#[YieldReady]` to allow extensions to tell when they're ready for yielding #3999

Add `#[YieldReady]` to allow extensions to tell when they're ready for yielding #3999

nicolas-grekas commented Feb 12, 2024 •

edited

Loading

stof left a comment

stof commented Feb 12, 2024

nicolas-grekas commented Feb 13, 2024 •

edited

Loading

nicolas-grekas commented Feb 13, 2024 •

edited

Loading

nicolas-grekas commented Feb 13, 2024

fabpot Feb 14, 2024

nicolas-grekas Feb 15, 2024

stof Feb 15, 2024

fabpot Feb 15, 2024

nicolas-grekas Feb 15, 2024

nicolas-grekas Feb 15, 2024

stof Feb 15, 2024

fabpot commented Feb 15, 2024

fabpot commented Aug 19, 2024

Add #[YieldReady] to allow extensions to tell when they're ready for yielding #3999

Add #[YieldReady] to allow extensions to tell when they're ready for yielding #3999

Conversation

nicolas-grekas commented Feb 12, 2024 • edited Loading

stof left a comment

Choose a reason for hiding this comment

stof commented Feb 12, 2024

nicolas-grekas commented Feb 13, 2024 • edited Loading

nicolas-grekas commented Feb 13, 2024 • edited Loading

nicolas-grekas commented Feb 13, 2024

fabpot Feb 14, 2024

Choose a reason for hiding this comment

nicolas-grekas Feb 15, 2024

Choose a reason for hiding this comment

stof Feb 15, 2024

Choose a reason for hiding this comment

fabpot Feb 15, 2024

Choose a reason for hiding this comment

nicolas-grekas Feb 15, 2024

Choose a reason for hiding this comment

nicolas-grekas Feb 15, 2024

Choose a reason for hiding this comment

stof Feb 15, 2024

Choose a reason for hiding this comment

fabpot commented Feb 15, 2024

fabpot commented Aug 19, 2024

Add `#[YieldReady]` to allow extensions to tell when they're ready for yielding #3999

Add `#[YieldReady]` to allow extensions to tell when they're ready for yielding #3999

nicolas-grekas commented Feb 12, 2024 •

edited

Loading

nicolas-grekas commented Feb 13, 2024 •

edited

Loading

nicolas-grekas commented Feb 13, 2024 •

edited

Loading