Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(sumtype): avoid quadratic memory complexity on opEquals #8607

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ljmf00
Copy link
Member

@ljmf00 ljmf00 commented Oct 16, 2022

Signed-off-by: Luís Ferreira [email protected]


Due to match template being quadratic/exponential in terms of memory complexity, (-vtemplate reports a rediculous amount of templates) using opEquals will generate a lot of unnecessary template instances. Removing match here to prevent that.

@ljmf00 ljmf00 requested a review from pbackus as a code owner October 16, 2022 19:29
@dlang-bot
Copy link
Contributor

Thanks for your pull request, @ljmf00!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + phobos#8607"

@pbackus
Copy link
Contributor

pbackus commented Oct 16, 2022

I would much, much rather improve match itself than avoid using it. Duplication like this increases maintenance burden and risks introducing bugs (in fact, I think it has introduced one already).

One simple optimization we can do is to replace the lambda handler in opEquals with a global function, so that instantiations will be shared between different SumType instances. This will also cut down on instantiations of canMatch.

@RazvanN7
Copy link
Collaborator

RazvanN7 commented Nov 1, 2022

@ljmf00 The style checker fails:



Enforce whitespace before opening parenthesis
--
  | grep -nrE "\<(for\|foreach\|foreach_reverse\|if\|while\|switch\|catch\|version)\(" $(find etc std -name '*.d') ; test $? -eq 1
  | std/sumtype.d:717:            final switch(tag)
  | std/sumtype.d:719:                static foreach(idx, T; Types)
  | std/sumtype.d:724:                            static foreach(ridx, U; typeof(rhs).Types)
  | make: *** [posix.mak:587: style_lint_shellcmds] Error 1

Copy link
Contributor

@pbackus pbackus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said in my previous comment: code duplication like this adds to our maintenance burden, and increases the risk of introducing bugs, like the one pointed out in my review comment.

I would strongly prefer it if we could try a less drastic alternative, like replacing the lambda with a global function (as has already been done in SumType's destructor), before resorting to this.

Finally, if the justification for this change is memory usage, it would be nice to see before-and-after measurements demonstrating the reduction in memory usage. In spite of what the PR title says, both the original and new code have quadratic complexity.

static foreach(ridx, U; typeof(rhs).Types)
{
case ridx:
static if (is(immutable T == immutable U))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line introduces a bug. In the original version, the following unit test passes:

@safe unittest
{
    int* p = null;
    const int* cp = null;

    SumType!(int*, const(int*)) x = p;
    SumType!(int*, const(int*)) y = cp;

    assert(x != y);
}

With this PR, the unit test above will fail.

@pbackus
Copy link
Contributor

pbackus commented Nov 21, 2024

Update: I tried replacing the lambda with a global function in a local git branch. Unfortunately, for reasons I don't fully understand, this ended up causing forward reference errors for recursive sum types that have This[] as one of their member types. So it looks like that approach is off the table, at least for now.

@LightBender
Copy link
Contributor

@pbackus In that case would you recommend moving forward as-is? I know that code duplication is bad, but quadratic memory complexity is worse IMO.

@pbackus
Copy link
Contributor

pbackus commented Nov 22, 2024

@LightBender I would be ok with moving forward as long as

  1. the bug I pointed out is fixed, and
  2. we have before-and-after measurements demonstrating the reduction in memory usage.

As I've pointed out before, both the original and the new version have quadratic complexity, which means that the gain from switching to static foreach is "only" a constant-factor reduction. That doesn't mean it isn't worth it, but it does mean I'm going to need some actual data to convince me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants