addressed: query filter behaves differently inside WSL Ubuntu environment than on Windows and Mac

[GrayedFox]

Hello there! Big fan of mailpit, thanks.

The issue we've surfaced today is that for some reason, when we run specs on Windows (natively) or Mac OS we see the query string using addressed: $email works fine, but inside the WSL Ubuntu env it fails to match any messages. This issue occurs locally, that is, when running against a local mailpit API and against a local stack.

The search query string that doesn't work in the WSL Ubuntu env is:

const queryString = `tag:${tag} subject:"${subject}" addressed:"${to}" is:${readState}`;

Changing it to the following works around the issue:

const queryString = `tag:${tag} subject:"${subject}" to:"${to}" is:${readState}`;

We use this utility method to hit the search endpoint:

import { APIRequestContext, request } from '@playwright/test';

export const searchMail = async (query: string, attempts: number = 30): Promise<MessagesSummary> => {
    const context: APIRequestContext = await request.newContext({ baseURL: MAILPIT_API });
    const response = await context.get(`${MAILPIT_API}/search`, {
        params: { query, start: 0, limit: 50 }
    });

    const body = await response.body();
    const json = (await response.json()) as MessagesSummary;
    const status = response.status();

    if ((status !== 200 || json.messages_count === 0) && attempts <= 0) {
        await context.dispose();
        throw new Error(`Failed to get messages. Status: ${status}, Body: ${body}`);
    }

    // poll emails until we find one or run out of attempts
    if (json.messages_count === 0) {
        console.log(`MAILPIT no mail found, ${attempts} attempts left`);
        await new Promise(resolve => setTimeout(resolve, 1000));
        return await searchMail(query, attempts - 1);
    }

    console.log(`MAILPIT mail found`);

    await context.dispose();
    return json;
};

And it's called like so:

console.log(`Searching ${readState} emails with TAG and SUBJECT addressed TO`, [tag, subject, to]);
const messages = await searchMail(`tag:${tag} subject:"${subject}" addressed:"${to}" is:${readState} `);

If this really is a bug on the mailpit end I imagine it could be minimally reproduced simply by sending a test email with a specific subject, that uses a plus address, and using the following call:

const messages = await searchMail(`tag:plus subject:"testSubject" addressed:"[email protected]" is:unread `)

[axllent]

Hi @GrayedFox. I can't speak for WSL Ubuntu as I do not run Windows (but I do run Ubuntu), nor rule out whether WSL is or isn't the cause of your issue, or whether it is something in your JS library - but your report is very confusing for me to follow. I'm sure it makes perfect sense to you as you understand the context though ;-)

Firstly I do not understand why you would be looping the same search again and again for up to 30 times, and why the example of successfully finding an email only finds it on the 4th attempt? Is this because you're polling the API before the email arrives? Assuming that assumption is correct, does the message actually arrive in Mailpit in the WSL environment within the 30 attempts (ie: you can see it in the web UI)?

Secondly, to help me to help you, the test needs to be simplified, and we need to eliminate all third part tooling (ie: the JS library):

1. this is the email to send to Mailpit (you attach a sample raw message which fails the test)
2. this is the exact web UI API search which fails to find the message (you provide the exact search)

I this issue can be reproduced within Mailpit natively, then that narrows down the issue. If it can't be reproduced in Mailpit directly, then you can continue to debug your JS library etc.

Once I can reproduce the error then I should be able to say more, until then I'm just left with more and more questions :)

[GrayedFox]

@axllent cleaned up the original post a bit! Included logs the first time round just to demo that that query does indeed work but I see that this ended up being a bit noisy.

Firstly I do not understand why you would be looping the same search again and again for up to 30 times

Event based arch with a lot going on, at the moment there's no easy or straight forward way to signal to the client that all necessary backend async ops have finished that mean the "email has been sent" - polling mailpit means the specs are more agnostic about the AUT, hence choosing that approach.

Is this because you're polling the API before the email arrives?

Yes.

Secondly, to help me to help you, the test needs to be simplified, and we need to eliminate all third part tooling

Have eliminated all but PlayWright which runs the spec, I can write a sample PlayWright spec if you like? If this is a bug on the MailPit end I think any SMTP service will do, is that something you're fine to setup on your end?

I can provide a full minimal reproduction sample in a repo but likely not until some time next week - all good if you just want to wait until then before further debugging, I'm a bit time poor at the moment but happy to do that so you can spend your time/energy on the investigation part instead of the reproduction part 🙇🏽

[axllent]

Thanks for the response/clarification @GrayedFox - I totally get the "time poor" situation (I have the same) :)

If I understand the error correctly, your suspicion is that the Mailpit API is responding differently with addressed searches on WSL Ubuntu (compared to Windows and Mac). What I am trying to do is eliminate everything non-Mailpit if possible, and that would ideally exclude PlayWright too.

Maybe an easy test for you to do is to do the following after your search for the message fails in Ubuntu:

• Open up the Mailpit web UI in your browser
• Confirm the email you are searching for is actually there
• Do the same search in the web UI which is failing in your automated tests

If the web UI also returns no matches (while using addressed:"...") but the message is really there:

• Provide me with a copy of the raw email
• Provide me with the search you are trying

If, however, the web UI does return the match (which the automation fails to detect), then you should continue debugging the automation itself, as that's where the difference is. It could be something as simple as different utf8 encoded quotes in your scripts or a hidden character somewhere (just an example).

As far as I can tell, the addressed search should work exactly as expected (searches all of To, From, Cc, Bcc & ReplyTo data). Unlike regular searches however, all of the to:, cc:, addressed: searches do search raw converted JSON data (SQLite's JSON implementation), so it is still possible there's an internal issue with the SQLIte port, but I'm not able to see it here. If there was an issue however, then I'd also expect to: to fail (which you say works)... so I'm doubtful that is actually the problem.

Take your time next week is fine, and we can take it from there based on your findings.