Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: Add option to disable AudioContext autoplay #33590

Closed
andy-fillebrown opened this issue Nov 13, 2024 · 24 comments · Fixed by #33734
Closed

[Feature]: Add option to disable AudioContext autoplay #33590

andy-fillebrown opened this issue Nov 13, 2024 · 24 comments · Fixed by #33734
Assignees
Labels

Comments

@andy-fillebrown
Copy link

🚀 Feature Request

When creating a new AudioContext, its initial state is "running", which is good for testing audio, but does not reflect the initial state encountered by users of a website, which is "suspended" and requires a user interaction for the AudioContext.resume() promise to resolve.

If possible, please add an option that makes AudioContext's initial state "suspended", and require a user interaction for the AudioContext.resume() promise to resolve.

Example

No response

Motivation

This would make it possible to test the user interaction requirements for autoplay implemented by browsers. See https://developer.chrome.com/blog/autoplay for more info on Chrome's requirements. Webkit's requirements are similar.

@pavelfeldman
Copy link
Member

Could you provide a full repro case and example of proposed API?

@andy-fillebrown
Copy link
Author

andy-fillebrown commented Nov 13, 2024

It may not be possible without changing the browser build settings. See bootstrap.diff:23122 for the webkit build for example.

I assume there's something similar for the chromium build. It might be better to leave the autoplay build settings alone so autoplay is disabled by default, and then add CLI flags for each browser to enable it. So for chromium the flag to set would be --autoplay-policy=no-user-gesture-required. I'm not sure if there are Firefox and Webkit equivalents.

If it were done that way then ignoreDefaultArgs could be used to disable autoplay: https://playwright.dev/docs/api/class-browsertype#browser-type-launch-option-ignore-default-args.

@yury-s
Copy link
Member

yury-s commented Nov 18, 2024

@andy-fillebrown can you share a repro where the behavior differs from the stock browsers?

@yury-s
Copy link
Member

yury-s commented Nov 18, 2024

The following test passes in all 3 browsers on macOS for me:

import { test, expect } from '@playwright/test';

test("context audio", async ({ page }) => {
  await page.route("**/*", async (route) => {
    console.log('server response');
    route.fulfill({
      status: 200,
      contentType: 'text/html',
      body: `<script> 
function onLoad() {
  const log = document.getElementById('log');
  const audio = new AudioContext();
  log.innerHTML = 'State: ' + audio.state;
}
</script>
<body onload="onLoad()">
  <div id="log"></div>
</body>`,
    });
  });
  await page.goto("http://127.0.0.1:8080/audio.html");
  await expect(page.locator('#log')).toHaveText('State: suspended', { timeout: 1000 });
});

@andy-fillebrown
Copy link
Author

The following test passes in all 3 browsers on macOS for me:

import { test, expect } from '@playwright/test';

test("context audio", async ({ page }) => {
await page.route("**/*", async (route) => {
console.log('server response');
route.fulfill({
status: 200,
contentType: 'text/html',
body: `<script>
function onLoad() {
const log = document.getElementById('log');
const audio = new AudioContext();
log.innerHTML = 'State: ' + audio.state;
}
</script>

`, }); }); await page.goto("http://127.0.0.1:8080/audio.html"); await expect(page.locator('#log')).toHaveText('State: suspended', { timeout: 1000 }); });

Changing that test to make it start a sound makes it pass for chromium and firefox, but fail for webkit on macOS for me:

test("context audio", async ({ page }) => {
    await page.route("**/*", async (route) => {
        console.log("server response");
        route.fulfill({
            status: 200,
            contentType: "text/html",
            body: `<script>
async function onLoad() {
  const log = document.getElementById('log');
  const audioContext = new AudioContext();
  const gainNode = new GainNode(audioContext);
  gainNode.connect(audioContext.destination);
  gainNode.gain.value = 0.025;
  const sineNode = new OscillatorNode(audioContext);
  sineNode.connect(gainNode);
  sineNode.start();
  await new Promise((resolve) => setTimeout(resolve, 2000));
  log.innerHTML = 'State: ' + audioContext.state;
}
</script>
<body onload="onLoad()">
  <div id="log"></div>
</body>`,
        });
    });
    await page.goto("http://127.0.0.1:8080/audio.html");
    await expect(page.locator("#log")).toHaveText("State: suspended", { timeout: 5000 });
});

And doing it more simply with page.evaluate makes it fail for chromium, webkit, and firefox on macOS for me:

test("AudioContext should stay suspended", async ({ page }) => {
    const audioContextState = await page.evaluate(async () => {
        const audioContext = new AudioContext();
        const gainNode = new GainNode(audioContext);
        gainNode.connect(audioContext.destination);
        gainNode.gain.value = 0.025;
        const sineNode = new OscillatorNode(audioContext);
        sineNode.connect(gainNode);
        sineNode.start();
        await new Promise<void>((resolve) => setTimeout(resolve, 5000));
        return audioContext.state;
    });
    expect(audioContextState).toBe("suspended");
});

@yury-s
Copy link
Member

yury-s commented Nov 19, 2024

Changing that test to make it start a sound makes it pass for chromium and firefox, but fail for webkit on macOS for me:

Thanks, I can reproduce it.

And doing it more simply with page.evaluate makes it fail for chromium, webkit, and firefox on macOS for me:

This is probably because each page.evaluate is marked as "User gesture" evaluate in the browser, the behavior is similar to that when the code is called from say onclick handler and is consistent with that.

@andy-fillebrown
Copy link
Author

Changing that test to make it start a sound makes it pass for chromium and firefox, but fail for webkit on macOS for me:

Thanks, I can reproduce it.

And doing it more simply with page.evaluate makes it fail for chromium, webkit, and firefox on macOS for me:

This is probably because each page.evaluate is marked as "User gesture" evaluate in the browser, the behavior is similar to that when the code is called from say onclick handler and is consistent with that.

Ok, thanks for the info. Is there a way to prevent page.evaluate from being marked as a user gesture? The test you posted earlier is good assuming the webkit difference is resolved, but audio tests would be cleaner if a page.evaluate option was available that prevents it from being marked as a user gesture.

@yury-s
Copy link
Member

yury-s commented Nov 19, 2024

Changing that test to make it start a sound makes it pass for chromium and firefox, but fail for webkit on macOS for me:

The test is actually consistent with vendor browsers behavior. Safari also creates the context in 'running' state on my macOS machine, so it seems to be working as intended.

Is there a way to prevent page.evaluate from being marked as a user gesture?

No. But you can probably trigger the code with intermediate setTimeout .

@andy-fillebrown
Copy link
Author

andy-fillebrown commented Nov 20, 2024

The test is actually consistent with vendor browsers behavior. Safari also creates the context in 'running' state on my macOS machine, so it seems to be working as intended.

How are you determining this? When I run a webpack dev server using that example, Safari creates the context as 'suspended', which is what I would expect. How are you getting Safari to create the context as 'running' without user interaction?

Is there a way to prevent page.evaluate from being marked as a user gesture?

No. But you can probably trigger the code with intermediate setTimeout .

Can you give an example of triggering the code with an intermediate setTimeout? I tried it with a setTimeout and got the same failure:

test("AudioContext should stay suspended", async ({ page }) => {
    const audioContextState = await page.evaluate(async () => {
        return new Promise<string>((resolve) =>
            setTimeout(async () => {
                const audioContext = new AudioContext();
                const gainNode = new GainNode(audioContext);
                gainNode.connect(audioContext.destination);
                gainNode.gain.value = 0.025;
                const sineNode = new OscillatorNode(audioContext);
                sineNode.connect(gainNode);
                sineNode.start();
                setTimeout(() => {
                    sineNode.stop();
                    resolve(audioContext.state);
                }, 3000);
            }, 1000)
        );
    });
    expect(audioContextState).toBe("suspended");
});

@yury-s
Copy link
Member

yury-s commented Nov 20, 2024

How are you determining this?

I open this page in Safari:

<script>
async function onLoad() {
  const log = document.getElementById('log');
  const audioContext = new AudioContext();
  const gainNode = new GainNode(audioContext);
  gainNode.connect(audioContext.destination);
  gainNode.gain.value = 0.025;
  const sineNode = new OscillatorNode(audioContext);
  sineNode.connect(gainNode);
  sineNode.start();
  await new Promise((resolve) => setTimeout(resolve, 2000));
  log.innerHTML = 'State: ' + audioContext.state;
}
</script>
<body onload="onLoad()">
  <div id="log"></div>
</body>

@mxschmitt
Copy link
Member

I ran this on my Mac with the following results:

Chrome: suspended
Safari: running

@andy-fillebrown
Copy link
Author

How are you determining this?

I open this page in Safari:

<script> async function onLoad() { const log = document.getElementById('log'); const audioContext = new AudioContext(); const gainNode = new GainNode(audioContext); gainNode.connect(audioContext.destination); gainNode.gain.value = 0.025; const sineNode = new OscillatorNode(audioContext); sineNode.connect(gainNode); sineNode.start(); await new Promise((resolve) => setTimeout(resolve, 2000)); log.innerHTML = 'State: ' + audioContext.state; } </script>

Right, but how are you serving the page and what URL are you pointing Safari to?

@yury-s
Copy link
Member

yury-s commented Nov 20, 2024

Just http://127.0.0.1:8081/audio.html, serving with npx http-server .

@andy-fillebrown
Copy link
Author

Just http://127.0.0.1:8081/audio.html, serving with npx http-server .

Thanks. Doing exactly that with the same html on my machine shows State: suspended, and again; this is exactly what I expect to see and matches what a user would see when trying to start audio without a user gesture.

Can you check your autoplay preferences in Safari and make sure 127.0.0.1 is set to "Stop Media with Sound"?
You can find this setting in Safari -> Preferences -> Websites -> Auto-Play.

@yury-s
Copy link
Member

yury-s commented Nov 20, 2024

Yeah, I'd expect it to be suspended as well, but for some reason it is not. Here is the settings window:

Image

@docEdub
Copy link

docEdub commented Nov 20, 2024

Yeah, I'd expect it to be suspended as well, but for some reason it is not. Here is the settings window:

Image

Maybe autoplay gets a free pass on 127.0.0.1? I have it explicitly set to "Stop Media with Sound" on my machine. I'll try it

@andy-fillebrown
Copy link
Author

andy-fillebrown commented Nov 20, 2024

On my machine the "Auto-Play" websites list is empty until I actually go to 127.0.0.1:8080/audio.html. Going back into the auto-play settings while that page is loaded shows it in the websites list set to "Stop Media with Sound". When I change it to "Allow All Auto-Play", I get the audio context State: running that you're getting. Can you verify your "Auto-Play" settings with 127.0.0.1:8080/audio.html loaded and see if it's set to "Allow All Auto-Play", too?

@yury-s
Copy link
Member

yury-s commented Nov 20, 2024

This is what I see when opening the settings while the page is loaded:

Image

@andy-fillebrown
Copy link
Author

Interesting. What version of Safari and macOS is this? I'm on Sonoma 14.7 running Safari 17.6 (19618.3.11.11.5)

@yury-s
Copy link
Member

yury-s commented Nov 20, 2024

Looks like it only starts in the "running" state when I paste the URL after Safari's cold start (e.g. after the computer's reboot), if I repeat the same action later the context starts in a suspended state! The settings look exactly the same in both cases.

@yury-s
Copy link
Member

yury-s commented Nov 20, 2024

Interesting. What version of Safari and macOS is this? I'm on Sonoma 14.7 running Safari 17.6 (19618.3.11.11.5)

Sequoia 15.1 Safari 18.1 (20619.2.8.11.10). But I believe this is just a race as described above, it might be that it starts in running state before the browser settings are fully loaded or something like that.

@andy-fillebrown
Copy link
Author

Interesting. What version of Safari and macOS is this? I'm on Sonoma 14.7 running Safari 17.6 (19618.3.11.11.5)

Sequoia 15.1 Safari 18.1 (20619.2.8.11.10). But I believe this is just a race as described above, it might be that it starts in running state before the browser settings are fully loaded or something like that.

Yes, this repros on my machine, too. Nice find!

@andy-fillebrown
Copy link
Author

I'm thinking the Playwright webkit browser is giving localhost a free pass on autoplay, but chromium and firefox are not. I'll try testing with a non-localhost server next to see if that changes things for webkit.

@andy-fillebrown
Copy link
Author

andy-fillebrown commented Nov 20, 2024

I'm thinking the Playwright webkit browser is giving localhost a free pass on autoplay, but chromium and firefox are not. I'll try testing with a non-localhost server next to see if that changes things for webkit.

Nope, this gives the same result. AFAICT the Playwright webkit browser always allows autoplay, so this feature request is only needed for the webkit browser.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants