Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: support stripANSIControlSequences in json reporter #33670

Open
gofr opened this issue Nov 19, 2024 · 5 comments
Open

[Feature]: support stripANSIControlSequences in json reporter #33670

gofr opened this issue Nov 19, 2024 · 5 comments
Labels
feature-test-runner Playwright test specific issues P3-collecting-feedback

Comments

@gofr
Copy link

gofr commented Nov 19, 2024

Version

1.48.2

Steps to reproduce

  1. Create simple test:
import { test, expect } from '@playwright/test';

test('color', () => {
expect('foo').toBe('bar');
})
  1. Run Playwright with colors turned off: FORCE_COLOR=0 DEBUG_COLORS=false npx playwright test

Expected behavior

I expect the output to lack any colors.

Actual behavior

Playwright's own output has no colors, but the output from expect does have colors. This part:

Error: expect(received).toBe(expected) // Object.is equality

Expected: "bar"
Received: "foo"

Additional context

This seems related to #32543.

#32764 changed DEBUG_COLORS to respect the environment, here (on line 40):

FORCE_COLOR: '1',
DEBUG_COLORS: process.env.DEBUG_COLORS === undefined ? '1' : process.env.DEBUG_COLORS,

If I make the same adjustment to FORCE_COLOR on line 39, the colors from expect disappear too.

Environment

System:
    OS: Linux 5.15 Ubuntu 22.04.5 LTS 22.04.5 LTS (Jammy Jellyfish)
    CPU: (8) x64 Intel(R) Core(TM) i5-10310U CPU @ 1.70GHz
    Memory: 1.29 GB / 11.69 GB
    Container: Yes
  Languages:
    Bash: 5.1.16 - /usr/bin/bash
  npmPackages:
    @playwright/test: ^1.48.2 => 1.48.2
@dgozman
Copy link
Contributor

dgozman commented Nov 19, 2024

@gofr Playwright wants expect to have color, even when running without tty or color support. This way, reports like an html report will still get nice expect error messages. What's your usecase for disabling expect coloring?

@gofr
Copy link
Author

gofr commented Nov 19, 2024

@dgozman I'm using the json reporter and am also getting the ANSI color codes in the error strings there.

That wasn't a problem until I ran into jestjs/jest#15384 which causes my JSON to be invalid.

I guess an alternative would be to add the stripANSIControlSequences option that the junit reporter has to json too.

@dgozman
Copy link
Contributor

dgozman commented Nov 20, 2024

@gofr Thank you for the details!

How does stripping ANSI control sequences help with unpaired surrogates in the text? These two problems look unrelated to me. Could you please explain in more details?

As a workaround, post-process your json with a regex to strip ansi. We do that quite often, here is an example from our source code:

const ansiRegex = new RegExp('([\\u001B\\u009B][[\\]()#;?]*(?:(?:(?:[a-zA-Z\\d]*(?:;[-a-zA-Z\\d\\/#&.:=?%@~_]*)*)?\\u0007)|(?:(?:\\d{1,4}(?:;\\d{0,4})*)?[\\dA-PR-TZcf-ntqry=><~])))', 'g');
export function stripAnsiEscapes(str: string): string {
  return str.replace(ansiRegex, '');
}

@gofr
Copy link
Author

gofr commented Nov 20, 2024

@dgozman Since Jest's diffing doesn't handle surrogates, it can put ANSI control sequences in the middle of a surrogate pair. So my JSON ends up containing an invalid string like this:

"Expected: \u001b[32m\"\ud83d\u001b[7m\ude04\u001b[27m\"\u001b[39m"

Stripping the ANSI makes it valid again:

"Expected: \"\ud83d\ude04\""

In my case this is just a workaround for the Jest bug. I also thought not being able to fully turn off colors was a bug, but if that's working as intended I guess this is more of a feature request.

Stripping the ANSI myself is an option. Thanks for that example. 🙂

@dgozman
Copy link
Contributor

dgozman commented Nov 21, 2024

@gofr Thank you for the explanation, this makes sense now! I'll repurpose this issue into stripANSIControlSequences feature request.

@dgozman dgozman changed the title [Bug]: expect() output is colored even with DEBUG_COLORS=false [Feature]: support stripANSIControlSequences in json reporter Nov 21, 2024
@dgozman dgozman added P3-collecting-feedback feature-test-runner Playwright test specific issues labels Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-test-runner Playwright test specific issues P3-collecting-feedback
Projects
None yet
Development

No branches or pull requests

2 participants