Support modern ECMAScript regexp syntax

[bd82]

For example:

• unicode flag
• lookbehind assertions.

The biggest question is to decide if to update regexp-to-ast to the newst spec
or replace it with regexpp.

At the moment I am leaning towards updating regexp-to-ast as it seems regexpp is only minimally maintained.

• See Investigate using regexpp Instead of regexp-to-ast #777

[likern]

@bd82 Hello, I've started using this awesome parser in React Native environment. Please, consider also supporting React Native Hermes engine, it probably has some limitations in RegEx compared to V8.

More information is on Hermes site.

[0xThiebaut]

For anyone ending up here and needing modern ECMAScript syntax, here is a quick workaround (in TypeScript) wrapping a RegExp into a Custom Token Pattern.

import { createToken, CustomPatternMatcherFunc } from "chevrotain"

function wrap(regex: RegExp): CustomPatternMatcherFunc {
    return (text: string, offset: number): RegExpExecArray | null => {
        const re = new RegExp(regex, 'y');
        re.lastIndex = offset
        return re.exec(text)
    }
}

export const ComplexIdentifier = createToken({
    name: 'variable.other.identifier.complex',
    pattern: wrap(/(?<=\[['"])[\w. -]{1,1024}(?=['"]\])/), // Wrap the modern ECMAScript RegExp syntax in the function
    line_breaks: false // A Custom Token Pattern should specify the <line_breaks> option
})

[bd82]

I've started using this awesome parser in React Native environment. Please, consider also supporting React Native Hermes engine, it probably has some limitations in RegEx compared to V8.

Hello @likern

Chevrotain is a JavaScript library, so it should not need any special support to run inside of hermes.
Generally it should run on any ES6 compatible JavaScript engine which I assume Hermes is...

[bd82]

Should evaluate the effort in updating regexp-to-ast to ECMAScript 2024 standard

[robocoder]

Are your maintenance concerns for regexpp alleviated by the fork?

• @eslint-community/regexpp

[bd82]

Are your maintenance concerns for regexpp alleviated by the fork?

Perhaps a little, it is not a very active fork, probably due to the small scope and the fact the regexp syntax does not
change on every ECMAScript spec version (yearly).
So its difficult to assess its health.

I need to map all the regexp features in newer ECMAScript versions to understand the scope of the work
for upgrading regexp-to-ast, it is possible replacing the regexp parser would be more work than upgrading the existing one...

The problem is that I'm only investing very little time in Chevrotain right now... So I don't know when (or if) I'll get around to it...