fix: identifier regexp supports unicode identifiers

[Ilya Trapashko (itrapashko)]

Fixes #2391

The new regular expression supports all JavaScript identifiers. It was taken from the following article: https://v8.dev/features/regexp-match-indices

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (2)

src/adapter/variableStore.ts:91

• privatePropertyRe is too permissive: it allows an empty private identifier ("#") and allows starting with ID_Continue characters (e.g. digits), which are not valid as the first character of a PrivateIdentifier. This can produce invalid this.#... evaluateNames.

const privatePropertyRe = /^#[$_\u200C\u200D\p{ID_Continue}]*$/u;

src/adapter/completions.ts:285

• Comment has a grammar typo: "properties are aren't" should be "properties that aren't".

    // For any properties are aren't valid identifiers, quote them as foo['bar!']

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/adapter/completions.ts:288

• The updated identifier validation now treats Unicode identifier characters as valid. Since there’s existing coverage for completion quoting behavior (see src/test/completion/completion.test.ts), consider adding a test case that verifies a non-ASCII but valid identifier (e.g. obj.π| or obj.привет|) is not converted to bracket access, while an invalid name (space/punctuation) still is. This helps prevent regressions in the new Unicode behavior.

    // For any properties that aren't valid identifiers, quote them as foo['bar!']
    for (const item of result) {
      if (!validIdentifierRe.test(item.label)) {
        item.text = `[${JSON.stringify(item.label)}]`;

[Ilya Trapashko (itrapashko)]

Connor Peet (@connor4312) I have added test to completion.test.ts as suggested by Copilot, but it did not pass because of completions sorting order. Current implementation uses leading ~ symbols to sort completion items. It works for ASCII letters, but identifiers with greater character codes (e.g. Cyrillic) sort after prototype members, because string comparison is lexicographic. For example, ~~привет sorts after ~~~constructor even though it has fewer ~. Should this issue be addressed in this PR? All other problems are fixed.

[Connor Peet (connor4312)]

THis is good, thanks!