Japanese characters in string literals are not highlighted correctly in VS Code

[Guest0301]

Problem:
When using the OCaml Platform extension in VS Code on WSL (Ubuntu), Japanese characters within string literals ("") are not highlighted with the correct string color. Instead, they appear with the default text color (e.g., white), while surrounding non-Japanese characters in the same string are correctly highlighted.

Steps to Reproduce:

1. Open VS Code on WSL (Ubuntu).
2. Open an OCaml file (.ml) containing a string literal with Japanese characters.
3. Observe the highlighting of the Japanese characters within the string literal.

Expected Result:
All characters within the string literal, including Japanese characters, should be highlighted with the designated string color (e.g., green, as seen with ASCII characters).

Actual Result:
Japanese characters within the string literal are not highlighted with the string color. They retain the default text color. (See attached screenshots).

Example Code (from hyoji.ml):
(* テスト *)
let test1 = hyoji {kanji = "若荷谷"; kana = "みょうがだに"; romaji = "myogadani"}
= "丸ノ内線, 茗荷谷 (みょうがだに)"
let test2 = hyoji {kanji = "鹿部"; kana = "しかべ"; romaji = "shikabe"}
= "函館本線, 鹿部 (しかべ)"
let test3 = hyoji {kanji = "志築"; kana = "しじみ"; romaji = "shizimi"}
= "栗生線, 志築 (しじみ)"

Environment Information:
・OS: Windows 11 (Version 21H2, OS Build 22000.2538) with WSL2 (Ubuntu 22.04 LTS)
・VS Code Version: 1.100.2
・OCaml Platform Extension Version: 1.30.0
・opam Version: 2.3.0
・OCaml Compiler Version: 5.1.1
・ocaml-lsp-server Version: 1.18.0

Troubleshooting Steps Already Taken (and their outcomes):

1. Full opam clean installation:
   ・Removed ~/.opam directory.
   ・Uninstalled and reinstalled opam using sudo apt remove opam and sudo apt install opam (or official install script).
   ・Re-initialized opam (opam init).
   ・Recreated OCaml switch (opam switch create 5.1.1).
   ・Executed eval $(opam env) or eval $(opam env --switch=5.1.1) after each relevant step.
   ・Outcome: ocaml-lsp-server was eventually installed successfully, but the highlighting issue persisted.

2. ocaml-lsp-server installation attempts:
   ・Initially, opam install ocaml-lsp resulted in [ERROR] No package named ocaml-lsp found..
   ・Later, opam install ocaml-lsp-server (as suggested by VS Code) successfully installed the package, after opam was re-initialized.
   ・Outcome: ocaml-lsp-server is now installed, and VS Code reports Failed to start the language server. ocaml-lsp-server is not installed... (initially) and later Unable to find 'ocamlformat-rpc' binary... (indicating ocaml-lsp-server is running).

3. Disabling Semantic Highlighting:
   ・Set editor.semanticHighlighting.enabled: false in VS Code settings.
   ・Outcome: The issue persists; Japanese characters are still not highlighted correctly.

4. VS Code Encoding Check:
   ・Confirmed file encoding is UTF-8 in VS Code status bar. Re-opened with encoding explicitly set to UTF-8.
   ・Outcome: No change.

5. Theme Change:
   ・Tried default VS Code themes.
   ・Outcome: No change.

[mnxn]

Hi @Guest0301,

I'm able to reproduce the issue, but only when semantic highlighting is enabled (editor.semanticHighlighting.enabled). That means it's likely a problem with the language server. Could you please submit a ticket to https://github.com/ocaml/ocaml-lsp?