[lsp][bug] LSP locks up for ~12s when navigating to files within the same project

[mjames-c]

Apologies if we're reporting this too early but we're excited to use the typescript-go LSP at Canva when it's ready.

We have a TS project with 90k+ files within it. The initial load time for the project is very impressive at ~20s (vs 2.5 mins with TSv5):

Info 8    [09:49:07.603] Project '<omitted>/tsconfig.json' (KindConfigured)
	Files (92699)

The problem we have is that navigating the code base causes the LSP to lock up for ~13s. According to the LSP logs it is dominated by getConfigFileNameForFile (5s) and updateGraph (7s).

Info 39   [09:52:20.374] getConfigFileNameForFile:: File: <omitted> ProjectRootPath: :: Result: <omitted>/tsconfig.json
Info 40   [09:52:25.344] Config: <omitted>/tsconfig.json : {
...
Info 41   [09:52:26.557] Starting updateGraph: Project: <omitted>/tsconfig.json
Info 42   [09:52:32.612] Finishing updateGraph: Project: <omitted>/tsconfig.json version: 5
Info 43   [09:52:32.612] Different program with same set of files
Info 44   [09:52:32.627] Project '<omitted>/tsconfig.json' (KindConfigured)
	Files (92699)

I assume there's a caching layer that still needs to be ported to tsgo that already exists TSv5.

[mjames-c]

sharing some additional investigation i did today -- i manually wrapped the handleDidOpen() function (LSP command textDocument/didOpen) with the pprof profiler and took a profile of an incremental update (opening a previously unopened TS file within the same project).

tsgo --lsp seems to be spending most of its time stat'ing files:

Duration: 19.29s, Total samples = 71.61s (371.23%)
Showing nodes accounting for 71.61s, 100% of 71.61s total

[jakebailey]

FYI we released a preview extension: https://marketplace.visualstudio.com/items?itemName=TypeScriptTeam.native-preview

I am curious if this still happens; a lot of stuff has been optimized. Maybe not this, since we haven't stuck FS caching in the LSP yet.

[mjames-c]

@jakebailey just tried it out now (Identifier: typescriptteam.native-preview, Version: 0.20250601.1)

at a high level the problem still persists, here are a subset of the logs after using go-to def on a symbol within the same project:

Info 17   [10:00:17.762] Starting updateGraph: Project: <omitted>/web/tsconfig.json
Info 18   [10:00:17.762] Program 2 used 1 checker(s)
Info 19   [10:00:29.053] Different program with same set of root files
Info 20   [10:00:44.289] createGlobMapper took 6.702289805s to create 8 globs for 2733778 failed lookups
Info 21   [10:00:44.293] createGlobMapper took 4.250189ms to create 5 globs for 1750 failed lookups
Info 22   [10:00:44.293] Finishing updateGraph: Project: <omitted>/web/tsconfig.json version: 2 in 26.531445762s
Info 23   [10:00:44.293] Project '<omitted>/web/tsconfig.json' (KindConfigured)
	Files (98150)

Info 26   [10:00:50.829] Starting updateGraph: Project: <omitted>/web/tsconfig.json
Info 27   [10:00:50.829] Program 3 used 2 checker(s)
Info 28   [10:01:01.016] Different program with same set of root files
Info 29   [10:01:16.437] createGlobMapper took 6.803180171s to create 8 globs for 2733778 failed lookups
Info 30   [10:01:16.441] createGlobMapper took 4.148414ms to create 5 globs for 1750 failed lookups
Info 31   [10:01:16.441] Finishing updateGraph: Project: <omitted>/web/tsconfig.json version: 3 in 25.611635744s
Info 32   [10:01:16.441] Project '<omitted>/web/tsconfig.json' (KindConfigured)
	Files (98150)

The update graph now takes ~26s and, interestingly, it seems to run twice -- once when i first initiate the go-to def and then again once VS Code opens the file with the symbol definition.

It's probably also worth pointing out that the tsgo LSP uses 22.6 GiB of memory (vs ~17ish for TSv5) -- the initial startup updateGraph seems to be better though taking ~37s in the tsgo LSP vs ~2m15s in TSv5.

Let me know if another pprof profile on handleDidOpen() would be helpful 🙂

[mjames-c]

here's an updated pprof profile from handleDidOpen() -- the critical path seems to be dominated by updating file watchers (~15s) and visiting directories as part of tryFindDefaultConfiguredProjectForOpenScriptInfo() (5s).

naively, both of these things sound like they should be skipped when the user is performing actions which don't mutate FS state

[jakebailey]

It's possible #996 will help, but it does clear the cache between changes, so maybe not, actually.

We just aren't doing any sort of caching when it comes to module resolution, which means no caching of globs either.

[jakebailey]

What OS is this?

[mjames-c]

What OS is this?

@jakebailey it's linux (remote dev env)

[sheetalkamat]

I added back the cache for config file for open file in #1007 and that might help with tryFindDefaultConfiguredProjectForOpenScriptInfo
Also we dont have proper program reuse code (its on TODO) and that means if you open and close file, it would mean program update for now. (because the text on disk could be different - optimization that is not yet done)