Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cpptools crash - The language server crashed. Restarting... #10651

Open
Colengms opened this issue Mar 10, 2023 · 71 comments
Open

cpptools crash - The language server crashed. Restarting... #10651

Colengms opened this issue Mar 10, 2023 · 71 comments
Assignees
Milestone

Comments

@Colengms
Copy link
Contributor

image

We're very interested in fixing crashes. Information that could help us fix crashes include:

  • Crash stacks.
  • Specific guidance on how to repro a crash.
  • Isolated examples that reproduce a crash and can be shared with us directly.

If you're experiencing a crash, any of the above information could be extremely useful to us to investigate and fix it. Please consider opening a new issue in this repo. Or, add a comment on this issue.

I'm pinning this issue, to make this more obvious to users who might be browsing our repo due to experiencing a crash.

@Colengms Colengms added this to the Backlog milestone Mar 10, 2023
@Colengms Colengms self-assigned this Mar 10, 2023
@Colengms Colengms pinned this issue Mar 10, 2023
@H-G-Hristov
Copy link

Just to remind that on macOS: #9810

@sean-mcmanus
Copy link
Contributor

sean-mcmanus commented Mar 10, 2023

@H-G-Hristov Yes, but that issue mentions a work around via running lldb -p <processId>, then continue (assume it gets into a stopped state), and then repro the crash, and use bt to get the full stack of the crashing thread (maybe bt all if bt isn't sufficient for some reason).

@sean-mcmanus
Copy link
Contributor

I've reproed multiple crashes. I'm tracking it with #10636 .

We may not need this issue anymore.

@sean-mcmanus sean-mcmanus unpinned this issue Mar 17, 2023
@bobbrow bobbrow closed this as not planned Won't fix, can't repro, duplicate, stale Apr 25, 2023
@Colengms
Copy link
Contributor Author

Colengms commented Apr 27, 2023

Reopening. The intent of this issue was to be pinned so users experiencing a crash can find the issue and easily find instructions. I'd like to keep this open and pinned for now.

@sean-mcmanus
Copy link
Contributor

@Colengms Yeah, that seems fine, but user should know that we don't currently have a widespread crashing issue (as of https://github.com/microsoft/vscode-cpptools/releases/tag/v1.14.5 or later).

@OnielN14
Copy link

OnielN14 commented Aug 9, 2023

I also encountered similar issue.
OS: Windows 11
VS Code version : 1.81.0
C/C++ Extension version: 1.16.3

here is what I found when I check the log in developer tools :
image

ERR [Extension Host] (node:20304) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `Code --trace-deprecation ...` to show where the warning was created)

@OnielN14
Copy link

OnielN14 commented Aug 9, 2023

I also encountered similar issue. OS: Windows 11 VS Code version : 1.81.0 C/C++ Extension version: 1.16.3

here is what I found when I check the log in developer tools : image

ERR [Extension Host] (node:20304) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `Code --trace-deprecation ...` to show where the warning was created)

I just try to manually install the extension into fresh VSCodium v1.81.0.. the extension works

@chris0306
Copy link

image
大佬,这个问题怎么破?

@doxxx
Copy link

doxxx commented Aug 14, 2023

Working with a large Visual Studio solution of ~70 C# projects and one C++/CMake project. Also had the same solution open in Visual Studio at the same time. Performed a Rebuild in Visual Studio and then came back to 10000+ duplicate errors in VS Code like this:

[Error - 9:24:53 AM] Sending notification cpptools/fileDeleted failed.
Error: Cannot call write after a stream was destroyed
	at new NodeError (node:internal/errors:387:5)
	at _write (node:internal/streams/writable:323:11)
	at Writable.write (node:internal/streams/writable:336:10)
	at c:\Users\gordon.tyler\.vscode\extensions\ms-vscode.cpptools-1.16.3-win32-x64\dist\main.js:77222:29
	at new Promise (<anonymous>)
	at WritableStreamWrapper.write (c:\Users\gordon.tyler\.vscode\extensions\ms-vscode.cpptools-1.16.3-win32-x64\dist\main.js:77212:16)
	at StreamMessageWriter.doWrite (c:\Users\gordon.tyler\.vscode\extensions\ms-vscode.cpptools-1.16.3-win32-x64\dist\main.js:76424:33)
	at c:\Users\gordon.tyler\.vscode\extensions\ms-vscode.cpptools-1.16.3-win32-x64\dist\main.js:76415:29
	at runNextTicks (node:internal/process/task_queues:61:5)
	at process.processImmediate (node:internal/timers:437:9)

@sean-mcmanus
Copy link
Contributor

@OnielN14 @chris0306 @doxxx That info is insufficient. We more info, particular the info that's described in the description for this issue, such as a call stack of cpptools or a repro.

@sean-mcmanus
Copy link
Contributor

@OnielN14 @chris0306 @doxxx Also, 1.17.1 and 1.17.2 of our extension has some crash fixes, so that may be worth trying.

@vineetsoni
Copy link

vineetsoni commented Aug 17, 2023

I start to have the same issue since the extension auto-updated to 1.17.3 today. I'm running AlmaLinux 8.8 with kernel 4.18.0-477.15.1.el8_8.x86_64. The VSCode version is 1.81

Not sure if this is any useful, but the output message is:

[Error - 2:14:08 PM] The language server crashed. Restarting...
[Error - 2:14:08 PM] Sending document notification textDocument/didOpen failed
Error: Attempting to use languageClient before initialized
	at DefaultClient.get languageClient [as languageClient] (/u/vinson3z/.vscode-server/extensions/ms-vscode.cpptools-1.17.3-linux-x64/dist/main.js:47283:19)
	at DefaultClient.sendDidOpen (/u/vinson3z/.vscode-server/extensions/ms-vscode.cpptools-1.17.3-linux-x64/dist/main.js:48450:20)
	at DefaultClient.takeOwnership (/u/vinson3z/.vscode-server/extensions/ms-vscode.cpptools-1.17.3-linux-x64/dist/main.js:48439:20)
	at processDelayedDidOpen (/u/vinson3z/.vscode-server/extensions/ms-vscode.cpptools-1.17.3-linux-x64/dist/main.js:53325:30)
	at runMicrotasks (<anonymous>)
	at processTicksAndRejections (node:internal/process/task_queues:96:5)
	at async Function.dispatch (/u/vinson3z/.vscode-server/extensions/ms-vscode.cpptools-1.17.3-linux-x64/dist/main.js:48476:44)

Is there any known workaround for this problem?

Fixed the problem by downgrading to 1.16.3. Downgrading from 1.17.3 to 1.17.2 didn't work.

PS: This happens only when I try to use VSCode with a very large project. For a medium-sized project, it works fine.

@bobbrow
Copy link
Member

bobbrow commented Aug 17, 2023

@vineetsoni it looks like the extension isn't recovering from a crash properly. There were some code changes in this area recently, so thank you for reporting it. I'll move your report into a new issue.

@borjamunozf
Copy link

borjamunozf commented Nov 29, 2023

@borjamunozf were you able to try 1.19.1 yet? Your screenshots suggest you are attempting to attach to 1.18.5. Do you happen to be working on anything open source that we can clone and test locally?

Same. Attach to process, 2-3 minutes showing lot of messages

[Detaching after fork from child process 1163080]
[Detaching after fork from child process 1163081]
[Detaching after fork from child process 1163082]
[Detaching after fork from child process 1163083]
[Detaching after fork from child process 1163084]
[Detaching after fork from child process 1163085]
[Detaching after fork from child process 1163086]
[Detaching after fork from child process 1163087]

and finally the crash.

Sadly, we work on propietary code. It's all code from my company, complex and huge projects I guess.

imagen

@borjamunozf, also I see that you opened microsoft/vscode-cmake-tools#3469. Not sure if these issues are related but I'm guessing you have a pretty big compile_commands.json file. Would you have time this week or next week to jump on a video call and walk me through the crash(es)?

It's not the same project, but yes, we have big compile_commands.json for all projects I have seen in my division. In that issue we have found that disabling the loadCompileCommands at least allows to run CMake Configure & build the project, but it's only a workaround. Just as reference, for this project the compile_commands.json is 47.56MB size., ~83000 lines

In this project, as I mentioned above, removing or renaming c_cpp_properties seems to avoid the crash, but the IntelliSense misses stuff, of course. Also if comment out the "compileCommands" only it seems to not crash, or not crash so fast, but it also does not work any Show Call Hierarchy, for example.

About the video call, that would be great. However I have to check given the nature of the code. Will let you know.

@Colengms
Copy link
Contributor Author

Hi @borjamunozf . I see a number of clues in the information you've posted so far.

In one of your images, I see that the IntelliSense process failed to be restarted. That would imply either an issue with our restart logic, or that something is immediately in a bad state when it starts back up, such as some anomolous configuration information.

The fact you were unable to detect a process crash from the debugger suggests that perhaps the process isn't actually crashing, but being terminated by another process. We've seen symptoms like this in the past with overzealous anti-malware software killing our process. To my knowledge, we do not have an influx of users reporting these symptoms on Linux, so something specific to your environment could potentially be involved.

Your C/C++ configuration refers to both use of CMake Tools as a configurationProvider, and a compile_commands.json file. Presumably, the compile_commands.json file is one being generated by CMake. This is superfluous, as use of CMake Tools as a custom configuration provider would provide configurations for all of the same files as in that compile_commands.json file. I'd suggest removing the compileCommands field from your config. (If that addresses the issue, could you help us investigate further, in case fixing this could help other compileCommands users?)

One of the most useful sources of information for investigating issues with the C/C++ extension is its debug output. Could you enable the setting "C_Cpp.loggingLevel": "Debug", monitor the C/C++ output channel leading up to the repro, and provide us with any pertinent content? Also, could you open a new issue here to track your specific repro, as this issue is more general and crashes often do not have the same cause.

I'm hoping there is something in your log output immediately before the issue that might point to what's going on here, or at least what it was in the process of doing when the problem started to occur.

Also if comment out the "compileCommands" only it seems to not crash, or not crash so fast, but it also does not work any Show Call Hierarchy, for example.

Given that this setting seems superfluous in your case, I would not expect any difference in behavior from removing it. Though, when using a compile_commands.json file, the C/C++ extension will analyze all of those command lines on startup (mainly to build a list of 'browse' paths to populate the browse database with), whereas it would only analyze those command lines as they are needed and provided for a file to configure IntelliSense for it. (Custom configuration providers provide browse paths up front, so all command lines do not need to be parsed in that scenario.) My guess would be that there is a failure parsing one of your command lines. Debug log output may refer to which one.

@sean-mcmanus
Copy link
Contributor

@bobbrow fileChanged/Created/Deleted triggers an IntelliSense update unless the file or the file's folder is filtered from a files.exclude setting. So sure you can put the cmake build folder in your workspace folder, but you should also add it to the C_Cpp.files.exclude to avoid unnecessary IntelliSense updating.

@bobbrow
Copy link
Member

bobbrow commented Nov 30, 2023

@bobbrow fileChanged/Created/Deleted triggers an IntelliSense update unless the file or the file's folder is filtered from a files.exclude setting. So sure you can put the cmake build folder in your workspace folder, but you should also add it to the C_Cpp.files.exclude to avoid unnecessary IntelliSense updating.

Can we change that behavior? If it's obviously not a C or C++ file, I don't see why we would want to trigger an IntelliSense update. We have filters for non-C/C++ files (based on file extension) in the TypeScript code so it doesn't seem wrong to do the same thing in the language server.

@Zes-MinKey-Young
Copy link

How to get a call stack? I cannot attach a debugger because the crash arises before that. If I open cpptools.exe outside VS Code, I cannot reproduce the crash. The executable does nothing.

@sean-mcmanus
Copy link
Contributor

@SemenAntonov Your issue is with the clangd extension and not our extension.

@godlikeue
Copy link

Please tell me which previous version did not have this issue.

  1. I install a clean Ubuntu system on a new computer.
  2. sudo snap install --classic code
  3. install cpptools in extensions

Open a cpp file , this issue will occur!

@godlikeue
Copy link

I have tried many times and finally located the problematic version to be v1.19.2.
If I revert to v1.19.1, as well as the earlier v1.18.5, this issue will not occur, and the IntelliSense is working now!

My vscode version is 1.89.1.

@smp73
Copy link

smp73 commented Jul 10, 2024

@microsoft-github-policy-service agree company="Microsoft"

@hbina
Copy link

hbina commented Jul 20, 2024

I am remotely developing on my RPI5, it appears that the LSP server is crashing consistently. This is the only output of cpptools.

[Error - 4:01:48 PM] The language server crashed. Restarting...
[Error - 4:01:48 PM] Server process exited with signal SIGSEGV.

@abhishekd-1
Copy link

abhishekd-1 commented Jul 23, 2024

I am remotely developing on my RPI5, it appears that the LSP server is crashing consistently. This is the only output of cpptools.

[Error - 4:01:48 PM] The language server crashed. Restarting...
[Error - 4:01:48 PM] Server process exited with signal SIGSEGV.

I'm seeing the same thing as well on a Raspberry Pi 4 with extension version 1.21.2. Downgrading the C/C++ extension to v1.20.5 fixed the problem for me.

I'm running VS Code version 1.89.1 and have attached the crash stack here as instructed in the first post.

@dkvg-77
Copy link

dkvg-77 commented Jul 25, 2024

I have been receiving the following error message ever since the update was released 2 days ago.

[Error - 7:36:32 PM] Server initialization failed.
  Message: Pending response rejected since connection got disposed
  Code: -32097 
[Error - 7:36:32 PM] The language server crashed 5 times in the last 3 minutes. It will not be restarted.
[Error - 7:36:32 PM] cpptools client: couldn't create connection to server.
  Message: Pending response rejected since connection got disposed
  Code: -32097 
[Error - 7:36:32 PM] Server process exited with code 3221226505.

I just open up the VS code and this error is thrown.
I tried reinstalling the extensions, reinstalling VS Code, and many other things.
But there has been no solution.
Please help me.

I am using Windows.

@imanabu
Copy link

imanabu commented Jul 26, 2024

On Windows 11, the extension is now crashing on Language server in the past few days. What can I provide you to help diagnose this?

@sean-mcmanus
Copy link
Contributor

@abhishekd-1 That crash is fixed with 1.21.4.

@dkvg-77 @imanabu Are you able to attach a debugger to the cpptools.exe process to get a crash call stack? https://github.com/microsoft/vscode-cpptools/wiki/Attaching-debugger-to-cpptools-or-cpptools%E2%80%90srv

@dkvg-77
Copy link

dkvg-77 commented Jul 26, 2024

Hey @sean-mcmanus

I have been trying to attach a debugger to the cpptools.exe process following the steps in the given URL.
After clicking on the “Start Debugging” button, I get to attach a process.
But I can't find 'cpptools.exe' option in there.

And, to talk about the problem of the extension,
I am facing no issues in the 1.20.5 or below version.
I am facing issues only in the last 3 updates.

Also for an option called 'C/C++ tag parser status', it says 'initializing'.
Where as it says 'parsing complete' for the 1.20.5's version.

@imanabu
Copy link

imanabu commented Jul 26, 2024

@abhishekd-1 That crash is fixed with 1.21.4.

@dkvg-77 @imanabu Are you able to attach a debugger to the cpptools.exe process to get a crash call stack? https://github.com/microsoft/vscode-cpptools/wiki/Attaching-debugger-to-cpptools-or-cpptools%E2%80%90srv

Well, here is what happens so with my knowledge of VSC extensions, right now I am at a loss how to get this attach while the extension is starting up. I have a flutter project with Windows target and I think when that gets detected, the extension starts and then it crashes. I am on 1.21.4

@sean-mcmanus
Copy link
Contributor

@dkvg-77 @imanabu It sounds like it's crashing too early to attach. Are you able to set C_Cpp.logginglevel to "Debug" and look at the C/C++ logging before the 2nd "cpptools/initialize" logging (which occurs after it crashes and re-initializes).

Does it crash with an empty workspace folder without any C/C++ files open? Does opening a file trigger the crash or adding certain files to the workspace folder? If you can attach before it crashes and then do something to trigger the crash, then you can get a crash call stack that way.

@adrianstephens
Copy link
Contributor

adrianstephens commented Nov 18, 2024

I am getting this crash with the release and beta versions, and I would love to help you track this down because intellisense has always been hit or miss for me for (literally) decades!

I have attached to the 'ms-vscode.cpptools-1.22.11-win32-x64\bin\cpptools.exe' process, but it just eventually exits with exit code 0, so I can't get a stack trace from that.
Could you advise what I should attach to, and if there is a version of it with symbols?

This is the log from the Output window:

loggingLevel: Debug
LSP: (received) cpptools/initialize (id: 1)
LSP: (invoked) cpptools/initialize (id: 1)
cpptools version (TypeScript): 1.22.11
cpptools version (native): 1.22.11.0
Autocomplete is enabled.
Error squiggles are enabled.
Hover is enabled.
IntelliSense Engine = default.
LSP: Sending response (id: 1)
LSP: (received) cpptools/queryCompilerDefaults (id: 2)
LSP: (invoked) cpptools/queryCompilerDefaults (id: 2)
LSP: Message ignored due to no registered handler: $/setTrace
LSP: (received) cpptools/initialize (id: 1)
LSP: (invoked) cpptools/initialize (id: 1)
cpptools version (TypeScript): 1.22.11
cpptools version (native): 1.22.11.0
Autocomplete is enabled.
Error squiggles are enabled.
Hover is enabled.
IntelliSense Engine = default.
LSP: Sending response (id: 1)
LSP: (received) cpptools/queryCompilerDefaults (id: 2)
LSP: (invoked) cpptools/queryCompilerDefaults (id: 2)
LSP: (received - deferred) cpptools/getDocumentSymbols: file:///d%3A/dev/shared/isoeditor/devices/device_directories.cpp (id: 3)
LSP: (received - deferred) cpptools/getCodeActions: file:///d%3A/dev/shared/isoeditor/devices/device_directories.cpp (id: 4)
LSP: (received - deferred) cpptools/getDocumentSymbols: file:///d%3A/dev/shared/isoeditor/devices/device_directories.cpp (id: 5)
LSP: (received - deferred) cpptools/getFoldingRanges: file:///d%3A/dev/shared/isoeditor/devices/device_directories.cpp (id: 6)

Adrian Stephens

@sean-mcmanus
Copy link
Contributor

sean-mcmanus commented Nov 18, 2024

@adrianstephens You may be seeing a crash with cpptools-srv.exe. There info on getting a stack on Windows at https://github.com/microsoft/vscode-cpptools/wiki/Attaching-debugger-to-cpptools-or-cpptools%E2%80%90srv .

@adrianstephens
Copy link
Contributor

@sean-mcmanus I don't see a process called that, but the message about the language server crashing happens very soon after cpptools.exe exits. Is cpptools-srv spawned by cpptools.exe? If so, I can try and intercept where that happens

@sean-mcmanus
Copy link
Contributor

@adrianstephens Yeah, if you're getting the "language server crashed" notification and no cpptools-srv is launched, then it sounds like cpptools is crashing, but if you attach and it exits normally it sounds like it might not be crashing and instead exiting normally for some unexpected reason. Make sure you have "symbolSearchPath": "https://msdl.microsoft.com/download/symbols" set. If you attach and use the Pause button you should be able to see some info on the thread call stacks.

@adrianstephens
Copy link
Contributor

@sean-mcmanus Yes, I do see symbols; I added this to my launch script:

"symbolOptions": {
   "searchMicrosoftSymbolServer": true
}

which I suppose I'd assumed was the default

@sean-mcmanus
Copy link
Contributor

@adrianstephens But you don't get an exception notification? The command logged is LSP: (invoked) cpptools/queryCompilerDefaults (id: 2) so it sounds like it's trying to query a compiler and the next expected logging is Compiler info database not connected - skipping load., so it might be crashing while trying to query the database. Can you reset your database with the C/C++: Reset IntelliSense Database command or manually delete the file? If you revert to 1.21.6, which I'm assuming isn't crashing (because it doesn't use the database during queryCopilerDefaults), you can run the database reset command there.

But your crash issue sounds like a very particular crash scenario and it probably should be tracked in a separate issue instead of this "catch all" issue.

@adrianstephens
Copy link
Contributor

adrianstephens commented Nov 19, 2024

@sean-mcmanus The database is seemingly never created; the folder is there (C:\Users\adrian\AppData\Local\Microsoft\vscode-cpptools\d720ade87aeabca9e1a090a5f4fac8b8) but empty.
cpptools.exe exits (after a few minutes) with:
The program '[17524] cpptools.exe' has exited with code 0 (0x0).
and I have it set to break on any exception.

The furthest back I can go using marketplace is 1.22.9, and that exhibits all the same behaviour (no database, cpptools.exe exiting cleanly, 'language server crashed')

@sean-mcmanus
Copy link
Contributor

@adrianstephens The marketplace UI seems to limit the versions when it shouldn't. You can get 1.21.6 from within VS Code itself or from GitHub at https://github.com/microsoft/vscode-cpptools/releases/tag/v1.21.6 . Yeah, the failure to create the database is the root problem, and the queryCompilerDefaults code might not be handling that failure correctly -- we can look into simulating a database failure in that code path to see if we can reproduce the issue and/or find a cause. There might be some unexpected issue with that location so it might be fixed if you change the databaseFilename location. I'll create a new issue to track that.

@adrianstephens
Copy link
Contributor

I used to have the database location set to ${workspaceFolder}/.vscode, and changed it to the default in case that was the problem, so I don't think it's going to be location specific. 1.21.6 exhibits the same problem (though it creates an additional error popup 'Sending document notification textDocument/didOpen failed.').
Did you want to move this discussion to 12976, or is that a separate issue?
The failure to create the database is more important to me than how cpptools handles it. but if that's someone else's problem, can you advise where I should take this?

@sean-mcmanus
Copy link
Contributor

sean-mcmanus commented Nov 19, 2024

@adrianstephens The databaseFilename needs to have the file too, like ${workspaceFolder}/.vscode/vc.db.

Sending document notification textDocument/didOpen failed. is not an error I remember being reported for 1.21.6.

You may be able to get more database error logging by temporarily setting C_Cpp.loggingLevel to the hidden value of "7", string not a number, (and "8" might give more info too...I can't remember what logging level we set the sqlite error messages at).

Yeah, it would be better to discuss it in #12976 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests