Infinite recursion in scout_parser

[Ursescu]

Bug type: Language Service

Describe the bug

• OS and Version: Windows 10 19042.1466
• VS Code Version: Version: 1.64.2 (user setup) Commit: f80445acd5a3dadef24aa209168452a3d97cc326 Date: 2022-02-09T22:02:28.252Z Electron: 13.5.2 Chromium: 91.0.4472.164 Node.js: 14.16.0 V8: 9.1.269.39-electron.0 OS: Windows_NT x64 10.0.19042
• C/C++ Extension Version: starting with v1.8.0
• Other extensions you installed (and if the issue persists after disabling them): not related to other extensions
• If using SSH remote, specify OS of remote machine: not using
• A clear and concise description of what the bug is, including information about the workspace (i.e. is the workspace a single project or multiple projects, size of the project, etc).

I've updated the extension to the newest version (v1.8.4 - I was on v1.7.1) and my project didn't complete the IntelliSense step. I've checked the debug tab and found out that tag parser was stuck on a specific file (header file, .h). Also the RAM usage increased until the hole system crashed (12GB RAM usage).

In order to understand exactly the release number that introduced this problem I've checked older releases (insider releases) and found out that v1.8.0 introduced this.

Steps to reproduce

In order to simplify this scenario I took the file (a header file around 22k lines) and included it in a simple main.c file. The results was the same, tag parser never finished to parse the file and ram usage increased indefinitely. Even a 5k header file won't work if the parser is starting clean. I can't provide the header file (it's part of some internal code), but by cutting it off to 3k lines worked. Progressively adding lines until 22k worked.

Notes:

• Initially the bug was discovered on a gcc-arm project, but it's the same with default settings.
• The project that replicates this is made out of 2 files and is only written in C language.
• Not sure what changed from v1.7.1 and v1.8.0, but v1.7.1 will parse it in 2.5 seconds.

Expected behavior Expected to work as normal, tag parser would finish and RAM usage would not increase that much.

Code sample and logs

• Code sample main.c which includes main.h (5k lines header)

#include <stdio.h>
#include <main.h>

/* Some defines from main.h */
int a = EXTAPB_REGFILE_TRANSCEIVER_PLL_CP_D2R_IF_POL_INVERT_EN_RST;
int b = EXTAPB_REGFILE_TRANSCEIVER_R_RC_CAL_R2D_IF_POL_INVERT_EN_ADDR;

int main()
{
    printf("Hello\n");
}

• Default configuration.

• Logs from running C/C++: Log Diagnostics from the VS Code command palette (I've modified the user to FAKEUSER)

-------- Diagnostics - 2/21/2022, 1:09:36 PM
Version: 1.8.0
Current Configuration:
{
    "name": "Win32",
    "includePath": [
        "${workspaceFolder}/**"
    ],
    "defines": [
        "_DEBUG",
        "UNICODE",
        "_UNICODE"
    ],
    "windowsSdkVersion": "10.0.18362.0",
    "compilerPath": "C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30037/bin/Hostx64/x64/cl.exe",
    "cStandard": "c17",
    "cppStandard": "c++17",
    "compilerArgs": [],
    "intelliSenseMode": "gcc-x64",
    "intelliSenseModeIsExplicit": true,
    "cStandardIsExplicit": false,
    "cppStandardIsExplicit": false,
    "mergeConfigurations": false,
    "compilerPathIsExplicit": false,
    "browse": {
        "path": [
            "${workspaceFolder}/**"
        ],
        "limitSymbolsToIncludedHeaders": true
    }
}
No active translation units.

------- Workspace parsing diagnostics -------
Number of files discovered (not excluded): 4831

• Logs from the language server logging

loggingLevel: Debug
cpptools/didChangeCppProperties
Attempting to get defaults from C compiler in "compilerPath" property: 'C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30037/bin/Hostx64/x64/cl.exe'
Warning: Database safety check failed. Recreating database.
[rc=0x80004005] (DB: false, Expected: true)
Code browsing service initialized
Attempting to get defaults from C++ compiler in "compilerPath" property: 'C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30037/bin/Hostx64/x64/cl.exe'
  Folder: C:/PROGRAM FILES (X86)/MICROSOFT VISUAL STUDIO/2019/COMMUNITY/VC/TOOLS/MSVC/14.29.30037/ATLMFC/INCLUDE/* will be indexed
  Folder: C:/PROGRAM FILES (X86)/MICROSOFT VISUAL STUDIO/2019/COMMUNITY/VC/TOOLS/MSVC/14.29.30037/INCLUDE/* will be indexed
  Folder: C:/PROGRAM FILES (X86)/WINDOWS KITS/10/INCLUDE/10.0.18362.0/CPPWINRT/ will be indexed
  Folder: C:/PROGRAM FILES (X86)/WINDOWS KITS/10/INCLUDE/10.0.18362.0/SHARED/ will be indexed
  Folder: C:/PROGRAM FILES (X86)/WINDOWS KITS/10/INCLUDE/10.0.18362.0/UCRT/ will be indexed
  Folder: C:/PROGRAM FILES (X86)/WINDOWS KITS/10/INCLUDE/10.0.18362.0/UM/ will be indexed
  Folder: C:/PROGRAM FILES (X86)/WINDOWS KITS/10/INCLUDE/10.0.18362.0/WINRT/ will be indexed
  Folder: C:/USERS/FAKEUSER/TEST/ will be indexed
Discovering files...
cpptools/didChangeSettings
IntelliSense Engine = Default.
Enhanced Colorization is enabled.
Error squiggles are enabled if all header dependencies are resolved.
Autocomplete is enabled.
textDocument/didOpen: C:\Users\FAKEUSER\Desktop\Test\main.c
cpptools/textEditorSelectionChange
cpptools/activeDocumentChange: C:\Users\FAKEUSER\Desktop\Test\main.c
  Processing folder (non-recursive): C:/PROGRAM FILES (X86)/MICROSOFT VISUAL STUDIO/2019/COMMUNITY/VC/TOOLS/MSVC/14.29.30037/ATLMFC/INCLUDE
  tag parsing file: C:\Users\FAKEUSER\Desktop\Test\main.c
cpptools/getDocumentSymbols: C:\Users\FAKEUSER\Desktop\Test\main.c (id: 2)
cpptools/getCodeActions: C:\Users\FAKEUSER\Desktop\Test\main.c (id: 3)
cpptools/getDocumentSymbols
  Processing folder (non-recursive): C:/PROGRAM FILES (X86)/MICROSOFT VISUAL STUDIO/2019/COMMUNITY/VC/TOOLS/MSVC/14.29.30037/INCLUDE
cpptools/getFoldingRanges: C:\Users\FAKEUSER\Desktop\Test\main.c (id: 4)
  Processing folder (recursive): C:/PROGRAM FILES (X86)/WINDOWS KITS/10/INCLUDE/10.0.18362.0/CPPWINRT/
Checking for syntax errors: C:\Users\FAKEUSER\Desktop\Test\main.c
cpptools/getCodeActions: C:\Users\FAKEUSER\Desktop\Test\main.c (id: 5)
cpptools/getSemanticTokens: C:\Users\FAKEUSER\Desktop\Test\main.c (id: 6)
  Processing folder (recursive): C:/PROGRAM FILES (X86)/WINDOWS KITS/10/INCLUDE/10.0.18362.0/SHARED/
  Processing folder (recursive): C:/PROGRAM FILES (X86)/WINDOWS KITS/10/INCLUDE/10.0.18362.0/UCRT/
  Processing folder (recursive): C:/PROGRAM FILES (X86)/WINDOWS KITS/10/INCLUDE/10.0.18362.0/UM/
  Processing folder (recursive): C:/PROGRAM FILES (X86)/WINDOWS KITS/10/INCLUDE/10.0.18362.0/WINRT/
  Processing folder (recursive): C:/Users/FAKEUSER/DESKTOP/TEST/
  Discovering files: 4831 file(s) processed
  0 file(s) removed from database
Done discovering files.
Populating include completion cache.
Parsing remaining files...
  tag parsing file: C:\Users\FAKEUSER\Desktop\Test\main.h

Screenshots

[Colengms]

Hi @Ursescu . Thanks for reporting this.

Is the C/C++ extension otherwise unresponsive while this is occurring? If it's a busy lock, could you try attaching a debugger to the process and posting the stacks here?

If there is any way you could provide us with a file that repro's the issue, that would greatly improve our chances of isolating and fixing the issue.

[Ursescu]

Hi @Colengms,

I've modified the header file to hide any details, only the file structure is the same. I've also stripped down the file from 5k lines to ~2k (removing a lot of simple constant defines, comments). This is still reproducing the issue on my side. The interesting part is that by removing the last block of the header file, it will be able to parse the file:

main.h:1918

#ifdef RANDOM_BITFIELDS
typedef volatile union{
#else
typedef volatile struct{
#endif 
  unsigned int val;
#ifdef RANDOM_BITFIELDS
  struct{
    unsigned int  random_bit_field : 1;
    unsigned int  random_bit_field : 1;
    unsigned int  random_bit_field : 1;
    unsigned int  random_bit_field : 1;
    unsigned int  random_bit_field : 1;
  } fields;
#endif 
} data_struct;

The zip contains main.c and main.h files. BigFile.zip

If needed I can provide the stack as well by attaching a debugger to the running process, but I guess the header file should be enough if this is reproducible on your side.

[Colengms]

Hi @Ursescu . Thank you for reporting this, and for providing a reduced repro. I'm able to repro and will investigate further.

[Ursescu]

Hi @Colengms. I've tested today on a Linux machine and this issue is present on releases as old as 1.0.0

[sean-mcmanus]

@Colengms Was an internal bug filed or was this the issue we a way to preprocess the file?

@Ursescu You may be able to work around the issue by adding the file's folder to C_Cpp.files.exclude (or the file itself if C_Cpp.exclusionPolicy is set accordingly).

[Ursescu]

@sean-mcmanus Thanks, I've added that file to exclude list and it seems to work. Looking forward for the proper fix.

[sean-mcmanus]

A "proper fix" is available with 1.14.0 (pre-release): https://github.com/microsoft/vscode-cpptools/releases/tag/v1.14.0