C++ syntax highlighting is very slow in this example

[alexr00]

Originally from @JacquesLucke in https://github.com/microsoft/vscode/issues/137106

Issue Type: Bug

1. Copy the text below.
2. In VSCode, hit Ctrl+N to create a new editor.
3. Paste the previously copied text. Note how VS Code automatically detects that this is C++ code.
4. VS Code freezes for a couple of seconds.

Since the text is quite short, I would not expect a noticeable freeze.

I have been sent here from the c++ extension repo: https://github.com/microsoft/vscode-cpptools/issues/8392.

libc.so.6!sched_yield (Unknown Source:0)
tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::receive_or_steal_task(long&, long) (Unknown Source:0)
tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::local_wait_for_all(tbb::task&, tbb::task*) (Unknown Source:0)
tbb::internal::generic_scheduler::local_spawn_root_and_wait(tbb::task*, tbb::task*&) (Unknown Source:0)
tbb::task::spawn_root_and_wait(tbb::task & root) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/task.h:809)
tbb::interface9::internal::start_for<tbb::blocked_range<GeometrySet**>, tbb::internal::parallel_for_each_body_for<GeometrySet::modify_geometry_sets(GeometrySet::ForeachSubGeometryCallback)::<lambda(GeometrySet*)>, GeometrySet**>, const tbb::auto_partitioner>::run(const tbb::blocked_range<GeometrySet**> &, const tbb::internal::parallel_for_each_body_for<GeometrySet::modify_geometry_sets(GeometrySet::ForeachSubGeometryCallback)::<lambda(GeometrySet*)>, GeometrySet**> &, const tbb::auto_partitioner &)(const tbb::blocked_range<GeometrySet**> & range, const tbb::internal::parallel_for_each_body_for<GeometrySet::modify_geometry_sets(GeometrySet::ForeachSubGeometryCallback)::<lambda(GeometrySet*)>, GeometrySet**> & body, const tbb::auto_partitioner & partitioner) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/parallel_for.h:95)
tbb::parallel_for<tbb::blocked_range<GeometrySet**>, tbb::internal::parallel_for_each_body_for<GeometrySet::modify_geometry_sets(GeometrySet::ForeachSubGeometryCallback)::<lambda(GeometrySet*)>, GeometrySet**> >(const tbb::blocked_range<GeometrySet**> &, const tbb::internal::parallel_for_each_body_for<GeometrySet::modify_geometry_sets(GeometrySet::ForeachSubGeometryCallback)::<lambda(GeometrySet*)>, GeometrySet**> &)(const tbb::blocked_range<GeometrySet**> & range, const tbb::internal::parallel_for_each_body_for<GeometrySet::modify_geometry_sets(GeometrySet::ForeachSubGeometryCallback)::<lambda(GeometrySet*)>, GeometrySet**> & body) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/parallel_for.h:201)
tbb::internal::parallel_for_each_impl<GeometrySet**, GeometrySet::modify_geometry_sets(GeometrySet::ForeachSubGeometryCallback)::<lambda(GeometrySet*)>, std::random_access_iterator_tag>::doit(GeometrySet **, GeometrySet **, const struct {...} &)(GeometrySet ** first, GeometrySet ** last, const struct {...} & f) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/parallel_for_each.h:79)
tbb::parallel_for_each<GeometrySet**, GeometrySet::modify_geometry_sets(GeometrySet::ForeachSubGeometryCallback)::<lambda(GeometrySet*)> >(GeometrySet **, GeometrySet **, const struct {...} &)(GeometrySet ** first, GeometrySet ** last, const struct {...} & f) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/parallel_for_each.h:114)
tbb::parallel_for_each<blender::Vector<GeometrySet*>, GeometrySet::modify_geometry_sets(GeometrySet::ForeachSubGeometryCallback)::<lambda(GeometrySet*)> >(blender::Vector<GeometrySet*, 4, blender::GuardedAllocator> &, const struct {...} &)(blender::Vector<GeometrySet*, 4, blender::GuardedAllocator> & rng, const struct {...} & f) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/parallel_for_each.h:120)
blender::threading::parallel_for_each<blender::Vector<GeometrySet*>, GeometrySet::modify_geometry_sets(GeometrySet::ForeachSubGeometryCallback)::<lambda(GeometrySet*)> >(blender::Vector<GeometrySet*, 4, blender::GuardedAllocator> &, const struct {...} &)(blender::Vector<GeometrySet*, 4, blender::GuardedAllocator> & range, const struct {...} & function) (/home/hans/Blender-Git/blender/source/blender/blenlib/BLI_task.hh:55)
GeometrySet::modify_geometry_sets(blender::FunctionRef<void (GeometrySet&)>)(GeometrySet * const this, GeometrySet::ForeachSubGeometryCallback callback) (/home/hans/Blender-Git/blender/source/blender/blenkernel/intern/geometry_set.cc:592)
blender::nodes::geo_node_curve_to_mesh_exec(blender::nodes::GeoNodeExecParams params) (/home/hans/Blender-Git/blender/source/blender/nodes/geometry/nodes/node_geo_curve_to_mesh.cc:64)
blender::modifiers::geometry_nodes::GeometryNodesEvaluator::execute_geometry_node(blender::modifiers::geometry_nodes::GeometryNodesEvaluator * const this, const blender::nodes::DNode node, blender::modifiers::geometry_nodes::NodeState & node_state) (/home/hans/Blender-Git/blender/source/blender/modifiers/intern/MOD_nodes_evaluator.cc:929)
blender::modifiers::geometry_nodes::GeometryNodesEvaluator::execute_node(blender::modifiers::geometry_nodes::GeometryNodesEvaluator * const this, const blender::nodes::DNode node, blender::modifiers::geometry_nodes::NodeState & node_state) (/home/hans/Blender-Git/blender/source/blender/modifiers/intern/MOD_nodes_evaluator.cc:905)
blender::modifiers::geometry_nodes::GeometryNodesEvaluator::node_task_run(blender::modifiers::geometry_nodes::GeometryNodesEvaluator * const this, const blender::nodes::DNode node, blender::modifiers::geometry_nodes::NodeState & node_state) (/home/hans/Blender-Git/blender/source/blender/modifiers/intern/MOD_nodes_evaluator.cc:723)
blender::modifiers::geometry_nodes::GeometryNodesEvaluator::run_node_from_task_pool(TaskPool * task_pool, void * task_data) (/home/hans/Blender-Git/blender/source/blender/modifiers/intern/MOD_nodes_evaluator.cc:707)
Task::operator()(const Task * const this) (/home/hans/Blender-Git/blender/source/blender/blenlib/intern/task_pool.cc:178)
tbb::internal::function_task<Task>::execute(tbb::internal::function_task<Task> * const this) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/task.h:1059)
tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::process_bypass_loop(tbb::internal::context_guard_helper<false>&, tbb::task*, long) (Unknown Source:0)
libpthread.so.0!__lll_lock_wait (Unknown Source:0)
libpthread.so.0!pthread_mutex_lock (Unknown Source:0)
__gthread_mutex_lock(__gthread_mutex_t * __mutex) (/usr/include/c++/11/x86_64-redhat-linux/bits/gthr-default.h:749)
std::mutex::lock(std::mutex * const this) (/usr/include/c++/11/bits/std_mutex.h:100)
std::lock_guard<std::mutex>::lock_guard(std::lock_guard<std::mutex> * const this, std::lock_guard<std::mutex>::mutex_type & __m) (/usr/include/c++/11/bits/std_mutex.h:229)
BezierSpline::evaluated_positions(const BezierSpline * const this) (/home/hans/Blender-Git/blender/source/blender/blenkernel/intern/spline_bezier.cc:614)
blender::bke::spline_extrude_to_mesh_data(const blender::bke::ResultInfo & info, const bool fill_caps, blender::MutableSpan<MVert> r_verts, blender::MutableSpan<MEdge> r_edges, blender::MutableSpan<MLoop> r_loops, blender::MutableSpan<MPoly> r_polys) (/home/hans/Blender-Git/blender/source/blender/blenkernel/intern/curve_to_mesh_convert.cc:225)
operator()(const struct {...} * const __closure, blender::IndexRange profiles_range) (/home/hans/Blender-Git/blender/source/blender/blenkernel/intern/curve_to_mesh_convert.cc:749)
operator()(const struct {...} * const __closure, const tbb::blocked_range<long> & subrange) (/home/hans/Blender-Git/blender/source/blender/blenlib/BLI_task.hh:72)
tbb::interface9::internal::start_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)>, const tbb::auto_partitioner>::run_body(tbb::blocked_range<long> &)(tbb::interface9::internal::start_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)>, const tbb::auto_partitioner> * const this, tbb::blocked_range<long> & r) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/parallel_for.h:115)
tbb::interface9::internal::dynamic_grainsize_mode<tbb::interface9::internal::adaptive_mode<tbb::interface9::internal::auto_partition_type> >::work_balance<tbb::interface9::internal::start_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)>, const tbb::auto_partitioner>, tbb::blocked_range<long int> >(tbb::interface9::internal::start_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)>, const tbb::auto_partitioner> &, tbb::blocked_range<long> &)(tbb::interface9::internal::dynamic_grainsize_mode<tbb::interface9::internal::adaptive_mode<tbb::interface9::internal::auto_partition_type> > * const this, tbb::interface9::internal::start_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)>, const tbb::auto_partitioner> & start, tbb::blocked_range<long> & range) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/partitioner.h:423)
tbb::interface9::internal::partition_type_base<tbb::interface9::internal::auto_partition_type>::execute<tbb::interface9::internal::start_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)>, const tbb::auto_partitioner>, tbb::blocked_range<long int> >(tbb::interface9::internal::start_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)>, const tbb::auto_partitioner> &, tbb::blocked_range<long> &)(tbb::interface9::internal::partition_type_base<tbb::interface9::internal::auto_partition_type> * const this, tbb::interface9::internal::start_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)>, const tbb::auto_partitioner> & start, tbb::blocked_range<long> & range) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/partitioner.h:256)
tbb::interface9::internal::start_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)>, const tbb::auto_partitioner>::execute(void)(tbb::interface9::internal::start_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)>, const tbb::auto_partitioner> * const this) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/parallel_for.h:142)
tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::process_bypass_loop(tbb::internal::context_guard_helper<false>&, tbb::task*, long) (Unknown Source:0)
tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::local_wait_for_all(tbb::task&, tbb::task*) (Unknown Source:0)
tbb::internal::generic_scheduler::local_spawn_root_and_wait(tbb::task*, tbb::task*&) (Unknown Source:0)
tbb::task::spawn_root_and_wait(tbb::task & root) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/task.h:809)
tbb::interface9::internal::start_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)>, const tbb::auto_partitioner>::run(const tbb::blocked_range<long> &, const struct {...} &, const tbb::auto_partitioner &)(const tbb::blocked_range<long> & range, const struct {...} & body, const tbb::auto_partitioner & partitioner) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/parallel_for.h:95)
tbb::parallel_for<tbb::blocked_range<long int>, blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)>&)::<lambda(const tbb::blocked_range<long int>&)> >(const tbb::blocked_range<long> &, const struct {...} &)(const tbb::blocked_range<long> & range, const struct {...} & body) (/home/hans/Blender-Git/lib/linux_centos7_x86_64/tbb/include/tbb/parallel_for.h:201)
blender::threading::parallel_for<blender::bke::curve_to_mesh_sweep(const CurveEval&, const CurveEval&, bool)::<lambda(blender::IndexRange)>::<lambda(blender::IndexRange)> >(blender::IndexRange, int64_t, const struct {...} &)(blender::IndexRange range, int64_t grain_size, const struct {...} & function) (/home/hans/Blender-Git/blender/source/blender/blenlib/BLI_task.hh:70)

VS Code version: Code 1.61.2 (6cba118ac49a1b88332f312a8f67186f7f3c1643, 2021-10-19T14:58:13.605Z) OS version: Linux x64 5.14.15-arch1-1 Restricted Mode: Yes

System Info

Item	Value
CPUs	AMD Ryzen 9 3900X 12-Core Processor (24 x 3800)
GPU Status	2d_canvas: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: disabled_off opengl: enabled_on rasterization: disabled_software skia_renderer: enabled_on video_decode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	1, 1, 1
Memory (System)	31.26GB (25.58GB free)
Process Argv	--crash-reporter-id 5f76c6ba-dfb0-40a8-a382-b82424e84719
Screen Reader	no
VM	0%
DESKTOP_SESSION	plasma
XDG_CURRENT_DESKTOP	KDE
XDG_SESSION_DESKTOP	KDE
XDG_SESSION_TYPE	x11

Extensions (2)

Extension	Author (truncated)	Version
jupyter	ms-	2021.9.1101343141
jupyter-keymap	ms-	1.0.0

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythontb:30283811
pythonvspyt551:30345470
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
vscorecescf:30384386
pythondataviewer:30285071
vscod805:30301674
pythonvspyt200:30340761
binariesv615:30325510
vsccppwt:30382697
bridge0708:30335490
dockerwalkthru:30377721
bridge0723:30353136
pythonrunftest32:30373476
pythonf5test824:30373475
javagetstartedt:30391933
pythonvspyt187:30373474
vsqsis200cf:30386380
vsaa593cf:30376535
vssld246cf:30386378

[jeff-hykin]

It might not have many lines, but the number of lines doesn't really effect the performance all that much.

Sadly It's the length of a single line that significantly impacts performance. I wouldn't really call a +1,000 character line a "quite short" piece of code.

Parsing that line is an inherently hard task, I don't think it can be done both efficiently and correctly with a TextMate parser. The better tree sitter parser derives it's name from exactly this kind of situation; it "trims/caches branches" while the TextMate parser does not. TextMate ends up having to do permutations enumerating/checking all possible branches to try to figure out whether it's the start of a variable declaration or a function declaration. Even if some advanced regex optimizations are made in the syntax, the time complexity of the algorithm is still going to be terrible. It would just be a band-aid that pushed the freeze limit from 1000 characters to, I suppose, 1200 characters.

The default highlighting character limit is below 1000 characters, and if there are issues with it being slow I would recommend reducing the maximum character length down to 250. That should prevent the freezing. I think the only practical change I can make is having a built-in C++ specific version character cap. Although it wouldn't be doing anything special, and I'm certain people would report it as a bug.

[RedCMD]

Seems nested () and <> inisde word<( ... )> is causing the problem s<(()<>()<>()<>()<>()<>()<>()<>()<>()<>()<>()<>()<>()<>())> s<(<(<(<(<(<(<(<(<(<(>)>)>)>)>)>)>)>)>)>)

[RedCMD]

I am no longer getting any lag? @jeff-hykin can you confirm?

[jeff-hykin]

Wow, yeah absolutely 0 delay now. Glad to see that was fixed