sdk Extension type expression evaluation error when from closure parameter

trafficstars

Repro below (I could not find a way to remove more code than this).

import 'dart:typed_data';

void main() {
  var b = B(0);
  var list = [b];
  b.value2;
  list.forEach((b) {
    print(b.value2);
  });
}

extension type A._(Uint8List i) {
  A(int i) : this._(Uint8List.fromList([i]));

  int get value => i.elementAt(0);
}

extension type B._(A i) implements A {
  B(int i) : this._(A(i));

  bool get value2 => i.value == 0;
}

Steps:

In the above code, place a breakpoint on line 6 (b.value2) and line 8 (inside the closure)
Debug and add to watch b.value2
On the first break, you'll see true as the result (as expected).
On the second break, this is the output for expression evaluation:

Unhandled exception:
NoSuchMethodError: Class 'int' has no instance method 'elementAt'.
Receiver: 274334218320
Tried calling: elementAt(0)
#0      Object.noSuchMethod (dart:core-patch/object_patch.dart:38:5)
#1      A.value (package:bug/bug.dart:15:22)
#2      B.value2 (package:bug/bug.dart:21:24)
#3      Eval ()
#4      main.<anonymous closure> (package:bug/bug.dart:8:13)
#5      List.forEach (dart:core-patch/growable_array.dart:417:8)
#6      main (package:bug/bug.dart:7:8)
#7      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#8      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

CC @DanTup

Oct 17 '24 15:10 FMorschel

Summary: The issue is that extension type expression evaluation fails when the extension type is used as a parameter in a closure. The evaluation throws a NoSuchMethodError when trying to access a method of the extension type within the closure.

Oct 17 '24 15:10 dart-github-bot

Also, if you place a breakpoint on line 21 (bool get value2 => i.value == 0) and evaluate i.value you get that it is not defined. I think this is related to the problem.

evaluateInFrame: (113) Expression compilation error
org-dartlang-debug:synthetic_debug_expression:1:1: Error: The getter 'i' isn't defined for the class '_Uint8List'.
 - '_Uint8List' is from 'dart:typed_data'.
Try correcting the name to the name of an existing getter, or defining a getter or field named 'i'.
i.value
^

If you add to A the implementation:

extension type A._(Uint8List i) implements Uint8List {

When you evaluate on elementAt(0) (no i) at that line you get no errors. Maybe something like this is not being handled by the VM?

Oct 17 '24 16:10 FMorschel

cc @bkonyi

Oct 18 '24 06:10 mraleph

One more thing that I'm not sure it is related but it seems that this has lots of related problems:

Change B implementation for the following and add a breakpoint on the call for foo.

extension type B._(A i) implements A {
  factory B(int i) {
    foo();
    return B._(A(i));
  }

  static void foo() {}

  bool get value2 => i.value == 0;
}

When you evaluate foo you get:

evaluateInFrame: (113) Expression compilation error
org-dartlang-debug:synthetic_debug_expression:1:1: Error: Method not found: 'foo'.
foo()
^^^

If foo was a top level function with the same implementation you'd get null as a response.

Oct 21 '24 12:10 FMorschel

One last thing (I think 🙃) I noticed this when trying to hover a getter I had defined (like value/value2 above). I'm not sure if fixing the evaluation will fix the hover as well (it shows the same as when not debugging - the base identification), but just to be explicit that is not working.

Oct 21 '24 12:10 FMorschel

Thanks for the detailed report @FMorschel! Having a reproduction case is extremely helpful!

FYI @johnniwinther @jensjoha, I think this might be a CFE issue?

Oct 21 '24 15:10 bkonyi

Given that it happens when attempting to display a hover, I'm fairly sure it's an analyzer issue (either only or also).

Oct 21 '24 15:10 bwilkerson

Given that it happens when attempting to display a hover, I'm fairly sure it's an analyzer issue (either only or also).

@bwilkerson, I'm not sure I was clear enough (or I'm not aware of how it works). This is the hover output when not debugging:

This is the hover output when debugging of b (not really sure what was supposed to appear here, but maybe it is right?):

This is the hover output when debugging of b.value2:

I was expecting to see true on this last one (since this is what is printed on the console with print).

Edit

That's why I'm not sure I was clear. I don't see how the analysis could get me the current value for the running program on the hover.

Oct 21 '24 15:10 FMorschel

When not debugging, the analysis server provides the hover. When you are debugging, VS Code extracts the expression under the mouse and sends it to the debugger for evaluation. Based on the screenshot above, I suspect that when that fails, VS Code falls back to using the language-server hover.

So I think the issue here is in the debugger, not correctly evaluating b.value2 and that issue might be causing the analyzer's hover to be shown (whereas if the evaluation didn't fail, it would instead show the result of the evaluation, true).

Oct 21 '24 15:10 DanTup

@bkonyi while there is some inconsistency on the CFE side (the line 21 thing in https://github.com/dart-lang/sdk/issues/56911#issuecomment-2420045201 --- this is caused by being paused on the getter, not inside it, making the CFE side not find the #this as being an extension type --- stepping in to it and it works as expected. I'll (try to) fix that on the CFE side) --- I do believe this is mainly a VM issue.

It seems to me the CFE compiles the right thing correctly and sends it to the VM. The VM then somehow passes an int instead of a thing of the correct type (Uint8List) when actually calling the expression (despite correctly telling the CFE that b is a Uint8List).

I can even crash the VM by asking to compile something else:

$ p b.value2
Unhandled exception:
NoSuchMethodError: Class 'int' has no instance method 'elementAt'.
Receiver: 69940663282104
Tried calling: elementAt(0)
#0      Object.noSuchMethod (dart:core-patch/object_patch.dart:38:5)
#1      A.value (issue_56911.dart:15:22)
#2      B.value2 (issue_56911.dart:21:24)
#3      Eval ()
#4      main.<anonymous closure> (issue_56911.dart:8:13)
[...]

$ p () {b.value2;}()
Internal error: ServerRpcException(Bad state: The client closed with pending request "evaluateInFrame".)
[...]

and in the terminal I now have this:

===== CRASH =====
si_signo=Segmentation fault(11), si_code=SEGV_MAPERR(1), si_addr=0x7db000ab1
version=3.7.0-edge (main) (Unknown timestamp) on "linux_x64"
pid=3315045, thread=3315072, isolate_group=main(0x55e1d5495ff0), isolate=main(0x55e1d5491130)
os=linux, arch=x64, comp=no, sim=no
isolate_instructions=55e1bc32d160, vm_instructions=55e1bc32d160
fp=7f38a8c7a6c0, sp=7f38a8c7a6b0, pc=55e1bc5dba34
  pc 0x000055e1bc5dba34 fp 0x00007f38a8c7a6c0 dart::StackFrameIterator::NextFrame+0xb4
  pc 0x000055e1bc5dcdbd fp 0x00007f38a8c7a9d0 dart::StackTraceUtils::CollectFrames+0x97d
  pc 0x000055e1bc48b013 fp 0x00007f38a8c7aa40 dart::CurrentStackTrace+0xb3
  pc 0x000055e1bc4e79d3 fp 0x00007f38a8c7ac20 dart::ThrowExceptionHelper+0x313
  pc 0x000055e1bc4e76b9 fp 0x00007f38a8c7ac40 dart::Exceptions::Throw+0x29
  pc 0x000055e1bc5b89e6 fp 0x00007f38a8c7b170 dart::DRT_Throw+0xa6
  pc 0x00007f38ab502f73 fp 0x00007f38a8c7b1b0 Unknown symbol
  pc 0x00007f38ab508426 fp 0x00007f38a8c7b1e0 Unknown symbol
  pc 0x00007f38aa228f88 fp 0x00007f38a8c7b210 Unknown symbol
  pc 0x00007f38aa22876c fp 0x00007f38a8c7b250 Unknown symbol
  pc 0x00007f38aa228141 fp 0x00007f38a8c7b288 Unknown symbol
  pc 0x00007f38aa22806a fp 0x00007f38a8c7b2b8 Unknown symbol
  pc 0x00007f38aa232255 fp 0x00007f38a8c7b2e8 Unknown symbol
  pc 0x00007f38aa232185 fp 0x00007f38a8c7b320 Unknown symbol
  pc 0x00007f38ab503426 fp 0x00007f38a8c7b390 Unknown symbol
  pc 0x000055e1bc4d4b55 fp 0x00007f38a8c7b3f0 dart::DartEntry::InvokeFunction+0x165
  pc 0x000055e1bc541a0f fp 0x00007f38a8c7b770 dart::Instance::EvaluateCompiledExpression+0x3cf
  pc 0x000055e1bc4d98ee fp 0x00007f38a8c7b7f0 dart::ActivationFrame::EvaluateCompiledExpression+0x12e
  pc 0x000055e1bc5cdec1 fp 0x00007f38a8c7b870 dart::EvaluateCompiledExpression+0x291
  pc 0x000055e1bc5c4142 fp 0x00007f38a8c7be70 dart::Service::InvokeMethod+0x512
  pc 0x000055e1bc5c44b1 fp 0x00007f38a8c7be90 dart::Service::HandleIsolateMessage+0x11
  pc 0x000055e1bc4f3ad7 fp 0x00007f38a8c7c420 dart::IsolateMessageHandler::HandleMessage+0x1f7
  pc 0x000055e1bc5167aa fp 0x00007f38a8c7c490 dart::MessageHandler::HandleMessages+0x12a
  pc 0x000055e1bc516935 fp 0x00007f38a8c7c4d0 dart::MessageHandler::HandleOOBMessages+0x45
  pc 0x000055e1bc7c437a fp 0x00007f38a8c7c560 Dart_HandleServiceMessages+0x11a
  pc 0x000055e1bc4f834d fp 0x00007f38a8c7c5b0 dart::Isolate::PauseEventHandler+0xad
  pc 0x000055e1bc4d78a5 fp 0x00007f38a8c7c750 dart::Debugger::Pause+0xf5
  pc 0x000055e1bc4dfce2 fp 0x00007f38a8c7c890 dart::Debugger::SignalPausedEvent+0xb2
  pc 0x000055e1bc4e0404 fp 0x00007f38a8c7cd40 dart::Debugger::PauseBreakpoint+0x364
  pc 0x000055e1bc5b8e3d fp 0x00007f38a8c7d330 dart::DRT_BreakpointRuntimeHandler+0x12d
  pc 0x00007f38ab502f73 fp 0x00007f38a8c7d370 Unknown symbol
  pc 0x00007f38ab503e71 fp 0x00007f38a8c7d3a0 Unknown symbol
  pc 0x00007f38aa2236af fp 0x00007f38a8c7d3d0 Unknown symbol
  pc 0x00007f38aa228412 fp 0x00007f38a8c7d428 Unknown symbol
  pc 0x00007f38aa2235e4 fp 0x00007f38a8c7d470 Unknown symbol
  pc 0x00007f38aa22617b fp 0x00007f38a8c7d498 Unknown symbol
  pc 0x00007f38aa2260b4 fp 0x00007f38a8c7d4f8 Unknown symbol
  pc 0x00007f38aa224d5e fp 0x00007f38a8c7d538 Unknown symbol
  pc 0x00007f38aa224a87 fp 0x00007f38a8c7d5a0 Unknown symbol
  pc 0x00007f38aa224310 fp 0x00007f38a8c7d5f8 Unknown symbol
  pc 0x00007f38ab503426 fp 0x00007f38a8c7d670 Unknown symbol
  pc 0x000055e1bc4d4b55 fp 0x00007f38a8c7d6d0 dart::DartEntry::InvokeFunction+0x165
  pc 0x000055e1bc4d6473 fp 0x00007f38a8c7d710 dart::DartLibraryCalls::HandleMessage+0x123
  pc 0x000055e1bc4f3ba4 fp 0x00007f38a8c7dca0 dart::IsolateMessageHandler::HandleMessage+0x2c4
  pc 0x000055e1bc5167aa fp 0x00007f38a8c7dd10 dart::MessageHandler::HandleMessages+0x12a
  pc 0x000055e1bc516baf fp 0x00007f38a8c7dd60 dart::MessageHandler::TaskCallback+0x1ff
  pc 0x000055e1bc5ec397 fp 0x00007f38a8c7dde0 dart::ThreadPool::WorkerLoop+0x127
  pc 0x000055e1bc5ec5f2 fp 0x00007f38a8c7de10 dart::ThreadPool::Worker::Main+0x72
  pc 0x000055e1bc59e8c9 fp 0x00007f38a8c7ded0 dart::ThreadStart+0xd9
-- End of DumpStackTrace
  pc 0x0000000000000000 fp 0x00007f38a8c7b1b0 sp 0x0000000000000000 [Stub] CallToRuntime
  pc 0x00007f38ab508426 fp 0x00007f38a8c7b1e0 sp 0x00007f38a8c7b1c0 [Stub] Throw
  pc 0x00007f38aa228f88 fp 0x00007f38a8c7b210 sp 0x00007f38a8c7b1f0 [Unoptimized] Object.noSuchMethod
  pc 0x00007f38aa22876c fp 0x00007f38a8c7b250 sp 0x00007f38a8c7b220 [Unoptimized] [email protected]
  pc 0x00007f38aa228141 fp 0x00007f38a8c7b288 sp 0x00007f38a8c7b260 [Unoptimized] A|get#value
  pc 0x00007f38aa22806a fp 0x00007f38a8c7b2b8 sp 0x00007f38a8c7b298 [Unoptimized] B|get#value2
  pc 0x00007f38aa232255 fp 0x00007f38a8c7b2e8 sp 0x00007f38a8c7b2c8 [Unoptimized] :Eval.<anonymous closure>
  pc 0x00007f38aa232185 fp 0x00007f38a8c7b320 sp 0x00007f38a8c7b2f8 [Unoptimized] :Eval
  pc 0x00007f38ab503426 fp 0x00007f38a8c7b390 sp 0x00007f38a8c7b330 [Stub] InvokeDartCode
Segmentation fault

Oct 29 '24 10:10 jensjoha

I added these logging statements:

#if !defined(PRODUCT)
    for (intptr_t i = 0; i < eval_function.NumParameters(); ++i) {
      OS::PrintErr(
          "parameter_names[%ld]: %s\n", i,
          String::Handle(eval_function.ParameterNameAt(i)).ToCString());
      OS::PrintErr(
          "parameter_types[%ld]: %s\n", i,
          AbstractType::Handle(eval_function.ParameterTypeAt(i)).NameCString());
    }
    for (intptr_t i = 0; i < arguments.Length(); ++i) {
      OS::PrintErr("arguments[%ld]: address: %p, type: %s\n", i,
                   arguments.At(i).untag(),
                   Object::Handle(arguments.At(i)).JSONType());
    }
#endif  // !defined(PRODUCT)

under this line:

https://github.com/dart-lang/sdk/blob/75e6a748f7ca5f36ebb6c82a96e97714962ffa88/runtime/vm/object.cc#L4953

and then followed the reproduction steps in the issue description, and the output was:

> b.value2
parameter_names[0]: b
parameter_types[0]: Uint8List
parameter_names[1]: list
parameter_types[1]: List<Uint8List>
arguments[0]: address: 0x7f49cfdab598, type: TypedData
arguments[1]: address: 0x7f49cfdaef68, type: GrowableObjectArray
true
> b.value2
parameter_names[0]: b
parameter_types[0]: Uint8List
parameter_names[1]: list
parameter_types[1]: List<Uint8List>
arguments[0]: address: 0x7f49cfdab598, type: TypedData
Unhandled exception:
NoSuchMethodError: Class 'int' has no instance method 'elementAt'.
Receiver: 69977493467576
Tried calling: elementAt(0)
#0      Object.noSuchMethod (dart:core-patch/object_patch.dart:38:5)
#1      A.value (file:///usr/local/google/home/derekx/programming/dart-sdk/sdk/local_test_files/test_extension_types.dart:15:22)
#2      B.value2 (file:///usr/local/google/home/derekx/programming/dart-sdk/sdk/local_test_files/test_extension_types.dart:21:24)
#3      Eval ()
#4      main.<anonymous closure> (file:///usr/local/google/home/derekx/programming/dart-sdk/sdk/local_test_files/test_extension_types.dart:8:13)
#5      List.forEach (dart:core-patch/growable_array.dart:417:8)
#6      main (file:///usr/local/google/home/derekx/programming/dart-sdk/sdk/local_test_files/test_extension_types.dart:7:8)
#7      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#8      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

The only noteworthy observation is that the VM isn't supplying an argument for list when calling eval_function when the program is paused at the second breakpoint. I'm not sure if that's what's causing the problem. I'll have to look into this more.

Nov 05 '24 22:11 derekxu16

I'd say that the throwing one (i.e. the second evaluation) only being given 1 argument instead of 2 is pretty noteworthy?

Nov 06 '24 07:11 jensjoha

I believe that this goes back to https://github.com/dart-lang/sdk/issues/53996#issuecomment-1822897466. I added this logging statement:

OS::PrintErr("%s\n", js->ToCString());

under this line:

https://github.com/dart-lang/sdk/blob/be7a40dd78655709cb512a9758437d90182c2f42/runtime/vm/service.cc#L3275

and if I try to evaluate b.value2 when paused at print(b.value2) within the closure, the VM outputs

{"jsonrpc":"2.0", "result":{"param_names":["b"],"param_types":["dart:typed_data","_Uint8List","1","0"],"type_params_names":[],"type_params_bounds":[],"type_params_defaults":[],"libraryUri":"file:\/\/\/usr\/local\/google\/home\/derekx\/programming\/dart-sdk\/sdk\/local_test_files\/test_extension_types.dart","method":"<anonymous closure>","tokenPos":179,"scriptUri":"file:\/\/\/usr\/local\/google\/home\/derekx\/programming\/dart-sdk\/sdk\/local_test_files\/test_extension_types.dart","isStatic":true

So this is one of those cases that Alex mentioned in the linked comment where the problem is that "VM currently doesn't have information about variables inside closures which were not captured."

Nov 18 '24 21:11 derekxu16

A thing that I have completely missed - and maybe it's in the comments above and I'm just missing it - is that the CFE is told that there is 1 known variable (b), but the CFE generates an expression that takes 2 (b and list). This seems to be what causes this whole thing. I'm fixing it in https://dart-review.googlesource.com/c/sdk/+/400120.

Dec 11 '24 12:12 jensjoha

sdk sdk copied to clipboard

Extension type expression evaluation error when from closure parameter

Edit

sdk
sdk copied to clipboard