pub-dev
pub-dev copied to clipboard
dart-lang
Crashing isolate due to uncaught exception: ClientException
@isoos if we're using RetryClient from package:http am I correct in that it won't retry if partial response is received. Like in the case of ClientException?
I'm guessing this error isn't the end of the world, but we probably shouldn't crash due to this. I'm also guessing that this isn't the only place where we have this issue.
pub.analyzer: ERROR from index-builder isolate #1
Error:
[Exception: Crashing isolate due to uncaught exception: ClientException: Connection closed before full header was received, uri=https://storage.googleapis.com/resumable/upload/storage/v1/b/dartlang-pub--search-snapshot/o?name=snapshot%2F2024.02.13.json.gz&uploadType=resumable&alt=json, #0 _wrapper.<anonymous closure> (package:pub_dev/service/entrypoint/_isolate.dart:316:9)
#1 _RootZone.runBinary (dart:async/zone.dart:1666:54)
#2 StackZoneSpecification._handleUncaughtError (package:stack_trace/src/stack_zone_specification.dart:161:20)
#3 _Zone._processUncaughtError (dart:async/zone.dart:1081:14)
#4 _CustomZone.handleUncaughtError (dart:async/zone.dart:1285:5)
#5 Future._propagateToListeners (dart:async/future_impl.dart:760:16)
#6 Future._completeError (dart:async/future_impl.dart:655:5)
#7 Future._asyncCompleteError.<anonymous closure> (dart:async/future_impl.dart:745:7)
#8 StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:207:15)
#9 StackZoneSpecification._registerCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:114:48)
#10 _rootRun (dart:async/zone.dart:1399:13)
#11 _CustomZone.run (dart:async/zone.dart:1301:19)
#12 _CustomZone.runGuarded (dart:async/zone.dart:1209:7)
#13 _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1249:23)
#14 _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
#15 _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)
#16 _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
#17 _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:185:5)
]
Also related I think:
isolate.wrapper: Uncaught exception in isolate.
Error:
ClientException: Connection closed before full header was received, uri=https://storage.googleapis.com/resumable/upload/storage/v1/b/dartlang-pub--search-snapshot/o?name=snapshot%2F2024.02.13.json.gz&uploadType=resumable&alt=json
Stack:
package:http/src/io_client.dart 156:7 IOClient.send
===== asynchronous gap ===========================
dart:async _Completer.completeError
package:gcloud/src/storage_impl.dart 629:20 _MediaUploadStreamSink._completeError
===== asynchronous gap ===========================
dart:async Future.then
package:gcloud/src/storage_impl.dart 659:10 _MediaUploadStreamSink._startResumableUpload
package:gcloud/src/storage_impl.dart 541:9 new _MediaUploadStreamSink
package:gcloud/src/storage_impl.dart 199:16 _BucketImpl.write
package:pub_dev/shared/storage.dart 255:27 uploadWithRetry.<fn>
package:pub_dev/shared/utils.dart 186:24 retryAsync
package:pub_dev/shared/storage.dart 253:9 uploadWithRetry
package:pub_dev/shared/storage.dart 276:5 uploadBytesWithRetry
package:pub_dev/shared/storage.dart 300:13 VersionedJsonStorage.uploadDataAsJsonMap
package:pub_dev/search/backend.dart 212:32 SearchBackend.doCreateAndUpdateSnapshot
===== asynchronous gap ===========================
dart:async _CustomZone.registerUnaryCallback
package:pub_dev/search/backend.dart 159:5 SearchBackend.doCreateAndUpdateSnapshot
===== asynchronous gap ===========================
dart:async _CustomZone.registerUnaryCallback
package:pub_dev/search/backend.dart 159:5 SearchBackend.doCreateAndUpdateSnapshot
package:pub_dev/search/backend.dart 106:17 SearchBackend.updateSnapshotInForeverLoop.<fn>
package:pub_dev/task/global_lock.dart 81:22 GlobalLock.withClaim
===== asynchronous gap ===========================
dart:async _CustomZone.registerUnaryCallback
package:pub_dev/task/global_lock.dart 40:15 GlobalLock.withClaim
package:pub_dev/search/backend.dart 105:20 SearchBackend.updateSnapshotInForeverLoop
===== asynchronous gap ===========================
dart:async _CustomZone.registerUnaryCallback
package:pub_dev/search/backend.dart 105:9 SearchBackend.updateSnapshotInForeverLoop
package:pub_dev/service/entrypoint/analyzer.dart 78:23 _indexBuilderMain
===== asynchronous gap ===========================
dart:async _CustomZone.registerUnaryCallback
package:pub_dev/service/entrypoint/analyzer.dart 77:3 _indexBuilderMain
package:pub_dev/service/entrypoint/_isolate.dart 311:43 _wrapper.<fn>.<fn>
package:pub_dev/service/services.dart 295:28 _withPubServices.<fn>.<fn>.<fn>
dart:async _CustomZone.run
package:pub_dev/service/services.dart 291:10 _withPubServices.<fn>.<fn>
package:gcloud/service_scope.dart 205:19 _ServiceScope._fork.<fn>
dart:async runZoned
package:gcloud/service_scope.dart 204:12 _ServiceScope._fork
package:gcloud/service_scope.dart 102:30 fork
package:pub_dev/service/services.dart 288:18 _withPubServices.<fn>
===== asynchronous gap ===========================
dart:async _CustomZone.registerUnaryCallback
package:pub_dev/service/services.dart 225:7 _withPubServices.<fn>
package:gcloud/service_scope.dart 205:19 _ServiceScope._fork.<fn>
dart:async runZoned
package:gcloud/service_scope.dart 204:12 _ServiceScope._fork
package:gcloud/service_scope.dart 102:30 fork
package:pub_dev/service/services.dart 222:10 _withPubServices
package:pub_dev/service/services.dart 127:20 withServices.<fn>.<fn>
===== asynchronous gap ===========================
dart:async _CustomZone.registerUnaryCallback
package:pub_dev/service/services.dart 84:26 withServices.<fn>.<fn>
package:gcloud/service_scope.dart 205:19 _ServiceScope._fork.<fn>
dart:async runZoned
package:gcloud/service_scope.dart 204:12 _ServiceScope._fork
package:gcloud/service_scope.dart 102:30 fork
package:pub_dev/service/services.dart 82:18 withServices.<fn>
package:appengine/src/appengine_internal.dart 37:51 withAppEngineServices.<fn>
package:appengine/src/appengine_internal.dart 110:20 _withAppEngineServicesInternal.<fn>
===== asynchronous gap ===========================
dart:async _CustomZone.registerUnaryCallback
package:appengine/src/appengine_internal.dart 102:45 _withAppEngineServicesInternal.<fn>
package:gcloud/service_scope.dart 205:19 _ServiceScope._fork.<fn>
dart:async runZoned
package:gcloud/service_scope.dart 204:12 _ServiceScope._fork
package:gcloud/service_scope.dart 102:30 fork
package:appengine/src/appengine_internal.dart 101:13 _withAppEngineServicesInternal
package:appengine/src/appengine_internal.dart 37:5 withAppEngineServices
package:appengine/appengine.dart 150:29 withAppEngineServices
package:pub_dev/service/services.dart 78:10 withServices
package:pub_dev/service/entrypoint/_isolate.dart 311:22 _wrapper.<fn>
package:stack_trace Chain.capture
package:pub_dev/service/entrypoint/_isolate.dart 310:24 _wrapper
Normally this should have been caught by this block: https://github.com/dart-lang/pub-dev/blob/master/app/lib/search/backend.dart#L104-L110
The only thing so far that may be suspicious is the concurrent processing inside it: if some exception would not have been caught while using Pool + Future.wait...
The other potential cause could be the scheduled microtask in lock.withClaim:
https://github.com/dart-lang/pub-dev/blob/master/app/lib/task/global_lock.dart#L45