sitewhere icon indicating copy to clipboard operation
sitewhere copied to clipboard
verified publisher icon sitewhere

Client exception in call to com.sitewhere.grpc.service

Open ATM006 opened this issue 5 years ago • 5 comments

error 1、Client exception in call to com.sitewhere.grpc.service.DeviceManagement/ListDeviceSummaries.CLOSE error 2、Client exception in call to com.sitewhere.grpc.service.AssetManagement/GetAssetById.CLOSE 20210326134600

ATM006 avatar Mar 26 '21 05:03 ATM006

Can you provide details regarding the Kubernetes cluster where you are running SiteWhere? In particular:

  • What provider (Minikube, Google Cloud, etc)?
  • How many k8s nodes and how much CPU/RAM for each node?
  • What version of Kubernetes?
  • Which version of SiteWhere?
  • Was SiteWhere installed via swctl install or directly via Helm?

derekadams avatar Mar 26 '21 13:03 derekadams

  • What provider (Minikube, Google Cloud, etc)? （Naked metal deployment） https://github.com/kubeoperator/kubeoperator/
  • How many k8s nodes and how much CPU/RAM for each node? 3nodes（total：CPU12C、RAM24G）
  • What version of Kubernetes? v1.18.8
  • Which version of SiteWhere? -v3.0.5
  • Was SiteWhere installed via swctl install or directly via Helm? swctl install（v0.9.2）

ATM006 avatar Mar 27 '21 02:03 ATM006

Thanks, that configuration should work well. I was just making sure that the services were not starved for CPU or other resources. It looks like the root of the issue is failed interactions with the device management microservice when getting device and/or listing assignments. Are there exceptions in the logs for that microservice. Also, is there any particular use case that triggers these issues?

derekadams avatar Mar 27 '21 14:03 derekadams

Thanks for your reply.The current bug will appear after entering the tenant after each new login. This is the error log. logs-from-device-management-in-device-management-65c6795c77-w58qp.txt logs-from-instance-management-in-instance-management-59dd64f84b-hwt9d.txt

ATM006 avatar Mar 28 '21 03:03 ATM006

The root of this error seems to be the following lines:

2021-03-28 02:47:11,541 WARN  [org.hib.eng.jdb.spi.SqlExceptionHelper] (grpc-default-executor-7) SQL Error: 0, SQLState: 08006
2021-03-28 02:47:11,542 ERROR [org.hib.eng.jdb.spi.SqlExceptionHelper] (grpc-default-executor-7) An I/O error occurred while sending to the backend.
2021-03-28 02:47:11,542 WARN  [org.hib.eng.jdb.spi.SqlExceptionHelper] (grpc-default-executor-8) SQL Error: 0, SQLState: 08006
2021-03-28 02:47:11,542 ERROR [org.hib.eng.jdb.spi.SqlExceptionHelper] (grpc-default-executor-8) An I/O error occurred while sending to the backend.

It seems like in this case, the database had an error, which caused the device management APIs to fail. If you look in the sitewhere-system namespace at the logs for PostgreSQL, are there any indications of what is failing in the database? This could be another issue with running out of space on the PVC.

derekadams avatar Mar 28 '21 20:03 derekadams