linstor-server
linstor-server copied to clipboard
LINBIT
Snapshoot Restore failed with not enough free space
Try to restore an snapshot. We have enough free space on storage-pool ( Free capacity 19.69 TiB, total capacity 29.02 TiB). SNapshot has 2 volume, first (0) 20Gb, second (1) 1Tb. We have another pool to, vith smaller storage, but snapshot and original resource not stored there.
ERROR REPORT 66391CD0-00000-000004
============================================================
Application: LINBIT® LINSTOR
Module: Controller
Version: 1.27.1
Build ID: c6f8ceed9d50da2c4d37ae8ce20d09daf3046464
Build time: 2024-04-25T11:55:38+00:00
Error time: 2024-05-22 12:41:56
Node: hp01
Thread: grizzly-http-server-29
Access context information
Identity: PUBLIC
Role: PUBLIC
Domain: PUBLIC
Peer: RestClient(127.0.0.1; 'PythonLinstor/1.19.0 (API1.0.4): Client 1.19.0')
============================================================
Reported error:
===============
Category: RuntimeException
Class name: ApiRcException
Class canonical name: com.linbit.linstor.core.apicallhandler.response.ApiRcException
Generated at: Method 'checkIfStorPoolsAreUsable', Source file 'CtrlVlmCrtApiHelper.java', Line #303
Error message: Not enough free space available for volume 0 of resource 'mal-backup'.
Error context:
Not enough free space available for volume 0 of resource 'mal-backup'.
Call backtrace:
Method Native Class:Line number
checkIfStorPoolsAreUsable N com.linbit.linstor.core.apicallhandler.controller.CtrlVlmCrtApiHelper:303
createVlmImpl N com.linbit.linstor.core.apicallhandler.controller.CtrlVlmCrtApiHelper:152
createVolumeFromAbsVolume N com.linbit.linstor.core.apicallhandler.controller.CtrlVlmCrtApiHelper:129
restoreOnNode N com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotRestoreApiCallHandler:526
restoreResourceInTransaction N com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotRestoreApiCallHandler:265
lambda$restoreSnapshot$0 N com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotRestoreApiCallHandler:154
doInScope N com.linbit.linstor.core.apicallhandler.ScopeRunner:149
lambda$fluxInScope$0 N com.linbit.linstor.core.apicallhandler.ScopeRunner:76
call N reactor.core.publisher.MonoCallable:72
trySubscribeScalarMap N reactor.core.publisher.FluxFlatMap:127
subscribeOrReturn N reactor.core.publisher.MonoFlatMapMany:49
subscribe N reactor.core.publisher.Flux:8759
onNext N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195
request N reactor.core.publisher.Operators$ScalarSubscription:2545
onSubscribe N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:141
subscribe N reactor.core.publisher.MonoJust:55
subscribe N reactor.core.publisher.MonoDeferContextual:55
subscribe N reactor.core.publisher.Flux:8773
onNext N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195
request N reactor.core.publisher.Operators$ScalarSubscription:2545
onSubscribe N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:141
subscribe N reactor.core.publisher.MonoJust:55
subscribe N reactor.core.publisher.MonoDeferContextual:55
subscribe N reactor.core.publisher.Mono:4495
subscribeWith N reactor.core.publisher.Mono:4561
subscribe N reactor.core.publisher.Mono:4462
subscribe N reactor.core.publisher.Mono:4398
subscribe N reactor.core.publisher.Mono:4370
doFlux N com.linbit.linstor.api.rest.v1.RequestHelper:326
restoreResource N com.linbit.linstor.api.rest.v1.SnapshotRestoreResource:68
invoke0 Y jdk.internal.reflect.NativeMethodAccessorImpl:unknown
invoke N jdk.internal.reflect.NativeMethodAccessorImpl:62
invoke N jdk.internal.reflect.DelegatingMethodAccessorImpl:43
invoke N java.lang.reflect.Method:566
lambda$static$0 N org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory:52
run N org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1:146
invoke N org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:189
doDispatch N org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker:159
dispatch N org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:93
invoke N org.glassfish.jersey.server.model.ResourceMethodInvoker:478
apply N org.glassfish.jersey.server.model.ResourceMethodInvoker:400
apply N org.glassfish.jersey.server.model.ResourceMethodInvoker:81
run N org.glassfish.jersey.server.ServerRuntime$1:256
call N org.glassfish.jersey.internal.Errors$1:248
call N org.glassfish.jersey.internal.Errors$1:244
process N org.glassfish.jersey.internal.Errors:292
process N org.glassfish.jersey.internal.Errors:274
process N org.glassfish.jersey.internal.Errors:244
runInScope N org.glassfish.jersey.process.internal.RequestScope:265
process N org.glassfish.jersey.server.ServerRuntime:235
handle N org.glassfish.jersey.server.ApplicationHandler:684
service N org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer:356
run N org.glassfish.grizzly.http.server.HttpHandler$1:190
doWork N org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:535
run N org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:515
run N java.lang.Thread:829
END OF ERROR REPORT.
I'ts happand again. After restart of controller and satelit now working. What is wrong? This issue I never have before version upgrade to 1.27.1
ERROR REPORT 664DD1CA-00000-000026
============================================================
Application: LINBIT® LINSTOR
Module: Controller
Version: 1.27.1
Build ID: c6f8ceed9d50da2c4d37ae8ce20d09daf3046464
Build time: 2024-04-25T11:55:38+00:00
Error time: 2024-06-25 15:02:55
Node: hp01
Thread: grizzly-http-server-35
Access context information
Identity: PUBLIC
Role: PUBLIC
Domain: PUBLIC
Peer: RestClient(127.0.0.1; 'PythonLinstor/1.19.0 (API1.0.4): Client 1.19.0')
============================================================
Reported error:
===============
Category: RuntimeException
Class name: ApiRcException
Class canonical name: com.linbit.linstor.core.apicallhandler.response.ApiRcException
Generated at: Method 'checkIfStorPoolsAreUsable', Source file 'CtrlVlmCrtApiHelper.java', Line #303
Error message: Not enough free space available for volume 0 of resource 'backup_db088c31'.
Error context:
Not enough free space available for volume 0 of resource 'backup_db088c31'.
Call backtrace:
Method Native Class:Line number
checkIfStorPoolsAreUsable N com.linbit.linstor.core.apicallhandler.controller.CtrlVlmCrtApiHelper:303
createVlmImpl N com.linbit.linstor.core.apicallhandler.controller.CtrlVlmCrtApiHelper:152
createVolumeFromAbsVolume N com.linbit.linstor.core.apicallhandler.controller.CtrlVlmCrtApiHelper:129
restoreOnNode N com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotRestoreApiCallHandler:526
restoreResourceInTransaction N com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotRestoreApiCallHandler:265
lambda$restoreSnapshot$0 N com.linbit.linstor.core.apicallhandler.controller.CtrlSnapshotRestoreApiCallHandler:154
doInScope N com.linbit.linstor.core.apicallhandler.ScopeRunner:149
lambda$fluxInScope$0 N com.linbit.linstor.core.apicallhandler.ScopeRunner:76
call N reactor.core.publisher.MonoCallable:72
trySubscribeScalarMap N reactor.core.publisher.FluxFlatMap:127
subscribeOrReturn N reactor.core.publisher.MonoFlatMapMany:49
subscribe N reactor.core.publisher.Flux:8759
onNext N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195
request N reactor.core.publisher.Operators$ScalarSubscription:2545
onSubscribe N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:141
subscribe N reactor.core.publisher.MonoJust:55
subscribe N reactor.core.publisher.MonoDeferContextual:55
subscribe N reactor.core.publisher.Flux:8773
onNext N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195
request N reactor.core.publisher.Operators$ScalarSubscription:2545
onSubscribe N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:141
subscribe N reactor.core.publisher.MonoJust:55
subscribe N reactor.core.publisher.MonoDeferContextual:55
subscribe N reactor.core.publisher.Mono:4495
subscribeWith N reactor.core.publisher.Mono:4561
subscribe N reactor.core.publisher.Mono:4462
subscribe N reactor.core.publisher.Mono:4398
subscribe N reactor.core.publisher.Mono:4370
doFlux N com.linbit.linstor.api.rest.v1.RequestHelper:326
restoreResource N com.linbit.linstor.api.rest.v1.SnapshotRestoreResource:68
invoke N jdk.internal.reflect.GeneratedMethodAccessor5:unknown
invoke N jdk.internal.reflect.DelegatingMethodAccessorImpl:43
invoke N java.lang.reflect.Method:566
lambda$static$0 N org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory:52
run N org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1:146
invoke N org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:189
doDispatch N org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker:159
dispatch N org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:93
invoke N org.glassfish.jersey.server.model.ResourceMethodInvoker:478
apply N org.glassfish.jersey.server.model.ResourceMethodInvoker:400
apply N org.glassfish.jersey.server.model.ResourceMethodInvoker:81
run N org.glassfish.jersey.server.ServerRuntime$1:256
call N org.glassfish.jersey.internal.Errors$1:248
call N org.glassfish.jersey.internal.Errors$1:244
process N org.glassfish.jersey.internal.Errors:292
process N org.glassfish.jersey.internal.Errors:274
process N org.glassfish.jersey.internal.Errors:244
runInScope N org.glassfish.jersey.process.internal.RequestScope:265
process N org.glassfish.jersey.server.ServerRuntime:235
handle N org.glassfish.jersey.server.ApplicationHandler:684
service N org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer:356
run N org.glassfish.grizzly.http.server.HttpHandler$1:190
doWork N org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:535
run N org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:515
run N java.lang.Thread:829
END OF ERROR REPORT.