linstor-server icon indicating copy to clipboard operation
linstor-server copied to clipboard

nvme add-volume errors

Open Smithx10 opened this issue 3 years ago • 4 comments

[root@drbd-linstor-0 ~]# lg nvme create -r nvme4_group linbit:nvme:foo 10.91.230.250/32 20G
Created target "linbit:nvme:foo"

[root@drbd-linstor-0 ~]# lg nvme stop linbit:nvme:foo
Stopped target "linbit:nvme:foo"

[root@drbd-linstor-0 ~]# lg nvme add-volume linbit:nvme:foo 2 20G
Error: failed to add volume to resource: failed to update config: error setting promoter config in linstor: Message: 'Modification of external file '/etc/drbd-reactor.d/linstor-gateway-nvmeof-foo.toml' failed due to an unknown exception.'; Details: 'External file: /etc/drbd-reactor.d/linstor-gateway-nvmeof-foo.toml'; Reports: '[62B79578-00000-000011]'
Usage:
  linstor-gateway nvme add-volume NQN VOLUME_NR VOLUME_SIZE [flags]

Flags:
  -h, --help   help for add-volume

Global Flags:
      --config string     Config file to load (default "/etc/linstor-gateway/linstor-gateway.toml")
  -c, --connect string    LINSTOR Gateway server to connect to (default "http://localhost:8080")
      --loglevel string   Set the log level (as defined by logrus) (default "info")

failed to add volume to resource: failed to update config: error setting promoter config in linstor: Message: 'Modification of external file '/etc/drbd-reactor.d/linstor-gateway-nvmeof-foo.toml' failed due to an unknown exception.'; Details: 'External file: /etc/drbd-reactor.d/linstor-gateway-nvmeof-foo.toml'; Reports: '[62B79578-00000-000011]'

[root@ac-1f-6b-a5-ab-ea etc]# drbdadm status
foo role:Secondary
  volume:0 disk:UpToDate
  volume:1 disk:UpToDate
  volume:2 disk:Inconsistent
  ac-1f-6b-9e-e5-46 role:Secondary
    volume:0 peer-disk:Diskless
    volume:1 peer-disk:Diskless
    volume:2 peer-disk:Diskless
  ac-1f-6b-a4-df-ee role:Secondary
    volume:0 peer-disk:UpToDate
    volume:1 peer-disk:UpToDate
    volume:2 peer-disk:Inconsistent

Smithx10 avatar Jun 27 '22 23:06 Smithx10

Jul 05 14:21:05 drbd-linstor-0 Controller[84135]: 14:21:05.789 [grizzly-http-server-5] ERROR LINSTOR/Controller - SYSTEM - A Resource group: nvme_group already exists.
Jul 05 14:21:05 drbd-linstor-0 Controller[84135]: 14:21:05.797 [grizzly-http-server-7] ERROR LINSTOR/Controller - SYSTEM - The creation of a new resource definition failed due to a name collision
Jul 05 14:21:05 drbd-linstor-0 Controller[84135]: 14:21:05.819 [grizzly-http-server-6] INFO  LINSTOR/Controller - SYSTEM - New volume definition with number '2' of resource definition 'demo0' created.
Jul 05 14:21:06 drbd-linstor-0 Controller[84135]: 14:21:06.445 [grizzly-http-server-0] ERROR LINSTOR/Controller - SYSTEM - A volume definition with the number 1 already exists in resource definition 'demo0'.
Jul 05 14:21:06 drbd-linstor-0 Controller[84135]: 14:21:06.451 [grizzly-http-server-1] ERROR LINSTOR/Controller - SYSTEM - A volume definition with the number 2 already exists in resource definition 'demo0'.
Jul 05 14:21:06 drbd-linstor-0 Controller[84135]: 14:21:06.484 [grizzly-http-server-4] ERROR LINSTOR/Controller - SYSTEM - DatabaseException [Report number 62B79578-00000-000202]

Smithx10 avatar Jul 05 '22 14:07 Smithx10

Do you still have that error-report from LINSTOR by any chance? linstor error-reports show 62B79578-00000-000011

kermat avatar Jul 05 '22 17:07 kermat

[root@drbd-linstor-0 ~]# linstor error-reports show 62B79578-00000-000011
ERROR REPORT 62B79578-00000-000011

============================================================

Application:                        LINBIT® LINSTOR
Module:                             Controller
Version:                            1.18.2
Build ID:                           26945460e48d2b9e98f6e2163e05b722dd5ff3ca
Build time:                         2022-05-23T07:54:20+00:00
Error time:                         2022-06-27 23:09:45
Node:                               drbd-linstor-0
Peer:                               RestClient(0:0:0:0:0:0:0:1; 'Go-http-client/1.1')

============================================================

Reported error:
===============

Description:
    DatabaseException

Category:                           LinStorException
Class name:                         DatabaseException
Class canonical name:               com.linbit.linstor.dbdrivers.DatabaseException
Generated at:                       Method 'update', Source file 'SQLSingleColumnDriver.java', Line #113

Error message:                      DatabaseException

Error context:
    Modification of external file '/etc/drbd-reactor.d/linstor-gateway-nvmeof-foo.toml' failed due to an unknown exception.

Call backtrace:

    Method                                   Native Class:Line number
    update                                   N      com.linbit.linstor.dbdrivers.sql.SQLSingleColumnDriver:113
    set                                      N      com.linbit.linstor.transaction.TransactionSimpleObject:48
    setContent                               N      com.linbit.linstor.core.objects.ExternalFile:174
    setInTransaction                         N      com.linbit.linstor.core.apicallhandler.controller.CtrlExternalFilesApiCallHandler:174
    lambda$set$0                             N      com.linbit.linstor.core.apicallhandler.controller.CtrlExternalFilesApiCallHandler:130
    doInScope                                N      com.linbit.linstor.core.apicallhandler.ScopeRunner:147
    lambda$null$0                            N      com.linbit.linstor.core.apicallhandler.ScopeRunner:75
    call                                     N      reactor.core.publisher.MonoCallable:91
    trySubscribeScalarMap                    N      reactor.core.publisher.FluxFlatMap:126
    subscribeOrReturn                        N      reactor.core.publisher.MonoFlatMapMany:49
    subscribe                                N      reactor.core.publisher.Flux:8343
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Flux:8357
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Mono:4252
    subscribeWith                            N      reactor.core.publisher.Mono:4363
    subscribe                                N      reactor.core.publisher.Mono:4223
    subscribe                                N      reactor.core.publisher.Mono:4159
    subscribe                                N      reactor.core.publisher.Mono:4131
    doFlux                                   N      com.linbit.linstor.api.rest.v1.RequestHelper:304
    putFile                                  N      com.linbit.linstor.api.rest.v1.ExternalFiles:142
    invoke                                   N      sun.reflect.GeneratedMethodAccessor24:unknown
    invoke                                   N      sun.reflect.DelegatingMethodAccessorImpl:43
    invoke                                   N      java.lang.reflect.Method:498
    lambda$static$0                          N      org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory:52
    run                                      N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1:124
    invoke                                   N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:167
    doDispatch                               N      org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker:159
    dispatch                                 N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:79
    invoke                                   N      org.glassfish.jersey.server.model.ResourceMethodInvoker:469
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:391
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:80
    run                                      N      org.glassfish.jersey.server.ServerRuntime$1:253
    call                                     N      org.glassfish.jersey.internal.Errors$1:248
    call                                     N      org.glassfish.jersey.internal.Errors$1:244
    process                                  N      org.glassfish.jersey.internal.Errors:292
    process                                  N      org.glassfish.jersey.internal.Errors:274
    process                                  N      org.glassfish.jersey.internal.Errors:244
    runInScope                               N      org.glassfish.jersey.process.internal.RequestScope:265
    process                                  N      org.glassfish.jersey.server.ServerRuntime:232
    handle                                   N      org.glassfish.jersey.server.ApplicationHandler:680
    service                                  N      org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer:356
    run                                      N      org.glassfish.grizzly.http.server.HttpHandler$1:200
    doWork                                   N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:569
    run                                      N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:549
    run                                      N      java.lang.Thread:750

Caused by:
==========

Category:                           Exception
Class name:                         PSQLException
Class canonical name:               org.postgresql.util.PSQLException
Generated at:                       Method 'setObject', Source file 'PgPreparedStatement.java', Line #651

Error message:                      Cannot cast an instance of [B to type Types.BLOB

Call backtrace:

    Method                                   Native Class:Line number
    setObject                                N      org.postgresql.jdbc.PgPreparedStatement:651
    setObject                                N      org.postgresql.jdbc.PgPreparedStatement:908
    setObject                                N      org.apache.commons.dbcp2.DelegatingPreparedStatement:529
    setObject                                N      org.apache.commons.dbcp2.DelegatingPreparedStatement:529
    fillSetter                               N      com.linbit.linstor.dbdrivers.sql.SQLSingleColumnDriver:133
    update                                   N      com.linbit.linstor.dbdrivers.sql.SQLSingleColumnDriver:83
    set                                      N      com.linbit.linstor.transaction.TransactionSimpleObject:48
    setContent                               N      com.linbit.linstor.core.objects.ExternalFile:174
    setInTransaction                         N      com.linbit.linstor.core.apicallhandler.controller.CtrlExternalFilesApiCallHandler:174
    lambda$set$0                             N      com.linbit.linstor.core.apicallhandler.controller.CtrlExternalFilesApiCallHandler:130
    doInScope                                N      com.linbit.linstor.core.apicallhandler.ScopeRunner:147
    lambda$null$0                            N      com.linbit.linstor.core.apicallhandler.ScopeRunner:75
    call                                     N      reactor.core.publisher.MonoCallable:91
    trySubscribeScalarMap                    N      reactor.core.publisher.FluxFlatMap:126
    subscribeOrReturn                        N      reactor.core.publisher.MonoFlatMapMany:49
    subscribe                                N      reactor.core.publisher.Flux:8343
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Flux:8357
    onNext                                   N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:188
    request                                  N      reactor.core.publisher.Operators$ScalarSubscription:2344
    onSubscribe                              N      reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:134
    subscribe                                N      reactor.core.publisher.MonoCurrentContext:35
    subscribe                                N      reactor.core.publisher.Mono:4252
    subscribeWith                            N      reactor.core.publisher.Mono:4363
    subscribe                                N      reactor.core.publisher.Mono:4223
    subscribe                                N      reactor.core.publisher.Mono:4159
    subscribe                                N      reactor.core.publisher.Mono:4131
    doFlux                                   N      com.linbit.linstor.api.rest.v1.RequestHelper:304
    putFile                                  N      com.linbit.linstor.api.rest.v1.ExternalFiles:142
    invoke                                   N      sun.reflect.GeneratedMethodAccessor24:unknown
    invoke                                   N      sun.reflect.DelegatingMethodAccessorImpl:43
    invoke                                   N      java.lang.reflect.Method:498
    lambda$static$0                          N      org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory:52
    run                                      N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1:124
    invoke                                   N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:167
    doDispatch                               N      org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker:159
    dispatch                                 N      org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher:79
    invoke                                   N      org.glassfish.jersey.server.model.ResourceMethodInvoker:469
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:391
    apply                                    N      org.glassfish.jersey.server.model.ResourceMethodInvoker:80
    run                                      N      org.glassfish.jersey.server.ServerRuntime$1:253
    call                                     N      org.glassfish.jersey.internal.Errors$1:248
    call                                     N      org.glassfish.jersey.internal.Errors$1:244
    process                                  N      org.glassfish.jersey.internal.Errors:292
    process                                  N      org.glassfish.jersey.internal.Errors:274
    process                                  N      org.glassfish.jersey.internal.Errors:244
    runInScope                               N      org.glassfish.jersey.process.internal.RequestScope:265
    process                                  N      org.glassfish.jersey.server.ServerRuntime:232
    handle                                   N      org.glassfish.jersey.server.ApplicationHandler:680
    service                                  N      org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer:356
    run                                      N      org.glassfish.grizzly.http.server.HttpHandler$1:200
    doWork                                   N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:569
    run                                      N      org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker:549
    run                                      N      java.lang.Thread:750


END OF ERROR REPORT.

Smithx10 avatar Jul 05 '22 17:07 Smithx10

Thanks for the report. Bug found and fixed, will be included in next release

ghernadi avatar Jul 06 '22 08:07 ghernadi