NFVO NullPointerException during the docker tutorial

Hello,

I'm following the docker tutorial, the first steps go well without problems but when I try to launch the Mongo NS I got "an uncaught exception. Message is: java.lang.NullPointerException" from the NFVO.

I can't figure out what I did wrong. I'd like some help.

Log and explanations :

openbaton.log

2018-03-28 08:33:30.693  WARN 911 --- [http-nio-8080-exec-10] o.o.n.a.i.AuthorizeInterceptor           : AnonymousUser requesting method: GET on /error
2018-03-28 08:44:11.150  INFO 911 --- [http-nio-8080-exec-3] o.o.n.c.a.NetworkServiceRecordManagement : Removing NSR with id: dc621619-401a-4739-adb2-ec138de43e85
2018-03-28 08:44:11.164  INFO 911 --- [OpenBatonAsyncTask-30] o.o.nfvo.vnfm_reg.state.VnfStateHandler  : Executing Task releaseresourcesTask for vnfr MongoDB. Cyclic=false
2018-03-28 08:44:11.164  INFO 911 --- [OpenBatonAsyncTask-12] o.o.n.v.tasks.abstracts.AbstractTask     : Release resources for VNFR: MongoDB
2018-03-28 08:44:30.706  INFO 911 --- [http-nio-8080-exec-10] o.o.n.c.a.NetworkServiceRecordManagement : Looking for NetworkServiceDescriptor with ID: 2f7861f7-fbba-4ad0-8fa0-d6881801f882
2018-03-28 08:44:30.709  INFO 911 --- [http-nio-8080-exec-10] o.o.n.c.a.NetworkServiceRecordManagement : Checking if all vnfm are registered and active
2018-03-28 08:44:30.715  WARN 911 --- [http-nio-8080-exec-10] o.o.n.c.a.NetworkServiceRecordManagement : Quota check is disabled... Please enable for comprehensive grant operation
2018-03-28 08:44:30.733  INFO 911 --- [OpenBatonAsyncTask-26] o.o.nfvo.vnfm_reg.state.VnfStateHandler  : Executing Task grantoperationTask for vnfr MongoDB. Cyclic=false
2018-03-28 08:44:30.733  INFO 911 --- [OpenBatonAsyncTask-13] o.o.n.v.tasks.abstracts.AbstractTask     : Executing task: GrantOperation on VNFR: MongoDB
2018-03-28 08:44:30.743  WARN 911 --- [OpenBatonAsyncTask-13] o.o.n.v.tasks.abstracts.AbstractTask     : Checking quota is disabled, please consider to enable it
2018-03-28 08:44:30.749  INFO 911 --- [OpenBatonAsyncTask-13] o.o.n.core.core.VnfPlacementManagement   : Chosen VimInstance: Optional[BaseVimInstance{name='vim-instance', location=Location{n
ame='Berlin', latitude='52.525876', longitude='13.314400'} BaseEntity{id='506a52e5-c9c6-4b0f-b4e1-1e153d07ac10', version=2, projectId='', shared=false, metadata={}}, type='docker', active=tr
ue} BaseEntity{id='c74402fa-c0b7-4ba9-87c1-f3acb6aa6684', version=3, projectId='2e01fe52-fead-43d5-8adc-346db6c845a4', shared=false, metadata={}}]
2018-03-28 08:44:30.753  INFO 911 --- [OpenBatonAsyncTask-29] o.openbaton.nfvo.core.api.VimManagement  : Refreshing vim
2018-03-28 08:44:30.798  INFO 911 --- [OpenBatonAsyncTask-13] o.o.nfvo.core.core.NetworkManagement     : Creating network new-network on vim vim-instance
2018-03-28 08:44:30.859  INFO 911 --- [OpenBatonAsyncTask-13] org.openbaton.vim_impl.vim.GenericVIM    : Created Network with name: new-network on VimInstance vim-instance
2018-03-28 08:44:30.861  INFO 911 --- [OpenBatonAsyncTask-13] o.o.nfvo.core.core.NetworkManagement     : Created Network new-network_7168
2018-03-28 08:44:30.873  INFO 911 --- [OpenBatonAsyncTask-13] o.o.n.v.tasks.abstracts.AbstractTask     : Choose all Vim Instance for vnfr: MongoDB
2018-03-28 08:44:30.893  INFO 911 --- [OpenBatonAsyncTask-26] o.o.nfvo.vnfm_reg.state.VnfStateHandler  : Executing Task allocateresourcesTask for vnfr MongoDB. Cyclic=false
2018-03-28 08:44:30.894  INFO 911 --- [OpenBatonAsyncTask-14] o.o.n.v.tasks.abstracts.AbstractTask     : Executing task: AllocateResources for VNFR: MongoDB
2018-03-28 08:44:30.927  INFO 911 --- [OpenBatonAsyncTask-29] o.o.nfvo.core.core.ResourceManagement    : Checking if Flavor m1.small exists...
2018-03-28 08:44:30.927  WARN 911 --- [OpenBatonAsyncTask-29] o.o.nfvo.core.core.ResourceManagement    : Flavor creation is supported for OpenStack only at the moment
2018-03-28 08:44:30.930  INFO 911 --- [OpenBatonAsyncTask-29] o.o.nfvo.core.core.ResourceManagement    : FloatingIp chosen are: {new-network=}
2018-03-28 08:44:30.932 ERROR 911 --- [OpenBatonAsyncTask-14] o.o.n.v.tasks.abstracts.AbstractTask     : There was an uncaught exception. Message is: java.lang.NullPointerException
2018-03-28 08:44:30.936  INFO 911 --- [OpenBatonAsyncTask-14] o.o.n.v.tasks.abstracts.AbstractTask     : Saved the VNFR MongoDB with status error after an exception
2018-03-28 08:44:30.954  INFO 911 --- [OpenBatonAsyncTask-30] o.o.nfvo.vnfm_reg.state.VnfStateHandler  : Executing Task errorTask for vnfr MongoDB. Cyclic=false
2018-03-28 08:44:30.954  INFO 911 --- [OpenBatonAsyncTask-30] o.o.nfvo.vnfm_reg.state.VnfStateHandler  : Sent INSTANTIATE to VNF: MongoDB
2018-03-28 08:44:30.954 ERROR 911 --- [OpenBatonAsyncTask-15] o.o.n.v.tasks.abstracts.AbstractTask     : ERROR from VNFM: Unable to allocate Resources
2018-03-28 08:44:30.956 ERROR 911 --- [OpenBatonAsyncTask-15] o.o.n.v.tasks.abstracts.AbstractTask     : Received ERROR message from VNFM related to VNFR: MongoDB
2018-03-28 08:44:30.960 ERROR 911 --- [OpenBatonAsyncTask-15] o.o.n.v.tasks.abstracts.AbstractTask     : There was an uncaught exception. Message is: Object of class [org.openbaton.catalogue
.mano.record.VirtualNetworkFunctionRecord] with identifier [79675b3d-7437-43be-bd8e-be7a2ce4e8fb]: optimistic locking failed; nested exception is org.hibernate.StaleObjectStateException: Row
 was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [org.openbaton.catalogue.mano.record.VirtualNetworkFunctionRecord#79675b3d-7437-43be-bd8e-be7a2ce4e8
fb]

no log in the plugin directory for docker (only for plugin-openstack and plugin-test)

dockerVNFM

08:44:11 [DEBU] docker-vnf -> handleNfvM ▶  Received Message RELEASE_RESOURCES
08:44:11 [NOTI] docker-vnf -> (*VnfmImpl ▶  Remove container for vnfr: MongoDB
08:44:11 [DEBU] docker-vnf ->  getConfig ▶  Getting config with id: 46d50827-43fa-4915-bd59-6f17dacb10c9
08:44:11 [ERRO] docker-vnf -> (*VnfmImpl ▶  Error while getting config: EOF
08:44:11 [ERRO] docker-vnf -> (*VnfmImpl ▶  Probably not found
08:44:30 [DEBU] docker-vnf -> handleNfvM ▶  Received Message INSTANTIATE
08:44:30 [DEBU] docker-vnf -> (*worker). ▶  received extensions: map[brokerPort:5672 monitoringIp: nsr-id:9df2b168-2897-47a0-8355-4dccc7030770 brokerIp:172.17.0.1 timezone:CET]
08:44:30 [DEBU] docker-vnf -> (*worker). ▶  received keys: []
08:44:30 [INFO] docker-vnf ->        Rpc ▶  Executing RPC to queue: vnfm.nfvo.actions.reply
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Getting Channel for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Got Channel for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Declaring Queue for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Declared Queue for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Registering consumer for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Registered consumer for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Publishing message to queue vnfm.nfvo.actions.reply
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Published message to queue vnfm.nfvo.actions.reply
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Received Response
08:44:30 [DEBU] docker-vnf -> (*worker). ▶  Received VNFR after GRANT_OPERATION
08:44:30 [DEBU] docker-vnf -> (*worker). ▶  allocating resources for the VNFR
08:44:30 [DEBU] docker-vnf -> (*worker). ▶  will send to NFVO UserData
08:44:30 [INFO] docker-vnf ->        Rpc ▶  Executing RPC to queue: vnfm.nfvo.actions.reply
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Getting Channel for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Got Channel for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Declaring Queue for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Declared Queue for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Registering consumer for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Registered consumer for RPC
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Publishing message to queue vnfm.nfvo.actions.reply
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Published message to queue vnfm.nfvo.actions.reply
08:44:30 [DEBU] docker-vnf -> handleNfvM ▶  Received Message ERROR
08:44:30 [ERRO] docker-vnf -> (*worker). ▶  received an error from the NFVO
08:44:30 [ERRO] docker-vnf -> (*VnfmImpl ▶  Recevied Error for vnfr: MongoDB
08:44:30 [DEBU] docker-vnf ->        Rpc ▶  Received Response
08:44:30 [ERRO] docker-vnf -> (*worker). ▶  exchange error
08:44:30 [ERRO] docker-vnf -> handleMess ▶  &{Unable to allocate Resources 0xc420267d40 9df2b168-2897-47a0-8355-4dccc7030770}
08:44:30 [DEBU] docker-vnf -> handleNfvM ▶  Received Message ERROR
08:44:30 [ERRO] docker-vnf -> (*worker). ▶  received an error from the NFVO
08:44:30 [ERRO] docker-vnf -> (*VnfmImpl ▶  Recevied Error for vnfr: MongoDB

docker driver

08:44:30 [INFO] docker-dri -> PluginImpl ▶  Listed 20 images
08:44:30 [INFO] docker-dri -> PluginImpl ▶  Listed 5 networks
08:44:30 [DEBU] docker-dri -> PluginImpl ▶  Networks are [host bridge none ob_default new-network_4612]
08:44:30 [DEBU] docker-dri -> PluginImpl ▶  Received DockerNetwork &{BaseNetwork:{ID: HbVersion:0 ProjectID: Shared:false Metadata:map[] Name:new-network ExtID:} Scope: Driver: Gateway: Subnet:}
08:44:30 [DEBU] docker-dri -> PluginImpl ▶  Creating network [new-network_7168] with config {false bridge  false <nil> false false false false <nil> map[] map[]}
08:44:30 [INFO] docker-dri -> PluginImpl ▶  Created ob network [new-network_7168] with ext id [e8750ba5df89581cc41bb2da270bb25991d286755b8af68e14d9adf9079eba33]
09:32:22 [ERRO] docker-dri -> PluginImpl ▶  Error while retrieving the image by id
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x757ee6]

goroutine 125 [running]:
github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-docker-driver/handler.PluginImpl.ListServer(0xc42016fb00, 0x897ea0, 0xc420024018, 0xc4203236e0, 0x0, 0x8581b6, 0x2a, 0x793120, 0xc4203f62a0, 0x0, ...)
	/go/src/github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-docker-driver/handler/handler.go:487 +0x806
github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-openbaton/pluginsdk.(HandlerVim).ListServer-fm(0x793120, 0xc4203f62a0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-openbaton/pluginsdk/worker.go:208 +0x46
reflect.Value.call(0x7bcb60, 0xc42011ece0, 0x13, 0x8496ae, 0x4, 0xc42011ed00, 0x1, 0x1, 0x0, 0x793120, ...)
	/usr/local/go/src/reflect/value.go:447 +0x969
reflect.Value.Call(0x7bcb60, 0xc42011ece0, 0x13, 0xc42011ed00, 0x1, 0x1, 0x0, 0xc420432000, 0xa)
	/usr/local/go/src/reflect/value.go:308 +0xa4
github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-openbaton/pluginsdk.worker.handle(0xc42016fb00, 0x89bd60, 0xc4202cc680, 0x7edec0, 0xc4203023f0, 0x812d80, 0xc4202e4820, 0xc420432000, 0xa, 0xc4204465a0, ...)
	/go/src/github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-openbaton/pluginsdk/worker.go:86 +0xc2e
github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-openbaton/pluginsdk.handlePluginRequest(0xc420392000, 0x265b, 0x2800, 0x83a2a0, 0xc4202cc680, 0x0, 0xc42034c140, 0x812d80, 0xc4202e4820, 0x7edec0, ...)
	/go/src/github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-openbaton/pluginsdk/handler.go:28 +0x2f9
github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-openbaton/sdk.(*Manager).Serve.func1.1(0xc420320200, 0xc4200be640)
	/go/src/github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-openbaton/sdk/commonsdk.go:311 +0xb6
created by github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-openbaton/sdk.(*Manager).Serve.func1
	/go/src/github.com/golang/openbaton/go-docker-driver/vendor/github.com/openbaton/go-openbaton/sdk/commonsdk.go:310 +0x10a

All my docker images are listed and I can run manually the mongo:lastest docker

I use those command to launch my environement :

docker run --name openbaton -d -p 8080:8080 -p 5672:5672 -p 15672:15672 -p 8443:8443 -e RABBITMQ_BROKERIP=172.17.0.1 openbaton/standalone

docker run -v /var/run/docker.sock:/var/run/docker.sock -v /mypath/Openbaton:/root/ openbaton/vnfm-docker-go  -conf /root/config.toml

docker run -v /var/run/docker.sock:/var/run/docker.sock -v /mypath/Openbaton:/root/ openbaton/driver-docker-go  -conf /root/config.toml

with this config.toml

type        = "docker"
workers     = 10
username    = "openbaton-manager-user"
password    = "openbaton"
logLevel    = "DEBUG"
brokerIp    = "172.17.0.1"
brokerPort  = 5672

(I needed to increase the "workers" value to avoid having an error "JSON malformated" )

The vnfpakage files are the same I just change the image upload parameter to "false"

I created the NSD using the webui and just by selecting mongodb from the VNFDs Catalogue menu

Mar 29 '18 09:03 JohanPy

There is a nil pointer in the docker driver so this issue should be moved to the docker driver repo. i suppose from the logs, the problem is while retrieving the image id from the tag defined in the vnfd.json. Most probably because was never pulled in the docker engine. Although the pull should be done while uploading the package, please try to pull manually the mongodb image and try again, but this time using the right docker images. You do not need the full standalone openbaton, but the nfvo, vnfm and plugin.

you can use this compose file:

version: '3'
services:
  nfvo:
    image: openbaton/nfvo:latest
    depends_on:
      - rabbitmq_broker
      - nfvo_database
    restart: always
    environment:
      - NFVO_RABBIT_BROKERIP=rabbitmq_broker
      - NFVO_PLUGIN_INSTALLATION-DIR=/dev/null
      - NFVO_VIM_ACTIVE_CHECK=false
      - SPRING_RABBITMQ_HOST=rabbitmq_broker
      - SPRING_DATASOURCE_URL=jdbc:mysql://nfvo_database:3306/openbaton
      - SPRING_DATASOURCE_DRIVER-CLASS-NAME=org.mariadb.jdbc.Driver
      - SPRING_JPA_DATABASE-PLATFORM=org.hibernate.dialect.MySQLDialect
    ports:
      - "8080:8080"
  vnfm-docker:
    image: openbaton/vnfm-docker-go:latest
    depends_on:
      - nfvo
    restart: always
    command: -ip rabbitmq_broker
    volumes:
        - /var/run/docker.sock:/var/run/docker.sock:ro
  driver-docker:
    image: openbaton/driver-docker-go:latest
    depends_on:
      - nfvo
    restart: always
    command: -ip rabbitmq_broker
    volumes:
        - /var/run/docker.sock:/var/run/docker.sock:ro
  nfvo_database:
    image: mariadb
    environment:
      - MYSQL_RANDOM_ROOT_PASSWORD=true
      - MYSQL_DATABASE=openbaton
      - MYSQL_USER=admin
      - MYSQL_PASSWORD=changeme
  rabbitmq_broker:
    image: rabbitmq:3-management-alpine
    hostname: openbaton-rabbitmq
    environment:
      - RABBITMQ_DEFAULT_USER=admin
      - RABBITMQ_DEFAULT_PASS=openbaton
    ports:
      - "5672:5672"
      - "15672:15672"

the workers are completely unrelated with the json format issue you hit. would be good to have a way to replicate it

Mar 29 '18 10:03 lorenzotomasini

Thank you for your help !

I had already pulled the lastest mongo docker before starting OpenBaton and the image is in the list with the same tag ( ["mongo:latest"] ) as the config file :

image:
    upload: "false"
    names:
        - "mongo:latest"
    link: "mongo:latest"
image-config:
    name: "mongo:latest"

Furthermore with this compose file it works perfectly (thank you again) and I use the same pop.json and the same .jar for my VNF package (but I dind't use my config.toml for the driver and the vnfm).

My problem is sloved, but if you think there is a real bug and it's not just a misconfiguration in the dockers I used to run OpenBaton I can pursue the investigation.

I will try to reproduce the misconfiguration json and open an issue if I can do it.

Mar 29 '18 13:03 JohanPy

Hello,

I am following the same tutorial as you @JohanPy that is docker-tutorial .

I have already installed:

Openbaton (as a docker container) link here
Docker VIM driver link here
Docker VNFManager for OpenBaton link here With this installed and launched I can access to openbaton web and list my docker images, networks....

My problem is that when I try to launch a NSD, this mongo db for example, the VNFM driver sais [ERRO] docker-vnf -> handleMess ▶ ERROR: &{**Image with name or id** [mongo:latest] **not found** 0xc4200c78c0 88263e7d-eacd-4226-8cb9-0225e9aa1fd3}

Why not found? In the Point of Presence information it appears (see screen shot) Do you have any idea about what could be happening? @JohanPy @lorenzotomasini @raj2569 @mcilloni

Thanks in advance. Regards. screenshot from 2018-04-25 14-24-50_2

Apr 25 '18 12:04 prom450

Hi @prom450 , Could you solve your problem ?I have a similar issue like yours. I created the issue here. I want to to run Iperf NSD tutorial in two different docker containers as in this tutorial. But, the VNFM says [ERRO] docker-vnf -> handleMess ▶ ERROR: &{Image with name or id [iperfserver:latest] not found 0xc4200cf200 d0f4c0cc-ad9a-4a8e-81c5-630590db75a2} although In the Point of Presence information the image appears. Thanks !

Nov 15 '19 20:11 mehdishar