Issues in launching Iperf NSD docker tutorial, Errors: invalid CIDR address, and Image with name or id [iperfserver:latest] not found #312
Description
I installed Openbaton on Linux Ubuntu 16.04 using the bootstrap procedure here. I start NFVO service on each startup of my Linux (by adding the line "sudo systemctl start openbaton-nfvo.service" to /etc/rc.local). I can then access Openbaton dashboard GUI by entering http://localhost:8080 in my browser (it redirects me to https://localhost:8443/#/main).
My goal is to run Iperf NSD tutorial in two different docker containers. So, I built client and server docker files as in the tutorial. When I run "$ sudo docker images" command, I see client and server images in the output:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
REPOSITORY TAG IMAGE ID CREATED SIZE
iperfserver latest 1af1f8b733e5 45 hours ago 126MB
iperfclient latest ed9268f85309 45 hours ago 126MB
openbaton/vnfm-docker-go latest b0844aa8564c 7 months ago 21MB
openbaton/driver-docker-go latest 15e4eed916a5 11 months ago 15.3MB
networkstatic/iperf3 latest 6ea158fee1a7 3 years ago 126MB
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
As a requirement, for Docker VM driver, I ran it according to this doc; Specifically, I ran "$ sudo ./docker-driver -conf config.toml" with the following config :
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
type = "docker"
workers = 5
username = "openbaton-manager-user"
password = "openbaton"
logLevel = "DEBUG"
brokerIp = "localhost"
brokerPort = 5672
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The driver run-time shows this :
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
12:45:03 [INFO] docker-dri -> startWithC ▶ Starting Plugin of type docker
12:45:03 [DEBU] docker-dri -> startWithC ▶ Config are {
"Type": "docker",
"Workers": 5,
"Username": "openbaton-manager-user",
"Password": "openbaton",
"LogLevel": "DEBUG",
"BrokerIp": "localhost",
"BrokerPort": 5672
}
12:45:03 [DEBU] docker-dri -> getCreds ▶ Dialing amqp://openbaton-manager-user:openbaton@localhost:5672/
12:45:03 [DEBU] docker-dri -> setupManag ▶ dialing amqp://vim-drivers.docker.docker:TZ7QY2TBcHb17mmg@localhost:5672/
12:45:03 [DEBU] docker-dri -> setupManag ▶ got Connection, getting Channel
12:45:03 [DEBU] docker-dri -> setupManag ▶ got Channel, declaring Exchange ("openbaton-exchange")
12:45:03 [DEBU] docker-dri -> setupManag ▶ declared Exchange, declaring Queue "vim-drivers.docker.docker"
12:45:03 [DEBU] docker-dri -> setupManag ▶ declared Queue ("vim-drivers.docker.docker", 0 messages, 0 consumers), binding to Exchange
12:45:03 [DEBU] docker-dri -> setupManag ▶ Queue bound to Exchange, starting Consume
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
As another requirement, I downloaded and ran Docker VNFM according to this doc. This is my config :
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
type = "docker"
endpoint = "docker"
allocate = true
workers = 1
description = "The Vnfm description"
username = "openbaton-manager-user"
password = "openbaton"
logLevel = "INFO"
brokerIp = "localhost"
brokerPort = 5672
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
After I run "$ sudo ./docker-vnfm -conf config.toml ", this is the run-time output:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
12:47:55 [INFO] docker-vnf -> startWithC ▶ Starting VNFM of type docker
12:47:55 [DEBU] docker-vnf -> startWithC ▶ Config are {
"Type": "docker",
"Endpoint": "docker",
"Description": "The Vnfm description",
"Workers": 1,
"Username": "openbaton-manager-user",
"Password": "openbaton",
"Allocate": true,
"LogLevel": "INFO",
"BrokerIp": "localhost",
"BrokerPort": 5672
}
12:47:55 [DEBU] docker-vnf -> getCreds ▶ Dialing amqp://openbaton-manager-user:openbaton@localhost:5672/
12:47:55 [DEBU] docker-vnf -> setupManag ▶ dialing amqp://docker:LNsB0RaZOoAAeyzj@localhost:5672/
12:47:55 [DEBU] docker-vnf -> setupManag ▶ got Connection, getting Channel
12:47:55 [DEBU] docker-vnf -> setupManag ▶ got Channel, declaring Exchange ("openbaton-exchange")
12:47:55 [DEBU] docker-vnf -> setupManag ▶ declared Exchange, declaring Queue "docker"
12:47:55 [DEBU] docker-vnf -> setupManag ▶ declared Queue ("docker", 0 messages, 0 consumers), binding to Exchange
12:47:55 [DEBU] docker-vnf -> setupManag ▶ Queue bound to Exchange, starting Consume
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
In Openbaton dashboard => Catalogue => VNF Managers, I see the following list:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
_Id Type Active Enabled Endpoint Type Endpoint
121c3c14-d2ff-47ef-9d6b-8856edf41415 docker true true RABBIT docker
87bca443-abfc-487c-a7d3-7c53b4494544 generic false true RABBIT generic_
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
In the dashboard Installed drivers page, I see this list:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Name Type
docker docker
test test
openstack openstack
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Then, I registered the Docker Engine as a PoP via this setup:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
{
"name": "vim-instance",
"authUrl": "unix:///var/run/docker.sock",
"tenant": "1.32",
"username": "admin",
"password": "openbaton",
"type": "docker",
"location": {
"name": "Berlin",
"latitude": "52.525876",
"longitude": "13.314400"
}
}
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Then, I tried to launch the following NSD json from the tutorial :
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
{
"name": "iperf",
"vendor": "FOKUS",
"version": "docker",
"vld": [{
"name": "new-network"
}],
"vnfd": [
{
"name":"iperf-server",
"vendor":"FOKUS",
"version":"docker",
"deployment_flavour":[
{
"flavour_key":"m1.small"
}
],
"vdu":[
{
"vm_image":["iperfserver:latest"],
"scale_in_out":3,
"vnfc":[
{
"connection_point":[
{
"virtual_link_reference":"new-network"
}
]
}
]
}
],
"virtual_link":[
{
"name":"new-network"
}
],
"type":"server",
"endpoint":"docker"
},
{
"name":"iperf-client",
"vendor":"FOKUS",
"version":"docker",
"vdu":[
{
"vm_image":["iperfclient:latest"],
"scale_in_out":3,
"vnfc":[
{
"connection_point":[
{
"virtual_link_reference":"new-network"
}
]
}
]
}
],
"deployment_flavour":[
{
"flavour_key":"m1.small"
}
],
"virtual_link":[
{
"name":"new-network"
}
],
"requires": {
"iperf-server": {
"parameters":["hostname"]
}
},
"type":"client",
"endpoint":"docker"
}
]
}
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
However, in NS Records page, I observe the following errors (invalid CIDR address):
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
VNF Records
Id VNFR Name Last Executed Action VNFR Type Vendor Version Status Actions
82ac65f2-e4b1-4430-a728-e407cbf0aa04 iperf-server ERROR:Not created Network with name: new-network successfully on VimInstance vim-instance. Caused by: org.openbaton.exceptions.VimDriverException: invalid CIDR address: server FOKUS docker NULL
b979a1b4-c6ee-4b7e-bdba-ec07702da3d8 iperf-client ERROR:Not created Network with name: new-network successfully on VimInstance vim-instance. Caused by: org.openbaton.exceptions.VimDriverException: invalid CIDR address: client FOKUS docker NULL
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Then, I tried to use another virtual link to replace "new-network". Looking at my Vim-instance in PoP information page, I see the following list of Networks:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Name ExtId Shared Subnet
bridge 24235e0406c9f4e4591dd6ebc9819658aad15435706ec4ab839263d1c3e24b49 false 172.17.0.0/16
host 6a39c734675b1c14a56fdbefd714877c660c771ea233a61ec4bd5228bb6ec04a false
none 60c42ef4035e399533abc62f324f33e3067bd015c69904dc7f38bcd5910e89d1 false
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
As a trial, I replaced "new-network" with "host" in the NSD json and launched it again. However, I see the state in NS records in NULL. Clicking on one of VNF records there, I see:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Name iperf-server
Status NULL
Current Task grantoperation
ID 31d46f80-d091-4586-b8bb-bc199ff476fd
Updated 2019.11.14 at 13:09:56 EST
Type server
Version docker
Endpoint docker
Cyclic Dependency false
Provides {"configurationParameters":[],"name":"provides","id":"f97a65dc-343f-4b50-bd84-e582e06171a0","hbVersion":3,"projectId":"","shared":false,"metadata":{}}
Flavour m1.small
Parent NSR d0f4c0cc-ad9a-4a8e-81c5-630590db75a2
Project ID c0871523-8395-4a1e-b124-90945e53c809
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Here is the log from docker-vnfm output:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
13:09:56 [DEBU] docker-vnf -> handleNfvM ▶ Received Message INSTANTIATE
13:09:56 [DEBU] docker-vnf -> (*worker). ▶ received extensions: map[brokerPort:5672 monitoringIp: nsr-id:d0f4c0cc-ad9a-4a8e-81c5-630590db75a2 brokerIp:localhost timezone:CET]
13:09:56 [DEBU] docker-vnf -> (*worker). ▶ received keys: []
13:09:56 [DEBU] docker-vnf -> ExecuteRpc ▶ Declaring Queue for RPC tentative number: 0
13:09:56 [DEBU] docker-vnf -> ExecuteRpc ▶ Registering consume for RPC tentative number: 0
13:09:56 [DEBU] docker-vnf -> (*worker). ▶ received VNFR
13:09:56 [DEBU] docker-vnf -> (*HandlerV ▶ iperf-server VNF has 1 VNFC(s)
13:09:56 [DEBU] docker-vnf -> GetCPsAndI ▶ iperf-server: Fixed Ip is:
13:09:56 [DEBU] docker-vnf -> GetCPsAndI ▶ Adding New Connection Point: &{ 0 false map[] docker host random 0}
goroutine 59 [running]:
runtime/debug.Stack(0xc, 0x100, 0xc42569e8c0)
/usr/local/go/src/runtime/debug/stack.go:24 0xa7
runtime/debug.PrintStack()
/usr/local/go/src/runtime/debug/stack.go:16 0x22
github.com/openbaton/go-docker-vnfm/handler.(*HandlerVnfmImpl).Instantiate(0xc420251ba0, 0xc4200cf200, 0x0, 0x0, 0xc420082f30, 0x0, 0xc4201926f0, 0x0)
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/handler/handler.go:84 0xc17
github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk.(*worker).handleInstantiate(0xc420192870, 0xc420198070, 0xb, 0xc420278120, 0xc5ec00)
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk/worker.go:134 0x98f
github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk.handleMessage(0xc227a0, 0xc420082630, 0xc420192870, 0xc4201942b0, 0x1, 0x1)
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk/handler.go:72 0x79b
github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk.handleNfvMessage(0xc420328000, 0x1cf3, 0x2000, 0x94ba60, 0xc420192870, 0xc42008a370, 0x5, 0x2, 0x0, 0x0)
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk/handler.go:26 0x465
github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/sdk.(*VnfmManager).Serve.func1(0xc4201ce230, 0x94ba60, 0xc420192870)
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/sdk/commonsdk.go:320 0x3dd
created by github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/sdk.(*VnfmManager).Serve
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/sdk/commonsdk.go:301 0x7e
13:09:56 [ERRO] docker-vnf -> handleMess ▶ ERROR: &{Image with name or id [iperfserver:latest] not found 0xc4200cf200 d0f4c0cc-ad9a-4a8e-81c5-630590db75a2}
13:09:56 [DEBU] docker-vnf -> handleNfvM ▶ Received Message INSTANTIATE
13:09:56 [DEBU] docker-vnf -> (*worker). ▶ received extensions: map[timezone:CET brokerPort:5672 monitoringIp: nsr-id:d0f4c0cc-ad9a-4a8e-81c5-630590db75a2 brokerIp:localhost]
13:09:56 [DEBU] docker-vnf -> (*worker). ▶ received keys: []
13:09:56 [DEBU] docker-vnf -> ExecuteRpc ▶ Declaring Queue for RPC tentative number: 0
13:09:56 [DEBU] docker-vnf -> ExecuteRpc ▶ Registering consume for RPC tentative number: 0
13:09:56 [DEBU] docker-vnf -> (*worker). ▶ received VNFR
13:09:56 [DEBU] docker-vnf -> (*HandlerV ▶ iperf-client VNF has 1 VNFC(s)
13:09:56 [DEBU] docker-vnf -> GetCPsAndI ▶ iperf-client: Fixed Ip is:
13:09:56 [DEBU] docker-vnf -> GetCPsAndI ▶ Adding New Connection Point: &{ 0 false map[] docker host random 0}
goroutine 59 [running]:
runtime/debug.Stack(0xc, 0x80, 0xc42569e974)
/usr/local/go/src/runtime/debug/stack.go:24 0xa7
runtime/debug.PrintStack()
/usr/local/go/src/runtime/debug/stack.go:16 0x22
github.com/openbaton/go-docker-vnfm/handler.(*HandlerVnfmImpl).Instantiate(0xc420251ba0, 0xc4200ceb40, 0x0, 0x0, 0xc420193170, 0x0, 0xc420010c00, 0x0)
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/handler/handler.go:84 0xc17
github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk.(*worker).handleInstantiate(0xc420192870, 0xc420198a10, 0xb, 0xc420278e10, 0xc5ec00)
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk/worker.go:134 0x98f
github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk.handleMessage(0xc227a0, 0xc420192e70, 0xc420192870, 0xc420194910, 0x1, 0x1)
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk/handler.go:72 0x79b
github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk.handleNfvMessage(0xc420046000, 0x1dce, 0x2000, 0x94ba60, 0xc420192870, 0xc4201e7060, 0x5, 0x2, 0x0, 0x0)
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/vnfmsdk/handler.go:26 0x465
github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/sdk.(*VnfmManager).Serve.func1(0xc4201ce230, 0x94ba60, 0xc420192870)
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/sdk/commonsdk.go:320 0x3dd
created by github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/sdk.(*VnfmManager).Serve
/home/ubuntu/go/src/github.com/openbaton/go-docker-vnfm/vendor/github.com/openbaton/go-openbaton/sdk/commonsdk.go:301 0x7e
13:09:56 [ERRO] docker-vnf -> handleMess ▶ ERROR: &{Image with name or id [iperfclient:latest] not found 0xc4200ceb40 d0f4c0cc-ad9a-4a8e-81c5-630590db75a2}
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
As you see in the log, it complains about image name or id not found. But according to list of docker images, they exist!
And this is docker driver log:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
13:09:56 [DEBU] docker-dri -> handlePlug ▶ Returning {"answer":{"id":"06f8ad32-d406-4f9e-839b-52d8d5f39ffb","hbVersion":3,"projectId":"c0871523-8395-4a1e-b124-90945e53c809","name":"vim-instance","authUrl":"unix:///var/run/docker.sock","active":true,"location":{"id":"f2ac8092-3c40-4d85-a95b-ade8759fd77a","hbVersion":2,"projectId":"","name":"Berlin","latitude":"52.525876","longitude":"13.314400"},"type":"docker","ca":"","cert":"","dockerKey":"","images":[{"projectId":"","extId":"sha256:1af1f8b733e5a152c34d0ef76373c984181a191f6dac3fa72b6228bcdb5b8b62","tags":["iperfserver:latest"]},{"projectId":"","extId":"sha256:ed9268f85309f2e120d654b655bf6c7ec7126deb15260300b6245aa1f956f988","tags":["iperfclient:latest"]},{"projectId":"","extId":"sha256:b0844aa8564c2ee4bfe650b47d29643fa63193ab8b9e29696f7181e41cc96bcf","tags":["openbaton/vnfm-docker-go:latest"]},{"projectId":"","extId":"sha256:15e4eed916a5f2fa09bfd26920201f2bce991f8ea2edc55e2287082dc47c001a","tags":["openbaton/driver-docker-go:latest"]},{"projectId":"","extId":"sha256:6ea158fee1a75f82ccf4d9fed2165883ec91fb094a9ea87b5257691248fca58d","tags":["networkstatic/iperf3:latest"]}],"networks":[{"projectId":"","name":"none","extId":"60c42ef4035e399533abc62f324f33e3067bd015c69904dc7f38bcd5910e89d1","scope":"local","driver":"null","gateway":"","subnet":""},{"projectId":"","name":"host","extId":"6a39c734675b1c14a56fdbefd714877c660c771ea233a61ec4bd5228bb6ec04a","scope":"local","driver":"host","gateway":"","subnet":""},{"projectId":"","name":"bridge","extId":"24235e0406c9f4e4591dd6ebc9819658aad15435706ec4ab839263d1c3e24b49","scope":"local","driver":"bridge","gateway":"172.17.0.1","subnet":"172.17.0.0/16"}]}}
13:09:56 [DEBU] docker-dri -> handlePlug ▶ Returning {"answer":{"id":"06f8ad32-d406-4f9e-839b-52d8d5f39ffb","hbVersion":4,"projectId":"c0871523-8395-4a1e-b124-90945e53c809","name":"vim-instance","authUrl":"unix:///var/run/docker.sock","active":true,"location":{"id":"f2ac8092-3c40-4d85-a95b-ade8759fd77a","hbVersion":3,"projectId":"","name":"Berlin","latitude":"52.525876","longitude":"13.314400"},"type":"docker","ca":"","cert":"","dockerKey":"","images":[{"projectId":"","extId":"sha256:1af1f8b733e5a152c34d0ef76373c984181a191f6dac3fa72b6228bcdb5b8b62","tags":["iperfserver:latest"]},{"projectId":"","extId":"sha256:ed9268f85309f2e120d654b655bf6c7ec7126deb15260300b6245aa1f956f988","tags":["iperfclient:latest"]},{"projectId":"","extId":"sha256:b0844aa8564c2ee4bfe650b47d29643fa63193ab8b9e29696f7181e41cc96bcf","tags":["openbaton/vnfm-docker-go:latest"]},{"projectId":"","extId":"sha256:15e4eed916a5f2fa09bfd26920201f2bce991f8ea2edc55e2287082dc47c001a","tags":["openbaton/driver-docker-go:latest"]},{"projectId":"","extId":"sha256:6ea158fee1a75f82ccf4d9fed2165883ec91fb094a9ea87b5257691248fca58d","tags":["networkstatic/iperf3:latest"]}],"networks":[{"projectId":"","name":"none","extId":"60c42ef4035e399533abc62f324f33e3067bd015c69904dc7f38bcd5910e89d1","scope":"local","driver":"null","gateway":"","subnet":""},{"projectId":"","name":"host","extId":"6a39c734675b1c14a56fdbefd714877c660c771ea233a61ec4bd5228bb6ec04a","scope":"local","driver":"host","gateway":"","subnet":""},{"projectId":"","name":"bridge","extId":"24235e0406c9f4e4591dd6ebc9819658aad15435706ec4ab839263d1c3e24b49","scope":"local","driver":"bridge","gateway":"172.17.0.1","subnet":"172.17.0.0/16"}]}}
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Can you please help me to find the cause of the issue and how to resolve it ? and what is your idea about the connection point or virtual links should I set in my NSD json for this iperf docker tutorial ? Any kind of help is appreciated.
Thank you in advance,
Mehdi