dnsname icon indicating copy to clipboard operation
dnsname copied to clipboard

containers do not start after reboot while dnsname enabled

Open KlausThornProgrammfabrik opened this issue 4 years ago • 7 comments

I installed dnsname and tested it successfully: A PostgreSQL container "pgsql" is found by the other container "pgclient" - just by using the container name, cool! But after reboot I get lots of problems while starting the containers again. (Problems which I do NOT get without dnsname in /etc/cni/net.d/mynetwork.conflist ).

Errors 1:

reboot
podman ps
ERRO[0001] Error deleting network: stat /run/containers/cni/dnsname/mynetwork: no such file or directory
ERRO[0001] Error while removing pod from CNI network "mynetwork": stat /run/containers/cni/dnsname/mynetwork: no such file or directory
ERRO[0001] Error refreshing container 3dc5021379d9feac756e1c73daf87e94a53ee918447262f9160b270c375a6cd7: stat /run/containers/cni/dnsname/mynetwork: no such file or directory
ERRO[0001] Error deleting network: stat /run/containers/cni/dnsname/mynetwork: no such file or directory
ERRO[0001] Error while removing pod from CNI network "mynetwork": stat /run/containers/cni/dnsname/mynetwork: no such file or directory
ERRO[0001] Error refreshing container 8e034a3a3b503137e1f04aee29641e65627e53f2c8884aa404c9fcd57d017819: stat /run/containers/cni/dnsname/mynetwork: no such file or directory
CONTAINER ID  IMAGE  COMMAND  CREATED  STATUS  PORTS  NAMES
podman ps # just doing it a second time after the reboot
# no error any more

Errors 2:

podman start pgsql
ERRO[0000] Error adding network: failed to allocate for range 0: 10.89.0.24 has been allocated to 8e034a3a3b503137e1f04aee29641e65627e53f2c8884aa404c9fcd57d017819, duplicate allocation is not allowed
ERRO[0000] Error while adding pod to CNI network "mynetwork": failed to allocate for range 0: 10.89.0.24 has been allocated to 8e034a3a3b503137e1f04aee29641e65627e53f2c8884aa404c9fcd57d017819, duplicate allocation is not allowed
Error: unable to start container "pgsql": error configuring network namespace for container 8e034a3a3b503137e1f04aee29641e65627e53f2c8884aa404c9fcd57d017819: failed to allocate for range 0: 10.89.0.24 has been allocated to 8e034a3a3b503137e1f04aee29641e65627e53f2c8884aa404c9fcd57d017819, duplicate allocation is not allowed

Instead of just starting the container pgsql I have to remove it, recreate it, and then start it. During creation I get ...

Errors 3:

podman rm pgclient pgsql
podman run -d -ti \
    --name pgsql \
    --net mynetwork \
    --volume=/srv/myapp/config:/config:z \
    --volume=/srv/myapp/pgsql/etc:/etc/postgresql:Z \
    --volume=/srv/myapp/pgsql/log:/var/log/postgresql:Z \
    --volume=/srv/myapp/pgsql/var:/var/lib/postgresql:Z \
    --volume=/srv/myapp/pgsql/backup:/backup:Z \
    --restart=always \
    docker.easydb.de/pf/postgresql-11
ERRO[0000] Error adding network: failed to set bridge addr: could not set bridge's mac: invalid argument
ERRO[0000] Error while adding pod to CNI network "mynetwork": failed to set bridge addr: could not set bridge's mac: invalid argument
Error: error configuring network namespace for container c59bc99cc9f0de9b7d6f963c8a43ed1d052d8fdae90b5c91269a1a3303cb3359: failed to set bridge addr: could not set bridge's mac: invalid argument
podman start pgsql
# works then, finally

So the problems are not only lots of errors but foremost that I have to recreate the containers to get them started at all.

Environment:

  • running podman as root
  • RHEL 8.1

Installation of dnsname:

dnf install containernetworking-plugins dnsmasq
yum module install go-toolset
dnf group install "Development Tools"
cd /usr/src
git clone https://github.com/containers/dnsname
cd dnsname
make all
make install PREFIX=/usr
vi /etc/cni/net.d/mynetwork.conflist # adding the next block...
{
         "type": "dnsname",
         "domainName": "localdomain"
      },
vi /etc/containers/libpod.conf # was empty, so I put in the next block...
cni_default_network = "mynetwork"

Are these containers using static IPs, by chance?

mheon avatar Apr 28 '20 19:04 mheon

Thanks for the fast reply!

They do not use --ip= . Is that what you are referring to?

Network:

podman network ls
NAME             VERSION   PLUGINS
podman           0.4.0     bridge,portmap,firewall,tuning
mynetwork        0.4.0     bridge,portmap,dnsname,firewall
cat /etc/cni/net.d/mynetwork.conflist
{
   "cniVersion": "0.4.0",
   "name": "mynetwork",
   "plugins": [
      {
         "type": "bridge",
         "bridge": "cni-podman1",
         "isGateway": true,
         "ipMasq": true,
         "ipam": {
            "type": "host-local",
            "routes": [
               {
                  "dst": "0.0.0.0/0"
               }
            ],
            "ranges": [
               [
                  {
                     "subnet": "10.89.0.0/24",
                     "gateway": "10.89.0.1"
                  }
               ]
            ]
         }
      },
      {
         "type": "portmap",
         "capabilities": {
            "portMappings": true
         }
      },
      {
         "type": "firewall",
         "backend": "firewalld"
      }
   ]
}

... I also changed firewall-backend to firewalld.

Versions:

cd /usr/src/dnsname 
git log -1
commit 2e1d2e0732df49df460e0991f68bf2ccfcd102db (HEAD -> master, origin/master, origin/HEAD)
Date:   Thu Feb 27 16:24:42 2020 -0600
yum list podman
Installed Packages
podman.x86_64             1.6.4-4.module+el8.1.1+5885+44006e55
yum list containernetworking-plugins
Installed Packages
containernetworking-plugins.x86_64   0.8.3-5.module+el8.2.0+5201+6b31f0d9

Alright, does not sound like anything I'm familiar with, then.

@baude Mind taking a look?

mheon avatar Apr 28 '20 19:04 mheon

this should be reported as a bugzilla. please do so and let us know the number

baude avatar Apr 28 '20 20:04 baude

this should be reported as a bugzilla. please do so and let us know the number

Bugzilla is some fine software. But which instance of bugzilla are you referring to? Do you have a URL for me?

Not that I am eager to post all that a 2nd time, but if that is what it takes, fine.

bugzilla.redhat.com

rhatdan avatar Apr 28 '20 20:04 rhatdan

The Error 2 may be that you have enabled both firewall and iptable at the same time. disable firewall maybe it works

LY1806620741 avatar May 11 '20 16:05 LY1806620741