关于容器网络目前有两种标准：

  • CNM
  • CNI

Container Network Model(CNM)由Docker公司的提出容器网络规范。本人目前对它原理了解不多，有空再详细研究一下。

Container Network Interface(CNI)由CoreOS提出的一个容器网络规范。关于CNI的介绍可以看一下我另外一篇文章：深入理解CNI

关于CNM和CNI的对比网卡已经有很多资料，不是本文的关注重点，本文的关注点是：

既然Docker已经有了自己的CNM，那么还能在Docker中使用CNI吗？

答案当然是能的，kubernetes不是跑着好好的。

下面我们就来演示一下如何使用CNI给Docker container添加网口并分配ip。

环境准备

系统：Centos7

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

[root@xnile01 ~]# cat /etc/redhat-release
CentOS Linux release 7.3.1611 (Core)
[root@xnile01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:a7:9f:a0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.94/24 brd 192.168.122.255 scope global eth0
       valid_lft forever preferred_lft forever

Docker：18.09.9

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

[root@xnile01 cni]# docker version
Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.39 (downgraded from 1.40)
 Go version:        go1.12.12
 Git commit:        633a0ea
 Built:             Wed Nov 13 07:25:41 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.9
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.11.13
  Git commit:       039a7df
  Built:            Wed Sep  4 16:22:32 2019
  OS/Arch:          linux/amd64
  Experimental:     false

启动一个net=none的Docker容器

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

[root@xnile01 cni]# docker run --name cnitest --net=none -d praqma/network-multitool:latest
Unable to find image 'praqma/network-multitool:latest' locally
latest: Pulling from praqma/network-multitool
050382585609: Pull complete
d1e342a34d6b: Pull complete
079d5234adcd: Pull complete
1deecd267281: Pull complete
d9d9efb6576a: Pull complete
a84544204238: Pull complete
a67bb2f07ab1: Pull complete
Digest: sha256:69f3947ff89b80abd8bfad0d12047fec820f14f8184a9e6aa27487fa6df6a79c
Status: Downloaded newer image for praqma/network-multitool:latest
6b940f005cbf461e7a71b77a82327424cc899884652ab83dc24068bd552191c6

因为我用了--net=none参数，docker 会为容器创建network namespace,但不会做任何配置，所以容器只能看到一个lo环回接口。

1
2
3
4
5

[root@xnile01 cni]# docker exec cnitest ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever

使用CNI为容器添加网口并配置ip。

下载CNI二进制包

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

[root@xnile01 ~]# mkdir cni
[root@xnile01 ~]# cd cni/
[root@xnile01 cni]# curl -s -L https://github.com/containernetworking/cni/releases/download/v0.5.2/cni-amd64-v0.5.2.tgz|tar zxvf -
./
./macvlan
./dhcp
./loopback
./ptp
./ipvlan
./bridge
./tuning
./noop
./host-local
./cnitool
./flannel
[root@xnile01 cni]# ls
bridge  cnitool  dhcp  flannel  host-local  ipvlan  loopback  macvlan  noop  ptp  tuning

准备CNI配置文件

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

cat > mybridge.conf <<EOF
{
  "cniVersion": "0.3.1",
  "name": "mybridge",
  "type": "bridge",
  "bridge": "cni_br0",
  "isGateway": true,
  "ipMasq": true,
  "ipam": {
    "type": "host-local",
    "subnet": "10.0.1.0/24",
    "routes": [
      {
        "dst": "0.0.0.0/0"
      }
    ],
    "dataDir": "/tmp/container-ipam-state"
  }
}
EOF

获取container网络namespace和ID

1
2
3

[root@xnile01 cni]# docker inspect cnitest | grep -E 'SandboxKey|Id'
        "Id": "6b940f005cbf461e7a71b77a82327424cc899884652ab83dc24068bd552191c6",
            "SandboxKey": "/var/run/docker/netns/799f7b677e7a",

为容器添加网卡并配置IP

1
2
3
4
5
6
7

CNI_COMMAND=ADD \
CNI_CONTAINERID=1018026ebc02fa0cbf2be35325f4833ec1086cf6364c7b2cf17d80255d7d4a27 \
CNI_NETNS=/var/run/docker/netns/799f7b677e7a \
CNI_IFNAME=em1 \
CNI_PATH=`pwd` \
./bridge \
< mybridge.conf

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

[root@xnile01 cni]#CNI_COMMAND=ADD CNI_CONTAINERID=1018026ebc02fa0cbf2be35325f4833ec1086cf6364c7b2cf17d80255d7d4a27 CNI_NETNS=/var/run/docker/netns/799f7b677e7a CNI_IFNAME=em1 CNI_PATH=`pwd` ./bridge < mybridge.conf
{
    "interfaces": [
        {
            "name": "cni_br0",
            "mac": "0a:58:0a:00:01:01"
        },
        {
            "name": "veth398d8e21",
            "mac": "7a:cf:e1:04:34:01"
        },
        {
            "name": "em1",
            "mac": "0a:58:0a:00:01:04",
            "sandbox": "/var/run/docker/netns/799f7b677e7a"
        }
    ],
    "ips": [
        {
            "version": "4",
            "interface": 2,
            "address": "10.0.1.4/24",
            "gateway": "10.0.1.1"
        }
    ],
    "routes": [
        {
            "dst": "0.0.0.0/0"
        }
    ],
    "dns": {}
}

验证容器网络

查看容器ip

1
2
3
4
5
6
7
8
9

[root@xnile01 cni]# docker exec cnitest ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: em1@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 0a:58:0a:00:01:04 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.0.1.4/24 scope global em2
       valid_lft forever preferred_lft forever

测试容器网络可达性

1
2
3
4
5
6
7

[root@xnile01 cni]# ping -c 1 10.0.1.4
PING 10.0.1.4 (10.0.1.4) 56(84) bytes of data.
64 bytes from 10.0.1.4: icmp_seq=1 ttl=64 time=0.138 ms

--- 10.0.1.4 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.138/0.138/0.138/0.000 ms