Docker Etcd项目

发表于 | 分类于

Etcd 是 CoreOS 团队发起的一个管理配置信息和服务发现（Service Discovery）的项目。

它的目标是构建一个高可用的分布式键值（key-value）数据库，基于 Go 语言实现。在分布式系统中，各种服务的配置信息的管理分享，服务的发现是一个很基本同时也是很重要的问题。CoreOS 项目就希望基于 etcd 来解决这一问题。etcd 目前在 github.com/etcd-io/etcd 进行维护。

受到 Apache ZooKeeper 项目和 doozer 项目的启发，etcd 在设计的时候重点考虑了下面四个要素：

  • 简单：具有定义良好、面向用户的 API (gRPC)

  • 安全：支持 HTTPS 方式的访问

  • 快速：支持并发 10 k/s 的写操作

  • 可靠：支持分布式结构，基于 Raft 的一致性算法

  • Apache ZooKeeper 是一套知名的分布式系统中进行同步和一致性管理的工具。

  • doozer 是一个一致性分布式数据库。

  • Raft 是一套通过选举主节点来实现分布式系统一致性的算法，相比于大名鼎鼎的 Paxos 算法，它的过程更容易被人理解，由 Stanford 大学的 Diego Ongaro 和 John Ousterhout 提出。更多细节可以参考 raftconsensus.github.io。

一般情况下，用户使用 etcd 可以在多个节点上启动多个实例，并添加它们为一个集群。同一个集群中的 etcd 实例将会保持彼此信息的一致性。

安装

二进制文件方式下载

编译好的二进制文件都在 github.com/etcd-io/etcd/releases 页面，用户可以选择需要的版本：

下载解压：

1
2
3
4
5
6
7
8
9
[root@VM_0_3_centos ~]#curl -L https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz -o etcd-v3.4.0-linux-amd64.tar.gz
[root@VM_0_3_centos ~]#tar xzf etcd-v3.4.0-linux-amd64.tar.gz
[root@VM_0_3_centos ~]# ls
dockerfile  etcd-v3.4.0-linux-amd64  etcd-v3.4.0-linux-amd64.tar.gz
[root@VM_0_3_centos ~]# mv etcd-v3.4.0-linux-amd64 /usr/local/etcd3.4/

[root@VM_0_3_centos ~]# ls /usr/local/etcd3.4/
Documentation  etcd  etcdctl  README-etcdctl.md  README.md  READMEv2-etcdctl.md
[root@VM_0_3_centos ~]# cd /usr/local/etcd3.4/

复制到目录，并且复制到bin，etcd 是服务主文件，etcdctl 是提供给用户的命令客户端。

1
2
3
[root@VM_0_3_centos etcd3.4]# cp etcd* /usr/local/bin/
[root@VM_0_3_centos etcd3.4]# ls /usr/local/bin/
etcd  etcdctl

启动

默认 2379 端口处理客户端的请求，2380 端口用于集群各成员间的通信。启动 etcd 显示类似如下的信息：

1
2
3
4
5
6
[root@VM_0_3_centos etcd3.4]# etcd
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2019-11-05 15:19:01.909992 I | etcdmain: etcd Version: 3.4.0
.......
2019-11-05 15:19:02.634795 I | embed: ready to serve client requests
2019-11-05 15:19:02.635348 N | embed: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!

测试：

1
2
3
4
5
6
7
8
#使用 etcdctl 命令进行测试，设置和获取键值 testkey: "hello world"，检查 etcd 服务是否启动成功：
[root@VM_0_3_centos ~]# ETCDCTL_API=3 etcdctl member list
8e9e05c52164694d, started, default, http://localhost:2380, http://localhost:2379, false
[root@VM_0_3_centos ~]# ETCDCTL_API=3 etcdctl put testkey "hello world"
OK
[root@VM_0_3_centos ~]# etcdctl get testkey
testkey
hello world

也可以通过 HTTP 访问本地 2379 或 4001 端口的方式来进行操作，例如查看 testkey 的值：curl -L http://localhost:4001/v2/keys/testkey

Docker 镜像方式运行

镜像名称为 quay.io/coreos/etcd，可以通过下面的命令启动 etcd 服务监听到 2379 和 2380 端口。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
$ docker run \
-p 2379:2379 \
-p 2380:2380 \
--mount type=bind,source=/tmp/etcd-data.tmp,destination=/etcd-data \
--name etcd-gcr-v3.4.0 \
quay.io/coreos/etcd:v3.4.0 \
/usr/local/bin/etcd \
--name s1 \
--data-dir /etcd-data \
--listen-client-urls http://0.0.0.0:2379 \
--advertise-client-urls http://0.0.0.0:2379 \
--listen-peer-urls http://0.0.0.0:2380 \
--initial-advertise-peer-urls http://0.0.0.0:2380 \
--initial-cluster s1=http://0.0.0.0:2380 \
--initial-cluster-token tkn \
--initial-cluster-state new \
--log-level info \
--logger zap \
--log-outputs stderr

打开新的终端按照上一步的方法测试 etcd 是否成功启动。

macOS 中运行

$ brew install etcd

$ etcd

$ etcdctl member list

Etcd 集群

使用 Docker Compose 模拟启动一个 3 节点的 etcd 集群。

docker-compose.yml 文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
version: "3.6"
services:

  node1:
    image: quay.io/coreos/etcd:v3.4.0
    volumes:
      - node1-data:/etcd-data
    expose:
      - 2379
      - 2380
    networks:
      cluster_net:
        ipv4_address: 172.16.238.100
    environment:
      - ETCDCTL_API=3
    command:
      - /usr/local/bin/etcd
      - --data-dir=/etcd-data
      - --name
      - node1
      - --initial-advertise-peer-urls
      - http://172.16.238.100:2380
      - --listen-peer-urls
      - http://0.0.0.0:2380
      - --advertise-client-urls
      - http://172.16.238.100:2379
      - --listen-client-urls
      - http://0.0.0.0:2379
      - --initial-cluster
      - node1=http://172.16.238.100:2380,node2=http://172.16.238.101:2380,node3=http://172.16.238.102:2380
      - --initial-cluster-state
      - new
      - --initial-cluster-token
      - docker-etcd

  node2:
    image: quay.io/coreos/etcd:v3.4.0
    volumes:
      - node2-data:/etcd-data
    networks:
      cluster_net:
        ipv4_address: 172.16.238.101
    environment:
      - ETCDCTL_API=3
    expose:
      - 2379
      - 2380
    command:
      - /usr/local/bin/etcd
      - --data-dir=/etcd-data
      - --name
      - node2
      - --initial-advertise-peer-urls
      - http://172.16.238.101:2380
      - --listen-peer-urls
      - http://0.0.0.0:2380
      - --advertise-client-urls
      - http://172.16.238.101:2379
      - --listen-client-urls
      - http://0.0.0.0:2379
      - --initial-cluster
      - node1=http://172.16.238.100:2380,node2=http://172.16.238.101:2380,node3=http://172.16.238.102:2380
      - --initial-cluster-state
      - new
      - --initial-cluster-token
      - docker-etcd

  node3:
    image: quay.io/coreos/etcd:v3.4.0
    volumes:
      - node3-data:/etcd-data
    networks:
      cluster_net:
        ipv4_address: 172.16.238.102
    environment:
      - ETCDCTL_API=3
    expose:
      - 2379
      - 2380
    command:
      - /usr/local/bin/etcd
      - --data-dir=/etcd-data
      - --name
      - node3
      - --initial-advertise-peer-urls
      - http://172.16.238.102:2380
      - --listen-peer-urls
      - http://0.0.0.0:2380
      - --advertise-client-urls
      - http://172.16.238.102:2379
      - --listen-client-urls
      - http://0.0.0.0:2379
      - --initial-cluster
      - node1=http://172.16.238.100:2380,node2=http://172.16.238.101:2380,node3=http://172.16.238.102:2380
      - --initial-cluster-state
      - new
      - --initial-cluster-token
      - docker-etcd

volumes:
  node1-data:
  node2-data:
  node3-data:

networks:
  cluster_net:
    driver: bridge
    ipam:
      driver: default
      config:
      -
        subnet: 172.16.238.0/24

使用 docker-compose up 启动集群之后使用 docker exec 命令登录到任一节点测试 etcd 集群。

1
2
3
4
5
6
7
8
9
10
[root@VM_0_3_centos tmp]# docker ps
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS               NAMES
45c34fa7d4d5        quay.io/coreos/etcd:v3.4.0   "/usr/local/bin/etcd…"   25 minutes ago      Up 20 minutes       2379-2380/tcp       etcd_node1_1
b1d9ecc03d64        quay.io/coreos/etcd:v3.4.0   "/usr/local/bin/etcd…"   25 minutes ago      Up 25 minutes       2379-2380/tcp       etcd_node3_1
dae9d24f3197        quay.io/coreos/etcd:v3.4.0   "/usr/local/bin/etcd…"   25 minutes ago      Up 25 minutes       2379-2380/tcp       etcd_node2_1
[root@VM_0_3_centos tmp]# docker exec -it 45 /bin/sh
# etcdctl member list
daf3fd52e3583ff, started, node3, http://172.16.238.102:2380, http://172.16.238.102:2379, false
422a74f03b622fef, started, node1, http://172.16.238.100:2380, http://172.16.238.100:2379, false
ed635d2a2dbef43d, started, node2, http://172.16.238.101:2380, http://172.16.238.101:2379, false

使用 etcdctl

etcdctl 支持如下的命令，大体上分为数据库操作和非数据库操作两类。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
NAME:
    etcdctl - A simple command line client for etcd3.

USAGE:
    etcdctl

VERSION:
    3.4.0

API VERSION:
    3.4


COMMANDS:
    get            Gets the key or a range of keys
    put            Puts the given key into the store
    del            Removes the specified key or range of keys [key, range_end)
    txn            Txn processes all the requests in one transaction
    compaction        Compacts the event history in etcd
    alarm disarm        Disarms all alarms
    alarm list        Lists all alarms
    defrag            Defragments the storage of the etcd members with given endpoints
    endpoint health        Checks the healthiness of endpoints specified in `--endpoints` flag
    endpoint status        Prints out the status of endpoints specified in `--endpoints` flag
    watch            Watches events stream on keys or prefixes
    version            Prints the version of etcdctl
    lease grant        Creates leases
    lease revoke        Revokes leases
    lease timetolive    Get lease information
    lease keep-alive    Keeps leases alive (renew)
    member add        Adds a member into the cluster
    member remove        Removes a member from the cluster
    member update        Updates a member in the cluster
    member list        Lists all members in the cluster
    snapshot save        Stores an etcd node backend snapshot to a given file
    snapshot restore    Restores an etcd member snapshot to an etcd directory
    snapshot status        Gets backend snapshot status of a given file
    make-mirror        Makes a mirror at the destination etcd cluster
    migrate            Migrates keys in a v2 store to a mvcc store
    lock            Acquires a named lock
    elect            Observes and participates in leader election
    auth enable        Enables authentication
    auth disable        Disables authentication
    user add        Adds a new user
    user delete        Deletes a user
    user get        Gets detailed information of a user
    user list        Lists all users
    user passwd        Changes password of user
    user grant-role        Grants a role to a user
    user revoke-role    Revokes a role from a user
    role add        Adds a new role
    role delete        Deletes a role
    role get        Gets detailed information of a role
    role list        Lists all roles
    role grant-permission    Grants a key to a role
    role revoke-permission    Revokes a key from a role
    check perf        Check the performance of the etcd cluster
    help            Help about any command

OPTIONS:
      --cacert=""                verify certificates of TLS-enabled secure servers using this CA bundle
      --cert=""                    identify secure client using this TLS certificate file
      --command-timeout=5s            timeout for short running command (excluding dial timeout)
      --debug[=false]                enable client-side debug logging
      --dial-timeout=2s                dial timeout for client connections
      --endpoints=[127.0.0.1:2379]        gRPC endpoints
      --hex[=false]                print byte strings as hex encoded strings
      --insecure-skip-tls-verify[=false]    skip server certificate verification
      --insecure-transport[=true]        disable transport security for client connections
      --key=""                    identify secure client using this TLS key file
      --user=""                    username[:password] for authentication (prompt if password is not supplied)
  -w, --write-out="simple"            set the output format (fields, json, protobuf, simple, table)

数据库操作

数据库操作围绕对键值和目录的 CRUD （符合 REST 风格的一套操作：Create）完整生命周期的管理。

etcd 在键的组织上采用了层次化的空间结构（类似于文件系统中目录的概念），用户指定的键可以为单独的名字，如 testkey，此时实际上放在根目录 / 下面，也可以为指定目录结构，如 cluster1/node2/testkey，则将创建相应的目录结构。

注：CRUD 即 Create, Read, Update, Delete，是符合 REST 风格的一套 API 操作。

put/set

1
2
$ etcdctl put /testdir/testkey "Hello world"
OK

get

获取指定键的值。例如

1
2
3
4
5
$ etcdctl put testkey hello
OK
$ etcdctl get testkey
testkey
hello

支持的选项为

–sort 对结果进行排序

–consistent 将请求发给主节点，保证获取内容的一致性

del

删除某个键值。例如

1
2
$ etcdctl del testkey
1

update

当键存在时，更新值内容。例如

1
2
3
4
$ etcdctl set testkey hello
hello
$ etcdctl update testkey world
world

当键不存在时，则会报错。例如

1
2
$ etcdctl update testkey2 world
Error:  100: Key not found (/testkey2) [1]

非数据库操作

watch

监测一个键值的变化，一旦键值发生更新，就会输出最新的值。

例如，用户更新 testkey 键值为 Hello world。

$ etcdctl watch testkey

PUT

testkey
2

member

通过 list、add、update、remove 命令列出、添加、更新、删除 etcd 实例到 etcd 集群中。

例如本地启动一个 etcd 服务实例后，可以用如下命令进行查看。

1
2
$ etcdctl member list
422a74f03b622fef, started, node1, http://172.16.238.100:2380, http://172.16.238.100:23
分享到： 收藏夹 复制网址 邮件 微信 QQ空间 腾讯微博 豆瓣 一键分享 更多