Radosgw 多 zone 部署以及 radosgw-agent 同步

介绍

部署一个 ceph 集群,创建一个 federated radosgw,包含一个 region,两个 zone,
分别对应节点 cd-1 上的 Master rgw 和节点 cd-2 上的 Secondary rgw。

部署

前提已经部署了 ceph 集群,并分别在两个节点上 部署了 rgw。

  1. 修改部署 rgw 是生成的 region infile
    cd.json
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    { “name”: “cd”,
    “api_name”: “cd”,
    “is_master”: “true”,
    “endpoints”: [
    “http:\/\/cd-1:80\/“],
    “master_zone”: “cd-center1”,
    “zones”: [
    { “name”: “cd-center1”,
    “endpoints”: [
    “http:\/\/cd-1:80\/“],
    “log_meta”: “true”,
    “log_data”: “true”},
    { “name”: “cd-center2”,
    “endpoints”: [
    “http:\/\/cd-2:80\/“],
    “log_meta”: “true”,
    “log_data”: “true”}],
    “placement_targets”: [
    {
    “name”: “default-placement”,
    “tags”: []
    }
    ],
    “default_placement”: “default-placement”,
    “hostnames”: [“cd-1”]},
    2. 配置
    节点cd-1
    1
    2
    3
    4
    5
    6
    # scp cd-center1.json cd-2:/etc/ceph/
    # radosgw-admin –name client.radosgw.cd-center1-cd-1 region set –infile cd-v2.json
    # radosgw-admin zone set –rgw-zone=cd-center2 –infile cd-center2.json –name client.radosgw.cd-center1-cd-1
    # rados -p .us.rgw.root rm region_info.default
    # radosgw-admin –name client.radosgw.cd-center1-cd-1 region default –rgw-region=cd
    # radosgw-admin regionmap update –name client.radosgw.cd-center1-cd-1

    节点cd-2
    1
    2
    3
    4
    5
    6
    # scp /etc/ceph/cd-center2.json cd-1:/etc/ceph/
    # radosgw-admin region set –infile cd-v2.json –name client.radosgw.cd-center2-cd-2
    # radosgw-admin zone set –rgw-zone=cd-center1 –infile cd-center1.json –name client.radosgw.cd-center2-cd-2
    # rados -p .us.rgw.root rm region_info.default
    # radosgw-admin region default –rgw-region=cd –name client.radosgw.cd-center2-cd-2
    # radosgw-admin regionmap update –name client.radosgw.cd-center2-cd-2
    3. 重启 ceph-radosgw 服务
    cd-1
    1
    2
    # /etc/init.d/ceph-radosgw restart
    # ssh cd-2 “/etc/init.d/ceph-radosgw restart”

配置 radosgw-agent

radosgw-agent 的默认配置文件是 /etc/ceph/radosgw-agent/default.conf

新建文件/etc/ceph/radosgw-agent/default.conf,在节点 cd-1 上

1
2
3
4
5
6
7
8
9
10
src_access_key: NDOO5JZAAM2Z6WA9FIXG
src_secret_key: 4rWHptoevPLPlOwDITljxPuJTqoMQCMM9ICBjGgv
#source: http://cd-1:80
#src_zone: cd-center1
dest_access_key: FLDVEMPT8PKY1F2UD3F7
dest_secret_key: 8HnGUZIveuHAWePuGXEDXYTR985m7voSUuQDGPkx
destination: http://cd-2:80
log_file: /var/log/radosgw/radosgw-sync-cd-center1-center2.log
#verbose: True
#versioned: True

拷贝到节点 cd-2

1
# scp /etc/ceph/radosgw-agent/default.conf cd-2:/etc/ceph/radosgw-agent/

启动服务

1
2
3
4
# mkdir /var/run/ceph/radosgw-agent/
# ssh cd-2 “mkdir /var/run/ceph/radosgw-agent/“
# /etc/init.d/ceph-radosgw start
# ssh cd-2 “/etc/init.d/ceph-radosgw start”

可以查看同步日志

1
# tailf /var/log/radosgw/radosgw-sync-cd-center1-center2.log