cloudfoundry & diego
introduction
cloudfoundry 现已支持 diego,目前官方有两种部署方式
- cf-deployment (较新)
- cf-release (将被弃用,不过相对充分验证<包括 openstack>)
部署过程太复杂,翻开官网一看,我靠
撸起袖子开搞
环境描述
- opnstack: ocata
- cf 版本: 截至目前的 release 版本 v280(https://github.com/cloudfoundry/cf-release/releases)
stemcell 和其他需要的 release 使用其推荐的兼容版本,如下:
name | version | url |
---|---|---|
cf release | v280 | https://bosh.io/d/github.com/cloudfoundry/cf-release?v=280 |
Diego release | v1.30.0 | https://bosh.io/d/github.com/cloudfoundry/diego-release?v=1.30.0 |
Garden-Runc release | v1.9.6 | https://bosh.io/d/github.com/cloudfoundry/garden-runc-release?v=1.9.6 |
cflinuxfs2 release | v1.168.0 | https://bosh.io/d/github.com/cloudfoundry/cflinuxfs2-release?v=1.168.0 |
etcd release | v117 | https://bosh.io/d/github.com/cloudfoundry-incubator/etcd-release?v=117 |
stemcell | 3468.5 | https://s3.amazonaws.com/bosh-core-stemcells/openstack/bosh-stemcell-3468.5-openstack-kvm-ubuntu-trusty-go_agent-raw.tgz |
openstack 环境准备
不具体描述,参考 http://docs.cloudfoundry.org/deploying/cf-release/openstack/index.html, 主要步骤包括
- 设置安全组
- 设置 DNS(用于解析 cf app)
- 确认设置 flavor
部署 bosh
bosh 是一种部署工具,cf 可以通过 bosh 来部署。通常 bosh 包括两部分
- bosh director(这里是在 openstack 中的一个虚拟机,用来接受 bosh client 的命令,并操作集群,如 cf)
- bosh client(任意可访问 openstack 的一个机器,安装 bosh cli 工具,后续的部署工作都是在该机器进行)
准备 openstack 环境
这里的准备工作不冲突之前的,这是为部署 bosh director 准备,不具体描述,参考 https://bosh.io/docs/init-openstack-v1.html
准备 bosh director 的配置文件
根据需求修改,如 openstack 环境认证、虚拟机的密码等,参考 https://gist.github.com/ly798/624c964dffcbd86d4f9fa01a7d5c4dba
部署
安装 bosh-init,参考 https://bosh.io/docs/install-bosh-init.html, 用来部署 bosh director
部署 bosh director
1 | bosh-init deploy ./bosh.yml |
安装 bosh-cli,参考 https://bosh.io/docs/bosh-cli.html, 用来操作 bosh director
登陆到 bosh director
1 | bosh target 173.81.16.12 #该 ip 是在配置文件中配置的 director 的 floatingip bosh vms #查询集群的 vm,现在为空 |
部署 cloud foundry
准备 cf 的配置文件
参考 http://docs.cloudfoundry.org/deploying/cf-release/openstack/cf-stub.html
cf 的配置文件(暂命令为 cf-deployment.yml)非常庞大,所以使用脚本工具来生成,不过也需要一个基本的 manifest stub 配置文件(咱命令为 cf-stub.yml)
首先获取 cf-release 代码仓库
1 | git clone https://github.com/cloudfoundry/cf-release.git git checkout v258 https://github.com/cloudfoundry/diego-release.git git checkout v1.30.0 |
新建配置文件 cf-stub.yml,参考 https://gist.github.com/ly798/1a393a7450b54a6e78bf62ba6c00a249, 根据自己的环境配置,另外由于 bug 需要注意的地方
1 | properties: etcd: require_ssl: false |
关于上述配置文件中的 cert、key 需要使用仓库中 script 目录下脚本实现,可以参考 https://github.com/cloudfoundry/cf-release/blob/master/on-tls-certificates.md, 这些脚本有依赖,安装如下(ubuntu 为例):
1 | git clone https://github.com/square/certstrap cd certstrap ./build sudo cp ./bin/certstrap-v1.1.1-linux-amd64 /usr/bin/certstrap sudo apt install golang mkdir ~/go export GOPATH=~/go |
关于上述配置文件中的 dns 配置暂时使用内部解析的 dns 服务器,可以使用 dnsmasq,将配置文件中的 domain 与 floatingip 进行解析
使用脚本工具生成 cf-deployment.yml
1 | cd cf-release ./scripts/generate_deployment_manifest openstack ../cf-stub.yml ../diego-release/stubs-for-cf-release/enable_diego_ssh_in_cf.yml > ../cf-deployment.yml |
生成的 cf-deployment.yml 还需要一些修改(由于 bug)
1 | uaa: require_https: false |
替换虚拟机镜像
1 | sed -i 's/bosh-openstack-kvm-ubuntu-trusty-go_agent/bosh-openstack-kvm-ubuntu-trusty-go_agent-raw/g' ../cf-deployment.yml |
部署
参考 http://docs.cloudfoundry.org/deploying/cf-release/common/deploy.html
上传 cf-release、stemcell
1 | bosh upload release ~/cf-release-v280.zip bosh upload stemcell ~/bosh-stemcell-3468.5-openstack-kvm-ubuntu-trusty-go_agent-raw.tgz |
一切准备就绪,开始 装逼 安装
1 | bosh deployment ../cf-deployment.yml bosh deploy |
验证
验证 cf 服务信息
1 | curl api.INCORRECT-SYSTEM-DOMAIN/info # INCORRECT-SYSTEM-DOMAIN 死在 cf-stub.yml 中的配置 |
登陆 cf
1 | cf login –skip-ssl-validation -a https:/INCORRECT-SYSTEM-DOMAIN -u admin cf a |
部署 diego
diego 在 openstack 上配置资料较少,翻遍官网和网络都 emmmm..
准备配置文件
参考 diego-release 代码仓库 aws 和 bosh-lite 的配置文件整理了一份,具体如下
1 | cd diego-release mkdir myop_config cp manifest-generation/examples/openstack/iaas-settings.yml . cp manifest-generation/bosh-lite-stubs/property-overrides.yml . cp ./manifest-generation/bosh-lite-stubs/postgres/diego-sql.yml . cp ./manifest-generation/bosh-lite-stubs/instance-count-overrides.yml . |
需要根据自己环境配置这四个文件,需要注意
- property-overrides.yml 中的 cert 等还是需要参考部署 cf 是使用的 cert 等
- property-overrides.yml 中的 etcd.ssl 相关设置为 false(部署 cf 是也使用了 false)
- iaas-settings.yml 中 stemcell 使用部署 cf 的即可(raw)
- diego-sql 的信息取自 cf 配置文件
生成配置文件 diego-deployment.yml
1 | cd diego-release ./scripts/generate-deployment-manifest -c ../../cf-deployment.yml \ -i ./myop_config/iaas-settings.yml \ -p ./myop_config/property-overrides.yml \ -n ./myop_config/instance-count-overrides.yml \ -s ./myop_config/diego-sql.yml -r > ./diego-deployment.yml |
生成的 diego.yml 还需要进行修改(遗漏的 uaa.secret)
1 | diego.ssh_proxy.uaa_secret: xxx #取自 cf 部署文件中的 |
部署
上传 releases
1 | bosh upload release garden-runc-release-v1.10.0.zip bosh upload release cflinuxfs2-release-v1.170.0.zip bosh upload release diego-release-v1.30.0.zip bosh upload release etcd-release-v117.zip |
部署 diego
1 | bosh deployment diego-deployment.yml bosh -n deploy |
验证
打开 docker flag
1 | cf enable-feature-flag diego_docker |
部署一个 docker
1 | # cf push my-app –docker-image daocloud.io/library/nginx:latest name requested state instances memory disk urls my-app started 1/1 1G 1G my-app.test.cf-app.com |