# Kolla ## 简介 Kolla是OpenStack Big Tent Governace下的一个项目，项目的目标是 > To provide production-ready containers and deployment tools for operating OpenStack clouds. Kolla使用Docker容器和Anisble playbooks来实现这个目标。Kolla是开箱即用的，即使你是个新手也可以很快的使用kolla快速部署你的openstack集群。Kolla也允许你根据实际的需求来定制化的部署。 kolla目前已经可以部署以下openstack项目 * [Aodh](http://docs.openstack.org/developer/aodh/) * [Barbican](http://docs.openstack.org/developer/barbican/) * [Bifrost](http://docs.openstack.org/developer/bifrost/) * [Ceilometer](http://docs.openstack.org/developer/ceilometer/) * [Cinder](http://docs.openstack.org/developer/cinder/) * [CloudKitty](http://docs.openstack.org/developer/cloudkitty/) * [Congress](http://docs.openstack.org/developer/congress/) * [Designate](http://docs.openstack.org/developer/designate/) * [Glance](http://docs.openstack.org/developer/glance/) * [Gnocchi](http://docs.openstack.org/developer/gnocchi/) * [Heat](http://docs.openstack.org/developer/heat/) * [Horizon](http://docs.openstack.org/developer/horizon/) * [Ironic](http://docs.openstack.org/developer/ironic/) * [Keystone](http://docs.openstack.org/developer/keystone/) * [Kuryr](http://docs.openstack.org/developer/kuryr/) * [Magnum](http://docs.openstack.org/developer/magnum/) * [Manila](http://docs.openstack.org/developer/manila/) * [Mistral](http://docs.openstack.org/developer/mistral/) * [Murano](http://docs.openstack.org/developer/murano/) * [Neutron](http://docs.openstack.org/developer/neutron/) * [Nova](http://docs.openstack.org/developer/nova/) * [Rally](http://docs.openstack.org/developer/rally/) * [Sahara](http://docs.openstack.org/developer/sahara/) * [Senlin](http://docs.openstack.org/developer/senlin/) * [Swift](http://docs.openstack.org/developer/swift/) * [Tempest](http://docs.openstack.org/developer/tempest/) * [Trove](http://docs.openstack.org/developer/trove/) * [Vmtp](http://vmtp.readthedocs.io/en/latest/) * [Watcher](http://docs.openstack.org/developer/watcher/) * [Zaqar](http://docs.openstack.org/developer/zaqar/) 可以部署的基础组件包括 * [Ceph](http://ceph.com/) 做cinder/nova/glance存储后端 * \[collectd (), \[InfluxDB (), and \[Grafana () for performance monitoring. * \[Elasticsearch () and \[Kibana () 日志分析 * \[HAProxy () and \[Keepalived () 高可用 * \[Heka () 分布式可扩展的日志系统 * \[MariaDB and Galera Cluster () 高可用数据库 * \[MongoDB () Ceilometer和Gnocchi的数据库后端 * \[Open vSwitch () * \[RabbitMQ () 消息队列 ## Kolla体验 可以参照kolla官方文档 进行部署。 ## Kolla解决的问题 ### 可配置的灵活架构 可以看下默认的多节点架构 ```bash # These initial groups are the only groups required to be modified. The # additional groups are for more control of the environment. [control] # These hostname must be resolvable from your deployment host control01 control02 control03 # The above can also be specified as follows: #control[01:03] ansible_ssh_user=kolla # The network nodes are where your l3-agent and loadbalancers will run # This can be the same as a host in the control group [network] network01 [compute] compute01 # When compute nodes and control nodes use different interfaces, # you can specify "api_interface" and another interfaces like below: #compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1 [storage] storage01 [baremetal:children] control network compute storage # You can explicitly specify which hosts run each project by updating the # groups in the sections below. Common services are grouped together. [kibana:children] control [elasticsearch:children] control [haproxy:children] network [mariadb:children] control [rabbitmq:children] control [mongodb:children] control [keystone:children] control [glance:children] control [nova:children] control [neutron:children] network [cinder:children] control [memcached:children] control [horizon:children] control [swift:children] control [heat:children] control [murano:children] control [ironic:children] control [ceph-mon:children] control [ceph-rgw:children] control [ceph-osd:children] storage # Additional control implemented here. These groups allow you to control which # services run on which hosts at a per-service level. # # Word of caution: Some services are required to run on the same host to # function appropriately. For example, neutron-metadata-agent must run on the # same host as the l3-agent and (depending on configuration) the dhcp-agent. # Glance [glance-api:children] glance [glance-registry:children] glance # Nova [nova-api:children] nova [nova-conductor:children] nova [nova-consoleauth:children] nova [nova-novncproxy:children] nova [nova-scheduler:children] nova [nova-spicehtml5proxy:children] nova [nova-compute-ironic:children] nova # Neutron [neutron-server:children] control [neutron-dhcp-agent:children] neutron [neutron-l3-agent:children] neutron [neutron-lbaas-agent:children] neutron [neutron-metadata-agent:children] neutron ``` 默认我们会把haproxy放到network节点，如果我想把haproxy放到一个单独的节点，那么我只需要到这样修改 ```bash -[haproxy:children] -network +[haproxy] +haproxy01 +haproxy02 ``` ### 配置文件管理 每个openstack服务都运行在一个容器中，那kolla是怎么管理openstack的配置的呢? 我们拿nova-compute的配置管理来举例 #### 首先kolla会使用ansible为nova-compute生成一份配置文件放在/etc/kolla/nova-compute/目录下。 ```bash #nova_custom_config默认是/etc/kolla/configs/nova #node_config_directory默认是 /etc/kolla - name: Copying over nova.conf merge_configs: vars: service_name: "{{ item }}" sources: - "{{ role_path }}/templates/nova.conf.j2" - "{{ node_custom_config }}/global.conf" - "{{ node_custom_config }}/database.conf" - "{{ node_custom_config }}/messaging.conf" - "{{ node_custom_config }}/nova.conf" - "{{ node_custom_config }}/nova/{{ item }}.conf" - "{{ node_custom_config }}/nova/{{ inventory_hostname }}/nova.conf" dest: "{{ node_config_directory }}/{{ item }}/nova.conf" with_items: - "nova-api" - "nova-compute" - "nova-compute-ironic" - "nova-conductor" - "nova-consoleauth" - "nova-novncproxy" - "nova-scheduler" - "nova-spicehtml5proxy" ``` 大家可能会注意到kolla使用merge\_configs来完成配置文件的合并，那么merge\_configs是干什么的呢?顾名思义，merge\_configs就是把多个配置文件合成一个,kolla为什么要这样做呢？ openstack配置选项非常多但是真正需要管理的则很少，对这部分选项kolla使用模版的方式管理，同时由于merge\_configs的使用，使得用户可以非常方便的添加自己的定制化选项。比如你部署kolla在一台虚拟机上，你必须使用QEMU hypervisor来替代KVM hypervisor。那么你可以在/etc/kolla/config/nova/nova-compute.conf中添加以下配置 ``` [libvirt] virt_type=qemu ``` merge\_configs的代码在 ansible/action\_plugins/merge\_configs.py #### 启动容器时/etc/kolla以docker卷的形式挂载到/var/lib/kolla/config\_files目录下 ```bash - name: Starting nova-libvirt container kolla_docker: action: "start_container" common_options: "{{ docker_common_options }}" image: "{{ nova_libvirt_image_full }}" name: "nova_libvirt" pid_mode: "host" privileged: True volumes: - "{{ node_config_directory }}/nova-libvirt/:{{ container_config_directory }}/:ro" - "/etc/localtime:/etc/localtime:ro" - "/lib/modules:/lib/modules:ro" - "/run/:/run/" - "/dev:/dev" - "/sys/fs/cgroup:/sys/fs/cgroup" - "kolla_logs:/var/log/kolla/" - "libvirtd:/var/lib/libvirt" - "nova_compute:/var/lib/nova/" - "nova_libvirt_qemu:/etc/libvirt/qemu" when: inventory_hostname in groups['compute'] ``` #### 容器启动脚本会根据nova-compute.json来将配置文件拷贝到/etc并设置合适的权限 ```bash { "command": "nova-compute", "config_files": [ { "source": "{{ container_config_directory }}/nova.conf", "dest": "/etc/nova/nova.conf", "owner": "nova", "perm": "0600" }{% if nova_backend == "rbd" %}, { "source": "{{ container_config_directory }}/ceph.*", "dest": "/etc/ceph/", "owner": "nova", "perm": "0700" }{% endif %} ] } ``` 关于kolla配置文件的管理还可以参考[这里](https://github.com/openstack/kolla/blob/master/doc/deployment-philosophy.rst) ### nova-fake测试控制平台性能 [这里](https://github.com/openstack/kolla/blob/master/doc/nova-fake-driver.rst) ### compute节点升级问题 由于所有服务都运行在容器中，那么是不是我升级compute节点时，该节点的虚机都会进入关机状态呢，kolla使用super-privilege的容器来解决了这个问题具体可以参考kolla PTL的文章 ### 平滑升级 kolla为升级也编写了upgrade.yaml这个playbook,我们还是拿nova-compute的升级为例 ```bash # kolla/ansible/roles/nova/tasks/upgrade.yml --- # Create new set of configs on nodes - include: config.yml # TODO(inc0): since nova is creating new database in L->M, we need to call it. # It should be removed later - include: bootstrap.yml - include: bootstrap_service.yml - name: Checking if conductor container needs upgrading kolla_docker: action: "compare_image" common_options: "{{ docker_common_options }}" name: "nova_conductor" image: "{{ nova_conductor_image_full }}" when: inventory_hostname in groups['nova-conductor'] register: conductor_differs # Short downtime here, but from user perspective his call will just timeout or execute later - name: Stopping all nova_conductor containers kolla_docker: action: "stop_container" common_options: "{{ docker_common_options }}" name: "nova_conductor" when: - inventory_hostname in groups['nova-conductor'] - conductor_differs['result'] - include: start_conductors.yml - include: start_controllers.yml serial: "30%" - include: start_compute.yml serial: "10%" - include: reload.yml serial: "30%" ``` ## 使用 ### 查看log ```bash cd /var/lib/docker/volumes/kolla_logs/ ``` ### 进入容器调试 ```bash docker exec -it service_name bash ``` ### root权限问题 出于安全考虑很多kolla服务都是运行在非root下，进入容器后拿不到root权限，我们还以nova\_compute为例,可以修改/etc/kolla/nova\_compute/config.json改为以下 ```bash { "command": "nova-compute", "config_files": [ { "source": "/var/lib/kolla/config_files/nova.conf", "dest": "/etc/nova/nova.conf", "owner": "nova", "perm": "0600" }, { "source": "/var/lib/kolla/config_files/nova.sudo", "dest": "/etc/sudoers.d/nova.sudo", "owner": "root", } ] } ``` 然后在/etc/kolla/nova-compute添加nova.sudo ```bash nova ALL=(ALL) NOPASSWD: ALL ``` 重启容器后即可sudo到root用户下调试 ### 定制化build镜像 参考 ## 总结 优点 * 配置管理灵活方便 * 可以平滑升级 * 部署简单 * 环境隔离 * 多种安装源 * 支持的部署的服务多 缺点 * 对新手的友好程度 * debug不方便 --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://newptone.gitbook.io/deployopenstackwithpuppet/deployment_tool/kolla.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.