wordpress调用指定文章详情南阳本地网络推广优化公司
文章目录
- 主机准备
- 部署
- 编辑 docker-compose.yml
- controller
- broker
- 生成cluster_id
- 一篇完整的 docker-compose.yml 文件
- 查看集群状态
- 使用 kafka-ui 查看
- 拉取 kafka-ui
- 添加集群
- 查看集群状态
- 使用命令行查看
- 配置讲解
- controller
- broker
主机准备
| IP | controller id | broker id |
|---|---|---|
| 192.168.142.157 | 1 | 5 |
| 192.168.142.156 | 2 | 6 |
| 192.168.142.155 | 3 | 7 |
| 192.168.142.158 | 4 | 8 |
部署
创建文件夹
mkdir -p /docker/kafka/{data,data1,data2}
chmod 777 -R {data,data1,data2}
编辑 docker-compose.yml
controller
docker-compose.yml
services:controller:image: bitnami/kafka:3.8.0container_name: controllernetwork_mode: hostrestart: alwaysvolumes:- /docker/kafka/data:/data- /etc/localtime:/etc/localtime:roenvironment:KAFKA_CFG_NODE_ID: 1KAFKA_CFG_PROCESS_ROLES: controllerKAFKA_CFG_LISTENERS: CONTROLLER://192.168.142.157:9093KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXTKAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLERKAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@192.168.142.157:9093,2@192.168.142.156:9093,3@192.168.142.155:9093,4@192.168.142.158:9093KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS: 0KAFKA_CFG_LOG_DIRS: /data
broker
image: bitnami/kafka:3.8.0container_name: brokernetwork_mode: hostrestart: alwaysvolumes:- /etc/localtime:/etc/localtime:ro- /docker/kafka/data1:/data1- /docker/kafka/data2:/data2environment:KAFKA_CFG_NODE_ID: 5KAFKA_CFG_PROCESS_ROLES: brokerKAFKA_CFG_LISTENERS: PLAINTEXT://192.168.142.157:9092KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://192.168.142.157:9092KAFKA_CFG_INTER_BROKER_LISTENER_NAME: PLAINTEXTKAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLERKAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXTKAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@192.168.142.157:9093,2@192.168.142.156:9093,3@192.168.142.155:9093,4@192.168.142.158:9093KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS: 0KAFKA_CFG_LOG_DIRS: "/data1,/data2"
所有的主机这一段基本都是一样的 只有 id 不同，id 可以根据上面表格进行修改
生成cluster_id
root@master:/docker/kafka# docker run -it --rm bitnami/kafka:3.8.0 kafka-storage.sh random-uuid
kafka 06:43:36.79 INFO ==>
kafka 06:43:36.80 INFO ==> Welcome to the Bitnami kafka container
kafka 06:43:36.80 INFO ==> Subscribe to project updates by watching https://github.com/bitnami/containers
kafka 06:43:36.80 INFO ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
kafka 06:43:36.80 INFO ==> Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
kafka 06:43:36.80 INFO ==>PavLImfXSqaOOdrMgq1teA
拿到 id
PavLImfXSqaOOdrMgq1teA
新增到 docker-compose.yml 文件中
一篇完整的 docker-compose.yml 文件
services:controller:image: bitnami/kafka:3.8.0container_name: controllernetwork_mode: hostrestart: alwaysvolumes:- /docker/kafka/data:/data- /etc/localtime:/etc/localtime:roenvironment:KAFKA_CFG_NODE_ID: 1KAFKA_CFG_PROCESS_ROLES: controllerKAFKA_CFG_LISTENERS: CONTROLLER://192.168.142.157:9093KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXTKAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLERKAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@192.168.142.157:9093,2@192.168.142.156:9093,3@192.168.142.155:9093,4@192.168.142.158:9093KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS: 0KAFKA_CFG_LOG_DIRS: /dataKAFKA_KRAFT_CLUSTER_ID: PavLImfXSqaOOdrMgq1teAbroker:image: bitnami/kafka:3.8.0container_name: brokernetwork_mode: hostrestart: alwaysvolumes:- /etc/localtime:/etc/localtime:ro- /docker/kafka/data1:/data1- /docker/kafka/data2:/data2environment:KAFKA_CFG_NODE_ID: 5KAFKA_CFG_PROCESS_ROLES: brokerKAFKA_CFG_LISTENERS: PLAINTEXT://192.168.142.157:9092KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://192.168.142.157:9092KAFKA_CFG_INTER_BROKER_LISTENER_NAME: PLAINTEXTKAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLERKAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXTKAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@192.168.142.157:9093,2@192.168.142.156:9093,3@192.168.142.155:9093,4@192.168.142.158:9093KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS: 0KAFKA_CFG_LOG_DIRS: "/data1,/data2"KAFKA_KRAFT_CLUSTER_ID: PavLImfXSqaOOdrMgq1teA
启动
docker compose up -d
查看启动状态
docker ps -a | grep Up | wc -l
2
查看集群状态
使用 kafka-ui 查看
拉取 kafka-ui
只需要在一台主机上部署 ui 就可以了
docker run -d -p 8080:8080 -e DYNAMIC_CONFIG_ENABLED=true provectuslabs/kafka-ui:v0.7.1
通过 ip + 8080 端口访问
添加集群
查看集群状态
使用命令行查看
root@master:/docker/kafka# docker exec -it broker ./opt/bitnami/kafka/bin/kafka-metadata-quorum.sh --bootstrap-controller 192.168.142.157:9093,192.168.142.156:9093,192.168.142.155:9093,192.168.142.158:9093 describe --status
ClusterId: PavLImfXSqaOOdrMgq1teA
LeaderId: 1
LeaderEpoch: 22
HighWatermark: 7695
MaxFollowerLag: 0
MaxFollowerLagTimeMs: 0
CurrentVoters: [1,2,3,4]
CurrentObservers: [5,6,7,8]
到此集群搭建完成
配置讲解
controller
| 参数 | 解释 |
|---|---|
| KAFKA_CFG_NODE_ID | 节点 id |
| KAFKA_CFG_PROCESS_ROLES | 节点角色 |
| KAFKA_CFG_LISTENERS | 节点监听的 ip 和端口，并且标记为 controller |
| KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP | 定义不同监听器使用的加密协议 |
| KAFKA_CFG_CONTROLLER_LISTENER_NAMES | 控制器（Controller）用于接收来自其他控制器节点和 broker 节点请求的监听器（Listener）的名称 |
| KAFKA_CFG_CONTROLLER_QUORUM_VOTERS | 添加集群节点 |
| KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS | 当一个新的消费者加入到一个空的消费组时，协调者（Coordinator）将延迟多长时间再开始重新平衡（Rebalance）的过程 |
| KAFKA_CFG_LOG_DIRS | 日志目录 |
| KAFKA_KRAFT_CLUSTER_ID | 集群 id |
broker
| 参数 | 解释 |
|---|---|
| KAFKA_CFG_NODE_ID | 节点 id |
| KAFKA_CFG_PROCESS_ROLES | 节点角色 |
| KAFKA_CFG_LISTENERS | 节点监听的 ip 和端口，并且标记为 broker |
| KAFKA_CFG_INTER_BROKER_LISTENER_NAME | 集群中 broker 之间通信所使用的监听器名称 |
| KAFKA_CFG_CONTROLLER_LISTENER_NAMES | 监听器的名称 |
| KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP | 定义不同监听器使用的加密协议 |
| KAFKA_CFG_CONTROLLER_QUORUM_VOTERS | 添加集群节点 |
| KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS | 当一个新的消费者加入到一个空的消费组时，协调者（Coordinator）将延迟多长时间再开始重新平衡（Rebalance）的过程 |
| KAFKA_CFG_LOG_DIRS | 日志目录 |
| KAFKA_KRAFT_CLUSTER_ID | 集群 id |