在Docker下安装了Zookeeper，Kafka(单机)后遇到的一些问题。

Docker安装Zookeeper、Kafka

拉取镜像

docker pull wurstmeister/zookeeper
docker pull bitnami/kafka:latest

设置网络，把Zookeeper、Kafka设置在同一个网络里

docker network inspect kafka-net

docker network disconnect bridge zookeeper

docker network connect kafka-net zookeeper

如果报错无法断开，可能要先停止容器或者重新创建。

检查网络配置

启动Zookeeper

docker run --restart=always \
--network kafka-net \
--log-driver json-file \
--log-opt max-size=100m \
--log-opt max-file=2  \
--name zookeeper -p 2181:2181 \
-v /home/zookeeper/etc/localtime:/etc/localtime \
-d wurstmeister/zookeeper

启动kafka

docker run -d --name kafka \
  --network kafka-net \
  -p 9092:9092 \
  -e KAFKA_CFG_BROKER_ID=1004 \
  -e KAFKA_CFG_RESERVED_BROKER_MAX_ID=2000 \
  -e KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181 \
  -e KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://IP:9092 \
  -e KAFKA_CFG_SOCKET_REQUEST_MAX_BYTES=524288000 \
  -e KAFKA_CFG_MESSAGE_MAX_BYTES=524288000 \
  -e ALLOW_PLAINTEXT_LISTENER=yes \
  -v /home/kafka/kafka-data:/bitnami/kafka \
  bitnami/kafka:latest

kafka启动成功

没有任何报错

常见问题

IP指定错误,连接失败,连接超时等

ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
ERROR Exiting Kafka. (kafka.Kafka$)

需要指定正确的IP,看你的连接方是内网还是外网，外网的情况下需要设置安全认证

-e KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://IP:9092 \

客户端发送了非法大请求

org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 1195725856 larger than 104857600)

设置单个 socket 请求最大字节数（约 500MB），防止过大请求导致 OOM。
设置单条消息的最大大小（也约 500MB），需与 producer/consumer 配置协调

-e KAFKA_CFG_SOCKET_REQUEST_MAX_BYTES=524288000 \
-e KAFKA_CFG_MESSAGE_MAX_BYTES=524288000

Broker ID 注册失败

Configured end points 192.168.0.89:9092 in advertised listeners are already registered by broker 1004

删除Zookeeper里的临时节点

docker exec -it zookeeper bin/zkCli.sh
delete /brokers/ids/1004

Topic ID 冲突

ERROR [Broker id=1004] Topic ID in memory: t9yVGgkGRwmGE-SFcpfRQA does not match the topic ID for partition ZL003-0 received: _zzYKEnXTQiHYjLJPmhltg

停Kafka,并清理topic。

docker stop kafka
docker exec -it zookeeper bin/zkCli.sh
rmr /brokers/topics。

清磁盘

rm -rf /home/kafka/kafka-data/*

重启Kafka

常用命令

查看消息

docker exec -it kafka kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic ZL003 --from-beginning

查看所有消费组名

docker exec -it kafka kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list

查看某个组的消费情况

（如 default_consumer_group）

docker exec -it kafka kafka-consumer-groups.sh \
  --bootstrap-server localhost:9092 \
  --group default_consumer_group \
  --describe

Kafka常见问题