网站icp备案查询,南宁网站建设哪家公司好,平台建设内容,沈阳铁西做网站公司文章目录 1、Kafka作为Source【数据进入到kafka中#xff0c;抽取出来】2、kafka作为Sink 【数据从别的地方抽取到kafka里面】 1、Kafka作为Source【数据进入到kafka中#xff0c;抽取出来】
kafka源 -- memory -- 控制台#xff1a;
a1.sources r1
a1.sinks k1… 文章目录 1、Kafka作为Source【数据进入到kafka中抽取出来】2、kafka作为Sink 【数据从别的地方抽取到kafka里面】 1、Kafka作为Source【数据进入到kafka中抽取出来】
kafka源 -- memory -- 控制台
a1.sources r1
a1.sinks k1
a1.channels c1# Bind the source and sink to the channel
a1.sources.r1.channels c1
a1.sinks.k1.channel c1a1.sources.r1.type org.apache.flume.source.kafka.KafkaSource
a1.sources.r1.batchSize 100
a1.sources.r1.batchDurationMillis 2000
a1.sources.r1.kafka.bootstrap.servers node01:9092,node02:9092,node03:9092
a1.sources.r1.kafka.topics five
a1.sources.r1.kafka.consumer.group.id donghu# Use a channel which buffers events in memory
a1.channels.c1.type memory
a1.channels.c1.capacity 1000
a1.channels.c1.transactionCapacity 100a1.sinks.k1.type logger
a1.sinks.k1.maxBytesToLog 1282、kafka作为Sink 【数据从别的地方抽取到kafka里面】
netcat -- memory --kafka
##a1就是flume agent的名称
## source r1
## channel c1
## sink k1
a1.sources r1
a1.sinks k1
a1.channels c1# Describe/configure the source
a1.sources.r1.type netcat
a1.sources.r1.bind bigdata01
a1.sources.r1.port 44444# 修改sink为kafka
a1.sinks.k1.type org.apache.flume.sink.kafka.KafkaSink
a1.sinks.k1.kafka.bootstrap.servers node01:9092
a1.sinks.k1.kafka.topic five
a1.sinks.k1.kafka.producer.acks 1
a1.sinks.k1.kafka.producer.linger.ms 1# Use a channel which buffers events in memory
a1.channels.c1.type memory
a1.channels.c1.capacity 1000
a1.channels.c1.transactionCapacity 100# Bind the source and sink to the channel
a1.sources.r1.channels c1
a1.sinks.k1.channel c1
