Flume实现Spooling Directory Source 案例

Flume实现Spooling Directory Source 案例

先停掉 node1 上的 flume（ctrl+c）将 node1 上的 option拷贝 option_sdir，并修改：

[root@node1 ~]# cp option option_sdir [root@node1 ~]# vim option_sdir# Describe/configure the source a1.sources.r1.type = spooldir a1.sources.r1.spoolDir = /root/log a1.sources.r1.fileHeader = true # Describe the sink

启动 node1 上的 flume

flume-ng agent -n a1 --conf-file option_sdir -Dflume.root.logger=INFO,console

启动后抛出以下异常： java.lang.IllegalStateException: Directory does not exist: /root/log 这是因为目录没有创建，创建目录后再次启动，则解决了。

复制一个 node1 连接的 xshell 终端，/root/log 目录下拷贝文本文件

[root@node1 ~]# cp log.txt log/

然后在启动 flume 的终端中查看信息

21/11/20 14:52:07 INFO avro.ReliableSpoolingFileEventReader: Last read took us just up to a file boundary. Rolling to the next file, if there is one.21/11/20 14:52:07 INFO avro.ReliableSpoolingFileEventReader: Preparing to move file /root/log/wc.txt to /root/log/wc.txt.neusoft21/11/20 15:01:37 INFO avro.ReliableSpoolingFileEventReader: Last read took us just up to a file boundary. Rolling to the next file, if there is one.21/11/20 15:01:37 INFO avro.ReliableSpoolingFileEventReader: Preparing to move file /root/log/log.txt to /root/log/log.txt.neusoft21/11/20 15:01:41 INFO sink.LoggerSink: Event: { headers:{file=/root/log/log.txt} body: 68 65 6C 6C 6F 20 6E 65 75 73 6F 66 74 hello neusoft }21/11/20 15:01:41 INFO sink.LoggerSink: Event: { headers:{file=/root/log/log.txt} body: 68 65 6C 6C 6F 20 79 71 71 hello yqq }

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。