Programmable Per-Packet Network Telemetry: From Wire to Kafka at Scale

High Performance Distributed Computing(2021)

引用 3|浏览11
暂无评分
摘要
ABSTRACTEfficient and secure management of networks requires collecting and analyzing fine-grained telemetry data, preferably in real-time. Existing monitoring and analysis frameworks (e.g., Netflow, SNMP counters) do not provide fine-grained, per-packet information, are hard or not possible to customize, and do not provide an expressive programming interface to extract information. We present ESnet High Touch Services, a programmable, scalable, and expressive hardware and software solution that produces and analyzes per-packet telemetry information with nanosecond-accurate timing. We highlight our architecture, the most critical performance considerations that allow the processing of 10.4 million telemetry packets per second with only 5 CPU cores, which is more than enough to handle 127 Gbit/s of original traffic with 1512B MTU. We also present applications of the system that use real-time stream processing with elegant filtering, aggregation, and windowing functionalities. Our use-cases show that High Touch Services can support a variety of advanced performance monitoring, troubleshooting, and security tasks.
更多
查看译文
关键词
network,per-packet
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要