In most edge networks, especially those with global presence, observability of traffic flow is critical to successful operations. In addition to providing operational metrics to ensure nominal functionality, the ability to divine key information from the flow of traffic - from transport to application layer - can often prove the difference between quickly detecting and mitigating a DDoS attack or not.
But in at-scale networks, challenges abound: often there is a choice between overwhelming real-time, localized information and delayed, over aggregated, centrally collected information. Efficient analysis and collection of deeply inspected, high throughput traffic is hard. And dynamic configuration management for a fleet of global inspectors is equally challenging.
pktvisor3 is a new open source traffic analysis tool designed to address these challenges. Developed by NS1 for their globally anycasted DNS network, it is designed to be used as both a local, on-node traffic analyzer providing up to the second “top” like functionality (including console UI) as well as to be centrally collected, providing a global, near real time view of critical metrics. It makes use of sketch algorithms to efficiently identify top-k heavy hitters, set cardinality, and quantiles of various L3-L7 metrics. Finally, it aims to offer modularity for application level dissection (starting with but not limited to DNS) and provide a control plane for tuning analysis parameters centrally in real time.
The talk will discuss motivation and previous tool iterations, implementation details, current status, limitations, future goals, and real world use cases at NS1.