30–31 Jul 2022
Sheraton Philadelphia Downtown
US/Eastern timezone

On the edge of small data

31 Jul 2022, 14:30
25m
Liberty D (Sheraton Philadelphia Downtown)

Liberty D

Sheraton Philadelphia Downtown

201 North 17th Street Philadelphia PA 19103 United States
Standard Presentation Main Session OARC 38 Day 2

Speaker

Shannon Weyrick (NS1)

Description

At OARC 33, pktvisor (https://pktvisor.dev) was presented as a free and open source traffic analyzer that summarizes critical information from DNS edge networks in real time, making the resulting lightweight telemetry available locally on node and easily collectable into a central database for dashboarding and alerting.

Since then we’ve introduced Orb (https://getorb.io), a free and open source dynamic edge observability platform which builds on our earlier work. Orb acts as a control tower for a distributed fleet of pktvisor agents, combining dynamic orchestration of analysis policies with data collection and sinking functionality, accessible via web UI and REST API.

The agent can now analyze packet capture, dnstap, flow and other inputs, combining deep traffic analysis with data sketch algorithms to efficiently extract counts, top-k heavy hitters, set cardinality, quantiles and other key information from data streams directly on the edge. The result is lightweight time series metrics that plug into modern observability stacks.

Together the goal of Orb and pktvisor is to deliver immediately actionable insights local to the traffic source and simultaneously collected and integrated into global result sets.

This talk will introduce and advocate for the powerful “small data” use case, discuss the goals and status of the projects, and look to the future as they extend beyond traffic analysis and into general network debugging and analytics embedded at the edge.

Presentation delivery In-person at the workshop venue

Primary author

Shannon Weyrick (NS1)

Presentation materials