Speaker
Description
Large recursive resolvers see significant amounts of unique DNS name/qtype/rrdata tuples, and as part of threat intelligence or other analysis work it is necessary to process and store these unique tuples in an effective and economic manner. The use of cuckoo filters (similar to Bloom filters) is a useful tool in understanding what objects have been seen before, in a way that minimizes resource (memory, bandwidth, CPU, disk) use.
Quad9 has developed (with funding assistance from Nominet) a new OSS Rust library called "Cuckoo Clock" that extends the standard cuckoo filter to include TTLs, Least Recently Used cache deletions, and counters. This new library has been implemented in Vector (an OSS telemetry management tool) allowing DNS-specific usage at planet-scale query volumes without custom code.
This talk will briefly address the theory of use for cuckoo filters for DNS data, but will more directly address how the filter is used for Quad9's daily distillation of hundreds of billions of names into a much smaller set of around 2 billion UDRs (Unique DNS Responses.) This method can serve as a case study for any organization interested in how to quickly sift through large amounts of highly-duplicated DNS data using relatively inexpensive hardware and open-source software.
| Talk duration | 20 Minutes (+5 for Q&A) |
|---|
