OARC 36 (Online)

29 Nov 2021, 13:00 → 30 Nov 2021, 16:00 UTC

Jan Včelák (NS1), Keith Mitchell (DNS-OARC), Manu Bretelle (Facebook), Pallavi Aras (Salesforce), Willem Toorop (NLnet Labs)

OARC 36 will be an online Workshop starting at 14:00 UTC on Monday and 10:00 UTC on Tuesday.

DNS-OARC is a non-profit, membership organization that seeks to improve the security, stability, and understanding of the Internet's DNS infrastructure. Part of these aims are achieved through workshops.

DNS-OARC Workshops are open to OARC members and to all other parties interested in DNS operations and research.

Social Media hashtag: #OARC36

Mattermost Chatroom: Workshops on chat.dns-oarc.net (sign-up here)

---

WORKSHOP SPONSORS

---

 

Sponsorship opportunities for OARC 36 are available. Details at:

https://www.dns-oarc.net/workshop/sponsorship-opportunities

---

OARC PATRONS 2021

Your company name here?

Annual Workshop Patrons for 2021 are available. Details at:

https://www.dns-oarc.net/workshop/patronage-opportunities

 

---

 

Monday, 29 November

OARC Software AMA

Welcome to another OARC Software Ask Me Anything (AMA) which will be hosted by our Software Engineer Jerry Lundström.

In this session you can ask us anything about our open source software or services, and there is possibility to demonstrate features if so requested.

Details on how to join this meeting will be sent to all delegates before the conference starts.

Please see https://www.dns-oarc.net/oarc/software for more information about our Software Development & Funding opportunities, and feel free to join us on Mattermost.

OARC 36 Day 1: Session 1

1 Welcome

Speaker: Mr Keith Mitchell (DNS-OARC)

OARC36-combined.odp

Welcome and Introduction

2 DNS-OARC Privacy Committee Report

Speakers: Benno Overeinder (NLnet Labs), Sara Dickinson (Sinodun IT)

OARC Privacy Committee Survey Results Report 20211129.pdf

3 Characterization of Anycast Adoption in the DNS Authoritative Infrastructure

Anycast has proven to be an effective mechanism to
enhance resilience in the DNS ecosystem and for scaling DNS
nameserver capacity, both in authoritative and the recursive
resolver infrastructure. Since its adoption for root servers,
anycast has mitigated the impact of failures and DDoS attacks
on the DNS ecosystem. In this work, we quantify the adoption
of anycast to support authoritative domain name service for toplevel and second-level domains (TLDs and SLDs). Comparing two
comprehensive anycast census datasets in 2017 and 2021, with
DNS measurements captured over the same period, reveals that
anycast adoption is increasing, driven by a few large operators.
While anycast offers compelling resilience advantage, it also shifts
some resilience risk to other aspects of the infrastructure. We
discuss these aspects, and how the pervasive use of anycast merits
a re-evaluation of how to measure DNS resilience.

Speaker: Raffaele Sommese

OARC-Anycast.pdf

14:45 15 Minutes Break

OARC 36 Day 1: Session 2

4 Advertising DNS Protocol Use to Mitigate DDoS Attacks

The Domain Name System (DNS) has been frequently abused for distributed denial-of-service (DDoS) attacks and cache poisoning because it relies on the User Datagram Protocol (UDP). Since UDP is connection-less, it is trivial for an attacker to spoof the source of a DNS query or response. While other secure transport mechanisms provide identity management, such as the Transmission Control Protocol (TCP) and DNS Cookies, there is currently no method for a client to state that they only use a given protocol. This paper presents a new method to allow protocol enforcement: DNS Protocol Advertisement Records (DPAR). Advertisement records allow Internet Protocol (IP) address subnets to post a public record in the reverse DNS zone stating which DNS mechanisms are used by their clients. DNS servers may then look up this record and require a client to use the stated mechanism, in turn preventing an attacker from sending spoofed messages over UDP. In this paper, we define the specification for DNS Protocol Advertisement Records, considerations that were made, and comparisons to alternative approaches. We additionally estimate the effectiveness of advertisements in preventing DDoS attacks and the expected
burden to DNS servers.

Speaker: Casey Deccio (Brigham Young University)

2021-11-29-oarc-dpar.pdf

5 A Measurement-based Investigation of DNS Hijacking

Attacks against DNS have long plagued the Internet, requiring continual investigation and vigilance to prevent the abuse of this critical infrastructure. In recent years, the severity of DNS hijacking has motivated renewed interest in developing more robust defenses. The size, dynamism, and diversity of the DNS ecosystem present nontrivial challenges to crafting an effective and scalable defense. Further, the relative rarity of documented DNS hijacking attacks makes them difficult to study in-depth. In this study, we attempt to address the challenges in two thrusts. We first conduct an analysis based on the reports of confirmed DNS hijacking attacks and passive DNS records to characterize known DNS hijacking attacks and identify features for building defense mechanisms. Then we explore the extent to which the characteristic features can be used to build a DNS hijacking detection mechanism and evaluate its effectiveness from the perspective of a network gateway.

Speaker: Shuai Hao (Old Dominion University)

oarc36-hao.pdf

6 Slack’s DNSSEC Rollout: Third Time’s the Outage

On September 30th 2021, Slack had an outage that impacted less than 1% of our online user base, and lasted for 24 hours. This outage was the result of our attempt to enable DNSSEC, but which ultimately led to a series of unfortunate events.

On this talk we'll cover our DNSSEC rollout to all Slack critical domains and the three failed attempts to enable DNSSEC on slack.com ­– doing a deep dive into our third attempt (the Sept 30th outage) – where we'll cover what was done during the outage, why we did it and ultimately the root cause of the outage, which was a bug in the DNSSEC implementation on our cloud provider authoritative DNS server.

Speaker: Rafael de Elvira (Senior Software Engineer @ Slack)

Slack DNSSEC Rollout slides- OARC36_final_final.pdf

7 Day 1 Wrap Up

Speaker: Mr Keith Mitchell (DNS-OARC)

16:10 Let's Chat! Social Event

Tuesday, 30 November

OARC 36 Day 2: Session 1

8 Measurement of DNSSEC Validation with RSA-4096

IN this presentation we motivate why there is a need to consider using larger keys for RWSA in the context of DNSSEC. We then describe a measurement experiment that looks at the success rate of using 4,096 bit signing keys in DNSSEC. We conclude with some thoughts as to the option between using ECDSA P-256 and RSA-4096 to counter the potential threat of quantum computing to this form of application of cryptography.

Speaker: Geoff Huston (APNIC)

2021-11-30-rsa4096.pdf

9 An analysis of DNSSEC signed domains appearing within a large web crawl dataset

This presentation will discuss the analysis of domain names found within the Common Crawl dataset. It will cover the method of analysis and will focus on the characteristics of DNSSEC with regard to different types of web page content. It will include discussion on the proportion of websites that use content from only signed domains, the kinds of content in websites that are more likely to be delivered via a signed domain, and the variance in different DNSSEC algorithms seen across the dataset. It is hoped this analysis will offer an interesting perspective on the rate of DNSSEC deployment within different website components.

Speaker: James Richards (Nominet)

oarc-cc-presentation-james-richards.pdf

oarc-cc-presentation-james-richards.pptx

10:45 15 Minute Break

OARC 36 Day 2: Session 2

10 Status Update on Authenticated Bootstrapping of DNSSEC Delegations

We present an update to the development and implementation of the DNSSEC Bootstrapping protocol since OARConline 35a. Protocol developments include, in particular, some structural changes to enable straightforward discovery of delegations which are ready for bootstrapping, on a per-parent basis. On the implementation side, we show prototype implementations for both the child (DNS operator) as well as the parental (registry/registrar) sphere, and give an overview of the implementation plans and status at various DNS operators and registries/registrars.

Speaker: Peter Thomassen (deSEC)

2021-11-30 OARC 36, Status Update on Authenticated Bootstrapping of DNSSEC Delegations.pdf

11 The PROXY protocol

Over the years, DNS proxies (such as dnsdist) have cooperated with
their backends to pass the real IP of the client to those backends.
edns-client-subnet has been abused for this, there was an attempt in
the IETF to standardise a new EDNS option (XPF), but DNSOP did not like
it.

Based on that, and other operational insights, PowerDNS (in dnsdist,
the Authoritative, and the Recursor) has decided to adopt the PROXYv2
protocol from the HAproxy developers (
https://www.haproxy.org/download/2.4/doc/proxy-protocol.txt), and we
know ISC has support for it on the roadmap as well.

As it would be good to have interoperability between all vendors in
this area, this talk will give a quick overview of the existing solutions, their drawbacks, and an introduction to the PROXY protocol.

Speakers: Pieter Lexis (PowerDNS.COM / Open-Xchange), Peter van Dijk (PowerDNS)

dns-proxyprotocol.pdf

12 Querying the Public Suffix List via the DNS

Information from the Public Suffix List (PSL) is required in various contexts, for example for cookie scoping in browsers, for certificate issuance, and for the secure operation of authoritative multi-tenant nameservers. Applications depending on the PSL customarily bring their own copy of the list, and thus require mechanisms to parse and interpret the list, and to keep it up to date.

The PSL Query Service removes the need for applications to parse or refresh the PSL altogether. Based on a mapping of the PSL onto the DNS, it instead facilitates single-query lookups to immediately retrieve the public suffix that is associated with a given name.

In this short contribution, I discuss the motivation, design, and applications of the PSL Query Service and explain its internal structure, before reflecting on privacy concerns that may arise. The talk ends with considerations about whether and how to better integrate the service into the Internet community.

Speaker: Peter Thomassen (deSEC)

2021-11-30 OARC 36, Public Suffix List DNS Query Service.pdf

11:55 95 Minute Break

OARC 36 Day 2: Session 3

13 Understanding DNSSEC debugging patterns using DNSVIZ.

DNS Security Extensions (DNSSEC) were introduced nearly two decades ago to provide integrity and authenticity of DNS messages. There have been some studies focusing on how DNSSEC has been deployed over years using active scans, which commonly reported pervasive mismanagement such as missing DS records.

From the domain administrator perspective, however, it is hard to understand what makes it really challenging to deploy and "manage" DNSSEC, or to fix errors; for example, answering the question of "how long do usually take for DNS administrators to resolve a specific DNSSEC error?" is nontrivial.

To shed a light on these questions, we leverage DNSVIZ (dnsviz.net), which is one of the most extensive and popular tools for debugging DNSSEC errors. It helps domain name owners and others help understand the current DNSSEC status of a domain name and diagnoses the problem if exists.

With 7 years of DNSViz dataset that contain DNS debugging histories of domains, we would like to share our preliminary findings.

Speaker: Dr Tijay Chung (Virginia Tech)

DNS-OARC-final.pdf

14 A DNS Anomaly Detection Model to Identify Unusual Lagging in Zone Updates

In large-scale DNS deployments, zone updates are made by DNS hosting services on short timescales to large numbers of servers. It’s normal for the updates to be somewhat asynchronous from each other because DNS is “eventually consistent” by design. The question for DNS operations is how much lag is OK. For our organization, customer sensitivity to stale information is high and we want to take action by asking the hosting services to check for server problems when there are anomalous lags, but which lags are anomalous? What threshold of difference in the SOA serial numbers indicates a problem?

This talk presents a project to use an unsupervised machine learning algorithm to identify the anomalous points in lag data to help with this question of what is actionable. The approach was to use the PyCaret open-source Python machine learning library to train and build a model. We chose PyCaret to start with because it includes many unsupervised ML algorithms and then we selected the Isolation Forest (iforest) clustering ML algorithm, an algorithm commonly used for anomaly detection because it requires limited memory and is relatively fast in performance.
Additionally, after the model was trained using unsupervised learning, it was fine-tuned with the probe_time of the data points given as its supervised_target. Fine-tuning performs additional training by focusing on update times to improve the model’s performance. 

The model was trained on DNS monitor logs that include information about zone updates made by DNS hosting services. This model was trained on SOA serial number changes in the multiple anycast locations for three production zones, my.salesforce.com, salesforce.com, and force.com. The output from the model marks each data point as an anomaly or normal and also provides a specific anomaly score where the higher the score, the more anomalous the datapoint. Analyzing the model output gives information regarding zone updates. This information includes learning about trends and patterns of zones, ranges of lags, lag dependence on zone sizes, etc. 

One of the results of the project was to provide good numeric thresholds for how much lag was normal, now much anomalous. The thresholds were extremely different for the three zones (see slide #7). The thresholds indicated that lagging depended on zone sizes and how frequently the zones were updated, which is not surprising, but the actual numbers were surprising. These thresholds are accurate because they mark where truly anomalous behaviors begin. The original default thresholds that were in use prior to the model were guesstimates by the team and were the same for all three zones and had not proven predictive. 

The next step of the project is to automate the model prediction by periodically running the model and integrating the new thresholds into the monitoring system (as well as preserving the history). 

ML has been used on DNS data frequently, especially focusing on DNS attacks and security. DNS anomaly detection plays a big role operationally in DDOS mitigation for example. But this work covers an area in DNS availability and it seems likely that ML models, both unsupervised and supervised, can be valuable to many other areas where DNS operators have data in real-time in the future.

Speaker: Kundana Pillari

DNS-OARC Anomaly Detection Model.pdf

15 Disaster Recovery with DNSSEC

What if you loose all your DNSSEC keys for your zone, what is the impact and how can you recover from this?

Speaker: Mr Stefan Ubbink (SIDN)

DR_with_DNSSEC-dns-oarc36.20211129.pdf

DR_with_DNSSEC-dns-oarc36.20211129.pptx

14:15 15 Minute Break

OARC 36 Day 2: Session 4

16 Independent Performance Verification of XDP-accelerated Knot DNS

Over the last year, Knot DNS has added support for Linux’s eXpress Data Path. CIRA and CENGN, with the help of CZ.NIC, took a deep dive into characterizing Knot DNS’s performance with XDP on both UDP and TCP-based queries. On a 36-core server, 16M QPS was achieved on UDP-based queries, and 1.3M QPS on TCP-based queries.

This presentation will detail the hardware, software, configurations and testing methodologies used to achieve these results. We will share the successes and failures we experienced along the way. We also propose a unit of measurement to allow rough performance comparisons between different processor types: QPS/core-GHz.

Speaker: Mr Brad Ellis (CENGN (Centre of Excellence in Next Generation Networks))

Slide decks CENGN OARC-DNS 36.pdf CENGN OARC-DNS 36.pptx

17 Lightning Talk: NSEC breakage survey

Some DNS authoritative servers provide incorrect proofs of non-existence which correctly DNSSEC-validate but deny existence of data which actually do exist in the zone. Consequently, this causes silent resolution failures on resolvers which implement aggressive use of DNSSEC-validated cache (RFC 8198).

This talk aims to provide very short glimpse to where the breakage can be found in wild on the Internet.

Speaker: Petr Špaček (Internet Systems Consortium (ISC))

nsec-bitmaps.pdf

18 Lightning Talk: Bringing up DoH

Bringing up DoH services at OpenDNS

• Phase 1 - DoH as a proxy
• Phase 2 - Native support
• Phase 3 - Native deployment

Speaker: Brian Somers (OpenDNS/Cisco)

Bringing Up DoH.pptx

19 Wrap-Up

Speaker: Mr Keith Mitchell (DNS-OARC)

OARC36-wrapup.odp

Wrap-Up