OARC 41 (DaNang, Vietnam)Workshop
Meliá Danang Beach Resort
OARC 41 will be a hybrid in-person and online workshop co-located with ICANN DNS Symposium and hosted by VNNIC in Da Nang, Vietnam
 |
Dragon Bridge - Cầu Rồng
DNS-OARC is a non-profit, membership organization that seeks to improve the security, stability, and understanding of the Internet's DNS infrastructure. Part of these aims are achieved through workshops.
DNS-OARC Workshops are open to OARC members and to all other parties interested in DNS operations and research.
Social Media hashtag: #OARC41
Mattermost Chatroom: Workshops on chat.dns-oarc.net (sign-up here)
OARC ANNUAL WORKSHOP PATRON 2023
 |
Annual Workshop Patronages are available. Details at:
https://www.dns-oarc.net/workshop/patronage-opportunities
OARC 41 HOST
 |
 |  |
OARC 41 Workshop Patronages are available. Please contact us for details.
-
-
09:00
In-person attendees registration
-
10:00
Session 1 start
-
OARC 41 Day 1: Session 1
-
1
Welcome to OARC 41Speaker: Denesh Bhabuta (DNS-OARC)
-
2
Deleting your domain? Preventing data leaks at TLD scale
We present LEMMINGS (an acronym derived from "deLetEd doMain MaIl warNinG System"), which has been developed at SIDN for warning former owners of deleted domains when their domain is likely still being used for sending email. In this presentation, we present the system and results based on real-world data collected while running the system for a nine-month period and analysing over 600,000 domains.
When a .nl domain is deleted, it enters a 40-day grace period, after which it becomes available for general registration again. A malicious actor may re-register this domain with the intent of collecting email traffic still being sent to the domain. The received email may contain highly confidential and privacy-sensitive data, such as medical records. We have seen real-world examples in the Netherlands, where this was the case. For example, when domains were deleted by the Dutch Police and healthcare organizations.
We use DNS data captured at the authoritative DNS name servers for the .nl ccTLD to determine the probability that a deleted domain is still being used for email transactions. By analyzing over 4 billion DNS requests daily, we are able to calculate the probability that email is likely still being sent to deleted domains. There may also be other reasons for the mail-related DNS lookups, and there exists noise in the form of SPAM mail. The system we developed therefore contains filters for removing the noise and only sending a warning to former owners of deleted domains when there is a good indication email is still being sent to email addresses linked to a deleted domain.
Speaker: Maarten Wullink (SIDN) -
3
Introducing DNSWatch, Meta's DNS Snooping utility
Introducing DNSWatch, Meta's DNS snooping utility.
This tool is built on top of BPF and Go, and enables analysing DNS usage.
It comes with a very handy top-like interface, which can be used to get a high level overview of dns query activity on the system, and a detailed dig-like view, to be able to deep-dive into said queries.
Also comes with a handy Prometheus exporter, which exposes per process dns usage metrics in the Prometheus format.https://github.com/facebookincubator/dns/tree/main/dnswatch
Detailed slides will follow, alongside some live demos
Speaker: Balint Csergo (Meta)
-
1
-
11:00
Mid-morning break
-
11:30
Session 2 start
-
OARC 41 Day 1: Session 2
-
4
Automation of DNSSEC provisioning and maintenance
One area of frequent challenge is how to fully automate DNSSEC provisioning and maintenance operations (e.g. initial provisioning, SEP key rollover and maintenance, and when required, inter-provider transfer). Steve Crocker and Shumon Huque have been running a panel on this topic for a number of years at the ICANN DNSSEC and Security workshops, and we'd like to share the collected experience from those workshops with the DNS-OARC community. This talk will provide a technical overview of this area, a summary of the state of DNSSEC automation adoption and implementation in the industry, and a preview of upcoming relevant protocol enhancements. It will also cover potential policy and contractual impediments that have hampered more rapid progress in this space for the generic Top Level Domains (gTLD), and prospects for improving the situation. Lastly, we will suggest some ways the OARC community can actively participate and help with these efforts.
Speaker: Shumon Huque (Salesforce) -
5
March towards shorter DNSSEC outages
Time-To-Live values in DNS are a controversial topic, riddled with counter-intuitive behavior.
Recently, a desire to lower the mean time to recovery from DNSSEC-related problems reignited discussion about the TTL values of DS and DNSKEY records. Can DS TTL be lower? What if we tried just 5-minute TTL? How would it impact users (mainly response latency) and operators (query rate seen on authoritative servers)?
In this talk, we present a comparative analysis of DNS resolver performance. We use DNS Shotgun to replay real (anonymized) traffic and compare two configurations of a DNS resolver:
- The first configuration respects TTL values as supplied by authoritative servers;
- The second configuration artificially limits TTL values of DS and DNSKEY records to 5 minutes.
This experiment demonstrates the non-linear relationship between TTL values, response latency, and query rate.
Based on our data set, we conclude that the performance impact of 5-minute DS and DNSKEY TTL, in terms of DNS latency visible to end clients, is negligible. The number of DS and DNSKEY queries to authoritative servers is higher but well below the linear increase.
Speaker: Petr Špaček (Internet Systems Consortium (ISC))
-
4
-
12:25
Lunch
-
13:55
Session 3 start
-
OARC 41 Day 1: Session 3
-
6
Lessons learned from the 2nd DNSSEC Key Rollover in .vn.
1) DNSSEC Implementation for .VN In Brief
2) 2nd DNSSEC Key Rollover in .vn
3) Lessons Learned
4) Some IssuesSpeaker: Tri Nguyen (VNNIC) -
7
Migrate from PowerDNS to Knot with help of Catalog Zones
With "RcodeZero Anycast DNS" we provide Authoritative DNS for millions of zones. We used to use PowerDNS Authoritative Nameserver with Database Backend, which is a great tool for hosting millions of zones. But the ease of provisioning comes with the drawback of poor performance for random subdomain queries. In the last few years the number of these random subdomain queries increased from once a month to 24x7. To handle the query load of such random subdomain "attacks", we opted for Knot DNS, which has the reputation of fastest open source name server. At the same moment, luckily catalog zones popped up and we decided to use them for provision the zones on Knot. Although everthing sounded straight forward, migrating from PowerDNS to Knot was a long way with plenty of pitfalls. And even as we did it, operating Knot is very different than operating PowerDNS.
This talk will discuss the reason to migrate to Knot, the migration problems and how we solved them with the help of the Knot developers. We will also talk about unsolved problems and alternative solutions.
Speaker: Klaus Darilion (nic.at GmbH) -
8
Strengthening DNS Security and IPv6 Implementation in Bangladesh: Overcoming Operational Challenges
The digital landscape in Bangladesh, as a prominent AP regional country, presents a dynamic arena for the adoption of IPv6 and the implementation of DNS Security Extensions (DNSSEC).
This presentation focuses on the operational challenges faced within Bangladesh's network infrastructure and shares insights into navigating the complexities of DNS security and IPv6 integration.
Bangladesh's network ecosystem comprises a diverse range of stakeholders, including a prominent ccTLD registry, numerous hosting companies, ISPs, and data centers. Recent accomplishments include the successful DNSSEC deployment by the ccTLD registry and incremental IPv6 upgrades within select ISPs and data centers. While these milestones demonstrate progress, operational hurdles remain that necessitate strategic approaches and collaboration.Drawing inspiration from the OARC 41 call for contributions and tailored to the AP region, this session will dive into the following localized areas:
1. Operational Challenges: Highlighting operational lessons learned from past incidents, addressing operational obstacles specific to Bangladesh's context, and sharing strategies for enhancing operational efficiency and tooling.
2. DNSSEC Implementation: Showcasing the ccTLD registry's successful journey in deploying DNSSEC, and sharing insights.
3. IPv6 Integration: Demonstrating the gradual integration of IPv6 within Bangladesh's network landscape, leveraging case studies from local ISPs and data centers, and addressing local challenges such as skill gaps and funding constraints.
By spotlighting real-world examples and practical solutions from Bangladesh, this presentation will offer a nuanced understanding of the unique challenges faced within the country's network ecosystem. Attendees will gain insights into the strategies employed to overcome these challenges, foster operational excellence, and drive DNS security and IPv6 implementation forward in the AP region.With a focus on regional expertise, this presentation invites attendees to engage in discussions to share experiences, challenges, and solutions. By leveraging the power of collaboration and knowledge exchange, we can collectively contribute to a more secure, resilient, and future-ready digital infrastructure for Bangladesh.
Speaker: Mr Imran Hossen (EyHost Ltd.)
-
6
-
15:15
Mid-afternoon break
-
15:45
Session 4 Start
-
OARC 41 Day 1: Session 4
-
9
Intercept and Inject: DNS Response Manipulation in the Wild
DNS is a protocol responsible for translating human-readable domain names into IP addresses. Despite being essential for many Internet services to work properly, it is inherently vulnerable to manipulation. In November 2021, users from Mexico received bogus DNS responses when resolving whatsapp.net. It appeared that a BGP route leak diverged DNS queries to the local instance of the k-root located in China. Those queries, in turn, encountered middleboxes that injected fake DNS responses. In this paper, we analyze that event from the RIPE Atlas point of view and observe that its impact was more significant than initially thought—the Chinese root server instance was reachable from at least 57 probes (in 15 countries) several months before being reported. We then launch a nine-month longitudinal measurement campaign using RIPE Atlas probes and locate 11 probes outside China reaching the same instance and receiving bogus responses, although this time over IPv6. More broadly, motivated by the November 2021 event, we study the extent of DNS response injection when contacting root servers. While only less than 1% of queries are impacted, they originate from 7% of RIPE Atlas probes in 66 countries. We conclude by discussing several countermeasures that limit the probability of DNS manipulation.
Speaker: Ms Yevheniya Nosyk (Grenoble Institute of Technology) -
10
Transitioning Verisign's TLDs to Elliptic Curve DNSSEC
In this short presentation we describe plans to transition the .com, .net, and .edu zones to elliptic curve DNSSEC. We'll provide a very high level description of the algorithm rollover and an expected schedule for each of the TLDs.
Speaker: Duane Wessels (Verisign) -
11
Deploying ZONEMD in the Root Zone
Verisign and ICANN are planning to introduce a ZONEMD record to the root zone before the end of this year. In this short presentation we'll go over the deployment plan and expected schedule for deployment.
Speaker: Duane Wessels (Verisign) -
12
DNS4EU scope. timeline and challenges
DNS4EU will go far beyond another public DNS resolver. During the presentation the detailed scope and timeline will be introduced. Besides the public resolvers, the presentation will also focus on specific plans for telcos, governments and threat intelligence research and application.
Each use case will focus on different technology aspects around DNS resolver operations and the added layer of security protection. Individual challenges and approaches chosen by the DNS4EU consortium will be introduced. Topics will cover privacy aspects, data governance, operations, threat intelligence research and particular aspect of DNS standard usage.
Speaker: Robert Šefr
-
9
-
09:00
-
-
09:30
Registration
-
10:00
Session 1 start
-
OARC 41 Day 2: Session 1
-
13
The role of DNS in residential Internet use
DNS is designed to drive Internet traffic, directing users to the IP addresses hosting Internet services. Modern, replicated services use DNS to direct users to desired (low latency, available) servers, and DNS uses caches to limit DNS overhead on users and services. However, little is known about how these interactions play out in practice, partly due to the proprietary nature of traffic traces.
In this talk, we will discuss DNS in the context of the Internet service traffic that it does (and does not) drive for residential users. Our results are based on a trace of 500 apartments, for which we join DNS queries, traffic flows, and services (e.g., Netflix). We find that DNS caching is effective at reducing overhead, with little DNS latency relative to the overall service latency. However, that effectiveness comes at a cost of responsiveness, with significant volumes of traffic using IP addresses from very stale cached DNS records (e.g., 6.5% of flows start more than 1 minute after the DNS TTL expired, and 80% of Microsoft traffic is sent more than 5 minutes after TTL expiration), limiting the agility of services to steer traffic for performance, load balancing, and resilience to failures. This significant staleness occurs before the recursive resolver, meaning client-side caches are to blame. Further, we find that 12% of traffic does not use DNS lookups. We develop inference techniques to identify half of that traffic as peer-to-peer -- 17% of connections overall! -- pointing to a possible resurgence of P2P for emerging use cases.
We will conclude by briefly presenting service redirection techniques we have developed to provide services with better tradeoffs than unicast+DNS redirection (precise steering, but poor responsiveness due to DNS caching and TTL violations) and anycast (imprecise steering but good resilience).
Speaker: Shuyue Yu -
14
Implementing EDNS Client Subnet in a privacy preserving way
If you're operating a recursive DNS resolver, you're likely interested in implementing the EDNS Client Subnet, as it's a vital component many top-tier CDN providers use for geo-balancing.
However, simply forwarding the user's subnet to name servers carries serious privacy implications. I'll share how AdGuard DNS navigates this challenge, ensuring we protect user privacy, deliver the best possible responses, and keep the cache size within reasonable limits.
Speaker: Andrey Meshkov
-
13
-
10:55
Mid-morning break
-
11:25
Session 2 start
-
OARC 41 Day 2: Session 2
-
15
B.root-servers.net renumbering in Nov. 2023
USC/ISI is renumbering both its IPv4 and IPv6 addresses for b.root-servers.net on 2023-11-27. Our new IPv4 address will be 170.247.170.2 and our new IPv6 address will be 2801:1b8:10::b. USC/ISI will continue to support root service over our current IPv4 and IPv6 addresses for at least one year (until 2024-11-27) in order to provide a stable transition period while new root hints files are distributed in software and operating system packages.
We are renumbering to increase the resilience of the Root Servers System by further diversifying the number of Regional Internet Registries (RIRs) that have allocated IP addresses to Root Server Operators. Our addresses will be the first in the Root Server System to have been allocated by LACNIC and our routes will be verifiable through LACNIC’s Resource Public Key Infrastructure (RPKI) Trust Anchor Location (TAL). We thank LACNIC for helping make this renumbering possible, and ARIN for supporting our prior addressing assignments.
This presentation will discuss the technical details to make sure the OARC audience is informed, and will include some of the rational/details. It may be co-presented by USC/ISI and LACNIC members -- presenters still being worked out. USC/ISI members will need to be remote while presenting unfortunately. It should be a short presentation in general (10 minutes?).
Speaker: Robert Story (USC/ISI) -
16
InternetNZ .nz DNSSEC Incident
InternetNZ faced its biggest crisis involving DNSSEC.
Much has been learnt from this incident since it occurred.
This talk aims to share the pains and lessons learnt from this challenging situation, with the hope other DNS operators never experience a similar issue.
An overview of what we dealt with technically as well as giving more context with a macro view of the incident.Speaker: Mr Felipe Barbosa (InternetNZ) -
17
Silence is not Golden: Disrupting the Load Balancing of Authoritative DNS Servers
Authoritative nameservers are delegated to provide the final resource record. Since the security and robustness of DNS are critical to the general operation of the Internet, domain owners often deploy multiple candidate nameservers for load balancing according to the requirement of DNS specifications (RFC 1034 and RFC 2182). Once the load balancing mechanism is compromised, an adversary can manipulate a large number of legitimate DNS requests to a specified candidate nameserver. As a result, it may bypass the defense mechanisms used to filter malicious traffic that can overload the victim nameserver, or lower the bar for DNS traffic hijacking and cache poisoning attacks.
In this study, we report on a class of DNS vulnerabilities and present a novel attack, named Disablance, that targets the domains with different NS records severing to multiple sites of authoritative servers. The attack is made possible by a prevalent misconfiguration of nameservers that ignores domains outside their authority, combined with the mainstream implementation of recursive resolvers that use a globally shared status for nameserver selection. By targeting authoritative nameservers configured by a large number of domains (e.g., the nameservers owned by DNS hosting services), Disablance allows adversaries to stealthily sabotage the DNS load balancing for authoritative nameservers at a low cost. By simply configuring the DNS records for a domain under their control to point to the targeted nameservers and performing a handful of requests, adversaries can temporarily manipulate a given DNS resolver to overload a specific authoritative server. Therefore, Disablance can redirect benign DNS requests for all hosted domains to the specific nameserver and disrupts the load balancing mechanism. Our extensive study proves the security threat of Disablance is realistic and prevalent. First, we demonstrated that mainstream DNS implementations, including BIND9, PowerDNS, and Microsoft DNS, are vulnerable to Disablance. Second, we developed a measurement framework to measure vulnerable authoritative servers in the wild. 22.24% of top 1M FQDNs and 3.94% of top 1M SLDs were proven can be the victims of Disablance. Our measurement results also show that 37.88% of stable open resolvers and 10 of 14 popular public DNS services can be exploited to conduct Disablance, including Cloudflare and Quad9. Furthermore, the critical security threats of Disablance were observed and acknowledged through in-depth discussion with a world-leading DNS service provider. We have reported discovered vulnerabilities and provided recommendations to the affected vendors. Until now, Tencent Cloud (DNSPod) and Amazon have taken action to fix this issue according to our suggestions.
Speaker: Fenglu Zhang (Tsinghua University)
-
15
-
12:25
Lunch
-
13:55
Session 3 start
-
OARC 41 Day 2: Session 3
-
18
Building a DNS Resolver in Fedramp
This presentation will focus on our journey towards building a FedRAMP version of OpenDNS/Cisco Umbrella resolver, all the challenges we encountered along the way and the strategy we took to overcome them.
Agenda:
- Our experiences (struggles) with moving to OpenSSL3 and using the FIPS provider within it.
- How we support both commercial (openssl 1.1.1) and FedRAMP (openssl 3) resolvers from the same codebase
- What does FIPS means for DNSSEC?
- What does FIPS means for DoH, DoT and DNSCrypt?
- Restrictions imposed by the environment we have to operate in.Speaker: Prashanth Suvarna (Cisco Systems/OpenDNS) -
19
TsuKing: Coordinating DNS Resolvers and Queries into Potent DoS Amplifiers
In this talk, we present a new DNS amplification attack named TsuKing. Instead of exploiting individual DNS resolvers independently to achieve an amplification effect, TsuKing deftly coordinates numerous vulnerable DNS resolvers and crafted queries together to form potent DoS amplifiers. We demonstrate that with TsuKing, an initial small amplification factor can increase exponentially through the internal layers of coordinated amplifiers, resulting in an extremely powerful amplification attack. TsuKing has three variants, including DNSRetry, DNSChain, and DNSLoop, all of which exploit a suite of inconsistent DNS implementations to achieve an enormous amplification effect. We conducted comprehensive measurements and evaluations to demonstrate the feasibility of TsuKing. In particular, we found that about 11.7% of 1.3M open DNS resolvers are potentially vulnerable to being exploited by TsuKing. And real-world controlled evaluations indicated that adversaries can achieve an amplification factor of at least 3,700×. We have reported the above vulnerabilities to all relevant vendors and also provided them with our recommendations for mitigation. We have received positive responses from 5 vendors, such as Unbound, confirming the issues, and got 3 CVE numbers. Some of the vendors are actively implementing our recommendations.
Speaker: Fenglu Zhang (Tsinghua University) -
20
DNSSEC for end-users
In these slides, we describe some technical details about DNSSEC implementation and promotion DNSSEC to end-users.
We are giving brief overview of our online visualization ttools and browser extension and sharing future plans.Speaker: Azizbek Kadirov (Uzinfocom)
-
18
-
15:15
Mid-afternoon break
-
15:45
Session 4 start
-
OARC 41 Day 2: Session 4
-
21
XFR does not scale to millions of zones
The standard way for transmitting zone updates from a Primary Nameserver to a Secondary Namserver is NOTIFY+XFR, with out-of-protocal zone provisioning on the Primary and the Secondary. The rather new catalog zones helped to automate the zone provisioning on the Secondary, but zone updates still use NOTIFY+XFR with additional freshness checks performed by Secondary. This works fine when hosting hundreds or thousands of zones, but creates challenges when hosting millions of zones: NOTIFYs and XFRs may fail, leading to inconsistent Secondarys. And even worse, as the Primary does not know the state on the Secondary, and vice versa, out of protocol monitoring is needed to verify that a Secondary is in sync with the Primary and potentiall fix inconsistencies. A new approach should ensure that all zones updates reliable end up in the Secondary and ease the monitoring.
This talk will highlight the current problem when having Secondaries with millions of zones, and want's to start a discussion for new approaches to keep Secondaries in sync with their Primaries.
Speaker: Klaus Darilion (nic.at GmbH) -
22
Encrypted DNS in the Network of Deutsche Telekom: Status and Outlook
The talk with cover the DNS platform of Deutsche Telekom (architecture, performance, features) and the implementation status of encrypted DNS (mainly DoT and DoH) with Deutsche Telekom Group. It will give an overview about challenges arising from DNS discovery (DDR) with a focus on home networks. It will also cover the potential impact of Encrypted Client Hello in the context of encrypted DNS.
Speaker: Mr Nicolai Leymann (Deutsche Telekom) -
23
There Are Problems With DNS Recursive Service
There are problems brewing. On-going and increasing problems with privacy leaks is the most talked about but there is also a growing use of DNS queries for malicious software to communicate and ping home.
These problems prompt a response and various, mostly commercial, efforts are underway or have already been deployed. None of them, however, present a truly open and transparent approach which we argue is needed.
In Sweden the government recently approved funding for the DNS TAPIR project which aim at such an open solution. The project has barely been initiated, so this talk is mostly about the problem space rather than any particular results.
Speaker: Johan Stenstam (Swedish Internet Foundation) -
24
A view of NXDOMAIN and DNSSEC from public DNS
A view of dnssec usage in query stream of a popular open resolver
Speaker: Joao Luis Silva Damas (APNIC/Bond Internet Systems) -
25
OARC 41 - Closing RemarksSpeakers: Denesh Bhabuta (DNS-OARC), Shumon Huque (Salesforce)
-
21
-
09:30