OARC 2015 Fall Workshop (Montreal)

3 Oct 2015, 08:00 → 5 Oct 2015, 17:15 US/Eastern

St-Francois (Fairmont Queen Elizabeth)

Keith Mitchell (DNS-OARC), Sebastian Castro (NZRS)

DNS-OARC's Fall 2015 Workshop was co-located with the NANOG 65 meeting in Montreal and sponsored by:

 

Gold and Social Sponsor

 

Bronze Sponsor

 

DNS-OARC Workshop meetings are open to OARC members and to all other parties interested in DNS operations and research, with NANOG and ARIN attendees particularly welcome this time around. Attendance is free for OARC Members, Speakers and Sponsors. There are charges for other attendees and late registrations.

If your organization is interested in sponsoring OARC workshops, please see our Sponsor Benefits or e-mail sponsor@dns-oarc.net for more information.

AGM Documents

• 2014 AGM Minutes
• 2014 Financial Statements
• Proxy Voting Form

Slides

• 2015 Treasurer Report
• Social Information Slide
• Workshops static slides

Saturday, 3 October

10:00 → 12:30 Members Session

Agenda OARC 2015 AGM Agenda

10:00 Introduction from OARC Chairman

Speaker: Mr Ondrej Filip (CZ.NIC)

10:05 OARC President's Report

Speaker: Mr Keith Mitchell (DNS-OARC)

Slides OARC-AGM-report_15.odp OARC-AGM-report_15.pdf

10:25 OARC Treasurer's Report

Speaker: Duane Wessels (Verisign)

Slides OARC-2015-Treasurer-Report.pdf

10:35 OARC Board Elections

Web Page https://www.dns-oarc.net/oarc/agm-2015/elections

10:45 Morning Coffee Break

11:15 OARC Systems Report

Speaker: Mr William Sotomayor (DNS-OARC)

Slides 201510_OARC_Systems_Update.pdf

11:35 OARC Projects Update

Speaker: Ms Dalini Khemlani (DNS-OARC)

Slides DNS-OARC_Projects_Update_201510.pdf

11:55 DNS Software Test Centre Proposal

Speaker: Mr Ralf Weber (Nominum Inc)

Slides Weber-OARC-DNS-Testbed.pdf

12:30 → 14:00 Lunch

14:00 → 15:30 Public Workshop: DNSSEC Track

Convener: Paul Ebersman (Comcast)

14:00 An Overview of DNS Privacy Mechanisms

As part of the IETF's efforts to secure all protocols against pervasive surveillance, several privacy enhancements to the DNS are actively being developed with prototype implementations of such enhancements also emerging. This talk will provide a technical description of these mechanisms as well as deployment challenges and related considerations. Topics to be covered will include query-name minimization, DNS over TLS/DTLS and other encryption proposals, payload padding, etc. We will also cover related efforts to minimize leakage of DNS names in other protocols.

Speakers: Allison Mankin (Verisign Labs), Mr Shumon Huque (Verisign Labs)

Slides dns-privacy.pdf dns-privacy.pptx

14:30 Using TLS for DNS privacy in practice

This talk will focus on the existing prototype implementations of DNS-over-TLS and dive into some of the finer points of using TLS in practice. This will include authentication issues, performance considerations, TCP connection management, DoS mitigation and a demonstration. It will also discuss the current best practices for using TLS in applications and the upcoming developments in TLS 1.3.

Speaker: Sara Dickinson (Sinodun IT)

Slides OARC_Fall2015_DNS-over-TLS.pdf

15:00 Next Steps in DANE Adoption

This talk will discuss upcoming and future steps envisioned to increase the adoption of DNSSEC and DANE (DNS-based Authentication of Named Entities) by Internet applications. It will start by providing an overview of the current state of adoption of DANE, and then discuss challenges faced by some application communities and some applications for which DANE doesn't yet provide a solution. Among the topics covered will be a proposed new TLS extension to allow servers to deliver a DANE record and the associated DNSSEC chain to clients, a mechanism to allow the use of TLSA records for client authentication, and others.

Speaker: Mr Shumon Huque (Verisign Labs)

Slides huque-dane.pdf

15:30 → 16:00 Afternoon Coffee Break

16:00 → 17:50 Public Workshop: Benchmarking Track

Convener: Liam Hynes (Dyn)

16:00 Benchmarking of authoritative DNS servers and DNSSEC impact assessment

CZ.NIC Labs created and continues to actively develop Knot DNS authoritative DNS server. The development team puts substantial effort into optimizing the server performance and searching for new optimization opportunities. So we created a DISTEL-based lab for benchmarking not only our server but for comparing many different authoritative DNS servers and versions. The presentation shows our method for collecting data, explain statistics that we use for testing hypotheses about the server performance and presents results for Knot 2.0 and others with regard to mixed DNSSEC and non-DNSSEC traffic.

Speaker: Mr Tomas Hlavacek (CZ.NIC, z.s.p.o.)

Slides dns-benchmarking.pdf

16:30 Impact of unknown EDNS options on the DNS

The EDNS (Extension mechanisms for DNS) protocol allows us to add new features to DNS that were not envisioned when DNS was originally specified. DNSSEC, Client-subnet Identifier and DNS cookies are applications that use EDNS. It appears from ISC's testing that a significant percentage of sites that support EDNS do not respond well to unknown EDNS options. The failure mode can be as severe as disabling EDNS (breaking DNSSEC). We are reluctant to encourage the use of new EDNS options until there is better tolerance for unknown EDNS options in the DNS. We would like to raise awareness of the issue, and find out what the community thinks we should do to address it. This presentation will review the [results of our testing][1] and the current EDNS failure modes we see, and explain how to [test your own site][2] for compliance. [1]: http://ednscomp.isc.org/ [2]: http://ednscomp.isc.org/ednscomp

Speaker: victoria risk (isc)

Slides DNS-OARC-EDNS-Compliance.pdf

16:45 Benchmarking and profiling DNS systems with modern Linux tools

This talk will outline the use of tools from the netsniff-ng toolkit and the Linux kernel along with a home-grown benchmark harness to characterize UDP DNS performance. These tools operate very differently from "traditional" utilities like dnsperf/resperf and produce very different results, possibly contradicting conventional wisdom that UDP on Linux is slow.

Speaker: Robert Edmonds (Farsight Security, Inc.)

Slides benchmarking.pdf

17:00 Impact of DNS over TCP - a resolver point of view

Using traffic captured at two different ISP's recursive resolvers we analise the potential impact on the servers of long lived TCP sessions, investigating the effect of timeout settings, the total number of simultaneous connections that would be kept open and the potential benefits of connection reuse as proposed in the current version of draft-ietf-dnsop-5966bis, with the intent of offering simulated operational advice, based on observerd traffic. The study looks at the impact on the recursive server as it queries authoritative servers as well as while it talks to stubs, two very different aspects of the life of a recursive server.

Speaker: Mr Joao Luis Silva Damas (Bond Internet Systems)

Slides DNS_over_TCP_resolvers.pdf

17:30 Idea: DNS over QUIC / zone transfer over QUIC or TLS/TCP

The presentation discusses just an idea about DNS over QUIC and zone transfer over QUIC or TLS/TCP. The third transport of DNS may be QUIC. Both DNS and QUIC use UDP and port 53/UDP may be possible to share. (If possible, implementation status will be reported, but it seems hard.) And zone transfers may be performed over QUIC or TLS/TCP transport with server certificate authentication.

Speaker: Mr Kazunori Fujiwara (Japan Registry Services Co., Ltd)

Slides fujiwara-02.pdf

18:00 → 21:00 Social Event

Sponsored by CIRA

Sunday, 4 October

09:00 → 10:30 Public Workshop: Operations Track

Convener: Mr Sebastian Castro (NZRS)

09:00 OpenDNS; Managing DDoS Attacks

Open resolvers will always be a target for abuse either as an attack amplification point or as a mask of the attack source. This presentation discusses the measures that OpenDNS has put in place to ensure that their open resolvers contribute towards reducing or blocking DDoS attacks. It goes on to discuss future plans to identify limit or block DoS sources.

Speaker: Mr Brian Somers (OpenDNS, FreeBSD)

Slides dns-oarc-2015_-_Managing_DDoS_Attacks_-_Slides_-_NOLOGO_v2.pdf

09:30 Continuous Integration & Continuous Deployment - For the new nameserver infrastructures of DENIC eG

This Abstract describes the concepts, the development and the functionalities of the DENIC DNS Countinuous Integration and Deployment Pipeline. Furthermore the advantages you could receive through this technics and automated testing. More informations will follow in the summary form.

Speaker: Mr Christian Petrasch (DENIC eG)

Slides

• CI_CD_DENIC_dns_infrastructure.odp
• CI_CD_DENIC_dns_infrastructure.pdf
• CI_CD_DENIC_dns_infrastructure.ppt

10:00 Neutering ANY queries: how we did it

DNS ANY queries are a source of controversy and strong feelings. In practice ANY queries are used for debug purposes, but are frequently abused in amplification attacks, as they give the best amplification factor. In some non-traditional DNS authoritative servers the cost of answering ANY queries can be high due to multiple DB lookups and dynamic records. Once in a while someone thinks that using ANY query is a good way to reliably get all RRsets in one query, frequently without understanding the semantics or implications. We have explored a number of alternatives to answer ANY queries without breaking any deployed systems, and at the same time discourage the use of ANY query. In this talk we will cover the alternatives and present our solution to a cacheable, non-breaking “negative” answer to ANY queries.

Speaker: Mr Ólafur Guðmundsson (CloudFlare Inc.)

Slides Neutering_ANY_OARC-2015-10.pdf

10:30 → 11:00 Morning Coffee Break

11:00 → 12:30 Public Workshop: Data Analysis

Convener: Mr Sean Stuart (Verisign)

11:00 Internet Performance Impacts of Canadian Content Hosting

In addition to driving ccTLD growth, a strong national content hosting industry improves consumer experience by making content faster, cheaper, and more accessible. Ironically, the existence of a large content-hosting industry next door has tended to artificially reduce the percentage of both Canadian and Mexican content that is domestically hosted. In this talk, we'll examine the most popular domains in Canada (using Alexa ranking data), as well as the much broader spectrum of domains hosted in .CA. We'll utilize BGP routing and IP geolocation to assess the geographies and key providers that support Canada's current state of domestic versus international content placement. Beyond this basic census, we'll also examine some of the potential performance impacts of nonlocal hosting on content consumers in both Eastern and Western Canada.

Speakers: Jim Cowie (Dyn, Inc.), Matt Larson (Dyn, Inc.)

Slides Cowie_OARC_Oct_2015.pdf

11:30 DNS big data analytics

#Introduction As the operator of the .nl ccTLD, SIDN is very interested in keeping the .nl zone as safe as possible. Analyzing the query data can help to detect cybercrime activity in the .nl zone which we can than try to cleanup. Traditional DNS query data analysis done by storing data as PCAP's and analyzing them with tools such a tshark and wireshark is often a slow and painful process. When storing DNS query data as PCAP files makes you will quickly run into performance and scalability problems. Most tooling used to analyze PCAP's is single threaded and has limited or no sql compatibility. What is required is a system which can cope with large volumes of PCAP data and still offer good query performance. That's why SIDN developed a DNS big data platform called ENTRADA, this platform is built on top of the Hadoop stack using open source technology. DNS query data from our authoritative name servers is stored on this platform and can be analyzed using multiple interfaces and languages. The system supports SQL, which means that anyone with SQL knowledge can quickly start analyzing the query data. Currently the database contains over 64 billion DNS queries, each day some 130 million new queries are added and this number will grow as we hook up more name servers. In this presentation I will be talking about system design, use cases and our experiences. #Platform design The platform at SIDN is used by the R&D team and is quite small (5 nodes) The costs of setting up such a cluster are very modest, the main components are as expected hardware and people. The hardware does not have to be enterprise grade and much of the required knowledge is available for free online. Adding more storage and compute capacity is as simple as adding more disk drives or servers. The cluster storage capacity at the moment is about 100 billion DNS queries and this data can be queried very efficiently. Depending on the type of query and number of data partitions that have to be scanned, most queries will return a result within seconds. #Privacy Privacy is an important aspect when collecting DNS data, because the query data might reveal personal information about the users who are sending DNS queries. The IP address of a client can in some cases be used to identify and track users (for a home user operating a private resolver, or for small shared resolvers) We designed a novel privacy framework (1) because it introduces privacy management to the use of DNS data and (2) because, to that end, it integrates legal, organizational and technical aspects of privacy management. This is described in our paper: https://www.sidnlabs.nl/uploads/tx_sidnpublications/SIDN_Labs_Privacyraamwerk_Position_Paper_V1.4_ENG.pdf #Workflow The time it takes from a query being received on the name server until it is available in the database for analysis is just a couple of minutes. The steps involved are: - get pcap data every x minutes from NS - PCAP conversion - enrichment of data - storage - query! #Storage There are a lot of different storage technologies, we chose to use the Parquet format to encode the data and Hadoop HDFS as a distributed storage layer. This part explains why Parquet is such a good fit for storing DNS data. - Why we chose Parquet - Size difference ( pcap vs parquet, total database size) - How do you convert pcap data to parquet (write parquet with Avro schema (KiteSDK)) - Parquet format can be read by Impala but also by Spark, this makes it very flexible. #Query engines and interfaces The data stored in the system can be access through multiple query engines and interfaces. The support workloads from a simple sql query to advanced graph and machine learning jobs. Impala/Impyla (SQL engine) Spark (SQL/Graph/Machine learning engine) Hue (SQL web interface) Jupyter (python notebook) #use cases Focused on increasing the security and stability of .nl - DNS security App (visualize traffic patterns for phishing domain names) - Botnet detector (detect botnet infections and report these to abuse information exchange (https://www.abuseinformationexchange.nl/english)) - Real-time Phishing domain name detection - Statistics dashboard (stats.sidnlabs.nl) - Scientific research (collaboration with Dutch Universities) - Ad-hoc operational analysis (quick analysis of current issues in the DNS) #experiences Our experiences in working with this data: So much work to be done when this data is available, we hired an additional Data scientist. Future work: - Combine passive data from .nl authoritative name server with active scans of the complete .nl zone and ISP data. - Adding more name servers and resolvers. - Open data interface #Summary 1. We believe that our choice of technology combined with our privacy framework is quite novel. 2. Our setup proves that a big data platform can start small with limited costs and still be powerful. 3. We provide the rational behind our architectural decisions with regards to tools, workflow and data formats for storage. 4. We provide example use cases of what is possible when this data is available for analysis.

Speaker: Mr M Wullink (SIDN)

Slides DNS-OARC-Montreal-2015-final.pdf DNS-OARC-Montreal-2015-final.pptx

12:00 Cluster the long tailed domains base on passiveDNS.cn

Cluster the DNS domains is a basic but very important work in analyzing the dizzy businesses of the Internet. Only based on the accurate clustered domain result, we can discern and analyze all kinds of DNS data. Now, most of the works focus on the domain structure and hoping finding the relationships among kinds of domains. Recently, based on the largest public passiveDNS database in China, we are exploring some new but beneficial ways on cluster the long tailed domains(based on some filter rules). Except the domain structure, we add two dimensions: client and server data. Introduce the real data of up-down stream is a big extension, of course it's more accurate. From the test result, the two dimensions is helpful in clustering the domains and finding the both benign and malicous domain communities.

Speaker: Mr zaifeng zhang (QIHOO 360)

Slides Long_Tail_domain_Clustering_Analysis_v2.2.pdf

12:15 Publishing zone scan data using an open data portal

NZRS has been running zone scans on a monthly basis over the active .nz domain names for the past two years. We are using dnscheck with custom changes to collect DNS health information, as well as IPv6 deployment metrics. The data is of no use if it can't be made readily available to interested parties. To sort this out, NZRS will start using an open data portal provided by Socrata to allow open access to the zone scan and other datasets about the registry operations. The data portal will allow to download the data, but also to explore it in a visual way. This presentation will cover - Methodology and infrastructure to run the zone scan - Overview of the data collected - Introduction to the open data portal (possibly a demo) - Some interesting findings. Time required: Ideally 30 minutes, but can be adjusted to be a lightning talk.

Speaker: Mr Sebastian Castro (NZRS)

Slides Publishing_zone_scan_data.pdf Publishing_zone_scan_data.pptx

12:30 → 14:00 Lunch

13:00 → 14:00 PGP Signing Session

Speaker: Mr Mauricio Vergara Ereche (ICANN)

notes notes-pgp-session.txt

PGP Keyring oarc-montreal-2015.asc

14:00 → 15:30 Public Workshop: Root Data Analysis

Convener: Mr Mauricio Vergara Ereche (ICANN)

14:00 A study of caching behavior with respect to root server TTLs

The Root Server System Advisory Committee (RSSAC) within ICANN was recently tasked with considering the extent to which the current root zone TTLs are appropriate for today's Internet environment and the impacts of root TTL alterations on the wider DNS system. The historical DITL data from 2014 and 2015 was analyzed for trends in TTL adherence, answering some of the following questions: To what degree do root zone TTLs matter? Is there a difference in behavior for authoritative versus non-authoritative data? Do all TLDs exhibit similar inter-query time distributions? Do specific recursive implementations, ISPs, open resolvers, etc. diverge from general TTL adherence trends? How has inter-query time changed over the past two years? Would a change in root zone TTLs result in a change in traffic levels at root name servers?

Speakers: Duane Wessels (Verisign), Matthew Thomas (Verisign)

Slides ttlThomasWessels_finalPresentation.pptx

14:30 F-root Anycast Research using RIPE Atlas

ISC has been using data routinely collected by every RIPE Atlas node to research the effectiveness of F-root's current transit and peering arrangements. The presentation will show how visualisation of this data can identify issues that should be resolved, along with "before and after" pictures showing the effect of changes that we already made to our routing configuration based on this analysis.

Speaker: Mr Ray Bellis (Internet Systems Consortium, Inc.)

Slides Bellis-Anycast.pdf

15:00 Thirteen Years of "Old J Root"

Thirteen years ago Verisign renumbered j.root-servers.net so that it could be anycasted. Since that time, we have been continuing to answer queries sent to the old IP address. We have also been collecting some data on queries to old J-root. In this presentation we will explore such questions as: what do we know about the clients of old J-root? Do they overlap with clients of the real J-root? Are there noticeable differences in traffic characteristics (e.g., EDNS, DNSSEC, query types) between the two? Does old J-root traffic fluctuate in the same way as real traffic? When real J-root gets attacked, does old J-root also get attacked? If so, can this be used to identify attacks coming through recursive name servers?

Speaker: Duane Wessels (Verisign)

Slides 2015-old-j-root.pdf

15:30 → 16:00 Afternoon Coffee Break

16:00 → 17:45 Public Workshop: Resolvers Track

Convener: Duane Wessels (Verisign)

16:00 Analyzing the distribution of DNS clients to recursive name servers across the Internet

As a byproduct of our web-based Real User Monitoring (RUM) agent, Dyn obtains the IP addresses of individual hosts running web browsers all over the world as well as the IP addresses of the recursive servers queried by those hosts. We have analyzed a rich data set of over 110 million client IP address-to-recursive IP address mappings to research an area of DNS that we believe has not been sufficiently studied. For example, what is the distribution of the number of clients per recursive server? Where are clients located relative to the recursive servers they use, both from a geographic as well as network topological perspective? What query patterns do individual clients follow if they use multiple recursive servers? We report on these and other interesting findings.

Speaker: Matt Larson (Dyn, Inc.)

Slides Larson-DNS_client_distribution-DNS-OARC-Montreal-2015-10.pdf

16:30 Real World Impacts of EDNS Client Subnet

Client Subnet offers the ability to offer better geolocation of end-users via DNS responses. This talk will concentrate on what happens when Client Subnet is enabled on a public resolver. It will look at upstream traffic patterns, cache performance, and other factors that come into play with Client Subnet. At the end of this talk, DNS providers should have a better idea of how Client Subnet will impact their performance & network.

Speaker: Brian Hartvigsen (OpenDNS)

Slides EDNS_Client_Subnet_Preso.pdf EDNS_Client_Subnet_Preso.pptx

17:00 dnstap-whoami: one-legged exfiltration of resolver queries

A few existing "whoami" or "dnsecho" authoritative DNS services allow for limited extraction of information about the resolver to the original client that would normally be hidden. For example, querying an anycasted resolver like 8.8.8.8 with the command "dig @8.8.8.8 whoami.akamai.net" will return an address record revealing a unicast initiator address used by the anycast service. This is "one-legged", because the original client only has visibility into the stub/recursive "leg" of the DNS interaction. The DNS-OARC porttest tool is another example of a "one-legged" service. Similarly, many DNS research projects use special purpose zones with instrumented nameservers which capture incoming query packets for analysis. For example, scans for open recursive DNS servers typically control both the stub/recursive "leg" and the recursive/authoritative "leg" and are thus "two-legged". This requires a more heavyweight investment but results in a richer set of data. This talk will demonstrate an enhanced "whoami" authoritative DNS server that can exfiltrate more detailed information about the recursive/authoritative interaction to the original client, including the complete resolver query packet sent to the authoritative server, using the dnstap format to compactly tunnel structured information which can be decoded by the original client.

Speaker: Robert Edmonds (Farsight Security, Inc.)

Slides dnstap-whoami.pdf

17:15 Happy DNS Eyeballs?

Much work has been undertaken in the browser world to produce the so-called "Happy Eyeballs" outcome. This is an outcome where the client will detect if the service is a dual stack service and if so then use a connection process that slightly biases the client in favour of using IPv6 as the transport for the DNS. What evidence is there for a similar mode of behaviour of DNS resolvers? This presentation will report on a large scale measurement experiment that was intended to expose the protocol behaviour of resolvers and determine whether they have any protocol selection bias.

Speaker: Mr Geoff Huston (APNIC)

Slides 2015-10-04-dns-dual-stack.pdf

Monday, 5 October

15:15 → 17:15 NANOG65 DNS Track

Registered NANOG65 Attendees only

Conveners: Duane Wessels (Verisign), Paul Ebersman (Comcast)

summary https://www.nanog.org/meetings/abstract?id=2677

15:15 DNS-OARC Overview

Speaker: Mr Keith Mitchell (DNS-OARC)

Slides OARC-NANOG665-DNStrack.pdf

15:25 Managing DDoS Attacks

Speaker: Mr Brian Somers (OpenDNS, FreeBSD)

Slides https://indico.dns-oarc.net/event/24/session/8/contribution/13/material/slides/2.pdf

15:55 F-root Anycast Research using RIPE Atlas

Speaker: Mr Ray Bellis (Internet Systems Consortium, Inc.)

Slides https://indico.dns-oarc.net/event/24/session/10/contribution/22/material/slides/0.pdf

16:25 Impact of unknown EDNS options on the DNS

Speakers: Eddy Winstead, victoria risk (isc)

Slides Winstead-EDNS.pdf

16:35 Benchmarking of authoritative DNS servers and DNSSEC impact assessment

Speaker: Mr Tomas Hlavacek (CZ.NIC, z.s.p.o.)

Slides https://indico.dns-oarc.net/event/24/session/7/contribution/5/material/slides/0.pdf

17:05 Root KSK Rollover

Speaker: Adiel Akplogan (ICANN)

Slides NAnog65MontrealDNSTrackKSK-AAA.pdf