PING

In this episode of PING, APNIC Chief Scientist Geoff Huston and I continue a discussion about the repeat queries Geoff sees on the APNIC Labs experiment. Across time this number has changed, as DNS protocol usage has changed. We now have DNS records which didn't exist when the problem was first seen, and we therefore have multiple queries about multiple things. Geoff has been exploring two facets of the DNS. Firstly, Exploring the effects how you answer "NO" has on the amount of queries you see, as well as the amount of work that has to be done by authoritative server, intermediate resolvers in the chain and the client. There are some surprising outcomes in the various choices of how to say no, including simply not answering. Secondly, what impact would adoption of alternative DNS transports have on the repeats and query load, which may be a function of the UDP protocol and it's behaviours. Many people believe the overhead of maintaining TCP and related protocols in terms of state on the server was a reason to avoid it, but with the volume of uneccessary repeats rising, it is possible there is less work end-to-end, maintaining state for a protocol like TCP.

In this episode of PING, we talk with Willem Toorop from NLNet Labs and Ilyas Rahimi who recently completed a masters in Security and Network Engineering at the University of Amsterdam (UvA). They’ve been looking at the effects of local root serving. Ilyas was on a masters programme called “OS3” at UvA (from the triplet of Open Standards, Open Software and Open Security). NLNet labs is on the campus of UvA and Willem has been working with the masters programme for some time, pursuing research in DNS and DNSSEC with the students doing their masters thesis component. They conduct 2 1-month original research activities in the masters course, and write the second one up as their masters thesis. Locally served root is a model of operating a DNS resolver where it “pre-fetches” the state of the root zone, the apex of the entire DNS naming model. By fetching this data the idea is that both early termination of unresolvable domain names (names where the final terminal label can be seen not to exist under the root of the DNS) and a degree of privacy in fetching resolvable names can be gained, by looking in the local copy. There is a current proposal in the IETF to make this a “best current practice” or BCP. Ilyas looked at 3 different public code-bases (bind, unbound and the knot DNS resolver) and a total of 4 configurations, exploring both in-band and HTTPS bound fetch of the root zone. As well as identifying a bug in the model unbound used, which accounted for a spike in traffic, Ilyas has been able to show that under the update cycle typically seen for the root there is a surprising increase in apparent network traffic fetching the zone, compared to the many but far smaller events asking for uncached data from the roots. There is more work to do in this space, and with the OS3 masters thesis component now given more time in the degree program, there is hope that Willem can continue to work with UvA students, and carry forward the measurements. WIllem has been exploring how the system behaves if converted to incremental updates (IXFR) rather than an entire zone-fetch.

In this episode of PING, APNIC Chief Scientist Geoff Huston and I discuss the behaviour of the DNS system when you have to come up from nothing: the “cold start” where no data exists from prior queries in your “cache”. This stems from a talk given by Ondřej Surý, that Geoff saw at the recent RIPE-92 Meeting held in Edinburgh, in the DNS working group. Cache, is the things you hold on to, from prior work done. It’s a fundamental technique in computer science to avoid the massive disparity in speed between parts of the systems, keeping things you use a lot in “fast” memory close to the CPU, and avoiding having to go to “slow” memory or even worse disk or tape files, to find the data. The size of your cache and it’s speed can have a huge influence on the speed of your program. Comparing CPU performance with and without prior cached data can be very instructive to it’s benefits! -In the DNS, Cache is how you avoid having to wait for a remote system to answer over the network (with all the round-trip delay of question and answer) because you know parts of your answer from the prior queries and answers you hung onto. “Cold Start” is a well known problem in large distributed systems. The problem is not just that industrial systems like coal plants and gas turbines need time to warm up and get sufficient energy to drive the generator, in the so-called “black start” there is the problem of energising the magnetic coils associated with generation of electricity. It’s distinct DC energy, used to make the generator enter the state where the rotational force from the turbine actually makes power. Without a source of power, you cannot make the coils “excite” and so the spinning generator won’t actually make any electrical power. Typically, this DC voltage comes from an independent source like a small diesel generator, or a source of electrical power which hasn’t been affected by the outage. If you contract to supply this contingency power, failure to be available is a major issue. The DNS isn’t an electrical generation network, but it does have this kind of complex dependency, mapping from DNS fully qualified name, to the specific IP addresses bound to the name. The “D” part of DNS stands for “Domain” and the elements of the name in a sequence form “domains” of control (or at least potentially do) separated by the dots in the name which may have a distinct “name server” detailing how things are named under-neath that domain boundary. So a single question “what is www.potaroo.net” has the potential to invoke at least 3 if not more questions, for each “domain” in the name. When you have a cache. The cache of known name-to-address pairs includes the data you need to know the addresses of the nameservers for each domain you have seen, which can tell you the name-to-address pairs for the other things you are asking: either directly or indirectly by passing you on to another system. But if you have no cache, when you ask (for example) the nameservers for .NET and are told they are A.NET and B.NET you are no wiser: how do you find these hosts, when you don’t know how to find .NET? Geoff has been chasing down what he sees exploring this “cold start” behaviour, and what it tells us about how people chose to name their hosts and services, how they provide the “name servers” behind these hosts and services, and how intruding intermediaries, or even trying to work around the risks of a cold start can increase the query burden on the clients worldwide.

In this episode of PING, we talk with Job Snijders about secure Internet routing again, focusing on an approach to preserving the state of RPKI as a longterm historical record for research and analysis. Job was last on PING to discuss the “Erik” protocol. Job has been running an archive of RPKI state for some time, as a volunteer activity but an exercise which was feasible when BGP speakers producing ROA objects was measured in the hundreds to low thousands is significantly more expensive when the population of ROA producers is a more realistic percentage of the around 80,000 AS holders worldwide. Because Public Key Cryptography depends on a regular re-signing, and re-cataloging (as another signed object, the RPKI “Manifest” file) even when there is no substantive change in the state of signed information, a large amount of “churn” can be seen in the data, and even a compressed form of this state in turn incurs a huge overhead in storage of hard-to-compress data. Job reached the limits of his free activity, and started to explore a more compact and useful representation. This has emerged in the IETF standards process as two related activities. The Canonical Cache Representation or CCR, and the aggregation over this for data represented as the RPKI Spool data model for “materialising” the state of RPKI objects. Along with his fellow IETF draft authors Bart Bakker , Tim Bruijnzeels, Theo Buehler and Fedor Vompe Job has managed to define a remarkably compact, highly compressible representation of RPKI validated objects, and the cryptographic payloads. These systems have been designed to leverage well known UNIX and other techniques for data management such as the “tar” format for spooled data, ASN.1 for binary encoding, Merkle Trees to hold compact hash state, and data compression using the zstd encoding. It’s a nice approach to solving the scaling problem.

In this episode of PING, APNIC Chief Scientist Geoff Huston and I discuss a traffic behaviour in the DNS which Geoff has noticed in the labs advertising based experimental data capture. Virtually every DNS query Geoff sees, he sees twice (or more). For a cohort of about 150 million unique DNS labels on a given day, The Labs system is collecting 270 million incoming DNS queries. Thats a lot of duplication. What's going on? The advertising data collection depends on issuing unique DNS queries, which in turn generate unique web page serves. This allows measurement of internet-wide behaviour on about 30 million browsers, games, devices every day. Not to say that these names and web URLs are not routinely seen by more than one entity, intermediary systems such as caches and proxies as well as re-visiting old open website tabs on a browser can cause this. The point is that after the "first" fetch, the subsequent fetches can usually be held to be re-presentations of the same experiment and can therefore often be discarded (in the case of the web). But, for the DNS which has always had an element of unreliable transport, and which in turn invites measurement of features like DNSSEC which cause SERVFAIL messages, and demand repeated attempts to "find the DNS name-to-address mapping, re-fetching is itself something under test. How many resolvers lie behind a given users systems? How many kinds of resolver (DNSSEC enabled, or not) does the user depend on? Finding that almost all DNS queries are repeated, and it turns out repeated very quickly invited Geoff to have a look at what's going on "under the covers". There are some patterns behind what's being seen which Geoff explored in the APNIC Labs Blog.