PING

In this episode of PING we’re hearing from Duane Wessels, a fellow from Verisign who researches DNS and who we’ve had on PING before. This time Duane is discussing the history of changes of top level Key Signing Key (KSK) called “roll-over” of the KSK at the root of the DNS. This month on the 11th of October, we will complete deployment of the 3rd KSK, in the second rollover. It’s a multi-year process. This second key roll was delayed from the first rollover in 2017 by some unexpected circumstances in 2020 and 2023. The initial KSK was deployed in 2011. In October, the new key pair will enter service for the first time. The old key pair will no longer be used to sign, but will remain visible in the root until early 2027. Duane discusses some recent DNS research work by himself and Roy Arendts from ICANN, about the observations of resolver behaviour across a root KSK roll-over. Their most recent blog article “The 2024-2026 Root Zone KSK Rollover: Updates and Observations” from July 2026 is a follow-up to their original article “The 2024-2026 Root Zone KSK Rollover: Initial Observations and Early Trends” from March 2025. It’s worth reading both articles to see how experience gained in the initial observations of the key rollover have informed subsequent research. The DNS KSK at the root is the fundamental trust source for all subsequent DNSSSEC signed data in the global DNS. The processes around management of the private keys of this public-private key pair are described by IANA at their DNSSEC webpage and Verisign play several critical roles in the production, maintenance, distribution and use of the keying materials which come from this activity. Duane monitors this closely, both as a participant in the key ceremonies held under the auspices of ICANN/IANA, and from the vantage points Verisign has into global DNS from operations of the “J” root server anycast cloud. Duane and Roy had the opportunity of re-analysing the behaviour of resolver systems under the key roll with greater clarity, because since the original key roll in 2017 a larger community of resolvers now use a signalling method defined in RFC8145 which indicates which trust anchors they see and are using. This has improved visibility of the uptake of the new trust anchor and shows very clearly in their time-charts of deployment of the new keying materials. Key rolls in the DNS are not a “one and done” process, this is a multi-year activity which is about to reach it’s next most significant date later this month in October. On October the 11th, The new key will begin operations signing over the zone and the older keys will be withdrawn from signing. The new KSK has been published and listed in the root zone since January 2025, and was promoted by RFC5011 signalling in Feburary 2025.

In this episode of PING, APNIC Chief Scientist Geoff Huston and I continue a discussion about the repeat queries Geoff sees on the APNIC Labs experiment. Across time this number has changed, as DNS protocol usage has changed. We now have DNS records which didn't exist when the problem was first seen, and we therefore have multiple queries about multiple things. Geoff has been exploring two facets of the DNS. Firstly, Exploring the effects how you answer "NO" has on the amount of queries you see, as well as the amount of work that has to be done by authoritative server, intermediate resolvers in the chain and the client. There are some surprising outcomes in the various choices of how to say no, including simply not answering. Secondly, what impact would adoption of alternative DNS transports have on the repeats and query load, which may be a function of the UDP protocol and it's behaviours. Many people believe the overhead of maintaining TCP and related protocols in terms of state on the server was a reason to avoid it, but with the volume of uneccessary repeats rising, it is possible there is less work end-to-end, maintaining state for a protocol like TCP.

In this episode of PING, we talk with Willem Toorop from NLNet Labs and Ilyas Rahimi who recently completed a masters in Security and Network Engineering at the University of Amsterdam (UvA). They’ve been looking at the effects of local root serving. Ilyas was on a masters programme called “OS3” at UvA (from the triplet of Open Standards, Open Software and Open Security). NLNet labs is on the campus of UvA and Willem has been working with the masters programme for some time, pursuing research in DNS and DNSSEC with the students doing their masters thesis component. They conduct 2 1-month original research activities in the masters course, and write the second one up as their masters thesis. Locally served root is a model of operating a DNS resolver where it “pre-fetches” the state of the root zone, the apex of the entire DNS naming model. By fetching this data the idea is that both early termination of unresolvable domain names (names where the final terminal label can be seen not to exist under the root of the DNS) and a degree of privacy in fetching resolvable names can be gained, by looking in the local copy. There is a current proposal in the IETF to make this a “best current practice” or BCP. Ilyas looked at 3 different public code-bases (bind, unbound and the knot DNS resolver) and a total of 4 configurations, exploring both in-band and HTTPS bound fetch of the root zone. As well as identifying a bug in the model unbound used, which accounted for a spike in traffic, Ilyas has been able to show that under the update cycle typically seen for the root there is a surprising increase in apparent network traffic fetching the zone, compared to the many but far smaller events asking for uncached data from the roots. There is more work to do in this space, and with the OS3 masters thesis component now given more time in the degree program, there is hope that Willem can continue to work with UvA students, and carry forward the measurements. WIllem has been exploring how the system behaves if converted to incremental updates (IXFR) rather than an entire zone-fetch.

In this episode of PING, APNIC Chief Scientist Geoff Huston and I discuss the behaviour of the DNS system when you have to come up from nothing: the “cold start” where no data exists from prior queries in your “cache”. This stems from a talk given by Ondřej Surý, that Geoff saw at the recent RIPE-92 Meeting held in Edinburgh, in the DNS working group. Cache, is the things you hold on to, from prior work done. It’s a fundamental technique in computer science to avoid the massive disparity in speed between parts of the systems, keeping things you use a lot in “fast” memory close to the CPU, and avoiding having to go to “slow” memory or even worse disk or tape files, to find the data. The size of your cache and it’s speed can have a huge influence on the speed of your program. Comparing CPU performance with and without prior cached data can be very instructive to it’s benefits! -In the DNS, Cache is how you avoid having to wait for a remote system to answer over the network (with all the round-trip delay of question and answer) because you know parts of your answer from the prior queries and answers you hung onto. “Cold Start” is a well known problem in large distributed systems. The problem is not just that industrial systems like coal plants and gas turbines need time to warm up and get sufficient energy to drive the generator, in the so-called “black start” there is the problem of energising the magnetic coils associated with generation of electricity. It’s distinct DC energy, used to make the generator enter the state where the rotational force from the turbine actually makes power. Without a source of power, you cannot make the coils “excite” and so the spinning generator won’t actually make any electrical power. Typically, this DC voltage comes from an independent source like a small diesel generator, or a source of electrical power which hasn’t been affected by the outage. If you contract to supply this contingency power, failure to be available is a major issue. The DNS isn’t an electrical generation network, but it does have this kind of complex dependency, mapping from DNS fully qualified name, to the specific IP addresses bound to the name. The “D” part of DNS stands for “Domain” and the elements of the name in a sequence form “domains” of control (or at least potentially do) separated by the dots in the name which may have a distinct “name server” detailing how things are named under-neath that domain boundary. So a single question “what is www.potaroo.net” has the potential to invoke at least 3 if not more questions, for each “domain” in the name. When you have a cache. The cache of known name-to-address pairs includes the data you need to know the addresses of the nameservers for each domain you have seen, which can tell you the name-to-address pairs for the other things you are asking: either directly or indirectly by passing you on to another system. But if you have no cache, when you ask (for example) the nameservers for .NET and are told they are A.NET and B.NET you are no wiser: how do you find these hosts, when you don’t know how to find .NET? Geoff has been chasing down what he sees exploring this “cold start” behaviour, and what it tells us about how people chose to name their hosts and services, how they provide the “name servers” behind these hosts and services, and how intruding intermediaries, or even trying to work around the risks of a cold start can increase the query burden on the clients worldwide.

In this episode of PING, we talk with Job Snijders about secure Internet routing again, focusing on an approach to preserving the state of RPKI as a longterm historical record for research and analysis. Job was last on PING to discuss the “Erik” protocol. Job has been running an archive of RPKI state for some time, as a volunteer activity but an exercise which was feasible when BGP speakers producing ROA objects was measured in the hundreds to low thousands is significantly more expensive when the population of ROA producers is a more realistic percentage of the around 80,000 AS holders worldwide. Because Public Key Cryptography depends on a regular re-signing, and re-cataloging (as another signed object, the RPKI “Manifest” file) even when there is no substantive change in the state of signed information, a large amount of “churn” can be seen in the data, and even a compressed form of this state in turn incurs a huge overhead in storage of hard-to-compress data. Job reached the limits of his free activity, and started to explore a more compact and useful representation. This has emerged in the IETF standards process as two related activities. The Canonical Cache Representation or CCR, and the aggregation over this for data represented as the RPKI Spool data model for “materialising” the state of RPKI objects. Along with his fellow IETF draft authors Bart Bakker , Tim Bruijnzeels, Theo Buehler and Fedor Vompe Job has managed to define a remarkably compact, highly compressible representation of RPKI validated objects, and the cryptographic payloads. These systems have been designed to leverage well known UNIX and other techniques for data management such as the “tar” format for spooled data, ASN.1 for binary encoding, Merkle Trees to hold compact hash state, and data compression using the zstd encoding. It’s a nice approach to solving the scaling problem.