PING

In this episode of PING, we’re hearing from Darren Kara, who is a principal engineer at Quad 9. If you haven’t heard of Quad 9 It’s one of the oldest public DNS services, over 10 years old. An initiative of Packet Clearning House (PCH), IBM, The Global Cyber Alliance (GCA) and Switch Telecom in Switzerland. Quad9 now operates as a public-benefit not-for-profit foundation and is headquartered in Switzerland. Amongst the choices of public DNS from other providers like Google and Cloudflare, Quad 9 has two principal points: They focus on privacy of users, with a very strong prevention of information leakage from out of any given location they operate in, and they offer protection against Malware and Phishing. The first design goal means for measurement, they have a completely different model to the one we’re mostly used to, where packet samples or entire captures (as in the DNS OARC Day-in-the-life or DITL collection) are available for analysis: Nothing leaves the room which identifies the individual user. Only aggregates, and with changes to anonymise the source address. The second goal of protecting the user is the result of a collaboration with threat intelligence sources operating worldwide. Quad 9 is able to reflect on their feeds, and ensure DNS queries which pass through them don’t send people to known sources of attack, and is also able to feed back to the sources information which helps them refine their signal, and understand risks in different parts of the world. This too demands data anonymisation to ensure nothing about the end user leaks, and that the different intelligence feeds don’t get signals intended for another, with Quad 9 as the intermediary. Darren discusses their architecture, and how things work from a management and measurement perspective. He’d love to hear from sources of threat analysis in the Asia-Pacific region, to improve collaboration and coverage for the community at large. Why not get in touch at the Quad 9 contact page?

In this episode of PING, APNIC Chief Scientist Geoff Huston reflects on the tremendous economic consequences of the surge in capital investment in Artificial Intelligence (AI) and the associated Data Center (DC) build out. This not involves so much money, it represents a tenth or more of the US annual Gross Domestic Product (GDP) and has significance outside of the application of AI to problems in the world. As technology consumers we’re already used to the idea the cost of basic inputs like RAM, storage devices, the GPU cards which we need to run high speed gaming computers are both in short supply and gone up in price significantly. But the economic effects of the surge in AI go beyond the IT sector. They affect all of us fundamentally. Having a speculative activity like this represent so much of any economies activity means that as it ebbs and flows, or even worse crashes, the same amount of economic activity is removed from other productive investment. Some of the AI money is hypothetical, set into the future on the premise the investments being made now play out, but some of it therefore is highly risky, money which won’t actually exist when the time comes, but the debt side, the cost of raising all this money, spending all this money and time and effort building out the DC. That debt will exist, and will either be recovered in a fire sale of assets, or be distributed around as pain in the pockets of investors who put their money into the proposal. A problem for almost all of us, is that the investment decisions here have been taken out of our hands: Although our apetite for risk may vary, the decision around investment is made mostly on our behalf, by the banks, the investment companies, the “institutional investors” which may seem remote, but when you consider the Australian Superannuation sector represents $4.5 trillion Australian Dollars, and is the fourth largest investment pool in the world, for a nation of only 27 million people, our exposure to this risk is a lot larger than you might think. The same is true for the pension funds of Canada, of the United Kingdom, the European Union, Asia and especially the investment industry of the USA, because the AI companies have joined or are on the brink of joining the pool of investment which is held to be “normal” and a required part of a balanced portfolio. To build and operate a modern AI focussed DC demands a huge investment in construction, in power engineering, in air conditioning and related systems. This investment is so potentially profitable that it can command all the inputs, all the labour required to build it, and afford to pay ahead of the rates normal construction sector projects would expect. It’s contributing directly to a supply chain shortage across the civil engineering sector, and in unrelated construction activity like housing. The diversion of electricity supply to power a DC means that electricity prices for domestic consumers are being pushed up, and the social consequences of where these DC are built effects their immediate surroundings in terms of the amenity and viability as a place to live and work. As Geoff points out, these systems are so big that in the USA, they cannot co-exist in the same state as a competitive sector: They have to be sited independently without a comparable DC investment, lest they overburden the utllity supply systems. This isn’t something the worldwide economy sat down and decided to plan out, it’s emerged into the world with probably unstoppable momentum. The problem exists everywhere, all at once, and the problem is getting larger, as the investor side gets more concerned of their risk of exposure as the cracks in the model emerge. Who “wins and loses” here goes beyond the nameplate on the DC, or the name of the AI company. We’re all standing to suffer economic shocks, as this AI craze works its way through to the inevitable other side of the bubble.

In this episode of PING we’re hearing from Duane Wessels, a fellow from Verisign who researches DNS and who we’ve had on PING before. This time Duane is discussing the history of changes of top level Key Signing Key (KSK) called “roll-over” of the KSK at the root of the DNS. This month on the 11th of October, we will complete deployment of the 3rd KSK, in the second rollover. It’s a multi-year process. This second key roll was delayed from the first rollover in 2017 by some unexpected circumstances in 2020 and 2023. The initial KSK was deployed in 2011. In October, the new key pair will enter service for the first time. The old key pair will no longer be used to sign, but will remain visible in the root until early 2027. Duane discusses some recent DNS research work by himself and Roy Arendts from ICANN, about the observations of resolver behaviour across a root KSK roll-over. Their most recent blog article “The 2024-2026 Root Zone KSK Rollover: Updates and Observations” from July 2026 is a follow-up to their original article “The 2024-2026 Root Zone KSK Rollover: Initial Observations and Early Trends” from March 2025. It’s worth reading both articles to see how experience gained in the initial observations of the key rollover have informed subsequent research. The DNS KSK at the root is the fundamental trust source for all subsequent DNSSSEC signed data in the global DNS. The processes around management of the private keys of this public-private key pair are described by IANA at their DNSSEC webpage and Verisign play several critical roles in the production, maintenance, distribution and use of the keying materials which come from this activity. Duane monitors this closely, both as a participant in the key ceremonies held under the auspices of ICANN/IANA, and from the vantage points Verisign has into global DNS from operations of the “J” root server anycast cloud. Duane and Roy had the opportunity of re-analysing the behaviour of resolver systems under the key roll with greater clarity, because since the original key roll in 2017 a larger community of resolvers now use a signalling method defined in RFC8145 which indicates which trust anchors they see and are using. This has improved visibility of the uptake of the new trust anchor and shows very clearly in their time-charts of deployment of the new keying materials. Key rolls in the DNS are not a “one and done” process, this is a multi-year activity which is about to reach it’s next most significant date later this month in October. On October the 11th, The new key will begin operations signing over the zone and the older keys will be withdrawn from signing. The new KSK has been published and listed in the root zone since January 2025, and was promoted by RFC5011 signalling in Feburary 2025.

In this episode of PING, APNIC Chief Scientist Geoff Huston and I continue a discussion about the repeat queries Geoff sees on the APNIC Labs experiment. Across time this number has changed, as DNS protocol usage has changed. We now have DNS records which didn't exist when the problem was first seen, and we therefore have multiple queries about multiple things. Geoff has been exploring two facets of the DNS. Firstly, Exploring the effects how you answer "NO" has on the amount of queries you see, as well as the amount of work that has to be done by authoritative server, intermediate resolvers in the chain and the client. There are some surprising outcomes in the various choices of how to say no, including simply not answering. Secondly, what impact would adoption of alternative DNS transports have on the repeats and query load, which may be a function of the UDP protocol and it's behaviours. Many people believe the overhead of maintaining TCP and related protocols in terms of state on the server was a reason to avoid it, but with the volume of uneccessary repeats rising, it is possible there is less work end-to-end, maintaining state for a protocol like TCP.

In this episode of PING, we talk with Willem Toorop from NLNet Labs and Ilyas Rahimi who recently completed a masters in Security and Network Engineering at the University of Amsterdam (UvA). They’ve been looking at the effects of local root serving. Ilyas was on a masters programme called “OS3” at UvA (from the triplet of Open Standards, Open Software and Open Security). NLNet labs is on the campus of UvA and Willem has been working with the masters programme for some time, pursuing research in DNS and DNSSEC with the students doing their masters thesis component. They conduct 2 1-month original research activities in the masters course, and write the second one up as their masters thesis. Locally served root is a model of operating a DNS resolver where it “pre-fetches” the state of the root zone, the apex of the entire DNS naming model. By fetching this data the idea is that both early termination of unresolvable domain names (names where the final terminal label can be seen not to exist under the root of the DNS) and a degree of privacy in fetching resolvable names can be gained, by looking in the local copy. There is a current proposal in the IETF to make this a “best current practice” or BCP. Ilyas looked at 3 different public code-bases (bind, unbound and the knot DNS resolver) and a total of 4 configurations, exploring both in-band and HTTPS bound fetch of the root zone. As well as identifying a bug in the model unbound used, which accounted for a spike in traffic, Ilyas has been able to show that under the update cycle typically seen for the root there is a surprising increase in apparent network traffic fetching the zone, compared to the many but far smaller events asking for uncached data from the roots. There is more work to do in this space, and with the OS3 masters thesis component now given more time in the degree program, there is hope that Willem can continue to work with UvA students, and carry forward the measurements. WIllem has been exploring how the system behaves if converted to incremental updates (IXFR) rather than an entire zone-fetch.