Interview with Olaf Kolkman, ISOC

Listen to the around 32-minute interview here or download the audio for later.

DOTMAGAZINE: As one of the seven main keyholders protecting the DNS root zone, you literally and metaphorically have one of the keys to the Internet in your hands. What would be the repercussions of these keys fell into the wrong hands?

OLAF KOLKMAN: I’m going to push back a bit on the assertions made in the question. First, it’s not the keys to the Internet. What I’m involved in is the protection of the so-called DNS. DNS is the highly distributed system that translates names that we use on the Internet, for instance the domain names that you use in email addresses, or the domain names that you use in browsers, to the IP addresses and other resources that computers need to do their job. Metaphorically, people refer to it as a telephone book.

The structure of that system is such that it is highly hierarchical. There is a so-called root zone, which points to the location of the next level down, which are the top-level zones. The root zone knows where to point somebody – which is a machine that usually lives in your local network – and it refers that machine to the next level down, say, .nl or .com. Then .com will refer further down to the machines that know where, say, the cocacola.com company is, or where isoc.org is. So we’re talking about the root of that system, and the signatures that are provided there are signatures that provide authenticity and integrity of the information. They basically sign off that the information you pass by has not been modified and is published by the right entity. 

That system is in place because otherwise, the information that traverses the Internet – all the questions and answers – might be replaced by somebody in the middle who tries to muck with that resolution. If that happens – if somebody mucks with that resolution process – you might end up with the wrong party to talk to.  For instance, you think you are talking to your bank, but instead you might be talking to another service. You type in your bank address in your web browser, there’s a look-up, the information is being replaced, and you talk to the wrong entity. There are other protection mechanisms that will flag that, but you want to have that protection throughout the Internet architecture, at every layer of this structure.

This is what we’re trying to protect. The keys that are used to generate these signatures are stored in machines called hardware signing modules. They’re baked into a chip, so to speak. And there’s a little bit of a layering here: there’s a master key, which lives in a machine that lives in a vault, and that master key is removed from that vault once every six months to generate signatures of the production keys that are maintained to be more accessible by somebody who operates the root zone. 

So, these keys are sort of master keys, they live in a device that sits in a vault. That vault has very high security perimeters. In fact, there are seven layers of security, and you cannot get into that vault without being seen, without having to sign in, without collaborating with other people.  My role in that whole process (and I’m not one of seven, I’m one of fourteen, because there are seven on the east coast and seven on the west coast) is to carry around a small key which can open a de facto safety deposit box. And in this safety deposit box in a tamper-evident bag, is a smart card. 

There are seven smart cards in that vault, in that facility, locked behind seven doors. In fact, my role is – with my little physical key, because I have a physical key – to open that little vault, take out the smart card, look at whether that smart card or the vault has been tampered with, very carefully study that tamper-evident bag, very carefully look that it’s not ruptured, and make sure that there is a continuity in the procedure so that if somebody ever had their hands in the vault and drilled out the lock or opened it, I can see that. I am an observer of a process that is meant to audit the integrity of that root key, and audit that nobody has gotten access to that key in the last six months. 

DOT: Have you ever had any evidence of tampering for any of the keys?

KOLKMAN: No. That is an unequivocal no. There has not been any evidence of that. And obviously, if that ever happened, any of the keyholders would be extremely vocal about it. 

The reason why we have those keyholders is that three out of seven independent people need to get together in that vault, get through those seven layers of security – which they can only do with other people being present, like staff from ICANN – to first open all the locks that get us into that vault so that we can open our little boxes and take out the cards. 

Those cards are then presented in a ceremony which is precise and very well structured. And by the way, publicly broadcast: everybody can watch the broadcast of these ceremonies. The goal is to be extremely transparent in the management of the key, so that at the first suspicion that something went wrong, we can expose it. 

If that ever happened, that would be a blow to the trust in the Internet.  What we’re trying to do collectively as a community is to be as transparent as we can in showing that there is continuity and that the key is secure. While protecting the bits that generate the signature. That is something that is very important.

DOT: How is the Domain Name System protected?  

KOLKMAN: It’s a fairly complex system that as I have described, has these layers of delegation. If you look at a domain name, www.internetsociety.org, the root is responsible for maintaining who has the information on the org level.  At the org level there’s an organization that is responsible for maintaining the information at the layer below it.  An organization like the Internet Society maintains the information about the internetsociety.org domain. As you see, there are already a few entities that have responsibility. If you, as a customer, use your iPhone or your computer, there is a machine in the local network that will go and hunt for the answers that you are asking. Those machines are called resolvers. 

The way that we have protected the Internet Domain Name System, metaphorically, is that the messages that get passed by all these machines –or the information that is transferred over the Internet can be considered as if they are postcards. And a postcard can easily be lost if we don’t protect against that. The other thing is that everybody can read what is on the postcard while it’s in transfer.  For example, the postman can look at the postcard and see the address and the little memo that you’ve put on the postcard. It’s not something that we protect against – at least, not with DNSSEC. 

In essence, what we are doing is putting those postcards in transparent envelopes, closing the envelope and putting a seal on it. And by looking at the stamp of the seal, you can tell who sent that envelope. You know who the holder of the seal is, and you can look at the seal and say, ok that was the .org domain that sent me that information. And I know, because this envelope is completely intact, that the message has not been tampered with. Nobody tried to write something else on the postcard that would cause confusion on the resolver or customer end. 

That is the concept. In this system, the root zone puts a seal on the message to say “the org zone has information about internetsociety.org, go there.” And on the postcard, inside the envelope, it puts what the seal that verifies .org looks like. So, the resolver then knows –look at the postcard, the next seal that I’m expecting looks like this, I can verify the next signature that I’m expecting against this image. At the top-level domains that really works well. There are a few hundred domains in the world that are signed and that deploy DNSSEC. On the lower level, however, there is not that much uptake. In the Netherlands, the uptake is very high – a few million domains are signed. In other parts of the world, the success rate of creating and maintaining those signatures, is not very high. 

So that answers the question of how well the DNS is secured: not very well, because there are not that many signatures around. And this is a point of concern. In the Internet, we sometimes find it hard for people to invest in the security of others. And this is an investment. Creating signatures is really an investment for any of the entities that need to do it. It doesn’t introduce immediate visible effects –having a signature doesn’t change the quality of your service and it’s not even visible to the end-user. I as an end-user, you as an end-user, and all of our readers and listeners – they don’t see anything about the DNS. And that’s good – they shouldn’t need to know about the inner workings of the system. But it also means that there’s no financial benefit in applying those seals, because they are not visible to the end-user. I think that is an issue because this whole infrastructure – almost everything we do on the Internet – starts with a DNS query. Being able to validate that nobody tampered with that is incredibly important. 

And once you have that infrastructure in place, you can actually innovate on top of that. And what you see nowadays is that there are innovations: for example, when I want to set up an encrypted connection with you, and I don’t have any a prior knowledge about you, how do I do that? Where do I get that information? If we have a secured DNS, it could be a source for that information. In some cases, it could be a secondary source of information so that you compare the results that you get from one source with another – which enhances technical trust. I think we need to collectively pay attention to these technologies and take our collective responsibility in deploying them. 

DOT: How often do you attend the Key Signing Ceremony, and what is your most memorable incident relating to it?

KOLKMAN: The Key Signing Ceremony takes place every year: once on the east coast, and once on the west coast. I’m a ceremony participant keyholder for the east coast facilities. The Key Signing facilities are in Culpepper, Virginia – a big data center where there are also a lot of federal and payment industry facilities.  It was put there to be outside of the nuclear blast zone for Washington DC.

We need three out of seven participants in the key ceremony in order to do a successful transaction. Personally, I believe that we should have five out of seven to validate that no three keys have been used in combination in the interim. I try to attend as many times as my schedule permits. There will be a key ceremony in April (27th and 28th) 2017, and there will be six keyholders there. I attend as much as possible, because I feel my responsibility there is to audit the processes and to make sure that they are followed.

Obviously, the first ceremony was very memorable –as a sort of a piece of history. That was a very important piece – the introduction of the key. The last was also memorable because we introduced a new key into the system. One thing you have to know about cryptography and cryptographic keys is that they have a lifetime. It is wise to replace a key that is used for creating signatures every now and then, because they might be brute-forced. At the last ceremony, we introduced a key that is going to be used to sign a new key set in April 2017, and that new key set will be introduced into the root zone on July 11, 2017. That is the first time that we’re going to roll the keys of the root zone, and that’s a somewhat scary moment. 

The reason being: I mentioned that at the customer side, we have these resolvers that do the validation of those seals. Now, they all have an idea of what the seal of the root – the highest point of the hierarchy – should look like. There is a procedure and a published technical specification that allows for replacement of the seal in the resolver through a special process. What we do not know is whether all of the resolvers out there actually implement that technical measure to track that change of the seal. What is a little bit scary in this whole thing is that when that seal is removed, maybe some of the old resolvers are still out there at the customer premises trying to validate against an old seal that they’ve never replaced. So it’s very important for the people who maintain these resolver boxes that do validation to be aware of this change and make sure that their resolver tracks that change.  Otherwise, their customers will not be able to validate answers, and will think that data has been tampered with. The result of this is that those customers will not be able to do look-ups against the Internet and will therefore de facto drop from the net. 

It’s also important for anybody who does DNSSEC validation to validate that they’re implementing the automatic roll-over technology, which we refer to as RFC 5011, and if not, that they do a manual reconfiguration. Obviously, this is an ask for specialists – this is not something that non-specialist users would need to do at home. It’s really the people at ISPs and mobile operators and whoever runs such validating resolvers. The people who run a validating resolver at home, I consider specialists – they know the plumbing [laughs]. And actually, that is possible – you can run this type of infrastructure yourself. If you don’t trust others to do the work for you, the beauty of the Internet is you can always do it yourself. 

DOT: Still talking about the Key Signing Ceremony, which security measures do you find most inconvenient?

KOLKMAN: The ceremony itself is a bit tedious as it’s long and it’s very precise. We’re following a script, and every exception to that script is noted and actually assessed – whether there’s a security implication, or whether we’re doing the right thing – and there is an auditor that follows the script and makes sure that those exceptions are noted. We have had cases where, for instance, the person who had to open the vault got a bit nervous. There are dial locks, and you have to be very precise – if you overshoot, you have to start again and, as far as I understand, if you do that a few times, you have a time-out of five minutes, and if you do it wrong again then the time-out is even longer. Sometimes those type of things might cause delays and cause an exception to the script – which is, again, very transparently assessed, written down for everybody to see, it’s on camera – but in the meantime, we’re all locked in this room. What we do, if these ceremonies are very long, then we make sure that people can go out of the room, and we actually bring in an armed guard that guards the key material – again, somebody who is completely independent of the process. 

DOT: So, how did someone who studied astronomy end up becoming a central figure in Internet administration? How and why did you become a keyholder?

KOLKMAN: A long story short: Starting with astronomy, it’s not uncommon for physicists and astronomers to work with modern computer technology – that’s always been the case. Astronomy has traditionally been a topic of Big Data – regardless of the times, it was always the biggest data around. Physics too. Whenever there was an opportunity to do Big Data and transfer data around, astronomers and physicists are the first to knock on the doors. You still see that academic networks are at the forefront of technology in that sense. And as an astronomer, as a student, we all had to put our weight behind maintaining some of the infrastructures, so I got interested in basically maintaining pieces of infrastructure. Got interested in the world-wide web – the lab introduced an http server, and volunteered to install that and maintain the information, and so on. So I slowly gained expertise with Internet technology, found it pretty interesting, applied for an Internet job while I was working on my PhD, walked away from my PhD, and got into an Internet environment. 

My second job in that environment was the RIPE NCC, which is one of the core Internet institutions, and I was very lucky to work with the right people to have access to knowledge, expertise, and leadership to learn on the job. One of the things that Daniel Karrenberg of the RIPE NCC felt very strongly about was leading by example. In early 2000, when DNSSEC was thought to be done, he said Olaf, I have a project for you – please go ahead and try to learn everything you can about DNSSEC and then teach that to our membership. I did that in cooperation with a group called NLnet Labs, and what we in essence discovered was that the DNSSEC protocol at that time was just not finished. So we had to go back to the drawing board, and I got involved more intensively in the work of the IETF – the Internet Engineering Task Force. I became Chair of the working group that worked on the DNSSEC extensions, also continued to work on the technical side trying to sort out solutions, worked with the people at NLnet Labs to build a name server – all those things happened sort of at the same time. 

Then I made a move and became Director of NLnet Labs, was also involved in the IETF leadership, member of the IAB [Internet Architecture Board], so sort of a visible person in the community. And since NLnet Labs did one of the first – actually the first – by design DNSSEC name server, we had the technical background and credibility in the community, and when the NTIA and ICANN at the time were looking for volunteers with a credible background and support of the community, that was a logical step. 

DOT: Tell us something about your work for the Internet Society – what does the Internet Society do, and what is your role?

KOLKMAN: The Internet Society is a global organization, and what we are about, essentially, is the open Internet and protecting the opportunities that the open Internet brings to society.  We are a membership organization – there are 234 countries and territories that have ISOC members, membership is well into 90,000. Members are often organized in chapters, which are sort of independent groups that subscribe to our mission – the open Internet – and try to work within their local communities to push and implement that agenda. There are 125 of them, in 105 countries. And then we have organizational members that support our cause – of which there are over 130. 

We have a staff that focuses on development and access, making sure that people have access to the Internet – an Internet that brings those opportunities – because we believe it is a vehicle for human development. We believe it’s a vehicle in achieving policy goals like the sustainable development goals set by the UN. We support a technical agenda – we’re the organizational home of the IETF, the standards organization for the Internet, and we provide funding for that and some organizational support. Obviously, the IETF itself sets and drives its own agenda – we’re not a part of that. It sets its own culture and standards and rules and internal procedures, but we do support them with finding sponsors, with financial continuity and so on. 

We have our own technical agenda: we’re trying to move technology in a way – or trying to convene technology communities in a way – that they enhance, for instance, the trust in the Internet. Our main areas of focus with respect to the Internet are trust and access. So, getting people connected, and making sure that they can keep trusting the Internet. For example, DNSSEC is one of the things that we continue to inform people about, we keep measurements on what the amount of deployment is, and try to give practical information to the people that have to deploy this technology through our Deploy 360 channel.  We also promote IPv6 – the new (not so new any more) addressing scheme that allows more than four billion people to connect with IP addresses. As you may know, the IPv4 addresses – which is the Internet’s addressing scheme on which the Internet has run for forty years – have run out, and the transition to a new set of addresses – IPv6 – is an important piece of our work. We try to inform people who have to do that work on how to do that best, and organize communities around that.  

Increasing the security of the Internet, paying attention to how people can build technical trust between each other: so that if I talk to you now, I actually know that I am talking to you, and if we want to keep our conversation confidential from others who might want to peek in – criminals, people that have access to communication on the wire – we can do that. So we can have safe conversations, secure bank transactions, we can do our business in a trustworthy way: those are things that we care about and we want to create a policy environment and a technical environment that allows that environment to prosper. 

The ability to innovate, the ability for new entrants, the ability for new ideas – those are things that we care about deeply at the Internet Society, and we are trying to do that through a development agenda, sponsoring local initiatives, a technical agenda, bringing communities together, organizing sessions around difficult questions, and by having a policy environment in which all this can work. We’re also informing and educating policy makers about the deep workings of the Internet. And that’s an incredibly fun job. My role, by the way is Chief Internet Technology Officer, so I’m looking at what we do in the technology agenda for that, and partly also how we bridge the technology into a policy message and what the relation is between those two. 

DOT: One last question, what are your hopes or vision for the further development of the Internet?

KOLKMAN: I see fundamentally that this comes back to access and trust, and while those are the keywords of the Internet Society, they are also my words. I think it’s incredibly important that anybody who wants to have access to the Internet can get access to the Internet. No matter where you live, what race, what color, political background, which nation state, whether you live on an island in the Pacific, whether you’re visually impaired. To put it another way, I would like to see the last billion connected. That’s very important. But I also want to do that in a way that provides them with protection from harm that the Internet could bring to them. A reasonable certainty that if you do business, your identity won’t be stolen. A reasonable certainty that your private details are not harvested in aggressive ways. A reasonable certainty that things work as you should expect. So, trust. Really trust. Based on technical means, so that you can have some certainty that the trust is not based on a perception, but actually on a technical foundation.