The Web Alienates Non-English Speakers — And We Need to Change That

Imagine you are trying to buy something online — plane tickets, an app subscription, a pair of sneakers. If you are an English speaker with an email address ending in .com, .org, .edu or .net, that process probably goes pretty smoothly for you.

But if your email address ends in .世界 or @डाटामेल.भारत or .photography, you might get an error message and be unable to complete your transaction. In this case, you have fallen victim to a lack of Universal Acceptance (UA).

The domain name space has experienced explosive growth since 2010, with the introduction of hundreds of new top-level domains (TLDs) that speak to interests (e.g., .ORGANIC) or which are in a language not based on the English script (e.g., .世界). So the problem is actually much bigger than just affecting non-English speakers.

These new TLDs give people greater freedom in their choice of Internet identity and ensure competition in the domain name world. Applications should no longer assume that the list of valid TLDs is fixed, or that a TLD will be just two or three ASCII characters. Likewise, email mailbox names can contain non-ASCII characters through the Email Address Internationalization (EAI) standards.

But while the domain name system (DNS) has changed, the rules used by many applications are from 20 years ago. Many systems don’t recognize or appropriately process new domain names, and not all online portals are primed for the opening of a user account with one of these new email addresses. While filling out online forms, new TLDs and email addresses containing Unicode are not always accepted (i.e., the user receives an error message).

Obviously, this creates problems for organizations and frustration for users who now are locked out of the organization’s offering. When systems don’t recognize or process the new domains and associated email addresses, users will experience a denial of service, and companies that do business online will leave revenue on the table.

UA prevents these problems and makes the Internet more accessible for the next billion users, most of whom will likely not speak or understand English or recognize the ASCII characters which are used by English speakers. It also enables organizations to better serve communities through their new domain identities, including language-specific domains in Arabic, Hindi, Cyrillic, and many other languages.

Why should developers care about UA?

If you’re a developer or system architect, and you’re wondering how this is relevant to you, there are three things to keep in mind:

First, you should adopt current tools and current standards whenever possible; this reduces compatibility bugs in general. The standards of the Internet are constantly evolving, and you need to keep your code up to date — regardless of the issue at hand.

Second, there are benefits for organizations that make UA a priority — both from a brand and business perspective. At Microsoft, where I work, the mission is to “empower every person and organization on the planet to achieve more,” and UA fits within this mission.

For example, we recently announced support for email addresses in Indian languages. Our commitment to UA opens doors in many markets around the world because we are seen as engaged and sharing in the priorities of local cultures and/or their governments.

For those more interested in tangible economic benefits, a 2017 study conservatively estimates a $9.8 billion USD annual boost in worldwide economic activity from both existing users using the new domain names, as well as new Internet users coming online in their native languages. Online spending from new users (predominantly Russian, Chinese, Arabic, Vietnamese, and Indic language speakers) could start at $6.2 billion USD per year.

The third argument for supporting UA is the simplest: it’s the right thing to do. There are millions of people around the world who can’t fully experience the benefits of the Internet because of the language they speak, or the domain name or email they’ve chosen. It’s another form of inequality that is within our power to fix.

So you want to help your organization become UA-ready. Now what?

If you’re interested in becoming UA-ready but don’t know where to start, there’s two pieces of good news. First, for many sites and services, UA is considered a “bug fix,” deliverable by a routine update to online systems.

The efforts invested by software and application owners to implement UA are not particularly complex, and are outweighed by the benefits that could be realized by doing so. One approach that works well is to set up standard processes so that UA is just another compliance activity, like localization or accessibility.

The second piece of good news is that organizations don’t have to do this alone. To help raise awareness of UA and provide support, stakeholders and industry leaders such as Apple, GoDaddy, Google, ICANN, Microsoft, and Verisign created the Universal Acceptance Steering Group (UASG).

The UASG exists to help educate and provide resources to organizations looking to ensure their systems are UA-ready and able to accept all domain names and email addresses in any valid script.

The UASG has developed a number of helpful guides and resources, which are available at https://uasg.tech/documents. Of particular note are the Quick Guide to Universal Acceptance, a technical guide for developers who want to test their systems for UA-readiness, and a guide to current standards.

Bottom line: Internet standards have changed and grown, and will continue to do so. Long-term success in a global world requires that organizations ensure their systems work with the common infrastructure of the Internet — email services and the DNS, which today are diverse and multilingual.

Developers and other IT professionals have an opportunity and a mandate to educate themselves on this important issue and help ensure their organizations are UA-ready.

Mark Svancarek has worked at Microsoft since 1993 and has worked across teams in the hardware, software, and online services business groups. Mark currently focuses on Internet Governance and Internet Technology Policy, including IPv6, WhoIs, generic Top-Level Domains (gTLDs), and Internet Internationalization (EAI/IDN).