Major Security Flaw Patched in X Window System

The U.S. Department of Homeland Security's (DHS) open source security audit program has identified the biggest X Window System security vulnerability in the last six years.

The X Window System is used in Unix and Linux operating systems. It also ships as an optional GUI with Apple's (Nasdaq: AAPL) Macintosh computers. Coverity, the company managing the DHS project under a US$1.25 million grant, detected the flaw using its Coverity Prevent technology.

The vulnerability was one of the most significant discovered in recent memory, according to Daniel Stone, a release manager for the X.Org Foundation. He referred to it as "something that we find once every three to six years and ... very close to X's worst-case scenarios in terms of security."

Small Flaw, Big Risk

The security hole resulted from a missing close-parenthesis symbol on a small piece of the program that checked the ID of the user. This seemingly harmless omission allowed local users to execute code with root privileges, giving them the ability to overwrite system files or initiate denial-of-service attacks.

The vulnerability was found in versions X11R6.9.0 and X11R7.0.0 during a security analysis of 31 major open source projects that Coverity undertook as part of a DHS initiative. These two X Window System versions marked a major milestone when released in December of 2005, as they were the first major updates in more than a decade. It took less than a week for the flaw to be repaired after the X.Org development team received the results of the analysis.

Unix-Linux Ripples?

Most highly publicized operating system security flaws are related to Windows, because it is the most prevalent system on the market, according to Pund-IT Principal Analyst Charles King. Coverity has indeed fixed an important flaw in the X Window System, he said, but it may not have made as much of an impact as a Windows flaw of the same magnitude would have, had it gone uncovered for a short while.

"With Unix you are talking about machines that are usually behind the walls of data centers. There are typically layers of security that would pick up hackers before they would get access to the server operating system," King told LinuxInsider. "Still, since probably more than half of the security breaches that are occurring are coming from inside the company, it is good that this was repaired ASAP."

Unix security fixes are a systematic part of regular maintenance by operating system vendors such as IBM (NYSE: IBM), Sun Microsystems (Nasdaq: JAVA) and Hewlett-Packard (NYSE: HPQ). Linux efforts, however, are a different story.

"Linux fixes are coming from the open source community, and there have been some questions raised in the past about exactly how effective the open source community has been at spotting these problems. I have to hasten to say that quite a few of those concerns have been voiced by Microsoft," King noted.

With its approach, Coverity seeks to help computer programmers automatically detect and remove software defects such as security vulnerabilities as the software is being built, according to the company.

Coverity was founded in 2002 by Stanford University computer scientists. Today its solution is used by more than 100 companies, including Juniper Networks (Nasdaq: JNPR), Symantec/Veritas, McAfee, Synopsys, NASA, PalmOne (Nasdaq: PALM), Sun and Wind River.