Windows hole discovered after 17 years

Microsoft isn't having an easy time of it these days. In addition to the unpatched hole in Internet Explorer, a now published hole in Windows allows users with restricted access to escalate their privileges to system level – and this is believed to be possible on all 32-bit versions of Windows from Windows NT 3.1 up to, and including Windows 7. While the vulnerability is likely to affect home users in only a minor way, the administrators of corporate networks will probably have their hands full this week.

The problem is caused by flaws in the Virtual DOS Machine (VDM) introduced in 1993 to support 16-bit applications (real mode applications for 8086). VDM is based on the Virtual 8086 Mode (VM86) in 80386 processors and, among other things, intercepts hardware routines such as BIOS calls. Google security team member Tavis Ormandy has found several vulnerabilities in this implementation that allow an unprivileged 16-bit program to manipulate the kernel stack of each process via a number of tricks. This potentially enables attackers to execute code at system privilege level.

Ormandy has also published a suitable exploit which functions under Windows XP, Windows Server 2003 and 2008, Windows Vista and Windows 7. When tested by the The H's associates at heise Security, the exploit opened a command prompt in the system context, which has the highest privilege level, under Windows XP and Windows 7. No patch has become available, although Ormandy reports that Microsoft was already informed of the hole in mid 2009. The developer decided to publish the information regardless because, in his opinion, there is a simple workaround: to disable the MS-DOS subsystem.

The workaround requires users to start the group policy editor and enable the "Prevent access to 16-bit applications" option in the Computer Configuration\Administrative Templates\Windows Components\Application Compatibility section. When tested with these settings by the heise Security team, the exploit no longer functioned. The settings reportedly don't cause any major compatibility problems for most users while no 16-bit applications are being used.

Update - The above option is only available through the group policy editor on Windows 2003 systems. Some versions of Windows do not include a group policy editor. As an alternative, users can also create a registry key under \HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\AppCompat with a D-Word value of VDMDissallowed = 1.

Under Windows XP, to prevent the system from being vulnerable to the exploit, users can place the following text:

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\AppCompat] "VDMDisallowed"=dword:00000001

into a file called vdmdisallow.reg and double click the file. Windows will then automatically import the key (admin rights are required to perform this action).

Update 2 - Microsoft has now confirmed the privilege escalation hole in Windows. The company says that it wants to complete its investigation of the vulnerability and will then decide whether, how and when to close it.