Written by Colin Finck
on 

Modern Visual Studio meets ancient Windows

Project on GitHub


In my previous blog series, I have shown how to write Win32 applications in 2020 using the amenities that modern C++ brings. Writing an application in 2020 doesn’t mean that we have to forget about 2000 though: The unrivaled compatibility of the Win32 API makes it possible to run your modern application even on Windows 2000 and older Windows XP versions – in theory at least! In practice, applications compiled with Visual Studio’s build toolset for Windows XP development (called v141_xp) require at least Windows XP with Service Pack 3.

I don’t want to advocate for using old and unmaintained operating systems here. But if you are working at a company like ENLYZE, you meet them at customers everyday: Industrial control systems have a lifetime of 20 years or more, so you will inevitably deal with an unsupported operating system at some point. To make matters worse, these systems are usually full of proprietary software that has never been tested in a different environment. Hence, upgrading them is no option either.

Anyway, if you just want to compile a modern C++ application for an old Windows version, look no further.

Tracking application dependencies

Why does an application actually run or not run under a certain Windows version?
The first answer to this question is always given by Dependency Walker. That tool has historically been shipped with multiple Microsoft development packages and is now available as a standalone download from the aforementioned link. Unfortunately, development on it has ceased and the last version exhibits poor performance on recent Windows versions. However, there is a modern reimplementation of it available on GitHub.

Both tools reveal the DLL files (shared libraries) on which an application depends, along with the called API functions, in a nice graphical user interface. Let’s try it out with the Wizard-2020 demo application from the previous blog post on Windows 2000:

First of all, Dependency Walker tells us that GDIPLUS.DLL is missing. This is an expected and minor issue though: I have deliberately used the GDI+ graphics library in Wizard-2020 to deal with PNG files, knowing that the library comes with Windows XP and later versions. Support for Windows 2000 was not a requirement back then. Fortunately, the DLL can be freely downloaded for older Windows versions (look for Platform SDK Redistributable: GDI+), and also comes preinstalled with Windows 2000 Service Pack 4. This time, Visual Studio is not at fault.

The situation is different with the APIs marked in red on the right side. These APIs have not been called from the Wizard-2020 source code, yet our final application somehow depends on them. The answer to this can be found in the sources of the various runtimes - the code that implements basics like printf or std::thread and is used by almost every C/C++ application.
The paths to these source files vary depending on the Visual Studio version. For my Visual Studio 2019 installation with the v141_xp toolset, they are in C:\Program Files (x86)\Windows Kits\10\Source\10.0.10240.0\ucrt (C Runtime) and C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.16.27023\crt\src (C++ Runtime).

Searching through these sources indeed reveals multiple calls to the DecodePointer and EncodePointer APIs highlighted in the screenshot above. Apart from that, the compiled Wizard-2020.exe also depends on GetModuleHandleExW and InitializeSListHead that Windows 2000 lacks.

More complex applications require even more operating system APIs. C++11 classes like std::mutex and std::thread are common culprits here. When using them in your code, Visual Studio’s Concurrency Runtime (ConcRT) gets imported and a lot more APIs from KERNEL32.DLL soon become requirements to run your application.

If you now think “Hey, let’s just rewrite the affected parts”, I have some more bad news for you: We can only view the runtime sources, but not rebuild changed versions of them.

So is there any other way to get rid of all these dependencies without sacrificing functionality?

How API calls work

Let’s have a look at the DecodePointer API that lives in KERNEL32.DLL. Its definition in the WinBase.h header (or utilapiset.h in newer SDKs) is as follows:

WINBASEAPI
__out_opt
PVOID
WINAPI
DecodePointer (
    __in_opt PVOID Ptr
    );

When hovering the mouse over WINBASEAPI in Visual Studio, it will resolve it to __declspec(dllimport). This tells the compiler to replace calls to DecodePointer by calls to _imp__DecodePointer. Likewise, WINAPI expands to __stdcall, which specifies the so-called calling convention. We only need to know that all stdcall functions are decorated with a leading underscore and the byte size of the passed parameters. Our _imp__DecodePointer calls therefore become calls to a symbol named __imp__DecodePointer@4 (PVOID is a pointer and pointers are 4 bytes on 32-bit CPUs).

That __imp__DecodePointer@4 symbol is provided by the import library file kernel32.lib (which every Win32 Visual Studio project links by default). Such an import library file doesn’t implement the actual API either. Instead, it tells the linker that the API can be found in KERNEL32.DLL and instructs it to add a corresponding entry to the import directory of the resulting executable file. When starting your application, the operating system parses the import directory, loads all required DLLs into memory, and resolves references to APIs in those DLLs. After that process, the __imp__DecodePointer@4 symbol contains the memory address of the DecodePointer function from KERNEL32.DLL and thereby becomes callable.

What if the linker wouldn’t take __imp__DecodePointer@4 from kernel32.lib, but from somewhere else?
Then no corresponding entry would be added to the import directory and the dependency to DecodePointer from KERNEL32.DLL would be gone. This is exactly how we’re going to deal with it!

Overwriting DLL imports

Now how can we actually create a symbol called __imp__DecodePointer@4? Defining that as a global variable in C code doesn’t work, because @ is not permitted inside a variable name.

It turns out that we have to move one layer deeper again and introduce some x86 assembly code. Thankfully, Visual Studio still comes with the Microsoft Macro Assembler. Adding support for x86 .asm source files is as simple as right-clicking the project → Build Dependencies → Build Customizations and ticking the masm checkbox there.

For diverting __imp__DecodePointer@4 and __imp__EncodePointer@4, I have created a winnt_51_forwarders.asm source file containing the following:

.model flat

EXTERN _LibDecodePointer@4 : PROC
EXTERN _LibEncodePointer@4 : PROC

.data

PUBLIC __imp__DecodePointer@4
__imp__DecodePointer@4 dd _LibDecodePointer@4

PUBLIC __imp__EncodePointer@4
__imp__EncodePointer@4 dd _LibEncodePointer@4

END

These lines define two public __imp__ symbols and set them to the function addresses of _LibDecodePointer@4 and _LibEncodePointer@4. Again, we have to use decorated symbol names throughout the entire assembly code.

Fortunately, this is the only excursion we have to do into assembly territory. Things now get a little more comfortable again in the accompanying “winnt_51.cpp” C++ source file, where we implement regular C functions called LibDecodePointer and LibEncodePointer. Let me exemplarily show you the LibDecodePointer implementation:

extern "C" PVOID WINAPI
LibDecodePointer(PVOID Ptr)
{
    if (!pfnDecodePointer)
    {
        // Check if the API is provided by kernel32, otherwise fall back to our implementation.
        HMODULE hKernel32 = GetModuleHandleW(L"kernel32");
        pfnDecodePointer = reinterpret_cast<PFN_DECODEPOINTER>(GetProcAddress(hKernel32, "DecodePointer"));
        if (!pfnDecodePointer)
        {
            pfnDecodePointer = _CompatDecodePointer;
        }
    }

    return pfnDecodePointer(Ptr);
}

I’m basically replicating the public interface of the DecodePointer function here. On the first call, my function calls GetProcAddress to check if the current operating system provides DecodePointer through its KERNEL32.DLL. If that is the case, I’m merely forwarding to the operating system API. Otherwise, I’m calling my own reimplementation in _CompatDecodePointer instead. The result of the check is stored in a global pfnDecodePointer variable, which is declared as follows:

typedef PVOID (WINAPI *PFN_DECODEPOINTER)(PVOID Ptr);
static PFN_DECODEPOINTER pfnDecodePointer = nullptr;

Finally, what do DecodePointer and EncodePointer actually do? :)
I need to know that for writing compatible reimplementations in _CompatDecodePointer and _CompatEncodePointer.

Fortunately, the related Microsoft documentation is enough this time, and Raymond Chen has some background details again. As long as we implement both functions symmetrically, we can go with the simplest possible implementation:

static PVOID WINAPI
_CompatDecodePointer(PVOID Ptr)
{
    // Just return the input pointer without any decoding.
    return Ptr;
}

static PVOID WINAPI
_CompatEncodePointer(PVOID Ptr)
{
    // Just return the input pointer without any encoding.
    return Ptr;
}

That’s it! Add all these files to your project, build it, and the resulting executable will no longer depend on DecodePointer and EncodePointer.

Doing things at scale

In a perfect world, it would be sufficient to just reimplement DecodePointer and EncodePointer. In fact, this already enables some applications to run under Windows XP and Windows XP SP1, when they previously required at least Windows XP SP2. However, this is not enough for Windows 2000 compatibility, and neither for applications using features of the Concurrency Runtime (e.g. std::thread).

I have therefore written a test application using std::mutex and std::thread, checked it with Dependency Walker on multiple operating systems, and ended up with the following list of APIs to reimplement:

Before reimplementing any of those, it’s important to first check their usages in the runtime code. For example, GetLogicalProcessorInformation is imported, but only used in the code path for Windows Vista or later. Hence, my “reimplementation” can simply return FALSE and that’s it.

Finding detailed explanations about the remaining APIs is also easier than you may think. Microsoft has documented a lot more APIs in the recent past, and also the ReactOS source code is a valuable treasure of information.

Rewiring the reported Windows version

Apart from relying on newer APIs, the Concurrency Runtime also checks the operating system version, and simply throws an exception when encountering Windows 2000. After reimplementing the APIs above, this remaining limitation is purely arbitrary and just needs a workaround.

We again cannot change the Concurrency Runtime, hence we somehow need to intercept that version check and make it believe that we’re on Windows XP. The guilty code is in C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.16.27023\crt\src\concrt\ResourceManager.cpp. Opening that file, we can see that it simply calls GetVersionExW and checks dwMinorVersion to decide between Windows 2000 and XP.

As you might already have guessed, the solution is again to divert an API call, this time GetVersionExW. However, we don’t want to reimplement it from scratch, but rather “correct” the reported version. My variant looks as follows:

extern "C" BOOL WINAPI
LibGetVersionExW(LPOSVERSIONINFOW lpVersionInformation)
{
    if (!pfnGetVersionExW)
    {
        // This API is guaranteed to exist.
        HMODULE hKernel32 = GetModuleHandleW(L"kernel32");
        pfnGetVersionExW = reinterpret_cast<PFN_GETVERSIONEXW>(GetProcAddress(hKernel32, "GetVersionExW"));
    }

    if (!pfnGetVersionExW(lpVersionInformation))
    {
        return FALSE;
    }

    // Check if we are running on Windows 2000.
    if (lpVersionInformation->dwMajorVersion == 5 && lpVersionInformation->dwMinorVersion == 0)
    {
        // Pretend to be Windows XP, which is the minimum version officially supported by the CRT.
        // If we don't do that, the CRT throws ::Concurrency::unsupported_os() in ResourceManager::RetrieveSystemVersionInformation.
        // Fortunately, this is the only function calling GetVersionExW.
        lpVersionInformation->dwMinorVersion = 1;
    }

    return TRUE;
}

This is where things start to get dirty though. While we previously only reimplemented non-existing APIs in a compatible manner, we’re now changing the behavior of an API call. Take into account that you now cannot easily distinguish between Windows 2000 and Windows XP in your own code, should you be using GetVersionExW somewhere else. Other means of querying the operating system version, such as GetVersion and WMI, are unaffected by this change though.

Introducing EnlyzeWinCompatLib

Nobody should ever be forced to do this work twice, which is why I have released the result as a reusable static library: The EnlyzeWinCompatLib.
Make sure to check out the msvc-v141-xp branch of it.

In Visual Studio, simply add its project file to your solution, let your application project depend on it (right-click References → Add Reference), and you are almost done. Almost, because by default, Visual Studio links all Windows import libraries first before even looking at EnlyzeWinCompatLib. To change that, you need to untick the Inherit from parent or project defaults option in Project Properties → Linker → Input → Additional Dependencies. Then make sure to include my EnlyzeWinCompatLib.h header file at the top of your application’s global header file to add the now missing import libraries back into the link process, but after linking EnlyzeWinCompatLib.

Patching the minimum OS version

Every Windows executable file carries Operating System Version and Subsystem Version fields in its header, which specify the minimum Windows version required to run this application. If Windows encounters an application with a version number higher than its own operating system version, it will simply refuse to run that application:

Any new application created with Visual Studio 2017 or 2019 sets that version number to 6.0 by default, which is the internal version of Windows Vista. If you select the v141_xp toolkit for building a Windows XP-compatible application, that version number is set to 5.01, the internal version of Windows XP. The related project properties field can be found in Linker → System → Minimum Required Version.

One might think that we can just go and change that field to 5.0, the internal version of Windows 2000. The reality is different though: Current versions of the Microsoft linker simply ignore any version setting below 5.01 silently. This can only be explained as a purely arbitrary decision from Microsoft’s side. The executable format hasn’t changed from Windows 2000 to XP and there are no other technical reasons for that.

Fortunately, the header of an executable file is fairly simple and not protected by any checksum. We can therefore just patch the relevant bytes after build.

For this task, I have written a PowerShell script that comes along with EnlyzeWinCompatLib: patch_exe_os_version.ps1
I found PowerShell to be the right tool for the job, as it integrates nicely into today’s CI systems for Windows.

Linking runtimes statically

It should be noted that my Wizard-2020 demo project has been configured to link the C and C++ runtimes statically. This is an important prerequisite to overwrite API calls of the runtimes at all, and also simplifies deployment of your application.

By default, Visual Studio creates new projects with a dependency on runtime DLLs. They have names like MSVCP140.DLL, VCRUNTIME140.DLL or api-ms-win-crt-runtime-…dll. The idea is that your application stays small and a single runtime can be shared by multiple applications and updated centrally. However, the downside is that your application won’t work at all before the runtime is installed. Additionally, my method for diverting API calls doesn’t work if a call originates from an external DLL.

Fortunately, Visual Studio makes it very simple to change from dynamic linking to static linking of the runtimes. Just open the Project Properties dialog, go to C/C++ → Code Generation and change Runtime Library from Multi-threaded DLL (/MD) to Multi-threaded (/MT) for the Release configuration. Likewise, you can also change from Multi-threaded Debug DLL (/MDd) to Multi-threaded Debug (/MTd) in the Debug configuration to statically link runtimes in your debug builds.

The reward for all the work

Here it is, Wizard-2020 running under Windows 2000:

Conclusion

Using EnlyzeWinCompatLib, you have a rather simple way to write standalone Win32 applications that leverage modern C++ features, but are nonetheless compatible down to Windows 2000. Unlike methods involving alternate compilers like MinGW/Mingw-w64, my solution uses the known Visual Studio compiler and is fully integrated into a Visual Studio project. Hence, applying this method to any existing Win32 C++ project should be straightforward.

This method has an expiration date though. While the Visual Studio 2017 toolset for Windows XP development (v141_xp) can still be used in Visual Studio 2019, this may not be the case for the next Visual Studio release. Furthermore, language features are limited to the C++17 subset that is supported by the Visual Studio 2017 compiler. Any new developments won’t find its way back to that compiler version. This also applies to Windows features, because the v141_xp toolset is similarly limited to the legacy Windows SDK 7.1. Finally, support for even older Windows versions (like NT 4.0) is off the table, due to the Concurrency Runtime using many APIs that can’t be easily reimplemented for such old releases.

In my upcoming post, I will therefore go with time – in both directions. I will present a new compiler that comes with Visual Studio 2019 and has not been abandoned, hence receiving new features every day. Surprisingly, all components of that new compiler together make it even easier to target old Windows versions. We will therefore dig a little further into archeology and build applications that are compatible down to Windows NT 4.0 from 1996. Stay tuned!


Colin Finck is working as a Software Engineer at ENLYZE. He has been digging into Windows internals for over a decade as a core member of the ReactOS Project.