14 Jun

11910

0

Machining Virtually

As I said in the last report, the week began by setting up a development environment in a Windows Server 2003 VM.

I had to install Visual Studio 2010 and RosBE, and then download and compile the source code to verify that everything was working correctly. When I tried to compile, I realized two things. First, that vs2010 is much buggier than I remembered, and second, that compiling in a Virtual Machine with one cpu core is very very slow.

So I spent some time installing the Service Pack to VS2010 (it takes around an hour to install, for some reason), and then an even longer time trying to convert the VM from single-core to multi-core.

In the end, I had to resort to finding my win2003 disk, and unpacking the SMP kernel file, the SMP ACPI kernel file, and the ACPI SMP HAL dll.

After all this was done, I took a snapshot of the VM, booted it, and asked Amine to guide me to how to install and use the leak detector he had been trying.

Getting the instrumented code to build, required a certain function that does not exist in the react msvcrt. With this function added, the code was able to compile correctly, but it would crash when trying to run it.

I was stumped for many hours, because the function that was having issues seemed to execute fine, yet there was some corruption right afterwards. It made no sense, until I remembered that the code was instrumented, which means it had some extra code at the end, which would run when the function exited. I used the debugger to step through the disassembly view of the function, and there I noticed that the EAX register (the one used to keep the return value of functions), was correct right until a function ran, and this function turned out to be the one Amine gave me to fix the build errors!

This function seemed to cause the issues by overwriting the contents of EAX, but the annoying part is that the generated code, if it needed the value in EAX, it should have saved it and then restored it, but that wasn’t happening. This meant one of two things: either the instrumentation was generating code with the assumption that the function wouldn’t touch EAX, or the compiler somehow bugged while generating the code. And I’m strongly tempted to assume it’s the former.

Now since the instrumented code is automatically generated by the tool, my solution was to manually save and restore the contents of EAX within the function, using inline assembly. The alternative was to write the whole function in assembly so that EAX is never actually corrupted, but this meant doing unnecessary work, since it wasn’t my goal to write a correct function, only to try to use the tool to find leaks.

With the register corruption out of the way, the instrumented filebrowser and explorer-new would start fine, unless I ran them through the analyser, in which case it’s not even able to complete the runtime initialization previous to starting the actual program.

I spent a whole lot of hours more trying to figure out why could that possibly happen, until I noticed it was the “final check” option of the leak analyser which, as long as it was enabled, would crash at startup.

Disappointingly, disabling said option allowed the program to run, but the tool was only able to capture basic memory leaks, and not the more complex leaks of COM references, or the ones I was truly interested in: USER handles.

It also didn’t help that the past week has been very hot, by spring standards, and the place I currently live in has no AC and a whole lot of mosquitos outside.

Again, this week has had no visual progress, so there is nothing to show. Hopefully it won’t be too boring if I leave the report image-free just this once?

Oh, what the heck. Thanks to The Cat API, every time you view this page you will get a completely random cat picture -- but to avoid spamming the site, you can also click on it to go to their website and watch them there.


Discussion: https://www.reactos.org/forum/viewtopic.php?f=2&t=13429

This blog post represents the personal opinion of the author and is not representative of the position of the ReactOS Project.