Fixed loading speed

I dunno who is reading 10 year old content, but this still gets a few hits... and it was loading extremely slowly due to some linked javascript. I removed it, makes it load much quicker (and given the js failed to load, the syntax highlighting was already useless).

Back from the dead!

Okay, not really. Had a request to write a complete program from the speedfan post I made forever ago.

Here is the main cs file for a simple console app that logs to stdout or to a file if one is given on the command line.
program.cs


EDIT: This is only meant as an example. I don't know how well this works, its been a long time since I've played around with this stuff. If you really need a decent logging tool, check out HWMonitor Pro. Nothing to do with me, and it will cost you a small amount, but its a solid program.

TreeView Scraping

Last time we looked at pulling text from an object... and noted that some more complex objects didn't respond to WM_GETTEXT. Well, today we are going to poke around in a TreeView (SysTreeView32 is what its reported as specifically) widget. Just so we are clear with what we are talking about:

When we use the code presented last time on the above application, we get a handle for the TreeView object, and RealGetWindowClass() returns "SysTreeView32", and WM_GETTEXT returns nothing. Because the nodes are not exposed as child windows, we only get a single handle for the whole thing... and WM_GETTEXT is not meaningful in that context. So what we actually want to do, is recognise we have a TreeView object, and send TreeView specific messages to it.

Lets get the DllImports out of the way soon. These are the Win32 API functions we will be using:

#region WIN32_DLLIMPORT
       [DllImport("user32.dll", EntryPoint = "SendMessage")]
       private static extern IntPtr SendMessage(IntPtr hWnd, int wMsg, int wParam, IntPtr lParam);
       [DllImport("user32.dll")]
       static extern IntPtr GetWindowThreadProcessId(IntPtr hWnd, out int lpdwProcessId);
       [DllImport("kernel32.dll")]
       static extern IntPtr OpenProcess(int dwDesiredAccess, bool bInheritHandle,int dwProcessId);
       [DllImport("kernel32.dll")]
       static extern bool CloseHandle(IntPtr hObject);
       [DllImport("kernel32.dll")]
       static extern IntPtr VirtualAllocEx(IntPtr hProcess, int lpAddress, int dwSize,int flAllocationType, int flProtect);
       [DllImport("kernel32.dll")]
       static extern bool VirtualFreeEx(IntPtr hProcess, IntPtr lpAddress, int dwSize, int dwFreeType);

       [DllImport("kernel32.dll")]
       static extern bool WriteProcessMemory(IntPtr hProcess, IntPtr lpBaseAddress, IntPtr lpBuffer, int nSize, IntPtr lpNumberOfBytesWritten);
       [DllImport("kernel32.dll")]
       static extern bool ReadProcessMemory(IntPtr hProcess, IntPtr lpBaseAddress, IntPtr lpBuffer, int nSize, IntPtr lpNumberOfBytesRead);


       const int PROCESS_ALL_ACCESS = 0x0008 | 0x0010 | 0x0020;
       const int MEM_COMMIT = 0x1000;
       const int PAGE_READWRITE = 0x04;
       const int LVIF_TEXT = 0x0001;
       const int MEM_RELEASE = 0x8000;
#endregion

Assuming that we are scraping a separate process, this is not quite as easy as just sending a WM_GETTEXT, as we need to send a structure which needs to be manipulated within the other process... this requires the other process to be able to access the memory, with a pointer that makes sense to it. Fortunately, we can create memory in another process with VirtualAllocEx()!

We also have some supporting const's, with values plundered from C header files.

Now, we also need to define some structures and constants that are specific to TreeView objects. It should be noted that the method to access ListView (and possibly other) objects is nearly identical.

       public const int TV_FIRST = 0x1100;
       public const int TVIF_TEXT = 0x0001;
       public const int TVIF_PARAM = 0x4;

       public enum TV_Messages
       {
           TVM_GETNEXTITEM = (TV_FIRST + 10),
           TVM_GETITEM = (TV_FIRST + 62),
           TVM_GETCOUNT = (TV_FIRST + 5),
           TVM_SELECTITEM = (TV_FIRST + 11),
           TVM_DELETEITEM = (TV_FIRST + 1),
           TVM_EXPAND = (TV_FIRST + 2),
           TVM_GETITEMRECT = (TV_FIRST + 4),
           TVM_GETINDENT = (TV_FIRST + 6),
           TVM_SETINDENT = (TV_FIRST + 7),
           TVM_GETIMAGELIST = (TV_FIRST + 8),
           TVM_SETIMAGELIST = (TV_FIRST + 9),
           TVM_GETISEARCHSTRING = (TV_FIRST + 64),
           TVM_HITTEST = (TV_FIRST + 17),
       }

       public enum TVM_EXPAND
       {
           TVE_COLLAPSE = 0x1,
           TVE_EXPAND = 0x2,
           TVE_TOGGLE = 0x3,
           TVE_EXPANDPARTIAL = 0x4000
       }

       public enum TVM_GETNEXTITEM
       {
           TVGN_ROOT = 0x0,
           TVGN_NEXT = 0x1,
           TVGN_PREVIOUS = 0x2,
           TVGN_PARENT = 0x3,
           TVGN_CHILD = 0x4,
           TVGN_FIRSTVISIBLE = 0x5,
           TVGN_NEXTVISIBLE = 0x6,
           TVGN_PREVIOUSVISIBLE = 0x7,
           TVGN_DROPHILITE = 0x8,
           TVGN_CARET = 0x9,
           TVGN_LASTVISIBLE = 0xA
       }

And the actual structure, complete with marshalling hints:

       [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
       public struct TVITEM
       {
           public int mask;
           public IntPtr hItem;
           public int state;
           public int stateMask;
           public IntPtr pszText;
           public int cchTextMax;
           public int iImage;
           public int iSelectedImage;
           public int cChildren;
           public IntPtr lParam;
       }

Okay, so we have everything set up, now lets run through the steps needed to actually pull this off! First, its assumed that our application has found the handle for a TreeView object in another application. Then we want to use the various flavours of TVM_GETNEXTITEM to get a handle for each node in the tree. Lastly, we want to query the node with TVM_GETITEM to find out what the actual text is. So a fair bit more involved than previously. But, one step at a time!

Lets get the root node to start with:

       retval = SendMessage(hWnd, (int)TV_Messages.TVM_GETNEXTITEM, (int)TVM_GETNEXTITEM.TVGN_ROOT, IntPtr.Zero);

Now we have a handle to the node, lets actually extract the information. I'll go through this step by step, as there is a lot in it:

First, we need to find the PID of the target application, and open it with certain access rights. Note that this will only work if we are running as administrator!

       threadId = GetWindowThreadProcessId(this.hWnd, out processID);
       if ((threadId == IntPtr.Zero) || (processID == 0))
           throw new ArgumentException("hWnd");

       hProcess = OpenProcess(PROCESS_ALL_ACCESS, false, processID);
       if (hProcess == IntPtr.Zero)
           throw new ApplicationException("Failed to access process");

Now that we can access the other process, we need to allocate a blob of memory that can fit TVITEM and whatever data we are retrieving.

       remoteBuffer = VirtualAllocEx(hProcess, 0, bufferSize, MEM_COMMIT, PAGE_READWRITE);
       if (remoteBuffer == IntPtr.Zero)
           throw new SystemException("Failed to allocate memory in remote process");

Before we call TVM_GETITEM we need to populate the TVITEM structure with some values to indicate what information we want and how much space is available to fill it. This is a two step operation. Firstly we want to allocate some memory locally (ie in our process), and fill it out with the values we want. We will then write this structure into our remotely allocated block of memory.

To complicate things, we need more space than just the structure, we also need a block of space for the string that will (hopefully!) get returned.

       tvItem = new TVITEM();
       int size = Marshal.SizeOf(tvItem);

       tvItem.mask = TVIF_TEXT;
       tvItem.hItem = hItem;
       tvItem.pszText = (IntPtr)(remoteBuffer.ToInt32() + size + 1);
       tvItem.cchTextMax = bufferSize - (size + 1);

For the sake of convenience, we are allocating a single block of memory, and going to write first the structure into it, and then use the remainder of the buffer for pszText. But since this structure is going to be written to the remote process, we need to make sure that pszText points to the right place, hence the above we set pszText to a position with removeBuffer.

We now have a tvItem that we want to write into the remote processes memory... however, its currently managed memory, so we need to figure out a way to marshal it and get it into unmanaged memory

       IntPtr localBuffer = Marshal.AllocHGlobal(Marshal.SizeOf(tvItem)) ;
       Marshal.StructureToPtr(tvItem, localBuffer, false);

Now to write that into the remote process:

       bSuccess = WriteProcessMemory(hProcess, remoteBuffer, ptr , size,IntPtr.Zero);
       if (!bSuccess)
           throw new SystemException("Failed to write to process memory");

Phew. Still with me? we are now, finally, ready to call SendMessage()!

       SendMessage(hWnd, (int)TV_Messages.TVM_GETITEM, 0, remoteBuffer);

Nothing we haven't seen before. But now we have another problem. We have (hopefully anyway) a structure in the remote processes memory that we want locally! So we need to reverse the above process:

       bSuccess = ReadProcessMemory(hProcess, remoteBuffer, localBuffer, bufferSize, IntPtr.Zero);
       if (!bSuccess)
           throw new SystemException("Failed to read from process memory");

       TVITEM retItem = (TVITEM) Marshal.PtrToStructure(localBuffer, (Type)typeof(TVITEM));

Cool, we have a TVITEM.... but its not quite right. It still contains a pointer to the unmanaged buffer (ie pszText), so lets marshall that too:

       String pszItemText = Marshal.PtrToStringUni((IntPtr)(localBuffer.ToInt32() + size + 1));

pszItemText is actually the only bit we are interested in... so we are done retrieving stuff! From here, using the above code and the right TVM_GETNEXTITEM calls, we can walk the whole tree view and pull out the text. Given the image at the start, this would end up with the word 'Root' in pszItemText.


Before we do that however, lets make sure we tidy up:

       if (localBuffer != IntPtr.Zero)
           Marshal.FreeHGlobal(localBuffer);
       if (remoteBuffer != IntPtr.Zero)
           VirtualFreeEx(hProcess, remoteBuffer, 0, MEM_RELEASE);
       if (hProcess != IntPtr.Zero)
           CloseHandle(hProcess);

Screen Scraping with C#

Something I have been messing around with lately is the ability to read other applications controls... or screen scraping. In general, screen scraping is a bad solution, using shared memory or some other IPC is much cleaner. However, there are times when screen scraping is the only practical solution (generally if the target application is closed source and the developers don't want to export the data).

So, how would we go about reading some values from another application? Windows actually makes it fair easy. As you may have seen in an earlier post, we have to use some Win32 API functions, so lets get the DllImport stuff out of the way first:

    [DllImport("user32.dll", CharSet = CharSet.Auto)]
    private static extern int SendMessage(IntPtr hWnd, int wMsg, int wParam, StringBuilder lParam);
    [DllImport("user32.dll", CharSet = CharSet.Auto)]
    private static extern int GetWindowText(int hWnd, StringBuilder lpString, int length);
    [DllImport("user32.dll", CharSet = CharSet.Auto)]
    private static extern int GetWindowTextLength(int hWnd);
    [DllImport("user32.dll")]
    static extern bool EnumChildWindows(IntPtr hWndParent, WindowEnumDelegate lpEnumFunc, int lParam);
    [DllImport("user32.dll")]
    static extern IntPtr FindWindow(StringBuilder lpClassName, StringBuilder lpWindowName);
    [DllImport("user32.dll")]
    static extern uint RealGetWindowClass(IntPtr hwnd, StringBuilder pszType, uint cchType);

    const int WM_GETTEXT = 13;
    const int WM_GETTEXTLENGTH = 14;

Okay, so if we want to find a particular window, we can use FindWindow(), like this:

    StringBuilder name = new StringBuilder("Notepad");
            
    hWnd = FindWindow(null, name);

Easy! we now have a handle for the window (or null if we couldn't find it). Now its a simple matter of finding all the child handles and getting their window text!

    public delegate bool WindowEnumDelegate(IntPtr hwnd, int lParam);

    private bool WindowEnumProc(IntPtr handle, int lParam)
    {
        int textLen;
        StringBuilder text;
        StringBuilder className = new StringBuilder(1024);
        RealGetWindowClass(handle, className, 1024);

        textLen = SendMessage(handle, WM_GETTEXTLENGTH, 0, null);
        if (textLen != 0)
        {
            text = new StringBuilder(textLen);
            SendMessage(handle, WM_GETTEXT, (textLen + 1), text);
        }

        ...

        return true;
    }

    ...
    
    WindowEnumDelegate del = new WindowEnumDelegate(WindowEnumProc);
    EnumChildWindows(hWnd, del, 0);        

The above function will be called once for each child of the window we found earlier, and it will find the type of object (className), and send a WM_GETTEXT message which will attempt retrieve text from the objects.

And there we have it! Code for a very basic screen scraper. There is, however, some issues with it. Some objects do not respond in useful ways to WM_GETTEXT messages. Some objects have a whole bunch of data contained under a single handle. For example, a TreeView object will return nothing when presented with a WM_GETTEXT message... special handling code is needed for this, and several other objects, but I'll leave that for another day (I have TreeView handling code I will post about later).

GPU-Z reader.... cause I can!

While I was looking for something, I found the data structure that GPU-Z uses in its shared memory file... and given I was doing this type of thing anyway, I figured why not?

So the following is more or less identical code to the SpeedFan code.. at least, identical in function. There were a number of little gotchas that made it a little harder than I was expecting. The documented structure is here. Kudos to the author for making it available. It is, of course, in C, and we want to access it in C#. So, we need to translate the structure into something that the Marshalling code will understand.

First a couple of constants. Nothing tricky here.

        const String SHMEM_NAME="GPUZShMem";
        const int MAX_RECORDS = 128;

Next up, the first structure, the 'record':

        [StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Unicode)]
        public struct GPUZ_RECORD 
        {
            [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
            public string key;
            [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
            public string value;
        };

        [StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Unicode)]
        public struct GPUZ_SENSOR_RECORD
        {
            [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
            public string name;
            [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 8)]
            public string unit;

            public UInt32 digits;
            public double value;
        };

A few important things to note here. The original type for key, value etc was WCHAR[], which means its a unicode string, not just a normal C string. So, its very important that we specify the CharSet explicitly. We haven't had to do that yet, but in this case, we need it.

Next, the WCHAR array is a fixed length and not just null terminated. So, to ensure we use enough bytes, we specify SizeConst, which indicates the number of array elements (not the number of bytes).

Lastly, we use string here rather than StringBuilder. We have specified the size, so the Marshalling code knows how many bytes to read, and knows how to create a string.

Now the main data structure:

        [StructLayout(LayoutKind.Sequential, Pack = 1)]
        public class GPUZ_SH_MEM
        {
            public UInt32 version;   // Version number, 1 for the struct here
            public Int32 busy;  // Is data being accessed?
            public UInt32 lastUpdate; // GetTickCount() of last update
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = MAX_RECORDS)]
            public GPUZ_RECORD[] data;
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = MAX_RECORDS)]
            public GPUZ_SENSOR_RECORD[] sensors;

            public GPUZ_SH_MEM()
            {
                data = new GPUZ_RECORD[MAX_RECORDS];
                sensors = new GPUZ_SENSOR_RECORD[MAX_RECORDS];
            }
        }

This isn't actually much different. No strings here, so we don't need to specify the CharSet, but once again we need to use SizeConst for the fixed length arrays. We also supply a constructor, to ensure the memory is allocated for the member arrays.

And that, believe it not, was the difficult bit. A failure in PtrToStruct can have some strange affects... presumably you end up overwriting random bits of memory if the sizes are not matched up, so its doubly important to get it right, as it could potentially mean random crashes in your application, without any indication why.

Full code is here. Note that in both this and the SpeedFan code, GetData() just returns the data structure. This isn't exactly the most encapsulated way to do it... but it keeps the example simple.

SpeedFanLogger

Today I wrote a bunch of code, but none of it very interesting. Basically I took the code I spoke about yesterday, called it once a second, and updated a form... and also, optionally, wrote the data to a file. In short, I created a program that will log SpeedFan data!

Yes yes, I am well aware that SpeedFan already has a logging capability! But I have bigger plans than just this...

Here is the full code for the SpeedFanReader class. It really doesn't do much... and could be re-factored to return the data a little better than just a 'Data' blob. But it serves its purpose.

Reading SpeedFan shared memory with C#

Building on what I spoke about in my previous post, lets say we want to access the data that SpeedFan provides from a C# application. As a small aside, reading information from the SMBus and other low level interfaces can only be done from the kernel. So applications like SpeedFan (HWMonitor, Everest, etc etc) generally run a driver at kernel level and then a front-end GUI to present the information.

In the case of SpeedFan, shared memory (actually its technically a memory mapped file on Windows I think) is used to communicate between the kernel driver and the userspace GUI application. Even better, the format of this file has been made public by the author of SpeedFan. So, enough talk, lets see some code!

First, we are going to need to access some Windows API functions that are not available via .net:

    public const int PROCESS_ALL_ACCESS = 0x1F0FFF;
    public const int FILE_MAP_READ = 0x0004;

    [DllImport("Kernel32.dll", CharSet = CharSet.Auto)]
    internal static extern IntPtr OpenFileMapping(int dwDesiredAccess,
            bool bInheritHandle, StringBuilder lpName);

    [DllImport("Kernel32.dll")]
    internal static extern IntPtr MapViewOfFile(IntPtr hFileMapping,
            int dwDesiredAccess, int dwFileOffsetHigh, int dwFileOffsetLow,
            int dwNumberOfBytesToMap);

    [DllImport("Kernel32.dll")]
    internal static extern bool UnmapViewOfFile(IntPtr map);

    [DllImport("kernel32.dll")]
    internal static extern bool CloseHandle(IntPtr hObject);

No dramas there.

Now lets do the dirty and actually access the file:

    StringBuilder sharedMemFile = new StringBuilder("SFSharedMemory_ALM");
    IntPtr handle = OpenFileMapping(FILE_MAP_READ, false, sharedMemFile);
    SpeedFanSharedMem sm;
    IntPtr mem = MapViewOfFile(handle , FILE_MAP_READ, 0, 0, Marshal.SizeOf((Type)typeof(SpeedFanSharedMem)));
    if (mem == IntPtr.Zero)
    {
        throw new Exception("Unable to read shared memory.");
    } 
          
    sm = (SpeedFanSharedMem) Marshal.PtrToStructure(mem, typeof(SpeedFanSharedMem));
    UnmapViewOfFile(handle);
    CloseHandle(handle);    

So there is a bit in there, lets break it down. First, we need to know the named of the file that has we are accessing. In this case its "SFSharedMemory_ALM". Next thing
to note is that we have a structure called 'SpeedFanSharedMem' which is specific to the file in question. So not only do you need to know the file name, but you also need to know the structure that resides there. Assuming you do, its pretty straight forward. Don't forget to tidy up once you are done!


The SpeedFan specific structure looks like this:

    [StructLayout(LayoutKind.Sequential, Pack = 1)]  
    private class SpeedFanSharedMem
    {
        ushort version;
        ushort flags;
        Int32 size;
        Int32 handle;
        ushort numTemps;
        ushort numFans;
        ushort numVolts;
        [MarshalAs(UnmanagedType.ByValArray, SizeConst = 32)]
        public Int32[] temps;
        [MarshalAs(UnmanagedType.ByValArray, SizeConst = 32)]
        public Int32[] fans;
        [MarshalAs(UnmanagedType.ByValArray, SizeConst = 32)]
        public Int32[] volts;

        public SpeedFanSharedMem()
        {
            temps = new Int32[32];
            fans = new Int32[32];
            volts = new Int32[32];
        }
    }

Things to note here are the attribute at the start, where we define the structure layout in memory, and the way we define the arrays with their appropriate attributes.
Nothing too tricky, just ensuring they get marshalled correctly.

That's it! Well nearly. Don't forget you need appropriate permissions to access the file (in this case, normal user permissions should be fine as we are only opening the mapping for reading).