Gem #79: Where did my memory go? (Part 3)

Let's get started…

Part III: External Tools

The previous two Gems on memory monitoring explained the use of GNAT.Debug_Pools and GNATCOLL.Memory to monitor memory access and detect memory leaks.

Some very powerful external tools exist on some systems that can be used to achieve similar goals. Very often, these tools will simply replace the system calls malloc and free, although in some cases they will in fact emulate a virtual machine in which your application is run.

One common kind of tool, available on most systems, indicates how much memory your application is using (such as the Process Manager on Windows or "top" on Unix systems). This is however a very limited tool, since memory that is properly freed by your application might in fact not be given back to the system (for performance reasons, malloc keeps it and reuses it for the next allocation). So it is very hard to discover memory leaks that way, and of course even if you can see that there is a leak, you have no way of knowing where it is in your code.

One very useful Linux application is valgrind. This is a virtual machine, that you start with various tools. One of them is a memory checker, which can detect invalid memory accesses, double deallocation, use of uninitialized variables, and memory leaks. You do not need to do anything special when compiling your application and you can simply run it as follows:

valgrind --tool=memcheck your_app app_arguments

If you also want to detect memory leaks, start your application with:

valgrind --tool=memcheck --leak-check=full --leak-resolution=med your_app

Compared to the techniques we discussed in the previous Gems, valgrind is much slower (since all code runs in a virtual machine), but more accurate. For instance, when it reports memory leaks, it will only report those chunks of memory that are no longer referenced anywhere in your application. For instance, if you have a global constant initialized once by calling "new ...", the Ada packages would report it as a leak, whereas valgrind by default knows it is still referenced and will not bother you with it (although, of course, you have an option to see it, namely --show-reachable). Since it is often very hard to free such memory (you have to do it at finalization time), and generally not worth it since the memory will be reclaimed by the system anyway, you generally want to ignore those chunks.

Likewise, if you have an allocated object that itself contains accesses to dynamically allocated memory, valgrind will by default only report the root object as a leak, since the other chunks are accessible from the first. When you fix the leak for the first, it is likely that you will at the same time fix the other leaks as well.

Other sources of memory leaks, much harder to detect, are those that occur in the graphical part of your application. On most systems, graphical rendering is a client-server process, and the memory is allocated on the server, not on the client. Therefore, tools like valgrind or the GNAT packages would not be able to report it as a leak (for instance if you have allocated a big pixmap but never freed it). On Windows these are called "GID", and are visible in the Process Manager, so that you can actually monitor whether your application seems to have such leaks. Once again, though, this provides no detail as to the origin of the leak.

On Unix, you can use the "xrestop" application, which can tell you the number of windows, cursors, graphic context, etc. that you have allocated.

Many such tools like this exist, both commercial and free. Let us know if you routinely use such tools, as they could be useful to other readers of this Gem.