While writing unit tests for my C code, I decided it would be nice to run the tests in Valgrind, so I could check for memory leaks too. It turned out to be easier than I expected, but still a lot harder than it should have been.

The problems I ran into are:

  • Autotools projects build with libtool by default, which means that running the tests against the created "binaries" actually runs them against libtool's shell scripts. Since bash uses memory in weird ways, that makes it hard to check for actual memory leaks in my code.

  • Valgrind makes the tests much slower. As much as I'd like to run Valgrind every time, it's not worth it, so I want a separate make check-valgrind.

  • My code frequently logs at critical level, and I don't want those logs to crash the tests (so don't run with G_DEBUG=fatal-criticals!). If an error is fatal, I already log it as an error.

The first problem makes it hard to add Valgrind tests without potentially messing up builds on other platforms, and the other two make AX_VALGRIND_CHECK useless to me.

What I ended up with was just defining a check-valgrind target in my Makefile.am:

.PHONY: check-valgrind ;
check-valgrind: $(TESTS)
    @for test in $$(echo $(TESTS) | sed 's/tests\//tests\/.libs\//g') ; do \
        CK_FORK=no LD_LIBRARY_PATH=h264bitstream/.libs $(VALGRIND) --error-exitcode=1 --leak-resolution=low --quiet \
            --leak-check=full --show-possibly-lost=no --suppressions=tests/valgrind.suppressions $${test} ; \
    done

The sed usage is a hack because I couldn't figure out how to get a variable passed through to the resulting Makefile. What I would have preferred to do is to use $(subst "tests/", "tests/.libs/", $(TESTS)). CK_FORK=no is there just so I'll get one valgrind report per suite rather than per test. I had to add this suppression file to ignore "possibly lost" memory in GLib:

{
   glib _dl_init
   Memcheck:Leak
   fun:*alloc
   ...
   fun:_dl_init
}

It's obviously not ideal, but it runs the tests exactly how I want, without messing with make check, and now I get nice valgrind checks:

$ make check-valgrind
Running suite(s): Bit Reader
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): CETS ECM
100%: Checks: 9, Failures: 0, Errors: 0
Running suite(s): Descriptors
100%: Checks: 9, Failures: 0, Errors: 0
Running suite(s): MPD
100%: Checks: 16, Failures: 0, Errors: 0
Running suite(s): Program Specific Information
100%: Checks: 13, Failures: 0, Errors: 0
==5972== 280 bytes in 5 blocks are definitely lost in loss record 6 of 13
==5972==    at 0x4A08946: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==5972==    by 0x406875: conditional_access_section_new (psi.c:572)
==5972==    by 0x406875: conditional_access_section_read (psi.c:605)
==5972==    by 0x4035AF: test_read_cat_no_descriptors (psi.c:147)
==5972==    by 0x4E6680A: ??? (in /usr/lib64/libcheck.so.0.0.0)
==5972==    by 0x4E66B56: srunner_run (in /usr/lib64/libcheck.so.0.0.0)
==5972==    by 0x4052E6: main (main.c:79)
==5972== 
Makefile:1460: recipe for target 'check-valgrind' failed
make: *** [check-valgrind] Error 1