Automated bug finding with git bisect and boost::test
Here we use boost::test to write test programs for our C++ code, and git as a distributed version control system to track revisions.
One day a test that used to pass stopped working so I decided to try git-bisect to identify the commit that broke the test. This took me along a journey of shell scripting and git magic that I’d like to share.
First I chdir’d to /tmp and cloned the repo:
cd /tmp<br /> git clone ~/my_repo<br /> cd my_repo<br />
Then I created a shell script that builds the test and runs it, and placed it in the root of the git repository. This is the script, aptly called my_script:
#!/bin/sh<br /> rm -rf bin<br /> cd src<br /> bjam release -j3<br /> cd ../test<br /> bjam release TestModel -j3<br /> cd ..<br /> bin/test/gcc-4.9/release/TestModel --build_info=yes --log_level=test_suite --report_level=detailed<br /> OUT=$?<br /> if [ $OUT -eq 0 ];then<br /> exit 0<br /> else<br /> exit 1<br /> fi<br />
The shell script checks the exit code of the boost::test executable, and if that is different from zero (some tests failed) it returns 1, or 0 if all is OK. This is based on the magic shell variable $? that returns the exit code of the last run command – a trick I always forget and that seems so difficult to find with a search engine.
Now I make the script executable:
chmod u+x my_script
Then I test it here where I am (the HEAD):
./my_script<br /> echo $?<br />
This returns 1, so the test is failing. Now we can start the git-bisect journey:
git bisect reset<br /> git bisect start<br /> git bisect bad
Next we look for a previous commit where the test used to pass; let’s rewind by 20 commits and try there:
git checkout HEAD~20<br /> ./my_script<br /> echo $?<br />
Now this passes, so let’s get back to HEAD and find the corresponding commit:
git checkout master<br /> git log --pretty=oneline | head -n 21<br />
The HEAD~20 has a hash of f6aabbe80766a9745447214a89c513de29cbbe70 so let’s mark that as a good one:
git bisect good f6aabbe80766a9745447214a89c513de29cbbe70
and now we’re ready to start:
git bisect run ./my_script<br />
after some time git-bisect returns with:
3c9b0231c2dbef5a7c6a490c877116e2c5bd89c2 is the first bad commit<br />
Done !