[ntp-GSoC] progress report

Mark Vander Voord mvandervoord at gmail.com
Fri Jun 26 13:42:26 UTC 2015


Harlan:

That's a good question.

CMock is particularly useful when you start working with interaction
testing. For example, if you have a unit "Coordinator" which makes calls to
other modules, like maybe "Messaging" and "Storage". Then, instead of
having to pull in live versions of Messaging and Storage and write tests
for all three at once, you can write tests just for Coordinator, specifying
how it interacts with Messaging and Storage.

This is REALLY handy when you are developing something with test-driven
development. It helps guide your design and allows you to test against
modules that haven't even been written yet (and therefore allows you to
specify what that module's API should look like in a real-world usecase
instead of just guessing at something generic).

It is far less useful when you are retrofitting unit testing onto an
existing project. There are a couple reasons for this (I'm not familiar
enough with your codebase to know if these are problems for you or not).

(1) Often the project wasn't designed with proper separation of concerns.
One module might contain hundreds of functions, many of which call
eachother, or it might contain functions which have many many nested levels
of decision structures. Neither of these are fun to test... and when it
comes to interaction testing, you often find yourself "verifying" behaviors
that you don't really care about for the purpose of that test. Your test
might be about replying with a version when asked through the Messaging
interface, but (because of the way the code is written), you also end up
having to write up expectations for many other things that are handled in
the same function.

(2) Often the project wasn't designed with low cohesion. CMock is very
useful when you have primarily one-directional linkage between modules.
Coordinator can call Messaging and Storage, causing things to happen or
requesting data... but if Messaging also directly calls Coordinator, things
get much more brittle. You can use CMock in these cases, but a small change
often requires you to rewrite a lot of tests.

(3) This is perhaps the biggest reason: It's easy for test writers to just
fall into a pattern of following the existing code logic and then writing
expectations to match. This is tedious and doesn't result in very good
tests. The point of mocking is to build up a set of expectations and then
to verify your code meets those expectations. It's like double-accounting
in the accounting world. When you're just copying your source code over and
then adding _Expect to the call, this isn't going to prove much.

So, there MAY be places where pulling in CMock makes sense for this
project... but I would see how far you can get with just logically testing
individual units. If the project has a good separation of concerns, you'll
find that you can test a lot of units on their own... leaving just the
"middle" coordinating modules. Those modules, then, are going to be perfect
for CMock.

Ok. I think I just brain-dumped a bunch of concepts and terms that may or
may not be familiar to you guys already. If any of this is unclear or you
would like further clarification or whatever, jump in. I'm happy to do my
best to explain better. :)

Mark


More information about the GSoC mailing list