R3T - Ruby Threading Test Tools

What is R3T?

R3T can help you to find, resolve and monitor race conditions.

Installing

You can install R3T as a gem:

sudo gem install R3T

Alternatively you can download the source code from RubyForge.

Example usage

The Test Case

Here is a quite simple example of how to use R3T. Can you spot the race condition?

require 'test/unit'
require 'rubygems'
require 'threading_test_tools'

class ThreadingTest < Test::Unit::TestCase
  include ThreadingTestTools
  
  class A
    attr_accessor :value
  end
  
  def test_object_race_condition
    mutex = Mutex.new
    a = A.new
    a.value = 0
    check_race_condition do
      2.threads do
        mutex.synchronize do
          b = a.value
          a.value = b + 1
        end
      end
    end
  end
end

This test will pass without complaints because the race condition is resolved by the Mutex. R3T has checked both, first that there is a race condition and second that it is resolved. I will illustrate this further by changing the test...

Removing the race condition

The first pass which R3T performs is to disable synchronization classes (Mutex, Monitor and Sync) and search for race conditions.
If we remove the race condition by commenting out the line "a.value = b + 1", the test will fail with the following message:

No race condition could be found (with disabled synchronizations).

Your conclusion could be now that you can remove the synchronization because it is of no need any more.

Removing the mutex

Now the other, more advanced way around: In the second pass R3T searches for race conditions with synchronizations enabled and expects to find any because they should all be resolved.
But what if this is not the case? We will comment out "mutex.synchronize do" and the appropriate "end" to see what happens:

An unhandled race condition has been found (with enabled synchronizations).
Exception raised:
Class: <ThreadingTestTools::RaceConditionFound>
Message: <"Race condition found when passing to next thread at threading_test.rb:20: a.@value should be 2, but was 1.">

R3T has successfully detected the race condition and gives you more information where you can find it:
First it tells you where it has forced a pass to the other thread to provoke the race condition. In this case this is the line "a.value = b + 1", sure.
Second it tells you what is different. Here it is "a.@value" which is 1 instead of 2 when the race condition occurs, because b is 1 in both threads.

ObjectGraph Support

Sometimes even these both hints might be not enough to spot an unknown race condition. You can use R3T's ograph support then (ograph is also available as a gem on RubyForge). Simply add the following flag:

check_race_condition :create_ographs => true do [...]

R3T will generate two graph images now when an unhandled race condition is found. They look like this:

ograph_expected.png

ograph_race_condition.png

You can see the differencing "2" and "1" in the lower right corner.

Comparison flags

Some times there may be some variables which are naturally different at each run (for example things concerning the current time). You can exclude them from the comparison by setting their data_equal_ignore attribute to true at the end of the check_race_condition block. Consider the use of instance_variable_get to get access to private object attributes in this case.
Additionally arrays are are by default only checked if they include all entries, not if they are in the same order. You can enforce this by setting the data_equal_check_order attribute of the array to true.

How does R3T work?

As I already wrote, R3T operates in two essential steps. In fact, there is a third, "analyze" step at the very beginning, where R3T runs the given test for the first time and collects the following data: The state of all test variables before the run (to be able to reset to them), the state of them after the run (considered as the expected result for later comparison) and a list of all Ruby interpreter events which occur during the run.
The next two steps are essentially the same, only that the first is done with synchronizations disabled and the expectation that a race condition will occur and the other way round. Both perform their search for the race condition in the same way:
For every event in the collected list, all test variables are first reset to their initial state and then the test is run. When it reaches the event of the current try, R3T injects a manual Thread.pass at this position which will force the thread scheudler to proceed to the other thread. At the end of the test it will compare its variables to the expected ones and raise a race condition exception if they differ. Thus, each possible race condition case will be tried.
Nevertheless, this is not everything needed to get it working properly, but keep this overall structure in mind, because it might help you to understand what is going on when writing some tests.

Links

RubyForge overview page

R3T is written by Richard Musiol