Continuous Testing using NCrunch

Long ago I had a discussion with Jesse Fewell who was coaching the teams in my previous organization on Agile adoption. During one of the casual talks he told me that if you are a developer you should do certain things. One of those very important thing is to automate as much as possible any repetitive task. I am always on the lookout for tools and utilities which can help me in getting the job easier and faster. Recently I came across a continuous testing tool called NCrunch. This post is about my experience in using NCrunch.

Get started with NCrunch

To get started, we need to install the Visual Studio add in from www.nchrunch.net. Go to the downloads section and install the MSI package. After the successful installation we get an additional menu in the VS IDE as shown below

By default NCrunch starts in disabled mode. At the time of taking this screenshot, it is already enabled and we just need to select the first option from the menubar to toggle the NCrunch. If we are enabling NCrunch for a solution for the first time, we’ll have to configure the options for the runner. We can also edit / update the configuration settings using the Run Configuration Wizard option. Here are some things which I found useful.

CPU Configuration allows us to assign the number of CPU cores we can use during the test execution. The description is self explanatory in the configuration screen. I used the default settings, if you wish you can assign more cores to the Visual Studio or NCrunch.

 

 

 

 

Max Processing Threads allows us to configure the number of threads NCrunch should use for test execution. Once again I had used the default option. If you wish to run the tests in parallel, you can increase the number of threads.

 

 

 

 

Parallel Test Execution option is used to set up NCrunch to run tests in sequential mode or parallel mode.

 

 

 

 

 

 

 

Engine Execution Mode is used to decide when the tests should be run depending on the changes to the source code. Once again the options are self explanatory.

 

 

 

 

 

 

We are ready to use NCrunch continuous test running service with these settings. Note: I have used a project from one of my previous posts on Strategy Design Pattern for demonstration purpose here.

Lets start by looking at the Calculator class without applying the strategy pattern from the earlier demo.

We see on the left hand side that most lines have green dots. That is one of the feature of NCrunch which I like. It shows the Code Coverage for a file. The colour coding is similar to that of Visual Studio code coverage which shows different colours for lines fully covered, partially covered and uncovered using different colours. No need to enable code coverage, run tests to see which lines are covered and which are not. NCrunch makes it dead simple. In this case all the lines are covered.

Lets comment one of the test related to this file and see the impact on the inline code coverage of NCrunch. I added the Ignore attribute to one of the test method and immediately the colour changes in the source file.

That is a big plus for me. I have been using various unit testing frameworks for close to 8 years and I don’t remember any of the test runners providing such features built into the product. Some may argue that Visual Studio has Test Impact Analysis feature. I have used it very limited number of times. My experience is that it is very slow and the overall performance is very bad. Instead of speeding things up it slowed me down when I used it.

We can hover over the dots and it provides additional context sensitive information.

As can be seen from the above screenshot, if I hover over a green dot it tells me how many passing / failing tests are covering this particular line of code. Once again I have not come across such feature in any other tool so far. What I like about this feature is the drill down facility. We can left click and double click on the test which covers the code and navigate to the test code.

We can also right click and choose from other options.

Along with the context based options we also get instant feedback after making changes to the code. This code can be either in the class under test or the test class. In both cases we get automatic updates in the IDE. Lets change one of the constant values for the interest rates and see how NCrunch adapts to it. I’ll change the value of 2nd constant from 10 to 20.

Immediately we get red dots on almost all lines of code because there is associated unit test or set of tests which are failing related to these lines of code. Lets look at the way the Test class is represented when NCrunch is enabled.

Here also the failing tests are represented by red dots. We can revert the change to the constant value and things would turn green again. Or else we can fix the related unit tests to take into account the refactoring to the code.

Conclusion

All this happens while we finish typing the changes in the IDE. There is no need to run the tests or even to compile the code. NCrunch takes care of that in the background and we get instant feedback. I see this as a definite time saver. On large projects you can imagine how much time we spent on compiling and running tests. A lot of that can be saved using parallel test execution using NCrunch. Imagine  the amount of time we spent in looking for tests which covers a piece of code. You can argue that Visual Studio provides a method using “Find All References” to get a list of all references to a method. But there is no built in way of navigating directly to the test code like the one provided by NCrunch.

Focussing on running only the impacted tests is a big advantage for NCrunch. It beats the Test Impact Analysis feature of Visual Studio comfortably in terms of performance and speed. And by the way if I have read correctly, then in VS 2011 Test Impact Analysis is already deprecated. I have not seen such quick feedback in other tolls that I have used for unit testing. 

There are other features of NCrunch which I like. But I’ll stop at this point and probably cover them in future. For time being if you are a fan of TDD I would suggest you to explore NCrunch. Its a free tool and very easy to setup.

Until next time Happy Programming

 

Further reading

 

Share: