xUnit vs. NUnit Demystified: A Comprehensive Explanation

Getting started with unit testing can seem like a daunting task. There’s plenty to learn, such as what unit testing is in the first place and why it’s valuable, what the best practices are when writing tests, and which tools to choose. We’ve already published a post covering the unit test frameworks for JavaScript; now, we’ll do the same for .NET, giving our two cents on the “xUnit vs. NUnit” dilemma.

NUnit and xUnit.NET—sometimes called simply xUnit, though that’s not entirely correct, as you’ll soon see—are the most popular unit testing frameworks for the .NET word. How do they differ? In what ways are they similar? Which one should you pick?

These—and more—are the questions we’ll tackle in this post.

xUnit vs. NUnit: The Fundamentals

Let’s open with some fundamentals regarding the frameworks. We’ll start by clearing up a misconception regarding the term “xUnit.”

xUnit and xUnit.NET: Two Different Beasts

You’ll often see people using “xUnit” and “xUnit.NET” as if they’re interchangeable terms, but they’re not—time for the history lesson.

It all started with SUnit, a unit test framework for the Smalltalk language, written by Kent Beck. Beck, along with Erich Gamma, then ported the software to Java, creating the JUnit framework.

Over the years, people created frameworks for more languages. These frameworks follow the design principles laid out by SUnit, so they’re all collectively known as the xUnit family of unit testing frameworks.

So, xUnit is the family of test frameworks inspired by SUnit and xUnit.NET is one specific framework for .NET that belongs to this family. For brevity’s sake, when I say xUnit, I mean the framework xUnit.NET, unless I explicitly say otherwise. With that out of the way, let’s continue.

Understanding NUnit

Let’s see a brief overview of NUnit.

Definition and History

According to Wikipedia:

NUnit is an open-source unit testing framework for the .NET Framework and Mono. It serves the same purpose as JUnit does in the Java world, and is one of many programs in the xUnit family.

NUnit was released in the early 2000s. It’s surprisingly hard to find an accurate source regarding the precise year—this article claims it was circa 2003, though I’ve read different claims. The project’s official site stated that NUnit was originally a port from JUnit. However, since version 3, the project has been rewritten from scratch.

How Does NUnit Work?

To write NUnit tests, you need to decorate your test classes and methods with NUnit attributes. The main attributes include:

• [Test] – Indicates a test method
• [TestCase] – Indicates a parameterized test method
• [TestFixture] – Marks a class as a test fixture
• [Setup] – Indicates a method to be run before all tests inside a class
• [TearDown] – Indicates a method to be run after all tests have been completed inside a class

Then, you have to write your tests, including one or more assertion methods. These are methods on the Assert class that actually perform the verification you need. Some of the most common assertions are:

• Assert.AreEqual
• Assert.AreNotEqual
• Assert.True
• Assert.False
• Assert.IsNull
• Assert.Throws

To have access to NUnit’s attributes and API, you need to install its Nuget package. Using .NET’s CLI—command-line interface—you can do that:

dotnet add package NUnit --version 3.13.2

Alternatively, you can install it using the Package Manager Console inside Visual Studio:

Install-Package NUnit -Version 3.13.2

To be able to run your tests inside Visual Studio, you need an additional package: the NUnit test adapter, responsible for making Visual Studio’s Test Explorer “see” NUnit tests:

dotnet add package NUnit3TestAdapter --version 4.2.1
Install-Package NUnit3TestAdapter -Version 4.2.1

Finally, you also need the Microsoft.NET.Test.Sdk package, which is required for test projects targeting .NET Core.

dotnet add package Microsoft.NET.Test.Sdk
Install-Package Microsoft.NET.Test.Sdk

However, there are simpler ways to achieve the same result. Using the CLI, you can create a new project using the NUnit template, and the CLI will ensure the installation of all the necessary packages:

dotnet new nunit

Alternatively, if you use Visual Studio, you can add a new project of type NUnit test project. Similarly, all of the essential bits and pieces will be installed for you:

Features

Here are some of the main features of NUnit:

• Inconclusive tests
• Parameterized tests
• Parallel test execution
• Multiple assertions

NUnit Test Examples

Let’s see a brief NUnit example. First, consider the following class, which is a possible solution for the StringCalculator Kata, a programming exercise proposed by Roy Osherov:

public static class StringCalculator
    {
        public static int Add(string numbers)
        {
            var integers = numbers
                .Split(',', StringSplitOptions.TrimEntries | StringSplitOptions.RemoveEmptyEntries)
                .Select(int.Parse);
            var negatives = integers.Where(x => x < 0);

            if (negatives.Any())
                throw new ArgumentException($"Negatives not allowed: [{string.Join(", ", negatives)}].");

            return integers.Where(x => x <= 1000).Sum();
        }
    }

The following are some tests for the class which showcase some of the features I mentioned earlier:

public class StringCalculatorTest
    {
        [Test]
        public void Add_EmptyString_ReturnsZero()
        {
            Assert.AreEqual(0, StringCalculator.Add(""));
        }

        [Test]
        public void Add_SingleNumber_ReturnsNumber()
        {
            Assert.AreEqual(1, StringCalculator.Add("1"));
        }

        [Test]
        public void Add_NegativeNumbers_ThrowsException()
        {
            var expectedMessage = "Negatives not allowed: [-4, -5].";
            Assert.Throws(
                Is.TypeOf<ArgumentException>().And.Message.EqualTo(expectedMessage),
                () => StringCalculator.Add("1,2,3,-4,-5"));
        }

        [Test]
        public void Add_NumbersGreaterThan1000AreIgnored()
        {
            Assert.Multiple(() =>
            {
                Assert.AreEqual(5, StringCalculator.Add("2, 3, 1001"), message: "Numbers greater than 1000 are ignored.");
                Assert.AreEqual(1005, StringCalculator.Add("2, 3, 1000"), message: "Values equal to 1000 aren't ignored.");
            });
        }
    }

Understanding xUnit

Having offered an overview of NUnit, let’s do the same for xUnit.

Definition and History

According to its official site, xUnit.NET is a “free, open-source, community-focused unit testing tool for the .NET Framework.” xUnit was first announced in 2007 as an alternative for NUnit.

How Does xUnit Work?

xUnit’s working is, in many aspects, similar to NUnit’s—which isn’t surprising, considering they both belong to the xUnit family.

In xUnit, you also decorate your class with attributes. xUnit considers some of NUnit’s patterns harmful, so it has fewer attributes. The attributes that do have a match often have different names, such as [Fact] and [Theory] instead of [Test].

To write tests in xUnit, you also use assertion methods, but the names aren’t the same as the ones from NUnit. Here are some of the most common assertion methods:

• Assert.Equal
• Assert.NotEqual
• Assert.True
• Assert.False
• Assert.Null
• Assert.Throws

There are many NuGet packages available for different aspects of xUnit’s working. Generally speaking, you need at least three of them:

• xunit. This is the main package.
• xunit.runner.visualstudio. This makes Visual Studio’s Test Explorer see and understand xUnit’s tests.
• Microsoft.NET.Test.Sdk. Necessary for tests projects targeting .NET Core and newer versions.

But of course, you don’t actually need to install all of those packages one by one. You can use the CLI and simply create a new xUnit project:

dotnet new xunit -n <PROJECT-NAME>

Using Visual Studio, you can also create a xUnit project:

Features

Among xUnit’s main features, we can mention:

• Great extensibility
• Parallel execution of tests
• Isolation between test methods

Perhaps xUnit’s main feature is the restrictions it brings to the table. You don’t have the concept of Setup or Teardown, as those are considered anti-patterns. You also don’t have a setup or teardown for the entire test execution.

xUnit Test Examples

Here’s the same test class from before, but this time using xUnit:

public class StringCalculatorTest
    {
        [Fact]
        public void Add_EmptyString_ReturnsZero()
        {
            Assert.Equal(0, StringCalculator.Add(""));
        }

        [Fact]
        public void Add_SingleNumber_ReturnsNumber()
        {
            Assert.Equal(1, StringCalculator.Add("1"));
        }

        [Fact]
        public void Add_NegativeNumbers_ThrowsException()
        {
            var expectedMessage = "Negatives not allowed: [-4, -5].";
            var exception = Assert.Throws<ArgumentException>(() => StringCalculator.Add("1,2,3,-4,-5"));
            Assert.Equal(expectedMessage, exception.Message);
        }

        [Theory]
        [InlineData("2, 3, 1001", 5)]
        [InlineData("2, 3, 1000", 1005)]
        public void Add_NumbersGreaterThan1000AreIgnored(string numbers, int expectedResult)
        {
            Assert.Equal(expectedResult, StringCalculator.Add(numbers));
        }
    }

xUnit vs NUnit: How Do They Compare?

Up to this point, we’ve covered both xUnit and NUnit in some detail. Let’s now talk about how the two compare regarding three main properties: features, documentation, and easiness of setup.

Features

When you google for “xUnit vs NUnit,” you see a lot of articles—including some published quite recently—making claims that aren’t true. Apparently, they compare xUnit to older versions of NUnit, causing the comparison to be much more favorable to xUnit.

At the time of this writing, both frameworks are comparable in terms of features. For instance, one of the most cited advantages of xUnit is its high level of isolation. xUnit creates an instance of the test class per test, which makes the tests highly isolated and prevents them from interfering with each other. Recent versions of NUnit also offer this capability, making it configurable.

Documentation

In terms of documentation, both projects seem to be well served at the moment. xUnit’s documentation is more oriented toward guides that teach specific tasks and FAQs. NUnit’s documentation is a better-structured wiki with comprehensive documentation regarding its APIs.

Difficulty Getting Started

Thanks to the .NET CLI, getting started in both frameworks is as easy as running a single command. So, no clear winner here.

No Matter Your Framework, Test Your Code!

What’s the verdict? Both frameworks are awesome and totally capable of serving your unit testing needs. In my view, they’re both mature tools and their features are comparable, so it’s mostly a matter of preference.

I personally like NUnit’s naming conventions more, both regarding its attributes and assertion methods. When it comes to NUnit 3, it doesn’t leave anything to be desired when compared to xUnit.

So, the final tip is: try both and pick whatever makes you most happy. As long as you’re writing plenty of high-quality unit tests, everything will be alright in the end.