Today's article is done in joint effort with my former colleague and mentor Guillaume Faas (🔹) in the form of an interview of a developer.
Please join me to thank him for his incredible involvement in the writing of this post! We both hope you are gonna love it as much as we loved writing it.
tldr; Do katas with the "Test Driven Development" methodology!
Presentation
🔸 Hey Guillaume ! Can you introduce yourself please?
🔹 Hello Tinaël! Thank you for inviting me to speak on your blog. My name is Guillaume Faas and I am a .NET expert / Software Craftsman, currently working at Squaremiled S.A.. I've been building softwares since 10 years or so, in various environments and business sectors.
The topic!
🔸 What are we going to talk about today?
🔹 What do you think? It seems someone did not read the post's title. We are going to talk about unit testing!
🔸 When did you first encounter unit testing?
🔹 It was a long time ago in a galaxy far, far away... I barely had a few years of experience. I've been working in the same company for most of my career and, compared to my day-to-day, I thought I had seen it all. I started looking online in guideliens, best practices, patterns and so on... I came across a lot of exciting and above all new topics. I seemed like I stepped in a new world that had nothing to do with my daily life in which I was stuck. The testing was obviously one of those topics. However, I quickly realised that I had to progress in other subjets before I could introduce tests in my projects. It took me quite a long time before I was really fit to work in a test-driven approach.
🔸 Ok! However, just not to lose our readers... can you explain what a unit test is please?
🔹 Yes, of course! A unit test is a test that verifies a unit of code. This concept of unit of code varies according to the testing schools. The London (or Mockist) School will see it as the smallest chunk of code so we are talking about a class or a method. On the other hand, the Detroit (or Classicist) School will see it as a behavior so a group of classes or methods. For the most curious of you, here's an article which gives details on the differences between both schools. I want to clarify that no school is better than the other, each one having its pros and cons. It's only a matter of preference and compromise. The main difference is mainly the size of your System Under Test (SUT) and the relationship between the different collaborators.
But in the end, a unit test is simply a piece of code validating that another piece of code has the expected result and/or side effect against a given scenario.
🔸 Alright. Where is it in this "test hierarchy"?
🔹 It looks like the Agile Testing Pyramid but from left to right instead of bottom to top. The more on the left, the smaller your scope is and the most numerous and faster your tests will be. If you read it the other way and apply a reverse logic, it works too.
We are on the "unit" part for this topic since there is technically nothing smaller than a unit. A unit test should be executed on a standalone basis in a sandboxed environment. That is, a unit test has no impact on the outside of its scope, and if you run several of them in parallel, they must not have any side effect between them.
This finally means that in a unit test: we avoid reaching a database, making HTTP requests, accessing file system, etc. In the same way, we are not going to contact other dependencies of the solution. We strictly stay "internal" to the unit.
Here are a few points on the functional testing part:
- unit testing is to check that a component works well. It is the fastest, we will talk about "fast feedback loop" ;
- integration testing is to check that several components work well together ;
- user acceptance testing consists of verifying the whole application while avoiding contact with external dependencies (eg: requests to external providers). This is the most representative test, probably the one that has the most value on the product scale as it validates business requirements.
Note that you don't have to implement them all. We could have a test suite only composed of tests from a single category. But it should be borne in mind that our test suite will not be the most effective.
🔸 Nice! But why is testing not used everywhere in the profesionnal world?
🔹 In reality, a large part of developers write little to no tests. In addition, integration tests are less common than unit tests because they are more complex to write. In the end, this task is often seen as a chore or an extra step that we will only do if we have time.
Uncle Bob also spoke about it at a conference in London in 2018:
🔸 What is the objective of unit testing and when should it be applied?
🔹 The goal is quite simple: it's to show that a method works as expected. That is, you are going to have an expected behavior. For example, a method getSomething should return you something.
You are thus going to test different scenarios and check that the method always behaves the right way.
As for the "when", it's even simpler: you need to apply unit testing whenever you have some logic somewhere.
🔸 Alrighty! What are the pros and cons of such thing?
🔹 As previously said, you write code to test code. Seen like this, it surely looks like additional workload but there are realy interests behind!
- The unit test acts as a safety net against regressions.
The refactoring stage systematically occurs in a projet. The problem is, when you start changing something, there is always a risk of regression. We talk about regression when something that used to work does not anymore due to some changes. It's precisely here that a unit test intervene: it guarantees that your components always work as expected. If anything does not work anymore after a refactoring, the test suite is going to display erroneous code with a beautiful red dot. And that, from the point of view of a developer, it's huge! It means you can be much more relaxed and spend less time checking that your changes did not have undesirable effect on the rest of the features.
- Writing tests improves the code quality of your app.
It is related to the above that we have just discussed. It is likely that the readers have already encountered a similar situation: when talking about refactoring to a Product Owner or a Product Manager, the first fear is always that is not working anymore. If you are covered by a test suite, you are not afraid of refactoring. It's even the opposite, you are encouraged to refactor regularly while being protected.
- A test suite becomes what we call a living documentation.
When we talk about documentation, we might all have commentaries in mind. The problem is that they are never up to date with the rest of the code. The code evolves, the documentation does not. However, your unit test will always be up to date. If it wasn't, your test suite wouldn't give you the green light to go further.
It's even more interesting when^in the context of the arrival of a new developer on the project. Rather than reading all the code of a method to know what it does, they can just look at the different tests for that specific method.
Each behavior will be represented by a test with an explicit naming on the scenario and expected result (eg: GetItem_ShouldReturnNotFoundResult_GivenItemIsMissing). It makes the onboarding process easier!
- It reduces the bug detection time.
We mentioned the concept of "short feedback loop" earlier. Unit tests are really fast to execute, they give an almost instantaneous feedback on the health of the solution. This means that we need to regularly execute them. Let's say we have a button (or a shortcut) that gives us a Green/Red status in a few seconds. Activating this button must become systematic.
It already has an interest for us developers but that's not all. We'll talk about it later!
- It is not a direct advantage but more like a side effect: testing makes you better.
To ensure you write tests that provide real value, there are certain principles that you need to follow. You must always have a certain layer of abstraction to mock your dependencies, you must be able to inject them, you must limit the responsibilities of your components, etc. In fact, you will force yourself to apply several principles regularly (eg: SOLID). Suddenly, it forces you to break your components, to decouple them, to think about their interactions and responsibilities. In short, to think and ask yourself a lot of questions. Casually, we are talking about code design! And so, action-reaction: you get better over time. Sounds very "Happy End" but you know where I'm going with this.
🔸 And as it is not that much introduced in companies, it's kinda "new" and motivates to learn more!
🔹 I agree with you on the novelty aspect but it only remains present when the subject is discovered. And there's an interpretive part to it all: some (like you) see it as something interesting. Others see it as a chore or extra pressure. You will always find people resistant to testing for a variety of reasons. Perhaps we will have the opportunity to address the reasons that are generally mentioned...
To get back to the question, here are the cons that come to my mind:
- Already mentioned, but there are pre-requisites: you have to understand the foundation of the Object-Oriented, dependency injection, SOLID principles, etc.
- The fact that there are few projects with real test suites makes learning less accessible. The same goes for a coach passionate about this subject.
- There's a fairly steep learning/progression curve. Everyone goes through a phase of frustration at the beginning because we are not comfortable and we have the impression of being slower. We must resist and persevere because the tests will actually make us go faster in the long go. We will talk about it with the TDD approach.
- It requires preparation: you have to think about the project's architecture, the relationships between the different components, etc. Seen like this, it isn't really a disadvantage but we can't (won't anymore?) go headlong in a development without a minimum of thought.
- There is a lack of comprehension on the side of other teams involved in the proect development, especially non-IT's. We always fall back on discussions about the "Return On Investment" (ROI) or the impact on velocity.
We often hear that "it takes time" or "it will be planned later" or "developers don't have time" but those arguments are not really valid. Indeed, the first one clearly indicates a lack of long term vision and understanding of testing. Because "later" never happens but above all, writing the tests at the end of a product's development doesn't make any sense. We lose all benefits of testing. I already said it but we will talk about it with the TDD approach. Then, the second indicates an organizational problem. The tests should be included in the time estimates of developping a feature and not as additional work to be done.
🔹 Testing is great, we feel the benefits. However, we is also necessary to make a status on the state of the test suite. This is when we start talking about "code coverage". It is an informative metric about your app's test coverage process. I really insist on the "informative" side. It would be a mistake to measure the quality of the test suite based on its coverage. It is a metric of quantity, not quality. I have already read articles about companies that have incorporated the value of code coverage into developer goals and this has prompted developers to use mock tests to boost the coverage.
The only way to check the quality of a test suite of a project is to ask yourself a few questions:
- Does the general feature development time stay approximately the same over time?
- Does the quantity of bugs found in production decrease over time?
- Do you easily manage to welcome new resource within the development team?
- Do the developers trust their test suite? Is it representative of the state of health of the solution? Does a green circle really guarantee that a component is working?
If you are able to answer "yes" to all these questions, congrats! You can be proud of the test suite that you have put in place. The problem? It is really difficult to have an answer to all these questions when you have to report day one... You also notice that the first three refer to time.
🔸 Well... Cost wise, where are we? Because ultimately, writing a test means writing code. It costs!
🔹 I see where you are going with this. No, it does not cost more unless you charge to characters! Even if you write more code, you really win and not only on the time aspect. Did I tell you we had to talk about the TDD approach? Because it even saves you time in short term. Anyway!
Developing a feature may take you a little longer, knowing that it will mostly depend on your ease to write tests. On the other hand, it will especially "save your life" a lot of times because you will avoid a lot of bugs which, in normal times, would have arrived much later in your process, during user testing on a QA environment or in production.
But you wanted to talk about money! The later a bug is discovered, the more it costs:
Cost of bug fix based on the moment it was detected, sourced from DeepSource!
And is totally normal.
We can talk again about the "fast feedback loop": if a bug is found by a unit test, it's in local, on your machine, just after a change (don't forget to rebuild and rerun your test suite!). The bug is quickly identified and fixed.
In contrast, a bug that goes to production... is discovered by a user who must report the problem to your product team who must analyze the feedback and open a ticket in your backlog. This ticket will be prioritized by your Product Owner to be included in the next iteration and then it will be assigned to a developer. Assuming that it is not you, there will be an investigation phase (reproduction of the bug), a bug correction phase and after that it goes through all environments and be validated by Quality Assurance Users.
I deliberately took an extreme case scenario to show the worst possible but it's also the best way to be explicit about the problem. The important thing to remember is that a test can save a lot of people a lot of time, no matter how simple it is.
Ed: to learn more about the reasons that softwares have bugs, do not hesitate to consult this page and others on the web!
In depth
🔸 Ok! What about we talk about black box and white box testing now?
🔹 I like diagrams, do you like diagrams? Diagrams are great!
The differences between black and white box testing
Black box testing consists in giving an input to the SUT and check the output. It's that simple: we don't take into account things happening inside the method. There's an accurate case scenario where this type of testing is mandatory: pure methods. Those having no dependencies or shared variables have no side effect. It's therefore obvious to use black box testing. It makes the test extremely robust because nothing impacts its result.
Let's take a Sum method from a Calculator class as an example. We are exactly on the scenario mentioned above:
[TestClass]
public class CalculatorTests
{
[TestMethod]
public int Sum_Should_ReturnTheSumOfTheTwoNumbers()
{
Calculator calculator = new();
int result = calculator.Sum(2,3);
Assert.AreEqual(expected: 5, actual: result);
}
}
We don't know the method's implementation, but we wrote a test. We give it input data, and check the output. For the other scenarios we haven't talked about yet, I find it a shame to stop there. It's a personal feeling, but I find white box testing more relevant as a mockist.
So yeah, on the other hand, we thus have white box testing. At first sight, it is the same process: we give an input and check the output. But we are also going to check what's going on inside the SUT: check that it called its dependencies correctly, that the value has been correctly cached or saved in a repository, that an event has been emitted, etc.
This allows us to check each behavior and their side effects.
🔸 The most awaited question ever... How do we write good unit tests?
🔹 I don't think there are good or bad unit tests... No, just joking! You need to think on what to do before actually doing it. That can sound silly but just think before you do. That's what I was explaining when I was talking about the fact that testing makes you better. If you want to write efficient tests, you need to think on the way your components are going to communicate with each other. Actually, your tests will be efficient once they are easy to do. And if you realize they are not, there is something wrong with your code.
eg: I've got a service that has to create a user. Before writing my test, I have to ask myself a few questions: what is the responsability of my service? Is it responsible or sending an HTTP request to an external provider to get information? Is it responsibile for database persistence? Is it responsible of the logging?
Divider and Conquer: a dependency here, one there, another there... Finally, what's left in your service? The orchestration of a process delegated to different dependencies (eg: http client, repository, logger, etc.) and possibly a modification of the state of an entity. That's it.
In the end, a "good" test must:
- protect you against regressions ;
- be resistant to refactoring ;
- give you a quick feedback ;
- be maintainable.
🔸 What about you give us some tips to start our journey?
🔹 I recommend people who want to start testing to start with the Test Driven Development approach. Writing the test of a method after its implementation is not really the goal because you already know the implementation so your test will be highly coupled to your implementation. In addition, your code already works so the test would probably be seen as a time loss. But above all: we don't benefit of any advantage of testing during the implementation phase.
You are not alone. There are tons of resources available to help you out. Here are some books that I wish I have read early in my career:
- "Test Driven Development - By Example" by Kent Beck ;
- "Unit Testing - Principles, Practices and Patterns" by Vladimir Khorikov.
And here's a website filled with tips and tricks on TDD with a large number of katas to progress: TDD Buddy.
Speaking of katas, do katas. Do a lot of katas and do them with other (pair and/or mob programming) if you can. It is fun and very educational, especially when starting with simple exercises and gradually increasing the difficulty until you end up with situations similar to what you can find in real projects. In addition to TDD Buddy, I would also suggest people to check out Code Wars if you are hungry for inspiration. And without necessarily doing self-promotion, you can also find some katas on my GitHub profile.
Last but not least piece of advice to start testing: we can refer to what is called the triple A (AAA), which means "Arrange, Act, Assert", in order to make the tests more clear and organized. The goal is to divide a unit test in 3 distinct parts:
- arrange: this is the scenario, the part where you prepare the input data of your method ;
- act: it is the action, the fact of carrying out the call to the method that you are going to test ;
- assert: this is the behavior check, the part where you check the output or the side effects.
🔸 What are the "bad smells" in unit testing?
🔹 I see a few of them...
- an "arrange" part that has like 15 lines... It is way too complicated. We clearly see that the tested method does too many things because the scenario is tough to prepare!
- We say that a test should have one and only one reason to fail. A test should contain a single assertion.
- The fact that you have trouble writing unit tests, not because you don't have the necessary knowledge but rather in relation to the code to be tested... this means there is a problem at the level of your component. So take a step back and think about the responsibilities.
To go further
🔸 Do you have interesting libraries in mind to ease our work?
🔹 Yes. We can consider 3 groups of libraries:
- testing ones which allow to generate tests ;
- mocking ones which allow to overload the behavior of your dependencies and monitor them ;
- data generation ones.
For me:
| Test Frameworks | Mocking Libraries | Data Generation Libraries |
|---|---|---|
| MSTest | Moq | AutoFixture |
| NUnit | NInject | |
| XUnit | WireMock |
🔸 I would like to retain the attention of the readers on this particular point... but there are librairies for the front-end too. In fact, unit testing is not reserved for back-end developers. We'll mention Jest, Mocha, Cypress and Jasmine as wellknown librairies for JavaScrit applications.
Well anyways! You can't stop mentioning it so here it is: what is Test Driven Development (TDD)?
🔹 Glad you finally asked! It's not like I gave you multiple openings... TDD is joy, happiness, the ultimate answer to the meaning of life, it's all of that! No, just kidding. In fact, it's more of a way of putting tests at the center of what you do. We talked about positive points of unit testing, and we also said that we would lose them by doing tests at the end, but we did not go into details.
The best way to benefit all the advantages of doing tests is to do them first but it is not just that. It doesn't mean first doing all the tests and then implementing all the methods. No, there's a kind of iterative aspect which we also find in Agility. You go step by step (baby steps), you add new behaviors ensuring the previously added ones still work. The safety net grows little by little naturally. We can even present this differently: think about a ladder. It will always be easier to climb it step by step instead of three by three.
I would add that contrary to popular belief, TDD does not make the development time longer. For example, it is not necessary to run the whole solution to know that the code is working because it has been developed entirely on a tests basis.
The TDD cycle
First thing first, you need to write one unit test. This test should normally have to fail since no implementation has been written for it to pass. The second step is to write the code which will make the test go green (pass). It is very important that this passage from red to green must be as short as possible. It's the moment when we have the right to write "ugly" code (hardcoding a value, to duplicate code, to paste a response from StackOverflow, etc.). It may seem weird at first but there is a real benefit: checking that adding a new behavior is possible without breaking everything that has been done before. The next step is the refactoring phase. We did write some horrible code, so now we need to make it clean. I spoke about refactoring being easier and safer: here we are! We have our "green light" and the behavior is guaranteed as long as the light remains green. We have our fast feedback loop at hand (or click, or shortcut) to know if everything's OK. Got it? And after that, we reach the end of the cycle. That means one thing: we start over again.
🔸 I'm gonna give a little example so that everyone fits! I am going to do it with a string length calculator because it's easy to do. So, we start by writing a unit test:
[TestClass]
public class StringCalculatorTests
{
[TestMethod]
public int Length_ShouldReturn_CorrectLength()
{
StringCalculator calculator = new();
int result = calculator.Length("string"); // "string" is 6 characters long, right?
Assert.AreEqual(expected: 6, actual: result);
}
}
The test fails because I haven't created the StringCalculator class yet. Next step!
public class StringCalculator
{
public int Length (string str)
{
return 6; // hardcoding the value for the shortest test resolution possible
}
}
Here, I'm passing at the green stage. So now, only the blue one remains.
🔹 I would like to say that this is a very ugly code because you hardcoded the value. And this is great! It's the goal!
🔸 Now the longest: carry out a code refactinrg which allows us to meet the requested need (calculate the length of a string) without breaking the test:
public class StringCalculator
{
public int Length (string str)
{
return str.Length; // cleanest way to return the length of a string in C#
}
}
Voilà! We can now start the writing of another unit test.
🔹 It's a really simple example but you did it. It should be noted that there are some rules to respect with TDD but I could talk about it for hours so we will stop here!
🔸 Nice! By the way, I heard you recently learnt about "Test && Commit || Revert" (TCR). What is it?
🔹 Exact, I had the chance to get to know about this practice via a workshop. To schematize, let's say it is an extreme vision of TDD. The best way to apply it is with a separated script. This script will analyze your solution at each save and then execute your test suite. If all the tests pass, it creates a commit representing a stable state of your branch (Test && Commit). If a single test does not pass anymore, it will roll back to the former commit (Revert), which is stable. This forces you to go in baby steps, and one thing is highlighted: it is your last change which broke something.
At the beginning, you go through a frustration phase because you lose some code but precisely this encourages you to go forward little step by step to limit your losses. The smaller your steps are, the less you risk to lose. It's a great teaching in addition to TDD. When you become relatively comfortable with all of this, you notice that you go faster and faster and above all: you always have a working branch.
Conclusion
🔸 Do you have a last word for this interview?
🔹 "Victoriae mundis et mundis lacrima", which does not make any sense but I feel it's relatively cool. More seriously, we have been discussing for a while now but we've only scratched the surface. There are still a lot of things to discuss about testing. I would urge your readers to be curious and the topic and to read and practice. And do not hesitate to ask for help around you!
And now, are you interested in testing?
