BDD approach in test automation

This is a reply for Vitalii’s blog post here.

Well, I also never saw BDD working.

The promise is to make everyone to be able to read automation tests and even write is never fulfilled and approach requires ideal conditions to work.

Why?

Not everyone wants to write test in BDD

1. Automation engineers often hate this approach, they like to write code. For automation engineers it is not working because they know that mapping 1 to 1 in automation test to test case is not often possible or makes sense. Why? Some things my be optimized in automation, there are also things you can check “for free” using automation and some ad-hoc manual checks can be made “for free” implicitly, like UI checks during manual execution.
2. Manual testers are more fluent in natural language steps since it is more expressive.
3. Business people just don’t do it.

On the same page

When you want everyone to be on the same page you create test case review procedure. Here it does not matter if it is BDD or not. Test cases are getting reviewed and even if this review happens(it really should) then there are also blind spots about how this is actually implemented in automation steps. You can take a look at given-when-then steps and understand what it does but you do not know HOW it does this. One line in BDD test will do several things and you just can guess what they are. The only difference between BDD and other approaches in this case is that you may think you understand how test is automated in BDD when you look at it, but you most likely will be wrong, and in case of regular codded tests you at least know from the start that you do not understand how it actually automated, I think that is better.

Misc. reasons

REASON 1: It requires a lot of things to be ideal and real life is not. REASON 2: This is a highly specialized tool like microscope, you have to know what you are doing, you are more likely just need a hammer. REASON 3: I imagine there is a lot of difficulties with simple stuff like parallelization and distributed execution and test data sharing. REASON 4: Requires specialized tooling

BDD’s main benefits: traceability and “shared language”

Let’s start with test “shared language”:

1. BDD steps are shared language for everyone on the team, but as I said before no one likes to use it really.
2. BDD steps are supposed to be descriptive for everyone. In reality you get roughly the same amount information from only test step names from regular non-BDD tests, fight me.

As for traceability, if you want tests to be more strictly mapped to automation tests you can achieve same results even in regular codded tests:

1. Explicitly map test case to automated test by adding test case number as attribute(or whatever you TAF allows) to automated test.
2. Make sure to mark test steps from Jira in automated test as well. If you have good reporting to go along with it you can click on test case id from logs and be taken to testcase itself in Jira.

It can look like this:

[Automates("TEST-ID-FROM-JIRA")]
public void Test_1234_TestName()
{
    // Step 1
    <Code that corresponds to step 1 from test case>

    // Step 2
    <Code that corresponds to step 2 from test case>
        
    // !!! If step would be skipped for some reason in automation you should still add comment or empty step like indicated bellow. This makes tracking changes easier and can look like this:
    // Step 3
    // Not automated because we dont't have DB credentials.

    // Step 4
    <Code that corresponds to step 4 from test case>
}

PROS

There a definitely some PROS, should be =)

1. It will be easier to write tests for newcomers, fair. But that is also true for well written keyword driven framework or any type of the framework that have steps defined. But again as Vitalii said is not something that comes for free with BDD.
2. It works good on level of acceptance testing, but in my mind requires some other tool to automate functional tests.

I definetly forgot something here, but I will leave this as is.

Conclusion

As I said BDD requires perfect conditions to work. In this conditions everyone (test, BA, automation) have to speak in in common given-when-then language, but language does not work outside acceptance tests, it works for simple tests, sure, but when you have to check corner cases you are done. Moreover it cannot work because there is always a gap between test case and actual implementation in automation, BDD tries to hide this, never a good idea. I’d rather everyone to be clear about this.

PS:

Also, now SpecFlow is dead, I think that is a trend, just like hate for pure OOP and clean architecture.

I really tried to not being a hater =)