A while back I had an interview where the interviewer asked me if I could write API-tests. I didn’t really think too much about the question, and thought: ‘Well, of course, I can!’.
So, as the good consultant that I am, I just answered ‘Yes, I do!’
Said and done, I left the interview hurrying back to my assignment. Already occupied with thoughts on how to solve my current problems. Later my boss called and informed me the interviewer had liked me and if I wanted the gig was mine. I accepted and went on vacation.
A couple of weeks later I was introduced to the team I was going to work with. Per usual, a friendly bunch. We started off great! Familiarizing myself with the domain using Postman, which is a fantastic tool. We did have some automated check such as unit tests and integration tests. What we did lack was something that tested a bit wider, that forced the system to communicate with other systems and the database. Introducing API-testing.
My earlier cocky “Yeah, sure. I can write API-tests!” was put to the test. All of a sudden I had to deliver on my claim. Yikes!
Just writing tests that fire towards an endpoint and verifies that the response is successful isn’t that hard. Heck, Postman is awesome for that and we already had a collection that ‘should’ include all the endpoints. We could use that, export the collections, and environments and run them using Newman as a build-step in our soon-to-be-born CI-chain.
However, if you want to do something fancier, Postman is probably the wrong tool for you. Looking at the system we had I wanted to write tests that would test the same endpoint with different values. Sure this can easily be done with Postman as well, we just add a data-file and read from it, iterating over the different values. I also wanted to be able to write scenarios or sequences testing the state transition of the objects we moved along in our chain. This was something that would be hard to do in a good way with Postman. Getting the flow is not a problem, but doing this with different flows touching on the same endpoints and then maintain this would be a hassle. So I didn’t want to use Postman for the automation part. I knew what I wanted to do, but not exactly how to do it.
Let’s go search the interwebs! The search for ‘API-testing’ didn’t really nett anything useful. I tried to tweak my search and ended up on different versions of the same posts (more or less). It seemed no one had the problem I had, or at least didn’t write about it. Perhaps I should get an F in Googling.
Oh well, I can’t very well spend the company’s money on trying to find the perfect solution, so I just went for it.
The solution
… or what I ended up with.
In the end, I opted to go for what I knew. I wanted something that would keep me DRY. I didn’t want to repeat code, reasoning for this was simple, maintainability. If the login method for the API changes, I don’t want to have to go thru the code and update each and every test that happens to use that method.
Starting my IDE of choice (or in this case the company’s) I created a C# unit test project and started coding. I wanted to use MS-Test because this had been recommended as the unit test framework to use by the company’s QA department who supported and maintained nifty projects and NuGet-Packages.
Since I’ve been working with Selenium earlier, and a commonly used pattern for writing maintainable code in these cases is to use the PageObject pattern. I figured I might as well try to apply this pattern to my current snag.
I started by setting up the structure of the project, I added a ‘TestBase’ file that would work as some kind of boilerplate. I could use this to pass along certain information that I would need later in the tests. I also created my folder structure, which was pretty slim. I wanted one folder to house my RouteObjects, one for my TestScenarios and one that would handle all the classes representing the JSON I would have to read and write. Working with C# before I feared that this would be one of the more cumbersome tasks. And it was. I eventually ended up only putting classes here that I needed to have in more than one class. Decreasing the number of files in my JSON folder by 90%.
The verdict
In the end, it turned out okay, I guess.
We’re using the test-set that I created, and I am still developing and maintaining it. It’s a part of our CI-chain and we run it on merge requests using Docker and all that fancy stuff to make sure things still work as it’s supposed to do. It’s been a very nice set of tools to have, notifying the team early when things are not as they’re supposed to be.
However, using PageObjects for an API where you’re not necessarily restricted in how you can navigate between different ‘pages’, or routes — in this case, might not be the best of ideas — and since I’m using the variation of PageObjects that returns itself in after every call so you get them nice readable chains — made it a bit clunky and as the amount of tests and endpoints grows, as does the complexity of the framework. It is still easy to see what a test does in how the scenarios are structured though.
My team consists of old C# devs gone F#. So perhaps writing the tests in F# for the team to maintain would have been the smarter choice. However, my lack of knowledge, when it came to F#, kinda deterred me from going that route. Rewriting this to F#, now that’s a project for the future!
On an ending note
How would you have solved this problem?
What do you think is a good setup when it comes to building and maintaining API-tests?
Are there any good patterns out there for this kind of problem?
