API testing using F#

Photo by Ilija Boshkov on Unsplash

After a while working in a team that develops in F# I converted to the dark side. Almost my entire first year with the team I wrote my API tests in C# because that was what I knew. It was easier.

During the summer months, I decided it was time to change that. So I started to port my C# tests to F#. But I ran into problems. C# is Object-oriented and F# is functional. I also wanted to be able to run my tests in parallel to get the execution time down. How do I do this the best way? Turns out the answer was simple; start over.

Since my team were using Expecto, I figured that it would be prudent to use that as well. I also went from DotNet framework to DotNet core. I went with the http.fs lib for making HTTP calls. The reasoning here, there was more documentation and looked more like what I was used to than the alternatives.

As I started out I wanted to separate the logic of the API calls from the test scenarios. I wanted to keep following the page-object model that I had used earlier, it seemed to do its job.

I created a folder called Abstractions, where I keep my RestMethods.fs and Response.fs files. I continued with the Tests folder where I wanted to keep my test-scenarios. I ended up with a structure that looked something like this:

Api-tests
  | Abstractions
    | Helpers.fs
    | RestMethods.fs
    | Response.fs
    | User.fs  
  | Tests
    | UserTests.fs  
  | Tests.fs

The thought with the RestMethods.fs file was to keep the HTTP verbs. Set up the verbs (Get, Post, Patch, Put, Delete) and an easy way to set the required headers. In our system, maybe not too surprisingly, all endpoints require authentication and some other arbitrary headers in order to respond with useful data. I designed the methods to be general, taking in the URI and making the call. Methods such as POST also take an additional argument, an arbitrary type, that it parses to JSON and attach to the call. ex:

let RestMethods =
    let Get (string: uri) (Authentication: auth) =
        http.Get (genURL uri) (Headers auth)
    let Post  (string: uri) (string: body) (Authentication: auth) =
        http.Post (genURL uri) body (Headers auth)

In my Response.fs I keep things such as common error codes and types expected with an error response. Such as 404. We always return a certain set of information to make it easier for the consumer to expose correct error messages for the end-user. Further, I added some assertion methods that would take two arguments, a response and an expected response. Again, this to make my verification of the responses easier. ex:

let Response =
    let expectBody (HttpResponseMessage: response) expected =
        Expect.equals (readBody response) expected
    
    let expectStatusCode (HttpResponseMessage: response) expected =
        Expect.equals response.StatusCode expected

Once that was done, I started with the individual endpoints. Since I had done the groundwork, this part was fairly straight forward. More or less, what I had to do was to create a function that used one of the HTTP verb functions I had created earlier, adding the required data. Since I wanted to be able to run the tests in parallel, I opted to control the authentication part by sending the authentication key from the test scenarios. So I had to update my functions to pass this along. Handling the types on the way.

I also realized that it was easier to handle the conversion to JSON in an earlier stage. So I moved it one level up from the functions that handled the calls to the functions that used these functions. Making it more organic. Some functions that would be used by a lot of functions were put into a helpers file. To allow easy access and keeping it DRY.

let info (Authentication: auth) =
    RestMethods.Get "/user/info" auth

Once this was done I went to town on the test scenarios. I already had a bunch in C# that I needed to convert to F# code. The job was monotone, but once done I was satisfied with it. ex:

testCase "Get user info" <| fun () ->
    let auth = Authentication.Testy
    expectedInfo = {
        userName = "Testy"
        address = "Exampleroad 1"
        state = "NY"
    }
    User.info auth
    |> expectStatusCode 200
    |> expectBody expectedInfo
    |> ignore

Once this was done and verified I replaced the old C# code test pipeline. Using Expecto and parallel execution I managed to get our API tests down from ~10 minutes to 24 seconds. This was huge, its a 95%+ reduction of execution time. One important reason for this was that I had cut time by only logging every user in once, rather than in every consecutive test. The reduction in time meant that we could use this in our CI pipeline. And the execution could be done on pull requests rather than after merge to the develop branch.

Said and done, updated our pipelines and added a system integrations step. So in order to get your pull request accepted, it needed to pass the unit tests and system integrations. If it didn’t it triggered an investigation into why it had failed.

This has shortened the feedback loops by up to 3 hours, and that is noticeable. Fixes are done while you have the solution fresh in the head. It’s not TDD, but we’re getting there. An added effect is that the tests are run locally on the developer's computers before the PR is even created, all thanks to DotNetCore and the appsettings.local.json file. My team have an additional feeling of security, an added layer.

Finally, they have also decided that they want to take a bigger ownership of the quality in the project and have opted to shoulder the responsibility of maintaining and updating the API test suit. With the support of QA of course. This brings me joy and frees up time to look into new avenues. Such as mainline development or testing in production!

Disclaimer: Code in this post is written from memory and may not be correct. Some functions used in the example are helpers created by me to be used in the project but are not described in code in this post. The name should be enough to give an understanding of their intended use.