We have a legacy system w a number of microservices that I am now looking at.Today our API verification for a service consists of
- one big postman collection
- some 200 requests they all have scripts for verifying the results
The collection is developed in postman as a big batch (by Jenkins/newman) we set env variables during the test run and use scripts for verification.The collection is developed in Postman and stored in Git (which requires export/import all the time)
I understand that we need to break up that collection; having all these tests in serial makes things harder to develop, some test cases just needs a few requests to verify the data. Also when developing I would prefer to be able to run a subset of these request but since the setup may be done way earlier in the collection chain I need to manipulate variables set by earlier requests. I might be able to put in some tests running locally (works on my machine) but then it fails on the CI server since the prereqs are a bit different.
I think Postman is a very good tool it has plenty of good functionality:
- pretty good UI
- pretty good script handling variables and verification
, but I experience some resistance in some tasks
Postman limitations
- online requirement
- Save requests and then Export collection in order to use Git
- sometimes does not save data (so when exporting to git the changes does not follow)
- the JS verification experience is subpar to a modern IDE w test frameworks. (Cant debug or develop DSL)
- Can't step a collection run (could be nice to debug and verify)
- Hard to run a sub-series of collection
- Can't set a path for file transfer (need to set a working dir in settings and then select the file from disk)
I understand that licensed versions of Postman can integrate w Git and also allows for "Flows" to be developed.Postman Flows seems like a very nice thing, but ATM I am a bit hesitant to tie us up further in this tool
Ideally I would develop the tests in an IDE and use xUnit test frameworks with nice Domain DSL:s.Since we want the test code in Git its clunky to Save and Export Postman collections, diff the changes and maybe revert some lines, re-import into Postman - you get the point...
I have looked at Bruno as an alternative its offline approach is appealing(it converted the postman requests but not the scripts...) but its another custom tool that we need to adapt for.
What approaches do you professionals out there do? I have read about Insomnia but it seems very much the same thing as Postman (or) ?