I am working on a C# Web API that performs a specific action using a third-party-api-1, executed concurrently for speed. Once I get my output from the third-party-api-1, I would further get additional processing using the third-party-api-2, which is again intended to be run concurrently, and its results would be appended to the results from third-party-api-1, and eventually the output from my API would be sent back to the caller.
So the path is:
- My API receives input
- My API calls third-party-api-1 concurrently, which I intend to keep for speed
- My API calls third-party-api-2, again wanting to keep it concurrent execution
I would like to identify my http requests after the calls to third-party-api-1 and third-party-api-2 are done so that I can tie back the outputs to input and return the result to the caller. If I had any control over third-party-api-1 & 2, I would have set some correlation-ids in the request header and persisted in the response too, but I can't do that with my third party APIs.
My issue is that once I construct my http get requests into a concurrent mode, I am no longer able to link back each response to its original request.
Is there any way to achieve what I am trying to do? Or is my attempt futile or bad design all together?
// I build my get query inputs from my user requestforeach (ParsedInputAddress ia in inaddr){... addresses.Add(addrbuilder.ToString()); addrbuilder.Clear();}.// I call the http concurrentlyIEnumerable<Task<HttpResponseMessage>> getTasks = addresses.Select(async ia => await httpClient.GetAsync(basepath + ia.ToString()));var gets = await Task.WhenAll(getTasks);//this tasks will run concurrentlyforeach (var getContent in gets){ jsonResponse = await getContent.Content.ReadAsStringAsync(); if (jsonResponse != null) vaddr = JsonSerializer.Deserialize<ValidatedAddress>(jsonResponse); if (vaddr != null && vaddr.address != null) vaddresses.Add(vaddr.address);}
I tried to set custom http request with an unique GUID but since I do not have control over third-party-api, I am unable to receive that ID in the response to tie them together.
My last resort is to make this a non-concurrent execution so that each request is processed by API-1 and 2 sequentially, preserving the link between request and response.