Making asychnronous requests appear real time.

For many business processes, an asynchronous approach makes a lot of sense, especially in cases when the user doesn't really care about the execution of a background process. Just the same, many processes need to provide real time or near real time feedback to the user. Async and Real Time don't have to be mutually exclusive, and a recent project challenge I faced illustrates one approach to blurring the line between these two concepts.

Situation
In brief, I was working on a project that retrieved data from an external API. Our requirements stated that we must execute this retrieval in both a batch and on-demand model. The implementation of the API was asynchronous, such that the request for a query would be submitted, and then a call back with the data would fire anywhere from 1 to 10 minutes later, depending on the volume of data being handled.

Batch operation was no problem, and we implemented a background process that ran overnight. In this case, the asynchronous nature of the API was a non-issue. Our process would handle incoming data as it was received.

However, the on-demand case was a little harder, due in part to the following challenges:

1. Response time for the callback was unpredictable. Some requests might return results in less than a minute, other may take longer.
2. We had to consider how long the user would wait for the data to be retrieved.
3. We had a requirement that on-demand data not be persisted to the data store (the batch process handled this task.)

Problem
How can we make an asynchronous process appear real time to a user?

Solution
There were several key components to our solution.

Created a Request Tracking Id
One of the first things we did was work with the API vendor to embed a tracking Id in to the API call and response. This Id allowed us to match up the request with the response, even though they were detached from one another. The Tracking Id itself had a marker in it that would designate a request as 'batch' or 'on-demand'.

The Tracking Id was a one time use value and needed to be 10 characters long (due to how the API vendor would embed it in the response.) I used a partial Guid and a leading character of 'B' for batch and 'D' for on demand:

Dim trackId As String = "D" + Left(Guid.NewGuid.ToString.Replace("-", ""), 9)

Updated Callback Receiver
Since all responses for both batch and on-demand requests we returned to the same callback (in our case a Web Service) we had to update the method to evaluate the Tracking Id in the response and handle the response according to whether it was triggered in a batch or on-demand.

Batch requests would be persisted to an interim data store that a separate ETL process would operate against.

On demand requests would be cached in-memory for 30 minutes on the web-server hosting the web services. The Tracking Id value became the key for the Cache entry.

HttpContext.Current.Cache.Insert(curCCR.TrackingId.ToString, curCCR, Nothing, System.DateTime.MaxValue, New System.TimeSpan(0, 30, 0))

Add a Cache Retrieval Method
Next, we created a new web service method that accepted a Tracking Id value and returned (if present) the cached object representing the response.

<webmethod()> _
Public Function RetrieveHistoryForTrackingId(ByVal trackingId As String) As BusinessObject

Dim newObj As BusinessObject = HttpContext.Current.Cache(trackingId)

Return newObj

End Function

Creating the User Experience
Now that all the back end handling of the response was in place, we turned to the User Interface for the on-demand request.

Our client was an ASP.NET page with some basic search values (first name, last name, date of birth, gender and zip code). We had a GridView control to display the history results if and when they returned, as well as a status Label control to show any messages to the user.

When the user would submit the search form, we would prepare the search and submit it to a web service that was a facade to the API call. Our Tracking Id was generated at this point and included in the request.

Dim response As webservices.Response = ws.SendRequest(encoded)

The next step would be to set up a polling mechanism to check for the response. We determined that 2 minutes would be the maximum tolerance to the user. We then used the following loop to poll the Retrieval web service and check for the received results.

      If response.Errors.Length = 0 Then
     Dim obj As webservices.BusinessObject = Nothing

     Dim timeout As DateTime = DateAdd(DateInterval.Minute, 2, Date.Now)

     While DateDiff(DateInterval.Second, Date.Now, timeout) >= 0
obj = ws.RetrieveHistoryForTrackingId(trackId)

         'if Object has data, exit loop
         If Not obj Is Nothing Then Exit While

         System.Threading.Thread.Sleep(30000)

     End While

     If Not obj Is Nothing Then
         grdHistory.DataSource = obj.HistoryItems
         grdHistory.DataBind()
     Else
         'no responses
         lblStatus.Text = "History records have not yet been received.  If you would like to check for results again, please use the link below.  History records are held for 30 minutes upon receipt."
txtTrackingId.Text = trackId
     End If
 Else
     lblStatus.Text = "Error submitting request: " + response.Errors(0).ToString
 End If

If the web service call yielded a result set, we loaded up our data grid and completed the rendering of the page. If the timeout occurred, then we simply posted a message to the user, but also put the Tracking Id into a secondary request form that the user could use to make a follow up attempt to retrieve the results.