Background processing

Sometimes an agent requires a long time to complete a request. This can happen with complex reasoning tasks, but interruptions may also occur due to network or client issues.

Background processing uses a continuation token, which can be used to:

• Poll for completion using the non-streaming agent

• Resume an interrupted stream with streaming agent

Only when the continuation token is null, we know the operation is complete.

Important note: currently, only agents that use the OpenAI Responses API support background responses

---

Using background responses

Let’s look at a simple example of how to start using background responses. The first step, of course, is to enable this option:

var agent = new AzureOpenAIClient(endpoint, new AzureKeyCredential(apiKey))
          .GetOpenAIResponseClient(deploymentName)
          .CreateAIAgent();

AgentRunOptions options = new()
{
    AllowBackgroundResponses = true
};

Next, we will start streaming a response and then intentionally break it, so we can simulate interruption.

AgentThread thread = agent.GetNewThread();
AgentRunResponseUpdate? latestReceivedUpdate = null;
int textCount = 0;

Console.WriteLine("===Generating Monthly Sales Performance Report...\n");

await foreach (var update in agent.RunStreamingAsync(
    "Generate a detailed business report analyzing last month's sales performance. "
    + "Include executive summary, regional breakdown, and recommendations for next month.",
    thread,
    options))
{
    if (!string.IsNullOrEmpty(update.Text))
    {
        Console.Write(update.Text);
        textCount++;

        if (textCount == 10)
        {
            Console.WriteLine("\n\n===Pausing report generation...\n");
            break;
        }
    }

    latestReceivedUpdate = update;
}

From the code, we can see that the prompt simply asks to generate a complex report, and when the text output count reaches 10, we simulate an interruption.

The output so far in my example is the following:

\===Generating Monthly Sales Performance Report...

Business Report on Last Month's Sales Performance

\===Pausing report generation...

To resume the streaming, we will use the continuation token we’ve received.

if (latestReceivedUpdate?.ContinuationToken is not null)
{
    options.ContinuationToken = latestReceivedUpdate.ContinuationToken;

    Console.WriteLine("===Resuming report generation from state...\n");

    await foreach (var update in agent.RunStreamingAsync(thread, options))
    {
        if (!string.IsNullOrEmpty(update.Text))
            Console.Write(update.Text);
    }

    Console.WriteLine("\n\nReport generation complete!");
}
else
{
    Console.WriteLine("No continuation token available to resume.");
}

The final output in my example is the following:

\===Resuming report generation from state...

Executive Summary

Last month's sales performance reflects a mixed but overall positive trend, with total revenue reaching $1.5 million, an increase of 10% compared to the previous month. Key drivers of this growth included successful marketing campaigns and increased demand for our flagship products. However, certain regions underperformed, which necessitates a closer examination and tailored strategies moving forward.

Key Highlights:

• Total Sales: $1.5 million

• Percentage Growth: 10% MoM

• Top Performer: Product Line A, accounting for 40% of total sales.

• Underperforming Regions: Northeast and Southwest.

Regional Breakdown

1. Northeast Region

• Sales Performance: $400,000

• Change from Previous Month: -5%

• Analysis: Decreased sales attributed to increased competition and stock shortages. Target demographics reported dissatisfaction regarding product availability.

2. Southeast Region

• Sales Performance: $500,000

• Change from Previous Month: +15%

• Analysis: Successful promotional strategies led to higher engagement and conversion rates, particularly in urban markets.

3. Midwest Region

• Sales Performance: $350,000

• Change from Previous Month: +8%

• Analysis: Steady growth sustained by strong community ties and local marketing efforts. Product Line A performed exceptionally well.

4. West Region

• Sales Performance: $250,000

• Change from Previous Month: +5%

• Analysis: Gradual increase attributed to new distribution channels. Potential for further growth exists with enhanced local initiatives.

5. Southwest Region

• Sales Performance: $300,000

• Change from Previous Month: -10%

• Analysis: Economic factors and reduced promotional activities contributed to lower sales. Customer feedback indicated a lack of awareness regarding recent offerings.

Recommendations for Next Month

1. Northeast Region

• Action: Conduct a stock audit to ensure product availability. Implement targeted marketing campaigns highlighting new arrivals to regain lost customers.

• Goal: Increase sales by 10% through outreach and stock replenishment.

2. Southeast Region

• Action: Continue leveraging successful marketing strategies while exploring partnership opportunities with local influencers to maintain momentum.

• Goal: Target an additional 10% growth by enhancing community engagement.

3. Midwest Region

• Action: Expand community events and workshops focusing on Product Line A to increase brand loyalty and cross-sell other products.

• Goal: Achieve a 12% increase in sales through direct customer interaction.

4. West Region

• Action: Strengthen online marketing efforts and social media presence to attract a broader audience. Collaborate with local businesses for cross-promotion.

• Goal: Aim for a 10% increase in sales via enhanced branding efforts.

5. Southwest Region

• Action: Reassess marketing strategies and implement a targeted advertisement campaign focusing on product benefits and customer testimonials.

• Goal: Reverse the negative trend by aiming for at least a 15% increase in sales.

Conclusion

The sales performance over the last month demonstrates both opportunities for growth and areas needing attention. By honing in on regional strengths and weaknesses, the company can optimize its strategies to ensure sustained growth. Implementing the recommendations outlined above will be critical for the upcoming month, with particular emphasis on addressing the underperforming regions to maximize overall profitability.

Next Steps:

• Regular monitoring of sales data will be essential to track the effectiveness of implemented strategies.

• Prepare a follow-up report to analyze the outcomes of these recommendations at the end of next month.

Report generation complete!

What we did here was use the continuation token to resume the stream from the interruption point. Keep in mind that in some situations you will need to store continuation tokens persistently.

Finally, if you are doing non-streaming, using the token will be similar to:

AgentRunOptions options = new()
{
    AllowBackgroundResponses = true
};

AgentThread thread = agent.GetNewThread();

AgentRunResponse response = await agent.RunAsync("Generate a detailed business report analyzing last month's sales performance. "
    + "Include executive summary, regional breakdown, and recommendations for next month.", thread, options);

while (response.ContinuationToken is not null)
{
    await Task.Delay(TimeSpan.FromSeconds(5));

    options.ContinuationToken = response.ContinuationToken;
    response = await agent.RunAsync(thread, options);
}