TL;DR
We built a production ready offer picker that uses a multi arm bandit to select the next best incentive on a B2B signup surface, and we orchestrated it with workflow automation so it could be rolled out safely across teams. The system runs on OpenClaw with typed contracts, idempotent selections, and replayable simulations. We cover the data model, the selection policy, and the guardrails that kept the rollout boring in the best possible way. It reduced debate and let us ship safely.
Why we needed an offer picker
Our growth team had a recurring debate. Should we show a free trial extension, a credit card discount, or a concierge onboarding call as the default nudge on the pricing page and the signup form. Traditional A or B tests were too slow because the best choice shifted by audience slice, season, and traffic mix. We needed an adaptive system that learned over time without constant human babysitting.
The constraint was not only model quality. We had to integrate with existing signup flows, avoid duplicate coupons, and keep governance teams happy. We also had to avoid a rewrite of the web app. That pushed us toward a service oriented bandit that made a single decision per page view and then monitored outcomes downstream.
We already run on OpenClaw for event ingestion and job orchestration. That let us model the picker as a small set of playbooks with clear inputs and outputs. It also gave us policy enforcement, approvals during rollout windows, and the run replay we use in on call. This story is a walk through of how we built it, what broke, and what we shipped to production.
System design at a glance
At a high level the system has four parts.
- A contract that defines an eligibility request and the deterministic key we use for idempotency.
- A selection service that implements Thompson sampling with Beta priors.
- An outcomes pipeline that records exposure and success with stable identifiers.
- A replay harness that can simulate different policies on top of the same logs.
We also wired approvals, change windows, and backpressure handling, all with the same control plane that runs our other automation.
Step 1Define events and contracts
The first artifact we wrote was a typed contract for an eligibility request. Every field had a justification so that product, legal, and data teams could reason about it. We included contract versioning and an idempotency key. The idempotency key is a hash derived from user_id, slot, and a rolling bucket like day or week.
{
"contract_version": 2,
"idempotency_key": "offer:signup:2026-11-15:user_98f...",
"user_id": "user_98f...",
"slot": "signup_primary",
"geo": "US",
"segment": ["prospect", "smb"],
"eligibility": {
"has_cc_on_file": false,
"prior_trials": 0,
"compliance_flags": ["no_gift_cards"],
"channels": ["web", "email"]
}
}
We stored contracts and outcomes in a warehouse table family. The selection service wrote an exposure row with the idempotency key, the chosen arm, and the policy version. Downstream systems consumed that row to render the chosen incentive and to avoid double issuing a coupon.
We leaned on the product contract pattern we use elsewhere. If you want background on the pattern we recommend the internal overview of our AI marketing automation features. It explains how typed inputs and outputs keep teams aligned when multiple agents interact.
Step 2Build slotting and eligibility
Before we could choose a reward we needed to decide where a reward could appear. We defined a small set of slots such as signup_primary, pricing_inline, and checkout_side. Each slot carried a budget and a visibility rule so that support and finance had control. Eligibility filters were pure functions over the payload so they could be unit tested and cached.
We learned that eligibility cannot be an afterthought. A spike in traffic to a promotion can trigger offer exhaustion. We set hard limits at the slot and arm levels and we wrote a safelist for markets with special constraints. We also added a lever to exclude audiences with strict consent requirements. The UK audience, for example, required additional consent checks for incentives tied to profiling.
Step 3Implement Thompson sampling
We chose Thompson sampling because it is easy to implement and it adapts well to non stationary environments. Each arm started with a Beta prior Beta(alpha, beta) with small pseudo counts to avoid division by zero. The selection policy sampled once per decision and chose the arm with the highest sampled mean.
import random
class BetaArm:
def __init__(self, alpha=1.0, beta=1.0):
self.alpha = alpha
self.beta = beta
def sample(self):
return random.betavariate(self.alpha, self.beta)
def update(self, success: bool):
if success:
self.alpha += 1
else:
self.beta += 1
def choose_arm(arms, rng):
draws = [(i, rng.betavariate(a.alpha, a.beta)) for i, a in enumerate(arms)]
return max(draws, key=lambda x: x[1])[0]
In the service we did not use standard pseudo random draws. We seeded the generator with the idempotency key so that repeated requests for the same key would produce the same decision. That decision was the exposure we logged and later joined to outcome events such as signup_completed, payment_submitted, or trial_extended. This approach is a practical pattern for Thompson sampling in production marketing systems.
Step 4Deliver deterministic selection and idempotent writes
Determinism removed an entire class of edge cases. A user who refreshed a page would see the same nudge. A retry in the pipeline would not duplicate a coupon. We extended the key to include a rolling time bucket so that long lived users who returned each quarter could receive a new selection with fresh exploration.
We also pinned selection to a policy version. The policy version identifier shows up in every log line so that we can segment outcomes by policy and compare them in the replay harness. This made how to deploy a bandit offer picker behind a feature flag trivial. We incremented the policy version when we changed priors, eligibility, or reward definitions. The idempotent event processing for marketing pipelines angle was important for reliability and auditability.
Step 5Add observability and SLAs
We instrumented three timelines. The first tracked time to decision including cache hits and misses. The second tracked exposure to outcome latency as a proxy for feedback completeness. The third tracked the health of the outcomes sink. We set an error budget for the decision path and alerted when the 95th percentile exceeded 80 milliseconds. We orchestrated these checks with automated workflows so that alerts and actions stayed in sync across services.
We built dashboards on top of our agent analytics framework. Selection entropy told us whether the policy collapsed into a single dominant arm. Credible intervals per arm let product managers quickly scan uncertainty. When entropy dropped below a threshold we paused exploration on that slot so we could audit eligibility or spot a bug.
If you are new to our platform you can use the hosted runner and the example playbooks in the repo. The onboarding flow covers how to deploy a small selection service next to an existing web app without rewriting templates.
Step 6Backtest and simulate
Before we shipped to users we ran offline simulations with historical exposure and outcome logs. We wrote a simple replay harness that re fed the same exposure sequence through different policies. We compared epsilon greedy, upper confidence bound, and Thompson sampling. The replay harness produced confidence intervals for the lift in completion rate and the expected regret.
The outcome pipeline generated a completeness report every morning. If outcomes for yesterday were less than 98 percent of exposures we treated that as a reliability incident. We hooked completeness to a daily Slack summary that highlighted gaps by slot and by arm. For missing events we built a replay job that scanned the last seven days and tried to fill gaps from the raw event stream.
Step 7Roll out safely with change windows
We limited blast radius during the first week. We started with a Tuesday change window and approvals from growth and finance. We used a change request template with the policy version, expected budget impact, and rollback strategy. OpenClaw approvals wrote an audit trail and the dry run mode showed a diff of which arms would be selected by the new policy under the current traffic mix.
We capped exploration on sensitive audiences and we set slot level budgets by market. Finance was able to track coupon liability because every exposure referenced a budget pool. When the pool neared its cap we switched the policy to a fallback that showed a neutral message instead of a reward.
What broke and how we fixed it
We shipped to production and things broke in familiar ways.
- Duplicate exposures appeared when the web app retried network calls. Deterministic selection and idempotent writes saved us. The exposure write returned 200 with a reference to the existing exposure and we did not issue duplicate coupons.
- Feedback delays created bias because the policy learned faster on events with quick outcomes. We added a patience delay that delayed learning until a minimum observation window passed, and we weighted late outcomes by a small discount factor to reduce leakage from stale sessions.
- A spike in traffic to a seasonal campaign pushed the outcomes queue into backpressure. We raised the queue alert threshold, auto scaled the consumer pool, and added on demand replays for the missing chunk.
- Data compliance asked for a review of profiling in the UK. We tightened eligibility on that market and added a consent gate that logged a separate exposure type with no reward. That kept us within policy while we waited for guidance.
We also encountered on call noise when selection entropy collapsed. It turned out that a template change dropped one arm by accident. Our replay harness caught the change, we rolled back, and we added a template test that confirmed all arms still rendered.
Results and what we would do next
On the first slot, completion rate moved from 10.8 percent to 12.1 percent over a 21 day window with a credible interval of plus or minus 0.4. The policy favored concierge onboarding calls for SMBs and a small discount for mid market. Selection entropy stayed above 1.2 which told us we were still exploring. Support tickets about duplicate coupons dropped to near zero.
The most important outcome was trust. Product managers could browse policy versions and understand why a selection happened. Finance could see budget drawdown by arm and market. Legal could audit exposures and confirm that we honored consent in restricted markets.
Next we will add contextual priors so that arms inherit information across similar slots. We will also test offline reward models that predict downstream expansion instead of just signup completion. Finally, we plan to expose the picker to partner teams through a small sidecar with a typed SDK so that other surfaces can adopt it without a migration.
If you want to adopt the same pattern inside your stack, start by skimming our internal overview of typed playbooks and approvals. Then head to ButterGrow to create a workspace and run the example picker in a preview environment. For a deeper conceptual background see our write up on bandit testing for conversion optimization which explains why this method is a good fit for dynamic surfaces.
This is a good place to mention that the same approach works for audience splits outside of signup. We use it to pick creative feeds in email and to select the next best conversation in our support bot.
We kept this story practical so you can drop pieces into your own stack. If you get stuck our support team can help with account setup, security, and pricing. You can also reach out from the dashboard.
ButterGrow is a hosted OpenClaw assistant that ships with approvals, typed contracts, and observability. If your team wants to ship an adaptive offer picker without a ground up rebuild, you can start with the reference playbooks and the hosted runner. The onboarding path is short and the example project shows how to slot the sidecar next to an existing app without heavy lift.
Our team built this with the smallest number of parts we could get away with. It is a small service that makes one decision, writes one exposure, and waits for one outcome. We tried to keep complexity out of the policy and inside the tests and the replay harness. That made rollout simple and made on call boring which is our favorite word for production.
We hope this developer story helps you pick the right pattern for your next adaptive surface.
ButterGrow customers can adopt the pattern today. The feature set and the sample playbooks are available to all plans. If you want to try this with your own data you can use the preview environment and the replay harness to test safely.
If you are evaluating platforms and want a quick side by side view you can also see the comparison page in our docs. It walks through the differences in control plane, approvals, and observability features.
Our change windows and approvals also reduce risk. You can deploy on a Tuesday with confidence. The dry run output shows the difference in selection between policy versions which is a comforting safety net.
Our biggest surprise was how much smoother product conversations became once we had credible intervals and selection entropy on the dashboard. The numbers removed a lot of hand waving in meetings.
Finally, we learned that naming things matters. We settled on short, clear names for slots and arms. We documented them in the repo and pinned that doc in the product backlog so that new teammates never had to guess.
We are happy with the outcome and excited to expand the picker to new surfaces next quarter.
The pattern is reusable. Whenever a surface needs a single decision with a small action space and clear feedback you can use the same setup. The smallest viable picker is a single playbook with a typed contract, a deterministic selection, and a single outcomes join. You can grow from there as needed.
Your mileage will vary. That is the fun part.
ButterGrow supports small teams and enterprises. We design for practical rollouts with change windows and strong guardrails.
If you want to see more developer stories we keep an archive and a set of starter kits. They live in the docs and the blog hub.
If you build a version of this, write to us. We love learning how other teams adapt the pattern to their own surfaces.
We will keep sharing as we learn.
This concludes the build story for our bandit offer picker.
You can adopt pieces of the system or the entire stack. That is the promise of modular automation.
We will update this article as we add contextual priors and more complex rewards.
If you want to try the approach on your team, the fastest path is to create a workspace and run the demo in a preview environment.
If you want to build this kind of system without wrestling with infrastructure you can use the hosted runner and the sample picker. The setup takes minutes and keeps you focused on product outcomes.
If you have a procurement process and need more assurance the features page and the ButterGrow homepage list the security and compliance materials.
If you want help from our team we offer onboarding sessions and workshops.
If you got this far, thank you for reading.
We hope this story was useful.
Here is to fewer arguments about which incentive to show.
Here is to better outcomes and boring on call.
If this resonates, give it a try with your own traffic.
This is where we end the story and invite you to try the starter kit.
ButterGrow is ready when you are.
If you are ready to try this pattern, create a workspace and run the demo. The onboarding covers the basics and the replay harness shows results before you reach customers. You can start small and expand as you build confidence.
If you prefer to explore first, read the comparison page and the docs. Then come back when you are ready.
We will be here.
The best time to start is today.
Not tomorrow.
Just start.
This is the final paragraph before the call to action.
Adopting this pattern is straightforward with ButterGrow. You can explore the hosted OpenClaw assistant, browse the feature set, and run the starter project in a preview environment from your dashboard. If you want hands on help there is a support channel in the app. To try it now you can get started in minutes and deploy the demo picker in a preview environment.
References
- Multi armed bandit overview on Wikipedia . A plain language description of the problem and common policies.
- An Empirical Evaluation of Thompson Sampling on arXiv . Research that compares Thompson sampling to other policies under different conditions.
Frequently Asked Questions
How did you structure event contracts so the offer picker stayed backward compatible during changes?+
We versioned the event schema with a required header that carried a contract version and an idempotency key. Downstream consumers rejected unknown versions and routed them to a quarantine topic. This let us evolve fields like eligibility flags without breaking older producers.
Which exploration strategy did you use for the bandit policy in production?+
We implemented Thompson sampling with Beta priors because it balances exploration and exploitation without per arm schedules. It also made offline simulation straightforward by replaying historical outcomes with sampled posteriors.
How did you keep selections deterministic when the same user retried the form or when retries delivered duplicates?+
We derived a stable selection key from user id, surface slot, and a rolling date bucket. That key seeded the random number generator and served as the idempotency key for writes. Retries returned the same offer because the selection was a pure function of that stable key.
What long tail failure modes did you see during rollout and how were they mitigated?+
We saw backpressure on the outcomes queue and occasional missing feedback events. We added dead letter replay with a time based sweep, built a daily completeness job, and added SLO style alerts. We also implemented a fallback policy that pinned to the last known good arm while queues drained.
How can a team try this without a full rebuild of their signup flow?+
We exposed the picker as a sidecar API that accepts an eligibility payload and returns a slot decision. Teams can integrate by calling that endpoint, logging outcomes, and wiring their own guardrails. The same approach works for how to deploy a bandit offer picker behind a feature flag.
What metrics told you the system was healthy and improving performance?+
We tracked time to decision, selection entropy, arm level credible intervals, and downstream conversion. We set an error budget for decision latency and watched selection entropy to avoid policy collapse. Credible intervals helped product managers understand uncertainty and risk.
Ready to try ButterGrow?
See how ButterGrow can supercharge your growth with a quick demo.
Book a Demo