Multi-Armed Bandit Simulation · Wayfair AI coding interview practice

Stage 1 of 3: UCB1

Background. A multi-armed bandit has K arms; pulling arm a pays 1 with probability probs[a] and 0 otherwise. A policy chooses an arm each round from what it has seen so far and tries to collect as much as it can over T rounds. UCB1 is the classic one: every arm is tried once, then the arm with the best mean plus an exploration bonus is pulled.

Implement total_reward(probs, rounds, seed) in bandit.py.

The RNG is random.Random(seed). Each round, after the arm is chosen, one call to rng.random() draws the reward: 1 when the draw is strictly less than probs[arm], otherwise 0. Nothing else consumes the RNG.
Rounds are numbered t = 1, 2, …, rounds. While some arm has never been pulled, the lowest-index such arm is pulled (so rounds 1..K pull arms 0..K-1 in order). Afterwards, the arm with the largest mean + sqrt(2 ln t / n) is pulled, where n is that arm's pull count and mean is its total reward so far divided by n, that one division (a running average rounds differently and can flip a near-tie pick); ties go to the lowest index.
Return the integer total reward. It must run for K = 10 arms and T = 100,000 rounds within the test's time budget.

Example

total_reward([0.1, 0.5, 0.9], 1000, 7)    # 858, the same every time for the same seed
total_reward([1.0, 0.0], 2, 5)            # 1: rounds 1 and 2 pull arms 0 and 1 once each

Done when Submit goes green.

Constraints

Python 3.12+ standard library only
Time limit: 10 seconds per test file
Editable: bandit.py
Read-only: all tests/test_*.py files

What comes after stage 1

The next stages open once the tests for the one before them pass. Each raises the bar, the way an interviewer does in the room.

Stage 2: Thompson sampling. A second policy that explores by sampling instead of by bonus, on the same RNG contract.
Stage 3: Arms that drift. The arms change under the policy, and a second policy forgets the distant past to follow them.

The files you get

Yours to edit: bandit.py.
Read-only: tests/__init__.py.
Tests you can read and run: tests/test_part1_ucb1.py. Hidden tests run on Submit.

The round: an agent in the editor

The assistant here works like the one in the interview IDE. It reads your files, runs code and proposes changes, and you accept or reject each one. It doesn't volunteer ideas, so what gets built is what you asked for. Along the way the interviewer asks about your decisions, and at the end the report says what you checked and what you accepted unread.

Frequently asked, briefly answered

Is Multi-Armed Bandit Simulation free?

It is a Pro problem. Free plan includes one complete problem, plus the first stage of several more, so you can see how a round works before you pay.

Which company asks something like this?

Wayfair. It is modelled on Wayfair's round: Final-round coding for machine-learning roles on HackerRank with the screen shared: AI assistance allowed, agent mode included, a small simulation to write, and most of the hour spent on follow-up questions about the strategies behind it.

Do I bring my own AI tool?

No. The assistant is in the editor, same as in the interview IDEs at Meta, Stripe and Amazon. What you choose at Begin is the format: an agent that proposes changes you accept or reject, an answer-only assistant that can't write code, or no assistant at all.

What does the interviewer ask?

3 questions along the way about your own code, and one at the end about the whole session. A weak answer gets one follow up. The answers go into the report, which ends with a hire or no hire verdict.

Start the problem

Sign in and the problem opens where you left it. New here? An account takes a minute to create, and the Pro plan starts at the first stage of this problem.

Start this problem → Sign in

More like it: every practice problem, how Wayfair runs its AI round, the AI coding interview guide.