ICE scoring: how to rank growth experiments with a worked backlog

ICE scoring multiplies Impact, Confidence and Ease, each scored 1 to 10: run the highest scores first and rescore every week. Picture a growth lead on Monday with 40 ideas in a Notion doc and engineering time for three.

The formula takes a minute to learn. The hard part is getting two people to give the same idea the same score.

What ICE scoring is

ICE scoring gives each idea three scores from 1 to 10 and multiplies them. Itamar Gilad writes the score as Impact times Confidence times Ease. He defines Impact as an estimate of how much the idea will move the key metric you are trying to improve. Gilad describes Ease as an estimate of the effort and resources an idea needs, "typically the inverse of effort". Confidence is how sure you are about the Impact estimate.

The origin explains the design. Gilad credits Sean Ellis with inventing ICE and says it was "originally intended to prioritize growth experiments". We read that as a hint about where it works best: a steady stream of small tests, not a two-year roadmap.

Multiplying and averaging produce different rankings. Multiplying punishes a low score on any factor, while averaging lets a big Impact hide a 2 on Ease. Pick one method and keep it, or this month's scores stop being comparable with last month's. The rest of this post multiplies.

Add three empty columns to your backlog this week: Impact, Confidence and Ease.

Score only ideas that attack the bottleneck

The bottleneck is the funnel step that loses the most potential customers right now. Scoring every idea across the whole funnel wastes the meeting. Impact means something different on each step, so the scores never line up.

Filter before you score. If you have not named the bottleneck yet, the SaaS growth strategy guide shows how to find it. Then write its metric at the top of the scoring sheet. In the worked example below, the metric is the share of people who start sign-up and go on to finish setup.

Every Impact score now answers one question: how much could this idea move that number? Ideas that touch other steps go to a parking list. They come back when the bottleneck moves.

Tag each idea in your backlog with the funnel step it touches, and move every off-bottleneck idea to the parking list before Monday.

Write anchors for each scale before you score

Scales drift without definitions. Without agreement on what a confidence of 7 means, the ProductPlan glossary warns, it "could lead to some very inconsistent assessments by various team members." In practice, the most optimistic person in the room ends up setting the scale. Anchors fix that: one plain line per score band, agreed before the first idea is scored.

Impact is relative to the bottleneck metric, not the company. For Masa, Kreto restructured the onboarding flow to show the core experience first and defer registration. The case page names that change as the primary driver of sign-up conversion rising from 32.7% to 63%. One case is not a benchmark, but it shows why reordering the step itself belongs in the top bands of the scale.

Paste the table into your backlog doc this week, rewrite each line for your product and agree on it before anyone scores.

Tie Confidence to evidence, not enthusiasm

Confidence is the score teams inflate most, because excitement feels like evidence. Gilad argues that you set Confidence by "looking for supporting evidence". His Confidence Meter lists evidence types and the confidence each one provides.

The meter is strict. On Gilad's scale, self conviction plus anecdotal support adds up to 0.5 out of 10, and opinions alone count as "Near Zero Confidence." Most teams score more generously on a simple 1 to 10 sheet. The principle still holds: a room full of opinions should not produce a 7.

Outcome data supports that caution. Kohavi and Thomke report that at Google and Bing, only about 10% to 20% of experiments generate positive results. At Microsoft as a whole, they write, one-third of experiments prove effective, one-third are neutral and one-third are negative.

The two figures cover different populations, two products versus a whole company, so they are not one benchmark. Both point the same way: many ideas that sound right in a meeting fail when tested.

Add an evidence column next to Confidence this week. Any Confidence score without a written source drops to 3.

A worked backlog: eight ideas for one bottleneck

Take a hypothetical B2B SaaS with a free trial. Its bottleneck runs from the sign-up form to finished setup: most people who start sign-up never finish setup. The growth team has one engineer and a part-time designer. Eight ideas survived the filter, most of them standard onboarding practices. Here they are, scored against the anchors and sorted by ICE score.

The reasoning behind four of the rows:

• Shorter sign-up form (360). Impact is 5 because the form is one friction point among several. Confidence is 8: funnel data shows the drop on that page, and users say why. Ease is 9, a form change that ships in a day.
• Template workspace (294). Users stall in an empty workspace, so Impact is 7. Funnel data and recordings show the stall: Confidence 7. Sample data takes most of the week, so Ease is 6.
• Defer sign-up (240). Moving the account step after first use earns an 8 on Impact. Confidence is 6: interviews support the idea, but nothing has been tested. Ease is 5 because it touches authentication and data storage.
• Dashboard redesign (32). Impact is 8 on paper, since setup happens on the dashboard. The only evidence is opinion, so Confidence is 2. Ease is 2 for a project of several sprints.

The top three ship this week. Defer sign-up shares the highest Impact on the list, yet it ranks fourth because its Confidence and Ease are lower.

One cheap test can change that. Show a slice of new visitors a demo workspace before the account step and count how many reach setup. If more of them do, a close test in your own product has won, so Confidence moves to 9. The score becomes 8 x 9 x 5 = 360, above every idea left on the list.

The bottom of the list teaches something too. The founder chat prompt is easy, but nobody has evidence it matters. The dashboard redesign shares the top Impact score and still lands at 32. Until someone finds a cheaper first step, it stays there.

Score your own eight bottleneck ideas against your anchors this week and commit to the top three.

Rescore after every test

Scores go stale as soon as a test ends. The tested idea retires: ship it if it won, archive it if not. Related ideas gain or lose Confidence, and Impact estimates shift toward the lift you saw.

Take the sign-up form test. If the shorter form wins, ideas that remove friction before first use gain Confidence, including deferring sign-up. If it comes back flat, form length was not the problem, and similar ideas lose Impact.

Ease deserves the same honesty. In Trustworthy Online Controlled Experiments, Kohavi, Tang and Xu describe a 2012 suggestion to change how Bing displayed ad headlines. The book says the feature "was prioritized low and languished in the backlog for more than six months." Then a software developer tried it, "given how easy it was to code." The authors report that Bing's revenue rose 12%, over $100M a year in the US alone at the time.

We do not know how Bing scored that idea. Our reading is narrower: when an idea is this cheap to try, a test teaches more than another debate about its Impact. Multiplication still ranks such an idea low, as the founder chat prompt shows, so keep one test slot each week for an untested idea that ships in a day.

Kreto runs this kind of weekly loop as an outsourced growth marketing team, with one backlog across ads, onboarding, pricing and analytics.

Add "result" and "rescored on" columns this week. Spend the first 15 minutes of Monday's meeting rescoring before new ideas get a turn.

ICE or RICE: when to switch

RICE adds a fourth factor. Sean McBride at Intercom lists its factors as "reach, impact, confidence and effort." ProductPlan notes that RICE also swaps Ease for Effort and divides by it. Intercom gives the reason for Reach: "To avoid bias towards features you'd use yourself, estimate how many people each project will affect within a given period."

Intercom scores Confidence as 100%, 80% or 50%, and Effort in person-months. Person-months suit quarterly roadmap planning better than a weekly test list.

ICE wins inside a bottleneck loop. Every idea targets the same step, so reach is roughly equal and a Reach column adds work without changing the order. ICE is also faster to rescore every week.

RICE wins when reach varies widely. Every buyer sees a pricing page change, while few users ever open a settings screen. Score both with ICE and a polished settings fix can outrank the pricing test. RICE also fits roadmap reviews that span the whole product.

Compare the reach of your top five ideas this week. If one touches far more users than the others, add a Reach column for that review.

Common ways ICE goes wrong

The formula rarely fails. The habits around it do.

• Score inflation. Everything drifts to 7 or 8, and the ranking stops meaning anything. Re-read the anchors before each scoring round.
• One person scores. The idea's owner rates it highest. Have two or three people score alone, then discuss any gap of three points or more.
• Ideas never retire. The backlog grows to 100 rows and nobody reads past row 20. Archive anything tested or untouched for a quarter.
• No metric at the top. Without a bottleneck metric, Impact turns into a vote on how much people like an idea.
• Averaging away a 2. See the multiply rule above.

Pick the habit your team shows most and fix that one at this week's meeting.

FAQ

How do you calculate an ICE score?

Score Impact, Confidence and Ease from 1 to 10 against written anchors, then multiply the three. The highest possible score is 1,000. In the worked backlog above, the sign-up form idea scores 5, 8 and 9, which gives 360. Sort the list by score and ship the top ideas that fit this week's engineering time.

Should you multiply or average the three scores?

Multiply. An idea with a 2 on Ease should not climb the list on Impact alone, and multiplication keeps it down. Averaging smooths those weak spots away and bunches most ideas in the middle of the range. Once you choose, keep it fixed, because switching makes old and new scores impossible to compare.

Who should score the backlog?

The people who will build and measure the tests: the growth lead, an engineer and whoever owns the data. Each person scores alone first, against the same anchors. Then the group discusses large gaps, because a gap often means one person holds evidence the others lack. The idea's owner should never score it alone.

How often should you rescore?

Rescore every week, before new ideas are scored. Each finished test changes the Confidence of related ideas and corrects your Impact estimates. Archive ideas that nobody has touched for a quarter instead of rescoring them. Once the anchors are written and the evidence column is filled in, the weekly rescore fits in the first part of the meeting.

Score your backlog against the bottleneck

ICE scoring is simple arithmetic wrapped around a hard habit: honest evidence. Name the bottleneck, write the anchors, ship the top three and rescore each Monday.

Start this week with the eight ideas closest to your bottleneck. If you want a second view on how to score them, talk to Kreto and we can go through your backlog against your bottleneck together.