How I run experiments like a Growth Scientist

When I first got into the growth part of marketing, I didn’t know how on earth to experiment, let alone start until I read Hacking Growth by Sean Ellis over 5 years ago.

After years of experience, I thought I would break down the framework or how to conduct experiments like a mad professor.

So let’s go, here is the framework:

Ideation > Experiments > Validation > Optimise > Scale

A Head of the Growth role is to draft up a tailored roadmap for the next 60-90 days which includes areas such as:

• Forecasting

• Budgeting

• Marketing funnels

• Channels

• Content strategy

And so on.

A key to success is experiments. Marketing never stops changing, you’re always playing catch up. To keep up in the game, you need to experiment with different tactics to drive the growth engine.

It’s all about speed and focus, the volume and velocity of testing always win. So the best way to speed things up is to test and learn what marketing actions drive revenue by:

• Defining the experiment's hypotheses.

• Defining the positioning and messaging

• Defining your audience

• Defining marketing channels

• Defining north-star metrics

Here are some strategies to include in your marketing roadmap.

Each tactic will help you experiment and build new revenue streams for your business.

Use these 3 frameworks to help you:

Framework #1: Organise your experiments

This is called a marketing backlog. Which is a list of tasks, experiments, and actions that are waiting to be completed.

It’s used to ideate, list, prioritise, and track the progress of your marketing actions.

Like any project, for example, SaaS project managers ship a new feature in their roadmap, a marketing project will go through several similar processes.

A marketing backlog is much like a product backlog in software development in that it is a prioritised list of features and enhancements for a product:

Ideation:

Brainstorm and generate ideas for new marketing strategies or campaigns. (such as identifying potential target audiences, channels, messaging, and creative concepts).

Market Experiments:

Implement marketing campaigns based on the ideas generated in the ideation stage, launched to a specific audience or market segment to gather data and insights about their effectiveness.

Validation:

Data is collected and analysed from the experiment to determine whether they achieve the desired results. KPIs are evaluated to validate whether the strategies are meeting their objectives.

Optimise:

Strategies/campaigns can be refined and optimised based on the insights gained from the validation stage. Such as making adjustments to messaging, targeting, creative elements, or other aspects to improve its performance/efficiency.

Scale:

Once proven to be effective, it can be scaled up to reach a larger audience or market segment. Allocating more resources and expanding the reach of the campaign to maximize its impact.

Example of a Marketing Backlog:

Here’s how to do each step.

• Action: explain the marketing action in detail

• Development: go through the 5 stages… Ideation > Market experiments > Validation > Optimise > Scale

• ICE Scoring: Using the ICE model, you need to assign a score from 1-10 for each of the three factors (Impact, Confidence, Ease)

• Action History: details of what has been conducted so far.

• Results: When the experiment or program is ongoing, record the results.

A well-done backlog will become your second brain and memory bank. It will help you to track 4 types of information:

• The history of what you tested

• The history of the A/B test results

• The history of the results, i.e. what worked best

• The history of what didn't work

Combine the results of all your marketing actions and you'll get a library of what worked for your company and what didn't.

** Before moving to step 2, let me show you how to THE ICE scoring model

The ICE scoring model is a simple tool made by Sean Ellis at GrowthHackers used to prioritise ideas and allocate resources towards actions that will generate the greatest return.

Most companies out there don't know how to prioritise marketing actions. Leading to 3 issues:

• Too much time on the wrong things.

• Plans are prematurely abandoned due to impatience.

• Chaotic v disordered and overwhelming marketing backlogs

Prioritisation and organisation are the roots of clarity and clarity produces results. Lacking this leads to bad outcomes. Use the ICE framework to prioritise.

ICE Scoring is a technique used to assess the potential impact and viability of marketing hypotheses. To use the model, just give each of the three factors a score from 1 to 10.

Here's an example of how it works:

You can use ICE scoring by either multiplying the scores or taking the average. But remember, don't add up the scores to get the final results. And don't forget, the final results are for comparing and deciding what's most important.

#2: Fast experimentation

The more you test, the more you increase your chances of success.

Top businesses commit to a weekly minimum of tests. Their growth is directly correlated to the number of tests conducted.

For example, Netflix found out that users user will browse the app for 90 seconds and leave if they find nothing. What did they do? Ran experiments by constantly changing their thumbnails every week.

To grow like Netflix, you need to launch experiments in the market and analyse the results.

Fast experimentation is about testing different marketing strategies or tactics quickly. The goal is to identify what works best for your company/product.

Testing different channels, value propositions, messages, types of content, pricing, or other hypotheses.

Allowing you to learn quickly and make informed decisions on how to allocate your resources.

Failing fast is a gift. It tells you what and when to stop. Saving both time and money. I'd like to share a strategy I've found highly effective for optimising your monthly marketing plans.

It involves breaking down your strategies into shorter, more manageable periods, typically in 2-week cycles.

Here's a straightforward 6-step process for conducting this type of marketing experiment:

Step 1: Define Your Focus

Start by clearly defining the problem or opportunity you want to tackle. Set SMART objectives for each challenge. These objectives will serve as your guide throughout the experiment.

For instance:

Identify a new acquisition channel.Launch into a new market.Increase revenue from a specific offer

Step 2: Produce a Hypotheses

Brainstorm and compile a list of potential strategies or tactics to address the challenge. Document these ideas in your marketing backlog and prioritise them using the ICE framework.

Step 3: Test and Monitor

Now, put your ideas into action during a 2-week sprint. Execute your campaigns and gather data on their performance. Ensure you have all the necessary content prepared for the experiment and closely monitor daily results.

Step 4: Analyse the Outcomes

After the 2-week sprint, dive into the data. Identify which ideas worked and which didn't by examining your North Star metrics. Compare the experiments and assign scores to determine the most effective strategies.

Step 5: Iterate and Improve

Use the insights gained from your analysis to refine successful strategies and develop new, improved tactics. This could involve creating better content, optimising your outreach, or making necessary adjustments to your campaigns.

For instance:

Enhance your content quality.Fine-tune your broadcasting methods.Launch and closely monitor the revised campaigns.

Step 6: Repeat the Process

Finally, don't stop here. Return to Step 2 and continue this process until you've effectively addressed your problem or seized the opportunity.

A few key points to keep in mind:

Embrace the concept of "failing fast" as a valuable learning experience.If something isn't working, don't hesitate to cut it from your strategy.This test-and-learn approach is one of the fastest paths to growth.

By following this framework, you'll gradually enhance your marketing strategy over time. It ensures that your marketing efforts are always data-driven and evidence-based, rather than relying on assumptions or guesswork.

Template for you

I've created a simple worksheet to help you 👌

If you want to duplicate this framework, you can steal my complete “Growth Expiremnt” Google Sheets template, which includes:

A reporting tracking sheet to track the experiment’s progress.An experiment project brief to write up each experiment

→ Create your sheet and Google doc (Click 'File' > 'Make a copy')

Summary

Smart marketing experiments stresses the need for constant testing in the ever-changing marketing world. The key is to experiment a lot and learn fast. Three practical frameworks are introduced: a marketing backlog, ICE scoring for prioritising ideas, and fast experimentation for quick testing. The simple 6-step process for optimising monthly marketing plans by focusing on short cycles, learning from failures, and making improvements.

TLDR

Three main frameworks are introduced:

• Marketing Backlog: It's like a to-do list for marketing ideas and experiments. It helps prioritize and track progress.

• ICE Scoring Model: A simple tool to rank ideas based on their impact, confidence, and ease of execution.

• Fast Experimentation: Test different marketing strategies quickly to find what works best.

The blog also gives a practical 6-step guide for monthly marketing experiments:

• Define Your Focus: Clearly state what you want to achieve.

• Produce a Hypothesis: Come up with ideas to tackle the challenge.

• Test and Monitor: Implement your ideas in a short sprint (2 weeks) and watch how they perform.

• Analyze the Outcomes: See what worked and what didn't.

• Iterate and Improve: Learn from the results and make things better.

• Repeat the Process: Keep trying and improving.

The bottom line: Experiment, learn from failures, and keep growing smarter over time.