Running growth experiments is a strict process.
It’s important to formalise your process before diving into one or multiple experiments. If you don’t, you may end up with completely irrelevant data, useless insights, or worse, no insight at all.
If you are a startup or running solo on your growth team for now, chances are you can’t afford fancy software to help you out. Don’t worry though, you can do lots without them. Definitely do not spank thousands on an app just yet.
Here I will share the process I use to craft, setup, and document my experiments at low costs. It’s not super efficient. It’s a bit frustrating. It could definitely be improved.
However, it does work and for now this is all you need.
My growth experiment stack
I prominently use three pieces of software to draft and report on experiments.
- Google Docs/Sheet
Documenting experiments with Google Docs
With every new experiment, I make a copy of that document and start filling it out.
The issue with Google Docs is they don’t integrate with anything easily. You will have to report manually. However, you can easily create folders and keep records of everything you’ve done. They are great to take extensive notes on what you’ve done and learned.
Experiment funnel with Trello
I use Trello to quickly dump growth ideas and move them down the experiment pipeline. This is how my pipeline looks:
My experiment funnel on Trello.
As you can see, it is very visual and easy to understand. Wherever I am, I can open my Trello app and quickly write an idea down in the left column. Then, at specific ideation or review days, I will mark up the ideas with the right lever (AARRR) and decide which ones go into preparation.
While Trello is great to move cards and keep an overview of the work ahead, it is terrible for other things. Two main issues I have identified:
- To-dos: Trello is terrible to keep a to-do list nice and tidy. If you need to go granular, you would be better off using yet another app like Todoist.
- Reminder: Trello is even worse with reminders. You can only put reminders on cards (as opposed to the to-dos, which would make them slightly more useful already). Reminders don’t work really well at… well, reminding you to do stuff. For actual reminders, I use Basecamp’s app.
Day-to-day structure with Basecamp*
Which is why I use Basecamp. Basecamp allows me to go into the nitty-gritty of each experiment, set up granular tasks, and reminders.
Once I have brought an experiment from Trello to the Up Next column, I will create a Google Docs experiment with an in-depth description of the idea, what I hope to get from the experiment, and specific goals.
Once that is done, I then create a Basecamp to-do list (under my ‘Growth’ project) and assign myself times and dates to achieve the different tasks.
Again, this is a bit time consuming and definitely not super efficient. But, it is easy to setup so if that is what you are looking for I would advise you follow this setup.
*Unfortunately, Basecamp is fairly expensive. You would have to spend $99/month on this app, which is very steep for any solopreneur. Some of you readers might have a main job and develop/grow side hustles. If that is the case you may have access to it through that. If not, use Trello, Todoist, and Docs.
Using the growth experiment template
At last, let’s go through the experiment template. Like I said previously, I set this up on Google Docs and you can get a copy of my template here.
Let’s go through each section one by one.
- Hypothesis table
- Experiment design
- Resource and probability
- Results table
- One sentence result
This is an important bit. Don’t just give your experiment any old name. I follow a strict pattern for my experiment titles which is:
Number – Lever – Location – Few descriptive words
I use a number first to keep my files in order. I usually just call my experiments E1, E2, E3, etc.
I use the AARRR levers; acquisition, activation, revenue, retention, referral. This gives you a quick view of what this experiment is supposed to improve.
Most of your experiments will be on your site. It is good practice to write down where. Simply use the name of the page, like ‘homepage’, ‘pricing page’, ‘blog’, etc.
Finally, a few descriptive words. Imagine you were to bring someone in to help you with your growth experiments. All she could see was your experiment titles. She should have a quick understanding of what you’ve done so far. Use words like ‘Implement testimonial feature’, or ‘display social shares’.
What is the goal of your experiment? This is not the place for numbers or metrics. This is where you write down what you hope to test, achieve, or prove with your experiment.
Example: To understand the impact of displaying the number of social shares an article has had on conversion rates from the blog.
Time for numbers and metrics. This is the place to put them. Use your better judgment to put a hypothesis (i.e. the result you expect from your experiment) with numbers.
Example: I predict that displaying social share numbers will increase conversions by 5%, adding 250 new blog subscribers per week.
This table should represent your experiment at a glance. Put the current numbers against your hypothetical numbers. I like to colour the cell of the unknowns (in the hypothetical numbers column), just so they stand out.
Example of a growth experiment hypothesis table.
A step by step of how you will create and launch this experiment. Start from the beginning. What needs to happen? Be as granular as you need.
If your experiment is fairly complex, you might transcribe these steps into your Basecamp todos.
- Design a new version of the White Paper XYZ.
- Get John in marketing to draw a new illustration.
- Make sure I triple check the copy.
- Make the design available as a template on Hubspot
- Ask Erica to show me how to do this again.
Resource Estimation and Probability
This section could be split in two, but I tend to keep them together.
Estimate the time it will take for each of the core team working on this experiment to make it happen. This is an estimation, don’t kill yourself trying to be too accurate.
Pro tip: most of my experiments do not involve any engineering or design time. I try to keep my experiments simple enough that I can get them done by myself.
Then, try to give a success probability to your experiment. How likely is it that your experiment is going to positively influence what you are trying to improve?
Use this section as a prioritisation tool. If you have experiments that will take 2hrs of your time and have a high probability of success, of course, put these ones up front.
Time to report on your success (or failure).
The results table should look pretty much the same as your hypothesis table. Enter your control version numbers vs. your new results. As simple as that.
At a glance, someone without any knowledge of your experiment should be able to say which version won.
Pro tip: your results table could have more rows than your hypothesis table. Let’s say you are trying to improve the conversion rate of a landing page. You could run ten experiments on the same landing page, changing the design/copy slightly, and report them all in there.
One sentence result
This is a tough one to master. If someone were to put a gun to your head and asked you to sum up your experiment results in one short but effective sentence, what would it be?
Trust me, it sounds easier than it actually is. Once again, this very simple sentence should get someone who doesn’t know anything about your experiment completely up to speed.
Use numbers, percentages, and strong verbs.
Example: The winning variation of the landing page has a 7% conversion rate, about 2% more than the control version, bringing an additional 127 leads per month.
Share what you have learned. You can use more words and space than with your one sentence result, so make sure you use it.
I like to keep each single learning point on its own, with some specification under it.
- Animals boost conversions
- We saw an incredible increase in conversion when we switched the baby pictures with images of dogs.
You have done all of this, and now what? Do you revert everything back to the variant? Do you push code to confirm the winning variation? Do you plan more tests?
Put together a post-experiment to-do list.
Pro tip: more often than not, your experiments will snowball. You will close one with actions to start new ones inspired by the one you just finished. This is the joy of growth experiments; it is a never-ending process 🙂
Conclusion: go experiment!
There you go! You now have an entire growth experiment canvas to use. This will take you far, trust me. I know it isn’t glamorous, fancy, or integrated; but it is free and easy to use. Your wallet will thank me later.
Now go and experiment, you crazy people.