The Anvil

A place to hammer out ideas

Video: Picking a data transform

October 16, 2020

Data transforms are super important for getting the most out of your forecasting. With Forecast Forge you need to use them if your data must be within a specific range (e.g. sessions must always be greater than zero, conversion rate must be between 0% and 100%) and they can also improve the accuracy of your forecasts too!

This video shows you how to test different data transformations and pick the one that performs the best.

All the examples in this video use the sidebar to configure the forecast rather than the FORGE_FORECAST custom function. This is because the using the sidebar enables us to automatically backtest the forecast method against historical data. You can do this with FORGE_FORECAST too but it is more effort.

Read more

A Forecasting Template with SEO Algorithm Updates

September 21, 2020

I recently read a very interesting post from Patrick Stox on the Ahrefs blog. Since it concerns itself with SEO forecasting I wanted to write something covering two main points:

  1. How to do this in Forecast Forge
  2. Different (perhaps better?) methods for dealing with algorithm updates

Patrick uses a forecasting algorithm which is part of an open source project called Prophet. Prophet was created by Facebook and it is the main algorithm used by Forecast Forge (at the time of writing). It is possible to produce a very similar forecast to what Patrick/Ahrefs have done using Forecast Forge.

You can see an example template of this in Google Sheets

Read more

Post-Bank Holiday Update

September 1, 2020

This last week has been all about usability.

I have tried to improve the error messages so that when the inputs are bad or something else goes wrong you have a better idea of what it is and how to fix it.

Believe it or not, this is way better than it was before

There is still a lot of work to be done here but these improved messages will hopefully help people use the tool; an important part of learning is making mistakes and experimenting but this doesn’t work if you can’t get good feedback - if you are just guessing and hoping for the best it will take a lot longer to learn.

Read more

Get Better Results from CausalImpact by Tweaking BSTS

August 26, 2020

CausalImpact is an R package that helps analyse the causal effect of an event on a metric of interest.

It is often used by media analysts trying to pinpoint the uplift caused by campaigns or by SEOs wanting to prove their impact to clients.

Mark Edmonson has made a tool called GA Effect where you can try it online with your Google Analytics data.

The output of the analysis can be visualised like this:

Textbook example from the CausalImpact documentation
  • The first plot shows the actual data (black line) and a forecasted estimate for what would have happened based on the historical data prior to the vertical dotted line.
  • The key assumption is that any difference between the forecast and the actuals is because of the change that happened on the day marked by the dotted line.
  • The following two charts show the size of the daily effect and the cumulative overall effect.
  • The pale blue ribbon is a predictive interval. If this is entirely above or below zero then people will reject the hypothesis that there was no change and conclude that there was a positive/negative impact.

Read more

Launch Week

August 24, 2020

On Tuesday last week I casually tweeted about my new Forecast Forge project.

I was hoping to line up a few beta testers to kick the tires and break the addon in new and unexpected ways.

It didn’t quite work out like that.

This is starting to sound like the kind of update where I revel stellar signup numbers off the back of going viral. Spoiler Alert that isn’t what happened.

The Good

I’ve been a fairly heavy Twitter user since late 2008 and the above launch tweet is by far the most popular one I’ve ever had on the platform.

Read more

Get updates in your inbox