Last month was an unusually social one for me with two virtual events.
First was a long term favourite of mine; MeasureCamp. And then a new one (for me); Bhav Patel’s CRAP Talks.
This was my first time attending a virtual MeasureCamp although I knew a bit of what to expect because I had been involved in a test event in lockdown number one.
I presented the following slides to introduce a discussion on “MLUX”; the user experience of working with machine learning.
The discussion that followed didn’t really talk about forecasting at all (which is fine - this is a bigger problem than one area). People expressed frustration with things like product recommendations that lag six months behind the current fashions and the difficulty of comparing the uplift from different ways of doing things.
From one angle it is easy to say that the problem lies in a lack of skills/training for the relevant people. But this will never be a complete solution because there are so few places where one can learn about the development and training of world class models.
In my (biased - look where you are reading this!) opinion improvements in the tooling is more likely to be the best way forward. Libraries like Keras or Huggingface’s transformers allow someone with my level of skill to get started in this area - I would not be able to code or train stuff like this from scratch. And I think similar tooling can and should exist for other people too.
In this case CRAP stands for Conversion Rate, Analytics and Product. I have been aware of it as a offline meetup for a few years now but because it is mainly in London and I don’t have a focus on conversion rate optimisation or product analytics I have never made the effort to attend. This is my loss as I thought the virtual event was excellent.
The main part of the event was a presentation from Facebook’s marketing science team about Robyn which is a new open source project they have launched to help with Marketing Mix Modelling.
I have done a small amount of MMM in the past and I wish Robyn had been available then. I was learning as I went along without the support of someone with more experience so I spent a lot of time thinking and worrying about the details of what I was doing and whether or not it was the right approach. A page like this would have saved me so much stress even if didn’t also come with code that does all the calculations for you.
Following this presentation there was a discussion on forecasting which Bhav was kind enough to invite me to be a part of. I think he was hoping for some juicy arguments but (boringly!) I mostly agreed with everyone else. A quick summary of some of the points covered:
I think I went a bit off piste talking about how your forecast is your “model of the world” but people seemed very tolerant of my ramblings ;-)
I look forward to seeing what else comes out of CRAP Talks in the months and years to come.