Knowing when to scale (and how to prove that you can do it)

When you’re talking to investors about a Series B, Series C or later round, one of the questions that will inevitably come up is “What are your CACs?”. It sounds like a simple question, but from the question of what costs to include and the right way to account for organic traffic to the pandora box of multi-touch attribution, there are lots of devils in the details.

What's more, the real question is not "What are your CACs?" but "What will your CACs be if you invest $10-20 million in sales & marketing?". It’s hard enough to calculate historic CACs for different acquisition channels with a high degree of accuracy. It’s much harder to predict future CACs at bigger scale.

And yet it shouldn’t come as a surprise that later-stage investors are so focused on this question. When you’re raising a Series B or later round, you’ve achieved Product/Market Fit (which is hard to define, see me attempt here) and you’ve got what Jason M. Lemkin calls “Initial Traction” and “Initial Scale”. At that point, the biggest thing standing between you and building a $100M+ business is finding scalable and profitable customer acquisition channels. Obviously you still have to overcome lots of other challenges along the way, but if you’re at $5-10M in ARR and you are confident that you’ve found scalable sales and marketing channels you are in an excellent (and rare) spot.

So how do you know if your customer acquisition channels will scale, that is, if a 10x increase of your sales and marketing spend will lead to a 10x increase in new customers? Consumer Internet startups are sometimes in the fortunate position to have found a profitable customer acquisition channel that offers huge potential for expansion. If ads on TV, YouTube or Facebook work for you, you might be able to increase your spending by 10x (and maybe much more) because these platforms have such a gigantic reach. In the B2B SaaS world this is very rare. Mass-market advertising won’t work because there’s way too much ad wastage, and targeted ads usually don’t give you the volume to easily 10x your spend.

Without a careful keyword volume analysis, being able to profitably spend $10k a month on AdWords doesn’t mean much in regards to your ability to spend $100k a month. If you spend small amounts on AdWords you will by definition (AKA by algorithm) capture the lowest-hanging fruits. As you’re trying to spend more, prices will go up. You might be able to offset the price increase by optimizing your campaigns, landing pages, onboarding, etc, but don’t take it as a given.

The underlying problem is that the existing “hot demand” for your product – people who are actively looking for a solution – is usually quite limited. The good news is that the amount of “lukewarm demand” – companies that would benefit from your product but aren’t aware of it yet – is usually much larger. That’s why content marketing is so critical in SaaS: it allows you to capture leads at a much earlier stage of the discovery process. But scaling up your content marketing by 10x is not as straightforward as simply 10x-ing your ad budget.

So how do you know, in B2B SaaS, if you’ve found scalable acquisition channels?

Nothing is completely certain here, but one great sign that should give you a lot of confidence is if you can hire new salespeople and the new hires (once they’re ramped up) are hitting their quota. If you add two AEs, add another two, and then another two, and most of them are hitting quota it shows that you’re able to increase the amount of high-quality leads. If that wasn’t the case, your growing sales team would quickly start fighting for the best leads and some of your salespeople wouldn’t be able to hit their quota any longer. Equally important, it also shows that you’ve managed to industrialize the sales process to a certain extent. Firstly, it doesn’t take the founders or superstar salespeople to sell your product, it can be sold by “normal” people. And second, you’ve managed to attract the right people, to set up the right processes and infrastructure and to create the right incentive structure and culture that is required to make a sales team successful.

Besides a growing, successful sales team, there are a few other factors that you can look at when you’re trying to decide if it’s time to put the pedal to the metal:

1. Are you able to make outbound sales work?
Doing outbound at reasonable CACs is usually very hard because you’re dealing with lots of unqualified leads. It requires lots of persistence from every AE and your sales leader as well as a strong commitment from the founders, since a serious attempt to make outbound work can cost a lot of money and time. The beauty of outbound sales is that if it works for you, you may have found a highly scalable customer acquisition channel: emailing or calling every single target customer in the world will keep your sales team busy for a while. :)

2. Have you managed to increase your SEM budget consistently and significantly without negative effect on CACs? What is your impression share, and how large is the search volume that you can still tap into?
As mentioned above, past performance in scaling an SEM budget from A to B alone is not a reliable indicator of future performance to scale from B to C. But in combination with a thorough analysis of the relevant search volume it can be a relevant data point.

3. Have you built a content marketing “machine” that consistently generates more leads month-over-month? 
If you can consistently increase inbound/content leads for some time, it means that you’ve found your narrative, or “North Star”; started to build content distribution channels; and managed to attract the right marketing people and make them effective. (Check out this great post from my colleague Clément for much more about this.)

If there are other aspects that you’re looking at to decide if you’re ready to scale, I’d love to hear about them in the comments below!

Thank you Rodrigo and Janis for reviewing a draft of this post and the valuable feedback.

A/B testing is like sex at high school

A few days ago I went on record saying that A/B testing is like sex at high school. Everyone talks about it, not very many do it in earnest. I want to follow up on the topic with some additional thoughts (don't worry, I won't stretch the high school analogy any further).

When talking to people about A/B testing I've noticed that there are four (stereo) types of mindsets which prevent companies from successfully using split tests as a tool to improve their conversion funnel.

1) Procrastinative

The favorite answer to suggestions for website or product improvements from people from this camp is "we'll have to A/B test that" – as in "we should A/B test that, some time, when we've added A/B testing capability". It is often used as an excuse for brushing off ideas for improvement, and the fallacy here is that just because the best way to test assumptions is an A/B test doesn't mean that all assumptions are equally good or likely to be true.

Yes, A/B tests are the best way to test product improvements. But if you're not ready for A/B testing yet, that shouldn't stop you from improving your product based on your opinions and instincts.

2) Naive 

People from this group draw conclusions based on data which isn't conclusive. I've seen this several times: Results are not statistically significant, A and B didn't get the same type of traffic, A and B were tested sequentially as opposed to simultaneously, only a small part of the conversion funnel was taken into account – these and all kinds of other methodological errors can lead to erroneous conclusions.

Making decisions based on gut feelings as opposed to data isn't great, but in this case at least you know what you don't know. Making decisions based on wrong data – thinking that you understand something which you actually don't – is much worse.

3) Opinionated

There's a school of thought among designers which says that A/B testing lets you find local maxima only. While I completely agree with my friend Nikos Moraitakis that iterative improvement is no substitute for creativity, I don't see a reason why A/B testing can't be used to test radically different designs, too. 

Designers have to be opinionated. Chances are that out of the 1000s of ideas that you'd like to test, you can only test a handful because the number of statistically significant tests that you can run is limited by your visitor and signup volume. You need talented and convinced designers to tell you which five ideas out of the 1000s are worth a shot. But then do A/B test these five ideas.

4) Disillusioned

The more you learn about topics like A/B testing and marketing attribution analysis, the more you realize how complicated things are and how hard it is to get conclusive, actionable data. 

If you want to test different signup pages for a SaaS product, for example, it's not enough to look at the visitor-to-signup conversion rate. What matters is the entire funnel conversion rate, starting from visitors all through the way to paying customers. It's well possible that the signup page which performs best in terms of visitor-to-signup rate (maybe one which asks the user for minimal data input only) leads to a lower signup-to-paying conversion rate (because signups are less pre-qualified) and that another version of your signup page has a better overall visitor-to-paying conversion. To take that even further, it doesn't stop at the signup-to-paying conversion step as you'll want to track the churn rate of the "A" cohort vs. "B" cohort over time.

If you think about complexities like this, it's easy to give up and conclude that it's not worth the effort. I can relate to that because as mentioned above, nothing is worse than making decisions which you think are data-driven but which actually are not. Nonetheless I recommend that you do use split testing to test potential improvements of your conversion funnel – just know the limitations and be very diligent when you draw conclusions.

What do you think? Did you already fall prey to (or see other people fall prey to) one of the fallacies above? Let me know!

Learning More About That Other Half: The Case for Cohort Analysis and Multi-Touch Attribution Analysis (Part 2 of 2)

Note: This is the second part of a post which first appeared on KISSmetrics' blog. The first part is here, and here is the original guest post on the KISSmetrics blog. Thanks go to Bill Macaitis, CMO at Zendesk, for providing extremely valuable input on multi-attribution analysis.

Multi-touch Attribution Analysis – Giving Some Credit to the “Assist”

Multi-touch attribution, as defined in this good and detailed post, is “the process of understanding and assigning credit to marketing channels that eventually lead to conversions. An attribution model is a set of rules that determine how credit for conversions should be attributed to various touch points in conversion paths.”

It’s easier than it sounds, and, since this is the year of the World Cup, let me explain it using a soccer analogy. Multi-touch attribution gives the credit for a goal to not only the scorer but also gives some credit to the players who prepared the goal. Soccer player statistics often calculate scores based on the goals and the assists of the players. That means the statistics are based on what could be called a double-touch analysis that takes into account the last touch and the touch before the last one.

Since the default model in marketing still seems to be “last touch” only, it looks like soccer has overtaken marketing in terms of analytical sophistication. :-)

Time for Marketing to Strike Back!

If you are evaluating the performance of a marketing campaign solely based on the number of conversions, you are missing a large piece of the picture. Like a great midfielder who doesn’t score many goals himself but prepares goals for the strikers, a marketing channel might not be delivering many conversions but could be playing an important role in initiating the conversion process or assisting in the eventual conversion.

This is especially true for B2B SaaS where sales cycles are much longer than in, say, consumer e-commerce. When you’re selling a SaaS solution to a business customer, it’s not unusual for there to be several touch points before a company becomes a qualified lead, and then many more before the lead becomes a paying customer. The process could easily look like this:

• A piece of content that you produced comes up as an organic search result and the searcher clicks on it
• A few days later, the person who looked at the content piece sees a retargeting ad
• A few days later, she sees another retargeting ad, visits your website, and signs up for your newsletter
• A week after that, she clicks on a link in your newsletter
• A few days later, she receives an invitation to a webinar, signs up for it, and attends the webinar
• After the webinar, she signs up for a trial
• The next day, one of your customer advocates gives her a call
• Close to the end of her trial, your lead does some more research, happens to click on one of your AdWords ads, and signs up for a paid subscription

If you look at this conversion path, it becomes clear that if you attribute the customer only to the first touch point (SEO) or to the last one (PPC), you’ll draw incorrect conclusions. And keep in mind that the example above is still quite simple. In reality, the number of marketing channels and touch points that contribute to a conversion can be much higher.

Data Integration in a Multi-device World

Maybe you use Google Analytics or KISSmetrics for Web analytics, Salesforce.com for CRM, and Zendesk for customer service. If you want to get a (more or less) complete picture of your user’s journey, you need to get and integrate the data from all of the major tools you’re using and track user interactions.

A big complicating factor here is that we now live in a “multi-device world”. It’s very possible that the person in the example conversion path above used a tablet device, a smartphone, and two different computers to access your content and visit your website. Since tracking cookies are tied to one device, there’s no simple way to know that all of these touch points belong to the same person, at least not until the person registers.

Going deeper into the data integration and multi-device attribution problem would go beyond the scope of this post, but there’s a lot of valuable information available on the Web. And, please feel free to ask questions or share experiences in the comments section.

Toward a Better Attribution Model

The next question to tackle is how credit should be distributed to touch points in a conversion path. A simple approach is to use one of these rules:

• Linear attribution – Each interaction gets equal credit
• Time decay – More recent interactions get more credit than older ones
• Position based – For example, 40% credit goes to the first interaction, 40% to the last one, and 20% to the ones in the middle

While using one of these rules is a big improvement over a “first touch only” or “last touch only” model, the problem is that all of the rules are based on assumptions as opposed to real data. If you’re using “linear attribution,” you’re saying “I don’t know how much credit each touch point should get, so let’s give each one equal credit.” If you’re using “time decay” or “position based,” you’re making an assumption that some touch points are more valuable than others, but whether that assumption is true is not certain.

A more sophisticated approach is to use a tool like Convertro, which takes a look at all touch points of all users (including those who didn’t convert!) and then uses a statistical algorithm to distribute attribution credit. The advantage of this approach is that the model gets continuously adjusted based on new incoming data. Explaining exactly how it works, again, would go beyond the scope of this post, but there’s more information available on Convertro’s website, and I assume there are additional tools like this on the market.

Is It Worth It?

Implementing a sophisticated multi-touch attribution model is obviously a large project, and so the next question is whether it’s worth it. The answer depends mainly on these variables:

• Product complexity and sales cycle – The more complex your product and the longer the sales cycle, the more likely you are to have several touch points before a conversion happens
• Number of simultaneous campaigns and size of marketing budget – The more campaigns you’re running in parallel and the more you’re spending on marketing, the more important it is to account for multi-touch attribution

While cohort analysis is something you should do as soon as you launch your product, I think multi-touch attribution analysis can usually wait until you’re spending larger amounts of money on advertising. Until then, spending too much money or time getting your attribution model right probably is not the best use of your resources. So, as an early-stage SaaS startup, don’t worry too much about it just yet. Just remember to take your single-touch attribution CACs with a grain of salt.

The 6th DO for SaaS startups – Fill the funnel

Here's another post in my series on DOs and DON'Ts for early-stage SaaS startups:

6th DO for SaaS startups

Fill the funnel
Or: Focus on inbound marketing, but
 try lots of things and double-down on what works

In this post I'm going to write about lead generation for SaaS startups. When I edit the series later on I might merge it into my 4th DO (Make your website your best marketing person) to have one post on marketing. Let's see.

To make it clear right away, unfortunately I can't tell you what's going to work for you in terms of getting a large number of potential customers to your site. In fact, my key message is that there is no magic bullet when it comes to lead generation and that you'll have to try lots of things, put in lots of time and effort, double down on what works and execute extremely well. I haven't seen a SaaS company yet which gets more than 50% of its leads from one particular distribution partnership or marketing channel (except maybe word of mouth if you want to count that as a lead source).

Compared to that, marketing for consumer Web startups can be relatively straightforward. If you are a travel startup or an online shop, for example, millions of people search for your products or services online so you can use SEM, affiliate marketing, banner ads and other proven tactics to acquire large numbers of customers. In addition you can do TV advertising as your products are interesting for a relatively large percentage of the population. Getting the economics right and making it work at scale is of course a huge task and a science of its own, don't get me wrong on that.

But the particular challenge in SaaS marketing is that in many cases there isn't a huge amount of demand (a.k.a. search volume on Google), so the number of customers that you can acquire via AdWords is often quite limited. And things like TV advertising obviously don't work because of the huge waste circulation ("waste circulation" was the best translation I could find for the German word "Streuverlust" – does anyone know a better one?).

Just because you have a great solution doesn't mean that people are actively looking for it. That's not to say that you have a solution in search for a problem, but people may not be aware that there is a better way of doing things. What that means is that you need to find – and be found by – the people who your product is geared towards, often at a stage when they are loosely interested but are not yet ready to try (let alone buy) your product. Give them something that is useful to them. Write about the topics that your target group is interested in and provide lots of useful high-quality content and tips and tricks in a variety of formats, e.g. blog posts, white papers, case studies, videos, webinars, infographics or podcasts. Make sure that you don't talk too much about your product and that what you're publishing is really interesting to your target group. Sooner or later, some of these people will try and eventually buy. That's the whole idea of inbound marketing and lead nurturing. If you're not yet familiar with those concepts you should start learning more about them. A good starting point is Hubspot.

Zendesk is of course a great example for excellent inbound marketing. On its site the company provides a wealth of resources that are valuable for anyone who's interested in customer service, everything from tips for hiring customer service reps, to a guide to multi-channel customer support to numerous case studies and much more (including funny videos like this one). All of this helps to establish Zendesk as the go-to site for the help desk industry.

As for other ways to fill the funnel, here are some thoughts on things that you can do (in no particular order and of course by no means exhaustive):

• PR: Very important, and can get you off the ground in the beginning. Build relationships with the important bloggers, journalists and opinion leaders in your space and supply them with news. In the long term try to become an opinion leader yourself. Use Facebook, Twitter, Quora, conferences and events to reach out to the important people in your space.
• Most products are not inherently viral, but think about whether there are (sensible) ways to build virality into the product. If you can't find any you can still launch a referral program and reward users for recommendations to increase referral rates at least a little bit.
• Marketplaces, app stores, API partnerships, integrations, partnerships with hosters and the like: Don't expect huge volumes of leads from them, but they can be a meaningful lead source (and add value to your product).
• SEM & SEO: While you shouldn't bet on it alone, this is a very significant lead source for almost all SaaS startups that I know, so it's worth spending time and money on it.
• Ads on Facebook and LinkedIn: Personally I haven't seen great results with Facebook or LinkedIn ads for SaaS companies, but given the vast targeting options that you have there I think it's worth trying. If you've made it work I'd love to learn more.
• Display ads: Similar story, most of the time it doesn't work very well, but if there are suitable industry sites or blogs you may want to try it. 
• Retargeting: Can work very well. Obviously rather a nice supplement than a real needle-mover since the amount of visitors that you can target is limited by the amount of visitors who you've attracted in the first place.
• Promoted tweets: I don't have a lot of experience with advertising on Twitter, but I think it's worth a try, too.
• Distributors, VARs and similar channels: Tends to work better for traditional software with high license fees, setup and training requirements etc., but I've seen some good success in SaaS as well. Usually better for satisfying existing demand than for generating the demand in the first place, i.e. don't expect your channel partners to create the awareness for you.
• Local meetups: Once you have a number of customers in a region, organize local meetups. Nothing beats putting a bunch of happy customers and prospective customers into one room!
• Telesales/telemarketing: Hard to make it work, but if you can pull it off it can scale extremely well.

Finally...if you have trouble reaching your target group, try to put yourself in the shoes of the persons that you're trying to reach. Imagine how a typical day looks like for them. What websites do they visit, what might they be looking for on the Web? What magazines do they read, which industry associations might they be part of, what other products do they use, which people do they spend time with? Thinking about it this way will hopefully spark your creativity and let you come up with some fresh ideas.