CRO is not new.The Scientific method and the mAyBe testing


Scan this post:


In one of my CRO's talks i always start with this slide:

Listing some of the acronyms used in our every day work is a trick/joke i use to swap the auddience'a attitude from "just another 3 letters acronym for my twitter bio", to a positive attitude.

And it works, just as in CRO when you use visitor's FUD's in your benefit.

The funniest thing here is that from all the stuff listed probably CRO is the oldest discipline we can find, as CRO is just the "online" form we use for applying the "Scientific Method" (which happens to be as old as Aristotle).

CRO is not about testing.

What? it's not about testing? No. Really. Testing is not the goal, if you are doing things right.

Testing is just a stage of the optimization process. To be more accurate, the stage between the hypothesis definition and the results analysis.

Testing is the experimentation stage in the Scientific Method.

The Scientific Method is formulated in dozens of forms and shapes, but the most used are the 11 and 14 steps. Summarized, they are all like this:


Let's put it this way: if there was a way to get our hypothesis falsificated (like some kind of all-knowing Oracle) testing wouldn't be needed and we just wouldn't test.

We'd just ask the Oracle what to change... and apply it to improve conversions. We test cause is the way to experiment if our hypothesis is true or false (this is: falsation of the hypothesis).

mice the ultimate oracle (hitchhikers guide to galaxy)


So, just to let this clear:

Testing is not the goal of CRO. It's just a needed step.

We are not paid to create tests, but to improve conversions, and we get there through several steps which include A/B testing. That's it.

What's your method?

Everyone has it's own method. Some begin with a survey, some with a Google Analytics report, some with a series of interviews with stakeholders... It may not seem the same, but at the end, if we are doing things properly, we just:

  1. Find out wether we have or not a problem on a page.
  2. If so, try to locate it and study it with data. Multiple data is welcomed.
  3. Once we know the problem, we form some hypothesis about it's cause and how to solve it: "By doing this..."
  4. We test the hypothesis by designing tests (experiments) that can falsate the hypothesis!
  5. Evaluate results and re-test or implement some kind of change that solves the problem.
  6. And start again

Well... that's just a simplification. The mix and use of tools, strategies, tactics, reports, metrics, to solve every stage is owned by every CRO as it's personal treasure.

The point here is: If you are doing this, you are doing the Scientific Method.

The Scientific Method is an empiric and mesurable methodology to adquire knowledge about a materia. In our case, what makes users convert more.

What are the benefits of sticking to the Scientific Method and not testing guided by Gurus, instinct or just experience?

You'll build over a solid and lasting basement. Testing without method can improve conversions, but understanding why, and the drivers that move the action of your visitors will make you reach the Conversion Nirvana, as you'll KNOW what to do in the future and apply it to every step of your sales cycle.

mAyBe testing

The guy who simply tests based on instinc, won't see improvements further than that page. He may find improvements someday, but won't be able to repeat it. They just do "mAyBe testing":

"mAyBe testing" is what some optimizers do, mostly on starting their careers. They don't care about why they are improving conversions and just launch random test: "this box here, this button there, i liked this competitor's call to action, let's see what happens if i put a different hero shot" based on god knows what. This is not the way the scientific method shows.

Here you can see an example of mAyBe testing, at the point 4. People who don't want to know the causation of the improvements they do.

So... read about it and get good on it. The better you get on Scientific Method, the less mistakes you'll make, the best your decisions will take, and better the CRO specialist you'll get.