Ton made a great comparison with Johan Cruijff. Cruijff was a master in innovating and improving his performance. Shouldn't every optimizer use a little Cruijff-attitude in his job?

• You are only able to see it, until you get it

• A/ b testing is easy to start, but so difficult to understand

• Regret: the money you are losing per year, not using our testcapacity

• Risk: you have to take risks, you do not have the data to start testing
• Optimization: you have a 1000 conversions a month, start testing
• Automation: you now have a lot of conversions, do some automation stuff like push notifications
• Rethink: Every good think ends, including your product

• With testing more then 3 weeks, there is a big chance you sample will be polluted

• Know you % winners bottom line
• Learn you false positieven, 10% of your test is a false positive

• T-testing is about significance, you can miss a winner because of a false positieve. Bayesian testing then is a better way of finding stuff that works better

• Celebrate a loser, you took a risk, how nice of you!