Deborah O’Malley is the Founder and Head of Product Strategy at GuessTheTest, a platform that offers gamified A/B test case studies to help experimenters get testing ideas, insights, and learn from real results. She is recognized as a leading Conversion Rate Optimization (CRO) specialist and top A/B testing influencer. With a master’s degree in eye-tracking technology, she blends research and practice in her work. Deborah also leads Convert Experts, where she runs optimization projects and helps clients improve their digital performance.
About the Episode
The world of A/B testing is changing faster than ever as artificial intelligence reshapes how marketers experiment and optimize. Traditional manual methods are being challenged by automation, new tools, and shifting consumer behaviors. Tyler Lane hosts Deborah O’Malley as they explore how to embrace AI while strengthening fundamental testing skills, why human intuition and strategic oversight remain critical, and how marketers can stay relevant and drive better results as the experimentation landscape transforms.
Key Moments
Racing to Keep Up With AI
- Why experimenters must keep up or risk being left behind in the rapidly changing AI landscape [02:01].
Gamified Learning at GuessTheTest
- What the platform offers as a fun, interactive way to learn from real A/B test case studies [04:36].
From Manual Testing to Managing AI Agents
- How experimenters can evolve their skills to manage AI agents and use human intuition strategically [12:11].
Cutting Through the Noise
- Why following thought leaders and key newsletters helps marketers stay up to date without overwhelm [17:22].
Teaching AI and Data Privacy
- How Deborah is updating her Queen’s University curriculum to reflect AI’s impact on marketing analytics and privacy [21:36].
Challenging CRO “Best Practices”
- Why best practices in CRO should be questioned and why the human element in experimentation remains essential [35:18].
Top Insights
- Embrace AI tools for experimentation – Leveraging AI-driven platforms streamlines A/B testing and uncovers insights faster than manual methods.
- Strengthen foundational CRO skills – Building expertise in hypothesis development and data analysis prepares you to oversee and guide AI outputs.
- Follow trusted thought leaders and newsletters – Staying updated through curated insights helps you cut through noise and avoid information overload.
- Prioritize data privacy and ethical considerations – Being mindful of how personal information is stored and used protects both your organization and users.
- Blend intuition with data-driven decisions – Combining human empathy with analytics ensures testing strategies remain relevant and impactful as technology evolves.
Notable Quotes
- “It’s keep up or die in this market because the world is going towards a web-based search, browsing web experience.”
- “Experimentation itself is not going away, and I don’t think A/B testing will go away either.”
- “The role of an experimenter is going to switch from manually doing it ourselves to actually being the manager of multiple AI agents.”
- “ChatGPT is my counselor, my astrologer, my content creator, my best friend, really with no judgment.”
- “We need to start first by having these conversations in places like classrooms and on podcasts and being aware.”
Resources
Industry Experts
- Tyler Lane on LinkedIn
- Deborah O’Malley on LinkedIn
- Shawn David on LinkedIn
- Craig Sullivan on LinkedIn
- Jeff Bullas on LinkedIn
Companies and Organizations
Tools and Platforms
Build Smarter Experiments With AI and Human Insight
AI is rewriting the rules of experimentation, but success depends on combining automation with human intuition. At Session Interactive, we help brands leverage cutting-edge analytics and conversion strategies to stay ahead of rapid change.
Ready to future-proof your marketing and testing? Contact Session Interactive to learn how we can help you scale with precision and confidence.
