Converio Infinity Logo

How can we help?

Tell us a little bit about your situation, so that we can do a little bit of homework before calling you right back.

We often do the following things, so if any of them particularly appeal to you then check the box and we’ll get something ready for you.

    We’re here:

    Somerset House,


    London WC2 1LA

    Statistical Significance, Simplified

    In the world of Conversion Rate Optimisation (CRO), the term ‘statistical significance’ is foundational. Yet, it’s a concept that often gets tangled in a web of complex explanations and statistical jargon, leaving many digital professionals scratching their heads. This post aims to demystify statistical significance, breaking it down into simpler terms and highlighting its crucial role in decision-making processes.

    The Essence of Optimisation

    At its core, CRO is about enhancing website experiences for customers through methodical testing. The goal is straightforward: if one version of a website experience proves superior, it should become the default customer experience. However, determining which version is truly superior introduces the need for a more scientific approach, where statistical significance takes the spotlight.

    What is Statistical Significance?

    A Google search on statistical significance yields the following definition:

    “Statistical significance refers to the claim that a result from data generated by testing or experimentation is likely to be attributable to a specific cause. A high degree of statistical significance indicates that an observed relationship is unlikely to be due to chance.”

    While accurate, this definition can seem daunting with its technicality. Yet, the concept can be simplified:

    “Statistical significance is a tool to support decision making, by giving an understanding of the risk of implementing a particular change.”

    The Holy Grail of CRO: 95% Statistical Significance

    In the realm of CRO, achieving a 95% statistical significance is often considered the gold standard. This means there’s only a 5% likelihood that the observed results occurred by chance. In practical terms, this level of confidence suggests that the outcomes we see in testing would likely persist if implemented across the website.

    Factors to Consider

    • Sample Size: The amount of data you need to ensure your results are reliable. A larger sample size can provide more confidence in the findings.
    • Test Duration: It’s essential to run tests for a sufficient period, ideally spanning a few business cycles, to account for any fluctuations in customer behaviour.

    Once a satisfactory level of statistical significance is achieved, the decision to implement changes, iterate further, or abandon the initiative rests on several considerations:

    1. Potential Impact: Evaluating the observed uplift and how it could improve customer experience or business metrics.
    2. Implementation Feasibility: Considering the cost and time required to make the changes permanent.
    3. Data for Further Optimisation: Deciding if there’s enough evidence to suggest modifications could yield even better results.

    Why Statistical Significance Matters

    Employing statistical significance effectively in CRO empowers businesses to make more informed decisions. It offers a buffer against the randomness of chance, providing a layer of confidence when deciding to roll out a new website feature or tweak an existing one. In essence, it helps businesses mitigate risk, ensuring that changes are genuinely beneficial to both the customer and the company.


    Statistical significance is more than a set of complex calculations—it’s a critical tool in the optimisation toolkit. By understanding and applying this concept, optimisation specialists can lead their businesses to make data-driven decisions that enhance customer experiences and, ultimately, drive success. While the science behind statistical significance can seem intricate, its application doesn’t have to be. Simplifying this concept down to its essence allows us to harness its power more effectively, making our digital environments better for users everywhere.

    In a landscape where data is king, understanding statistical significance is akin to holding the keys to the kingdom. It’s what allows us to discern between a fleeting anomaly and a genuine improvement, guiding our optimisation efforts towards meaningful, impactful changes.

    Re-defining Success

    When it comes to experimentation programs, defining and measuring success is critical to demonstrate return on investment and engage business stakeholders. Success metrics allow us to showcase value and monitor the performance of our experimentation program and should effectively empower us to make informed decisions to further strengthen our optimisation efforts. However, the win rate KPI often lies at the heart of this assessment and can sometimes be misinterpreted as the sole indicator of program success.

    The joy of victory…

    It makes sense, right? Winning is deeply embedded in human nature—we are programmed to associate success with winning. Winning provides a sense of accomplishment, boosting confidence and self-esteem. From childhood through to adulthood, we are driven to celebrate our wins.

    And we see it within our industry; case studies, sales calls, and conference talks filled with examples of ‘winning’ test results to showcase a successful approach to experimentation. Who among us hasn’t felt the satisfaction when your testing variant achieves a ‘winning’ result? We’ve all been there… we can thank the dopamine hit, as ‘winning’ activates our brain’s reward system and brings us joy.

    …But at What Risk?

    While our ambition may be to win, an exclusive focus on win rates can undermine the effectiveness of your experimentation program. This isn’t about spinning the cliché that ‘every test is a win’—it’s about recognising what a testing program can bring to your business beyond just a winning variant.

    Testing Innovative Ideas

    Testing bold ideas may come with risks, but it can lead to significant breakthroughs and substantial rewards. Innovation fosters creativity, resulting in a more dynamic and inventive work environment that has a ripple effect throughout an organisation. Addressing user challenges with a creative thinking approach sets your brand and website apart in a world where differentiation is invaluable.

    The Value of Truth

    Testing enables businesses to make rapid, evidence-based decisions, steering clear of investments in areas that don’t resonate with users or meet their needs. This agility allows teams to pivot to more impactful strategies quickly. Celebrating and valuing the rapid acquisition of actionable insights ensure that data-driven next steps are prioritised. Maximising efficiency requires avoiding the temptation to only test the obvious and investing the appropriate level of effort to validate strategies. By focusing on uncovering the truth, testing empowers stakeholders with the confidence to make informed decisions, driving investment into areas that positively impact both customers and the bottom line.

    The Long-term Goals

    Exploring a diverse set of testing strategies provides a broad landscape in which to learn and react. This, alongside understanding the context behind ‘losing’ tests, drives a more comprehensive understanding of user behaviour, preferences, and pain points that may be difficult to get to within your existing data. This context fuels more effective strategies in the long run, driving tighter hypotheses and a robust testing approach, setting you up for success.

    A Culture for Change

    Focusing solely on win rates can undermine the true essence of a testing culture, which should prioritise curiosity, continuous improvement, robust methodology, and learning. A balanced view of success is crucial for maintaining team morale and avoiding frustration. Emphasising fast learning, agility, and data driven hypotheses ensures that every effort is valuable and outcomes are viewed as opportunities for growth rather than sources of disappointment.


    Effective experimentation opens up opportunities beyond a winning test result and how we assess and communicate program success will influence program direction, quality and perception within your business.

    While win rate is a valuable metric, it should be considered within a broader context. A balanced approach showcases experimentation as a tool for making data-driven decisions, fostering a culture of innovation and continuous improvement, prioritising robust testing processes to successfully optimise the user experience.

    By looking beyond win rates, you can elevate your testing program, empower your teams, and embed experimentation into the core of your business culture, ensuring it is seen as a strategic, integral part of your organisation rather than a standalone function.

    The Power of Social Proof

    Social proof, in the form of testimonials and reviews, holds immense power in influencing customer behaviour and driving conversions. 95% of users read reviews online before making a purchase, with a further 58% confirming they would pay more for a product with positive reviews.

    In an age where consumers are inundated with choices and marketing messages, the authenticity and reliability of social proof can make or break a purchase decision.

    A study by Spiegel Research Center found that showing 5 product reviews can increase conversion rates by 190% for lower-priced products and 380% for higher-priced items.

    The Importance of Being Genuine

    Arguably, one of the most important aspects of leveraging social proof is maintaining authenticity.

    Potential customers are more likely to trust and engage with feedback that feels real and unbiased. This means using actual data gathered from genuine customer experiences, rather than artificially inflating ratings or cherry-picking only positive feedback.

    Displaying negative reviews alongside positive ones can bolster credibility. When both positive and negative reviews are displayed, 68% of users are more likely to find them trustworthy. Showing that a company does not hide or delete negative feedback, but rather addresses it constructively, not only builds trust, but helps customers make well-informed decisions.

    Encouraging customers to share their own photos in reviews also adds a layer of authenticity, making it easier to visualise how products will look and perform in real-life scenarios.

    The Role of A/B Testing

    Implementing social proof should not be a set-it-and-forget-it strategy. A/B testing is essential to understand how various elements of social proof impact customer behaviour.

    By comparing different versions of social proof—whether it be the placement of testimonials, the wording of messages, or the types of metrics highlighted—you can uncover valuable insights into what resonates best with their audience.

    Another important consideration is customising social proof for different markets. Using A/B testing, you can establish the optimum messaging and metrics that resonate best with users, and determine where on your site social proof has the highest impact.

    It also allows for optimisation of exposure levels, meaning businesses can determine whether social proof should be displayed on a select range of products or more broadly across an entire catalogue.

    Exposure Levels and Repetition Impact

    In addition to how frequently social proof is shown, it’s important to consider the impact of repeated exposure.

    Research suggests that there can be diminishing returns with repeated exposure.

    Therefore, it might be prudent to limit how often the same testimonial or review is displayed to a returning visitor, ensuring that the social proof remains impactful without becoming redundant.

    Leveraging Different Types of Social Proof

    Beyond customer reviews and testimonials, there are various other forms of social proof that can be leveraged.

    These include expert endorsements, celebrity influencers, media mentions, and even user-generated content such as social media posts.

    Businesses can also utilise metrics such as the number of products sold, the number of users registered, or high-profile clients to create a bandwagon effect.

    By diversifying the types of social proof employed, companies can appeal to a broader audience.

    The Importance of Context

    Context matters when it comes to displaying social proof.

    Testimonials and reviews should be relevant to the product or service being considered and should address the concerns or desires specific to that context.

    For instance, a testimonial about customer service might be more relevant on a product return page than on the product detail page itself.

    Moreover, dynamic social proof that updates in real-time—such as recent purchases or live customer activity—can create a sense of urgency and prompt immediate action.

    By carefully curating and continually testing the deployment of testimonials and reviews, businesses can harness the power of social proof to build trust, enhance credibility, and ultimately drive conversions.

    The authenticity of the feedback, coupled with strategic A/B testing and mindful exposure levels, ensures that social proof remains a potent tool in the marketing arsenal.

    Embracing the psychology of social validation and contextually relevant endorsements can further amplify its impact, turning prospective buyers into loyal customers.

    The Human Touch in E-commerce

    In the swiftly evolving world of e-commerce, artificial intelligence (AI) has emerged as a pivotal game-changer, revolutionising the customer experience with unprecedented personalisation, predictive analytics, and automation. The allure of AI in streamlining operations, understanding customer behaviour, and tailoring experiences is undeniable. Yet, amidst this digital transformation, the essence of human connection has become more valuable than ever.

    E-commerce entrepreneurs are at a crossroads, seeking the perfect equilibrium between leveraging cutting-edge AI capabilities and maintaining genuine customer interaction. This delicate balance is not just a preference but a necessity in distinguishing brands in a saturated market.

    The Potential Issues of Over-reliance on AI

    While AI’s contributions to e-commerce are vast and varied, its limitations highlight the indispensable value of human touch.

    Loss of Authenticity

    AI-driven responses, though efficient, often lack the warmth and personal touch that human interactions inherently possess. In an era where brand loyalty is closely tied to customer experience, the absence of authenticity could lead to a disconnect with customers. According to data[1], “Nearly half of customers, including three-fifths of millennials, are willing to pay extra for better customer service, underscoring the importance of customer experience”

    Customer Frustration

    Despite the sophistication of AI technologies, they can sometimes fall short in understanding the nuances of customer queries, leading to irrelevant recommendations or solutions. This can culminate in customer frustration and a tarnished brand image. For simpler tasks, most customers are happy to use tools such as chatbots. However, they do need careful implementation, as “over two-thirds of customers won’t use a company’s chatbot again after just one negative experience.” [2]

    Privacy Concerns

    The foundation of AI-driven personalisation is the collection and analysis of vast amounts of customer data. This raises significant privacy concerns, making customers hesitant to share their information. “For instance, a mere 37% of customers trust AI’s outputs to be as accurate as those of an employee. Accordingly, 81% want a human to be in the loop, reviewing and validating those outputs.”[3]

    Striking the Right Balance

    Achieving the right mix of AI efficiency and human empathy is the linchpin for e-commerce success. Here are strategies to ensure a harmonious blend:

    Personalise with Purpose

    Use AI to gather insights and tailor experiences but infuse these interactions with elements of human insight. Personalised emails, recommendations, and services should feel curated by a human touch, offering relevance and warmth that AI alone cannot mimic.

    Implement AI with Empathy

    Design AI systems with a focus on empathy, ensuring that automated responses and interactions are as thoughtful and considerate as possible. This includes programming AI to recognise when a customer’s queries surpass its capabilities and seamlessly transition them to a human representative.

    Prioritise Customer Privacy and Transparency

    Build trust by being transparent about how customer data is used to enhance their experience. Implement stringent data protection measures and give customers control over their information, reassuring them of their privacy.

    Foster Human Connections

    Encourage and facilitate direct interactions between customers and your team. Whether through personalised customer service, live chats, or community engagement initiatives, make sure there’s always a pathway for customers to connect with a human on the other side.

    In the dynamic landscape of e-commerce, AI offers unparalleled opportunities for innovation and efficiency. However, the heart of customer experience lies in authentic, empathetic interactions. By thoughtfully integrating AI with a human touch, e-commerce entrepreneurs can create meaningful connections, build trust, and ultimately, cultivate brand loyalty. The future of e-commerce isn’t just about technological advancement; it’s about how well we can balance these innovations with the irreplaceable value of human connection.

    [1][2][3]Salesforce State of the Connected Customer, 2023

    Unlocking Site Potential

    In today’s digital landscape, optimising your website’s performance is more crucial than ever. One of the most effective methods to achieve this is through A/B testing. A/B testing, also known as split testing, allows you to compare two versions of a webpage to determine which one performs better. Whether you’re looking to increase conversions, improve user experience, or boost engagement, A/B testing is a powerful tool to unlock your site’s potential. Here are easy steps to set up and run A/B tests.

    Step 1: Define Your Goals

    Before diving into A/B testing, it’s essential to define clear, measurable goals. These goals could range from increasing the click-through rate on a call-to-action button, reducing bounce rates, or improving the overall user experience. Having a specific objective will guide your test design and help you measure success accurately.

    Example Goals:

    Increase newsletter sign-ups by 20%
    Reduce cart abandonment by 15%
    Improve time spent on a specific landing page by 25%

    Step 2: Identify the Elements to Test

    Next, decide which elements of your webpage you want to test. These elements can include headlines, images, buttons, forms, and overall layout. Focus on areas that have the most significant impact on user behaviour and align with your goals.

    Common Elements to Test:

    • Headlines and subheadings
    • Call-to-action buttons (text, color, size, placement)
    • Images and videos
    • Product descriptions
    • Form fields and layout

    Step 3: Choose an A/B Testing Platform

    To simplify the setup and execution of A/B tests, using dedicated A/B testing platforms is highly recommended. Popular platforms like Optimizely, VWO, Dynamic Yield, AB Tasty, and Adobe Target provide user-friendly interfaces and powerful features to help you run tests efficiently.

    These platforms typically offer:

    • Easy integration with your website through a simple tag or plugin
    • Intuitive visual editors to create and modify variations without coding
    • Automated traffic distribution to ensure unbiased test results
    • Real-time analytics and reporting dashboards to track performance

    Setting up these platforms is straightforward. Generally, it involves:

    1. Signing Up: Create an account on your chosen platform.
    2. Integrating the Platform: Add a small snippet of code or use a plugin to integrate the platform with your website.
    3. Creating Your First Test: Use the visual editor to select elements and create variations.
    4. Launching the Test: Define your audience and start the test.

    Step 4: Create Variations

    Once you’ve identified the elements to test and chosen your platform, create different versions (variations) of these elements. For instance, if you’re testing a headline, create multiple versions with different wording or styles. Ensure that your variations are significantly different to detect meaningful changes in user behaviour.

    Example Variations:

    • Headline A: “Unlock Your Potential with Our Services”
    • Headline B: “Transform Your Business Today”

    Step 5: Set Up the A/B Test

    There are two primary ways to run A/B tests:

    • Modifying the Main Code on the Site: This method involves directly integrating variations into the existing codebase of your web application. It often requires collaboration with your development team to deploy different versions of elements or pages within your platform. This approach provides more control over customisation and performance but may require more development resources and time.
    • Using JavaScript and CSS to Modify the Existing Site: This approach involves adding a single tag to your site that loads external JavaScript and CSS to dynamically alter the appearance and behaviour of elements on the page. This method is often quicker to implement and allows for easier adjustments without altering the core site code.

    With your variations ready, set up the A/B test in your chosen platform. Ensure that your traffic is evenly split between the different versions to get unbiased results. This step involves:

    • Configuring the test settings, such as audience targeting and test duration
    • Defining goals and metrics to track
    • Previewing and launching the test to start collecting data

    Step 6: Run the Test

    With your test set up, it’s time to run it. The duration of the test will depend on your traffic volume and the statistical significance required to draw reliable conclusions. Make sure to monitor the test to ensure everything runs smoothly.

    Step 7: Analyse the Results

    After the test has run for a sufficient period, analyse the results to determine which variation performed better. Look at key metrics such as conversion rates, click-through rates, and other relevant data points. Many A/B testing tools provide detailed reports and insights to help you interpret the results. For more in-depth analysis:

    • Export Data: Export the test data to a spreadsheet or a data analysis tool like Google Analytics.
    • Segment Analysis: Analyse performance across different audience segments to understand how different user groups respond to variations.
    • Custom Events: Track custom events (e.g., form submissions, video plays) to get a deeper understanding of user interactions.

    Step 8: Implement the Winning Variation

    Once you’ve identified the winning variation, implement it on your site. Monitor the performance to ensure that the changes lead to sustained improvements. Continuously testing and iterating based on user feedback and data will help you refine your site further.

    Step 9: Iterate and Optimise

    A/B testing is an ongoing process. Continually test new ideas and variations to keep optimising your site’s performance. What works today might not work tomorrow, so maintaining a culture of continuous improvement is key to long-term success.


    A/B testing is a powerful strategy to enhance your website’s effectiveness and achieve your business goals. By defining clear objectives, identifying critical elements to test, using the right tools, and analysing the results meticulously, you can unlock your site’s full potential. Start small, test often, and iterate based on data-driven insights to create a superior user experience and drive better results.

    By following these easy steps, you can set up and run A/B tests that provide valuable insights and drive meaningful improvements to your website. Happy testing!

    Driving Digital Success

    In the fast-paced world of e-commerce, driving continuous growth and staying ahead of the competition is paramount. One of the most effective strategies to achieve this is through Conversion Rate Optimisation (CRO). While many businesses might think they can handle CRO internally, the truth is that independent CRO experimentation labs offer unique advantages that can significantly enhance digital success.

    In this blog post, we’ll explore why partnering with an independent CRO agency like Conversio can be a game-changer for your e-commerce business.


    Obviously, as a co-founder of an independent CRO agency, I am going to promote the benefits of using an expert partner like Conversio, but I thought like this even before starting this business. With minimal agency experience, apart from a brief stint at a large international full-service agency, I’ve spent most of my career on the client side at companies like and I’ve seen firsthand how challenging it can be to implement change, especially in CRO. Trying to pull resources from unconnected teams whose key objectives are not growth or CRO is like walking through treacle. This is why I firmly believe that having an external CRO partner is often the best way to drive growth through experimentation. Here are three key reasons why.

    BAU Blockers

    One of the biggest challenges in implementing a successful CRO program internally is the constant interference of Business-As-Usual (BAU) activities. Hiring a CRO manager and expecting them to drive growth with existing resources is an uphill battle. While BAU activities are essential, they often take priority over A/B tests and other CRO initiatives. This prioritisation can stifle innovation and limit the potential for making data-driven decisions based on customer behaviour.

    When you use an external CRO partner, these BAU blockers are significantly reduced. An external CRO team can focus solely on optimisation without being encumbered by trade, merchandising, release schedules, and other internal distractions. This allows your CRO manager to be an internal advocate for optimisation while the external team drives the program forward without interruptions.

    Cost Efficiency

    Supporting an internal CRO manager with a fully resourced independent team can be costly. Depending on the size of your business, you’ll ideally need a UX designer/researcher, an analyst, and a front-end developer. These resources can add up quickly, making it more cost-effective to use an external partner.

    An independent CRO agency like Conversio can provide a dedicated team of experts at a fraction of the cost of building an internal team. This not only saves you money but also ensures that you have access to top-notch talent with specialised skills in CRO.


    By partnering with an external CRO agency, you can hold them accountable for driving a successful CRO program. Agencies like Conversio are always on the cutting edge of analytics developments, optimisation technology, and changes in customer behaviour patterns. This commitment to staying on the front foot ensures that your CRO program is always evolving and adapting to new trends and technologies.

    External agencies feel the pressure to deliver results and maintain a competitive edge, which translates to better performance and outcomes for your business. This level of accountability is often difficult to achieve with an internal team that may be juggling multiple priorities.


    Using an expert external partner means leveraging CRO expertise that spans a wide range of businesses and sectors. This breadth of experience is invaluable, as it brings fresh perspectives and innovative solutions to your CRO program. An independent CRO team has seen what works and what doesn’t across different industries, allowing them to apply best practices and avoid common pitfalls.

    At Conversio, 90% of our team comes from a client-side background. This unique blend of client-side experience and agency expertise means we understand the challenges you face and have the skills to drive meaningful change. Our team’s diverse experience across multiple brands provides significant advantages over a siloed internal team working on a single set of customers.