Category: Strategy

Why Peak Season Testing Sparks Debate

Black Friday and other peak trading periods will always trigger one simple question. Should we be experimenting during peak when the “risks” are so high?

The main argument, and an understandable one is that a failed test could cost thousands in lost revenue. Others say it’s the best time to experiment, with bigger traffic volumes and the only chance for some ever 12 months to better understand this traffic set. The truth of course lies somewhere in the middle.

In this article, I’ve taken a look at the pros and cons of A/B testing during Black Friday, and share my thoughts on the approach for these key periods.

We Shouldn’t Test, it’s too risky!

Let’s start with the reasons I believe teams will ask this question, and why the decision can be to stop testing all together:

Higher stakes, bigger impact

A losing test in July might cost you a few thousand in missed sales. The same test in November could cost ten times as much. The financial impact of a “losing” experiment is much higher.

Noise and data volatility

Your peak period will differ from standard shopper behaviour as it brings in: gift buyers, more bargain hunters and the one-time seasonal shoppers. This will likely distort what we see as normal can make next step decisions more difficult, as can you guarantee these results outside of peak?

Short windows

For most Black Friday campaigns they only run for a few days, sometimes as things have shifted a couple of weeks. However, that may not be enough time for every test to reach statistical significance, especially for lower-traffic pages.

Teams are too busy 

Having worked in a toy retailer during peak, I can relate to this one. When there are 101 jobs to do every day to implement promotions, price changes, marketing campaigns and the rest. Experimentation will often fall to the bottom of the pile.

But we could reap huge rewards if we win

Now the flip side, I personally believe it’s a mistake to fully stop testing during your peak and here’s why:

Bigger upside

A winning test during peak has the potential to generate much larger incremental revenues. If you also can roll this change to 100% quickly, you can reap the rewards immediately with the large traffic pools. The positives could outweigh the negatives here.

Everyday changes are untested anyway

During peak, hundreds of untested decisions are made across our sites from promotions, homepage banners, email campaigns, product sorting rules. These all carry risk too, each one of these has the potential to decrease conversion or revenue the same way testing can.

Faster learning cycles

The surge in traffic means tests reach significance quicker. What might take weeks in March could take just days in November. 

My View: The Risks haven’t changed Only the Impact has

• The risks of A/B testing are present all year-round.
• A poor variant can underperform any month of the year.
• Traffic fluctuations can always distort results.
• Tests always require time, discipline, and monitoring.

What changes in Black Friday isn’t the type of risk, but the impact. A loss will cost more. But a win equally will deliver more too.

Best Practices for A/B Testing During Peak Season

If you decide to test during peak (and you should), approach it differently:

Prioritise ruthlessly

Focus only on tests with high potential impact or testing to understand the peak shopper better: recommendation algorithms, promotional content, delivery messaging, social proofing, etc

Monitor performance more regularly and set guardrails

Avoid “peaking” but keep your eyes open. If a test is underperforming and it’s slowly making its way to a level of loss you’re not comfortable with, pause it. You can always come back to it in the new year. 

Shorten decision cycles

Be ready to pause or roll out variants faster than you would at other times of year. Peak windows are short, don’t forget Experimentation in ecommerce is to support decision making, we’re not testing the effectiveness of medicines.

Run “safer” tests

Stick to experiences unlikely to break core functionality and lower risk from a development perspective. This isn’t the time for risky experiments and 100’s of lines of code that could break with a simple merchandising change.

Plan ahead

Have experiments built, UAT’d, and ready to launch ahead of your peak season.

Conclusion: Should You Test During Black Friday?

Short answer: yes, I believe you should, if you can prioritise well, monitor closely and be prepared to pause losses or launch winners quickly.

However, this is only a question that truly each business can answer on its own, based on workload, team capacity and ability to turn changes round quickly. Hopefully, this article has given you some thoughts on how it could be approached.

Black Friday is when every conversion counts most. And that’s exactly when your testing plan should be at its sharpest.

Convinced? Us too – book some time with our expert team to discuss your strategy for Black Friday or peak trading times in general.

Whilst the main focus of CRO (Conversion Rate Optimisation) is on increasing purchases, paying attention to the small steps that lead to order completion can have a bigger impact than you might think.

From revealing purchase intent, to highlighting friction points along the customer journey, tracking micro conversions allows you to see which customers are most likely to convert, and helps you to see where and why customers are dropping off.

What Are Micro-Conversions in eCommerce?

Macro conversions are the actions that directly drive revenue, the primary goal of your eCommerce site, such as purchase completion, or subscribing to a paid membership or subscription.

In comparison, micro-conversions support the customer towards macro-conversions:

• Adding a product to basket
• Interacting with the product gallery
• Using filters and/or search
• Beginning checkout

These conversions signal intent to buy, and can be analysed to optimise the journey to conversion.

How Micro-Conversions Drive Growth in eCommerce

Depending on the type of products you sell, customers can spend considerable amounts researching products, considering different options. Therefore micro-conversions tend to happen more often than macro-conversions.This can make macro-conversion rates slow to change, as customers don’t complete them as often. 

Micro-conversions fuel eCommerce growth by highlighting customer intent and friction points. For example, if products get high views, but low add-to-bag rates, it might be an indication that something’s wrong, be it price, trust signals or product information presentation. Optimising these steps improves the efficiency of the funnel, which in turn increases the likelihood of users converting.

Over time, small improvements in the early stages of the customer journey can accumulate into significant gains in overall sales performance.

5 Tips for Optimising for Micro-Conversions

1. A good starting point is to optimise navigation and filtering to help product discovery. Highlight recommendations, build intuitive filters, and a strong site search.
2. Add to bag rate is a crucial KPI, and optimising the product page experience can be pivotal in improving this metric. Making product information relevant and helpful, including clear and varied product images, and building trust through customer reviews are strong areas to improve PDPs.
3. Encourage users to take action by placing key CTAs above the fold, call attention to them, and make sure that users understand the action they will complete.
4. Personalisations can be a great way to add value to the user’s experience, by creating a more custom experience, based on their behaviour. Personalised recommendations, device specific layouts and flows, and serving different content for users who display exit intent, or returning customers are great ways to build a more bespoke journey.
5. Small actions can lead to great insights and opportunities, and whilst order completion might be your ecommerce site’s primary goal, it’s crucial to support your customers’ journey every step of the way.

Take a look at your site’s micro-conversions, identify points of friction and start testing improvements.

Better still, book some time with our expert team for a consultation around how we can support driving results for your experimentation programme as an extension of your team. 

An effective Conversion Rate Optimisation (CRO) strategy goes beyond tracking metrics, it’s about understanding the customer’s journey. While platforms like GA4 and Adobe Analytics remain critical for measuring performance, behavioural analytics platforms such as Contentsquare and Fullstory add a deeper layer of insight. These platforms bring customer context to the numbers, revealing why customers behave the way they do and enabling more targeted, impactful optimisation.

Core Capabilities of Behavioural Analytics Platforms

Behavioural analytics platforms offer a rich, visual lens into customer behaviour, helping you understand how people engage with your site, not just whether they convert. Through tools like heatmaps, session recordings, and customer journey mapping, you can quickly identify which areas of a page attract attention, where customers encounter friction, and how they move through the site.

• Heatmaps show where customers click, scroll, or hover, revealing which parts of a page draw attention and which are being overlooked.
• Journey mapping illustrates how customers navigate end-to-end, helping uncover drop-offs and opportunities to streamline the journey.
• Session recordings provide real-time playback of customer sessions, highlighting where customers hesitate, backtrack, or abandon tasks. re:member used Contentsquare session recordings to discover that customers were scrolling up and down instead of completing a credit card application form. Adding more benefit details to this led to a 17% increase in form conversions.

These insights create a baseline for identifying what’s working, and what’s getting in the way.

From Insight to Experiment: Using Behavioural Data to Drive Smarter Testing

Once you’ve mapped customer behaviour, the next step is turning those insights into focused, data-backed test hypotheses. Behavioural analytics platforms help uncover intent and friction that standard metrics often miss, giving you the depth to generate hypotheses grounded in real customer experiences rather than assumptions.

For example:

• Excessive clicking may signal frustration, such as on a broken or misleading CTA.
• High scroll depth might suggest strong interest, a subtle cue to signal future conversion potential even if the session doesn’t result in purchase.
• Repeated back-and-forth navigation could indicate customers are struggling to find what they need. User Conversion used FullStory to identify this friction in Travis Perkin’s navigation, leading to a redesign that drove a 26% conversion uplift among customers interacting with the new menu structure.

These behavioural signals help explain not just what customers are doing, but why. Informed with this context, you can design hypotheses that directly address customer pain points and motivations. For instance, if customers drop off at checkout, behavioural data might reveal that it’s due to an overwhelming form, not a lack of purchase intent. With that clarity, you can confidently test solutions like simplifying the process or redesigning the layout.

Behavioural insights make your experimentation strategy sharper, helping you run smarter, more meaningful tests that solve real problems.

Analysing Tests with Deeper Context

One of the key advantages of behavioural analytics platforms is their ability to bring deeper context to A/B test analysis. While traditional metrics tell you which variant performed better, behavioural data reveals why, showing how specific design or content changes impact customer behaviour. Did more customers engage with a streamlined CTA? Did fewer customers excessively click or drop off after a form update? These insights uncover the customer response behind the results.

Equally important, this behavioural context helps shape future testing strategies. By understanding not just what worked, but how customers interacted with each experience, you can drive a cycle of continuous learning, optimisation, and meaningful improvement.

Conclusion: Strengthening Optimisation with Behavioural Insight

As a CRO program matures and the need to go beyond clicks and surface-level metrics grows, behavioural analytics platforms become increasingly valuable. They provide deeper insight into how customers engage with your site, revealing what captures attention, where friction arises, and what behaviours indicate future intent. This level of understanding empowers more meaningful, customer-focused improvements.

From ideation and testing to post-test analysis, behavioural insights support smarter decisions at every stage of the optimisation process. When you combine quantitative data with behavioural context, you don’t just boost conversion rates, you build more intuitive, customer-first experiences.

Feeling overwhelmed? Get in touch to book a consultation with one of our optimisation experts via hello@conversio.com.

Great CRO (Conversion Rate Optimisation) doesn’t stop at the first impression; it’s about guiding the customer from landing to conversion with as little friction as possible.

Although landing pages are a vital first step for your eCommerce site, and good first impressions are important, it’s important to think beyond them. Your fully-optimised landing page means very little if the rest of the customer journey creates friction.

The reality is, CRO doesn’t end at the landing page. It only begins there.

First Impressions Matter, But They’re Not Everything

High-converting landing pages should make an immediate impact. Reassuring the customer they’re in the right place offers value and guides them towards the next step, as well as keeping bounce rates low.

But the effort you’ve spent creating a strong first impression is wasted if your customers abandon your site because of a confusing, difficult, or untrustworthy checkout process.

The Full Funnel Perspective

True CRO looks at the full customer experience, from the moment they land on your site, to the point of conversion, and beyond. Every touchpoint matters.

Brands often invest heavily in directing traffic to their site, and focusing optimisation efforts high up in the funnel, only to lose those hard-earned visitors due to poor experiences as they progress through the site.

Optimisation strategies should be interlinked and look at the full user journey.

1. Straightforward Navigation

Once a customer lands, how easy is it for them to find what they’re looking for? Can they refine product lists effectively and with minimal effort? Does search serve relevant products, and react to previous user behaviour?

Optimising the navigation structure ensures customers can move efficiently around your site with minimal frustration.

2. Product Pages That Build Confidence

Product pages are a key point in the decision-making process, where customers start to show higher intent to buy.

Product information and the way it is presented can be make or break. Are you building trust, creating clarity, and making it easy for customers to answer every question they might have?

Informative and persuasive copy, high-quality images, and clear delivery information are great starting points for optimisation efforts that can drastically affect conversion rates.

3. Cart & Checkout Experience

This is where many journeys end prematurely. High intent customers want to checkout as quickly and seamlessly as possible, so any points of friction can have a significant impact on conversion rate.

• Forcing account creation: give customers the option to checkout as a guest, or make the login journey as smooth as possible. You can encourage customers to sign up post-purchase with benefits like order management.
• Hidden fees at checkout: delivery fees can be a shock to users if the checkout is the first time they see them. Display delivery pricing throughout the site, from sitewide banners, to details on PDPs to help manage users expectations and avoid nasty surprises at the point of purchase.
• Lack of payment options: if you don’t let customers pay via their method of choice, they might not pay at all. Offer customers a variety of payment options, from debit and credit cards, to payment services like Apple Pay, Google Pay, and PayPal.

4. Post-Purchase & Retention

The customer’s journey doesn’t end after a successful checkout. Post-purchases experiences are often overlooked, but can significantly impact lifetime value.

Optimise order confirmation pages, account creation journeys, and customer support to encourage repeat purchases, reviews, and referrals.

Why This Holistic Approach Matters

When CRO efforts focus only on landing pages, you’ll see higher abandonment rates as users progress through the site. Click-through rates and engagement might look strong initially, but your conversions may not reflect that.

Instead, by aligning your optimisation efforts across the entire journey, you create a seamless, enjoyable experience that builds trust. It removes barriers, and increases conversions where they matter most, at the bottom of the funnel.

If you want to find out more, take a look at How we Work or drop us a quick message here for a chat. 

As an optimisation agency, we often champion the mantra:

“Test everything!”

But what does “everything” mean?

Well? Pretty much everything.

Why? There is the standard argument which I’ll get out of the way… if you’ve been in testing long enough, you’ll have countless stories where the “obvious” solution produced completely unexpected results. Even when confidence in a variant is high, the results often defy assumptions, reinforcing the notion that implementing without testing carries a business risk. And yes, risk mitigation is a major reason we advocate for testing.

Testing safeguards your business in several ways:

• Brand-driven changes: Ensuring updates don’t disrupt the customer journey.
• Budget measures: Measuring the impact of removing or investing in functionality and/or development.
• Non-funnel metrics: Exploring opportunities to influence metrics like engagement or loyalty while maintaining conversion.

But here’s the thing: testing isn’t just about risk mitigation or incremental gains. When decisions are reduced to a binary view of risk vs. reward, we lose sight of testing’s true purpose: learning about your customer.

This is where the risk of skipping testing really lies. When a change feels “too small to matter” or “too obvious to fail,” the temptation is to skip testing and go straight to development. While this approach might feel efficient, it overlooks the critical opportunity to learn from customer behaviour.

The purpose of A/B testing isn’t just to validate decisions. It’s to uncover what your customer values, what motivates them, and what drives their behaviour.

When a test idea is pushed straight to development, you miss the chance to:

• Understand why the change succeeded or failed.
• Identify hidden insights that could influence broader strategies.
• Leverage learnings for innovative on-site optimisations.
• Translate findings into other areas of the business, from marketing to product development.

For example, a seemingly small copy change might not only boost conversions but also reveal deeper insights into how customers perceive your brand or value proposition. These learnings can shape strategies beyond the test itself, delivering long-term value far beyond the immediate impact.

Testing is not just a decision-making tool – it’s a customer insight engine.

When we view testing through this lens, we move beyond the black-and-white view of risk vs. reward. Testing becomes about truly understanding your customer, so every decision, whether on-site or beyond, is grounded in data-driven insights about what matters most to them.

So, the next time a change feels “obvious” or “low-risk,” pause and ask:

What could we learn about our customers from testing this?

The answer might surprise you, and drive value in ways you hadn’t anticipated.

If you want to find out more, take a look at How we Work or drop us a quick message here for a chat. 

In the world of Conversion Rate Optimisation (CRO), the term ‘statistical significance’ is foundational. Yet, it’s a concept that often gets tangled in a web of complex explanations and statistical jargon, leaving many digital professionals scratching their heads. This post aims to demystify statistical significance, breaking it down into simpler terms and highlighting its crucial role in decision-making processes.

The Essence of Optimisation

At its core, CRO is about enhancing website experiences for customers through methodical testing. The goal is straightforward: if one version of a website experience proves superior, it should become the default customer experience. However, determining which version is truly superior introduces the need for a more scientific approach, where statistical significance takes the spotlight.

What is Statistical Significance?

A Google search on statistical significance yields the following definition:

“Statistical significance refers to the claim that a result from data generated by testing or experimentation is likely to be attributable to a specific cause. A high degree of statistical significance indicates that an observed relationship is unlikely to be due to chance.”

While accurate, this definition can seem daunting with its technicality. Yet, the concept can be simplified:

“Statistical significance is a tool to support decision making, by giving an understanding of the risk of implementing a particular change.”

The Holy Grail of CRO: 95% Statistical Significance

In the realm of CRO, achieving a 95% statistical significance is often considered the gold standard. This means there’s only a 5% likelihood that the observed results occurred by chance. In practical terms, this level of confidence suggests that the outcomes we see in testing would likely persist if implemented across the website.

Factors to Consider

• Sample Size: The amount of data you need to ensure your results are reliable. A larger sample size can provide more confidence in the findings.
• Test Duration: It’s essential to run tests for a sufficient period, ideally spanning a few business cycles, to account for any fluctuations in customer behaviour.

Once a satisfactory level of statistical significance is achieved, the decision to implement changes, iterate further, or abandon the initiative rests on several considerations:

1. Potential Impact: Evaluating the observed uplift and how it could improve customer experience or business metrics.
2. Implementation Feasibility: Considering the cost and time required to make the changes permanent.
3. Data for Further Optimisation: Deciding if there’s enough evidence to suggest modifications could yield even better results.

Why Statistical Significance Matters

Employing statistical significance effectively in CRO empowers businesses to make more informed decisions. It offers a buffer against the randomness of chance, providing a layer of confidence when deciding to roll out a new website feature or tweak an existing one. In essence, it helps businesses mitigate risk, ensuring that changes are genuinely beneficial to both the customer and the company.

Conclusion

Statistical significance is more than a set of complex calculations—it’s a critical tool in the optimisation toolkit. By understanding and applying this concept, optimisation specialists can lead their businesses to make data-driven decisions that enhance customer experiences and, ultimately, drive success. While the science behind statistical significance can seem intricate, its application doesn’t have to be. Simplifying this concept down to its essence allows us to harness its power more effectively, making our digital environments better for users everywhere.

In a landscape where data is king, understanding statistical significance is akin to holding the keys to the kingdom. It’s what allows us to discern between a fleeting anomaly and a genuine improvement, guiding our optimisation efforts towards meaningful, impactful changes.

When it comes to experimentation programs, defining and measuring success is critical to demonstrate return on investment and engage business stakeholders. Success metrics allow us to showcase value and monitor the performance of our experimentation program and should effectively empower us to make informed decisions to further strengthen our optimisation efforts. However, the win rate KPI often lies at the heart of this assessment and can sometimes be misinterpreted as the sole indicator of program success.

The joy of victory…

It makes sense, right? Winning is deeply embedded in human nature—we are programmed to associate success with winning. Winning provides a sense of accomplishment, boosting confidence and self-esteem. From childhood through to adulthood, we are driven to celebrate our wins.

And we see it within our industry; case studies, sales calls, and conference talks filled with examples of ‘winning’ test results to showcase a successful approach to experimentation. Who among us hasn’t felt the satisfaction when your testing variant achieves a ‘winning’ result? We’ve all been there… we can thank the dopamine hit, as ‘winning’ activates our brain’s reward system and brings us joy.

…But at What Risk?

While our ambition may be to win, an exclusive focus on win rates can undermine the effectiveness of your experimentation program. This isn’t about spinning the cliché that ‘every test is a win’—it’s about recognising what a testing program can bring to your business beyond just a winning variant.

Testing Innovative Ideas

Testing bold ideas may come with risks, but it can lead to significant breakthroughs and substantial rewards. Innovation fosters creativity, resulting in a more dynamic and inventive work environment that has a ripple effect throughout an organisation. Addressing user challenges with a creative thinking approach sets your brand and website apart in a world where differentiation is invaluable.

The Value of Truth

Testing enables businesses to make rapid, evidence-based decisions, steering clear of investments in areas that don’t resonate with users or meet their needs. This agility allows teams to pivot to more impactful strategies quickly. Celebrating and valuing the rapid acquisition of actionable insights ensure that data-driven next steps are prioritised. Maximising efficiency requires avoiding the temptation to only test the obvious and investing the appropriate level of effort to validate strategies. By focusing on uncovering the truth, testing empowers stakeholders with the confidence to make informed decisions, driving investment into areas that positively impact both customers and the bottom line.

The Long-term Goals

Exploring a diverse set of testing strategies provides a broad landscape in which to learn and react. This, alongside understanding the context behind ‘losing’ tests, drives a more comprehensive understanding of user behaviour, preferences, and pain points that may be difficult to get to within your existing data. This context fuels more effective strategies in the long run, driving tighter hypotheses and a robust testing approach, setting you up for success.

A Culture for Change

Focusing solely on win rates can undermine the true essence of a testing culture, which should prioritise curiosity, continuous improvement, robust methodology, and learning. A balanced view of success is crucial for maintaining team morale and avoiding frustration. Emphasising fast learning, agility, and data driven hypotheses ensures that every effort is valuable and outcomes are viewed as opportunities for growth rather than sources of disappointment.

Conclusion

Effective experimentation opens up opportunities beyond a winning test result and how we assess and communicate program success will influence program direction, quality and perception within your business.

While win rate is a valuable metric, it should be considered within a broader context. A balanced approach showcases experimentation as a tool for making data-driven decisions, fostering a culture of innovation and continuous improvement, prioritising robust testing processes to successfully optimise the user experience.

By looking beyond win rates, you can elevate your testing program, empower your teams, and embed experimentation into the core of your business culture, ensuring it is seen as a strategic, integral part of your organisation rather than a standalone function.

Social proof, in the form of testimonials and reviews, holds immense power in influencing customer behaviour and driving conversions. 95% of users read reviews online before making a purchase, with a further 58% confirming they would pay more for a product with positive reviews.

In an age where consumers are inundated with choices and marketing messages, the authenticity and reliability of social proof can make or break a purchase decision.

A study by Spiegel Research Center found that showing 5 product reviews can increase conversion rates by 190% for lower-priced products and 380% for higher-priced items.

The Importance of Being Genuine

Arguably, one of the most important aspects of leveraging social proof is maintaining authenticity.

Potential customers are more likely to trust and engage with feedback that feels real and unbiased. This means using actual data gathered from genuine customer experiences, rather than artificially inflating ratings or cherry-picking only positive feedback.

Displaying negative reviews alongside positive ones can bolster credibility. When both positive and negative reviews are displayed, 68% of users are more likely to find them trustworthy. Showing that a company does not hide or delete negative feedback, but rather addresses it constructively, not only builds trust, but helps customers make well-informed decisions.

Encouraging customers to share their own photos in reviews also adds a layer of authenticity, making it easier to visualise how products will look and perform in real-life scenarios.

The Role of A/B Testing

Implementing social proof should not be a set-it-and-forget-it strategy. A/B testing is essential to understand how various elements of social proof impact customer behaviour.

By comparing different versions of social proof—whether it be the placement of testimonials, the wording of messages, or the types of metrics highlighted—you can uncover valuable insights into what resonates best with their audience.

Another important consideration is customising social proof for different markets. Using A/B testing, you can establish the optimum messaging and metrics that resonate best with users, and determine where on your site social proof has the highest impact.

It also allows for optimisation of exposure levels, meaning businesses can determine whether social proof should be displayed on a select range of products or more broadly across an entire catalogue.

Exposure Levels and Repetition Impact

In addition to how frequently social proof is shown, it’s important to consider the impact of repeated exposure.

Research suggests that there can be diminishing returns with repeated exposure.

Therefore, it might be prudent to limit how often the same testimonial or review is displayed to a returning visitor, ensuring that the social proof remains impactful without becoming redundant.

Leveraging Different Types of Social Proof

Beyond customer reviews and testimonials, there are various other forms of social proof that can be leveraged.

These include expert endorsements, celebrity influencers, media mentions, and even user-generated content such as social media posts.

Businesses can also utilise metrics such as the number of products sold, the number of users registered, or high-profile clients to create a bandwagon effect.

By diversifying the types of social proof employed, companies can appeal to a broader audience.

The Importance of Context

Context matters when it comes to displaying social proof.

Testimonials and reviews should be relevant to the product or service being considered and should address the concerns or desires specific to that context.

For instance, a testimonial about customer service might be more relevant on a product return page than on the product detail page itself.

Moreover, dynamic social proof that updates in real-time—such as recent purchases or live customer activity—can create a sense of urgency and prompt immediate action.

By carefully curating and continually testing the deployment of testimonials and reviews, businesses can harness the power of social proof to build trust, enhance credibility, and ultimately drive conversions.

The authenticity of the feedback, coupled with strategic A/B testing and mindful exposure levels, ensures that social proof remains a potent tool in the marketing arsenal.

Embracing the psychology of social validation and contextually relevant endorsements can further amplify its impact, turning prospective buyers into loyal customers.