Enhancing Online Conversion Rates through Digital Marketing Interventions Based on A/B Testing: Evidence from a Three-Month Retail Pilot
DOI:
https://doi.org/10.54097/19bc9x66Keywords:
Digital transformation; personalized recommendation; A/B testing; conversion rate; retail; causal inference.Abstract
As the digital transformation of the retail sector deepens, rigorous empirical evidence regarding the causal relationship between digital initiatives and business performance remains scarce. This study conducts a three-month randomized controlled trial at the Shanghai online store of Dunelm Group, a leading UK home furnishings retailer. A total of 10,350 active users were randomly assigned to two groups: the treatment group (n = 5,200) received a personalized recommendation system integrating collaborative filtering and deep learning, while the control group (n = 5,150) retained a basic recommendation module. The study tests three core hypotheses: (1) personalized recommendations significantly increase online conversion rates; (2) this increase is driven by reduced drop-off rates at the add-to-cart stage; and (3) the conversion improvement is achieved without compromising order value while enhancing customer satisfaction. Results show that the treatment group achieved a conversion rate of 4.7%, representing a 20.5% increase compared to 3.9% in the control group (t = 3.45, p < 0.01, Cohen’s h = 0.42), supporting Hypothesis 1. Funnel analysis reveals that the conversion rate from product browsing to add-to-cart was 68.2% in the treatment group versus 50.4% in the control group, a 35% improvement (χ² = 45.2, p < 0.01, Cramér’s V = 0.21), validating Hypothesis 2. Average order value did not significantly differ between groups (£85.3 vs. £84.1, U = 0.98, p = 0.32), while customer satisfaction (CSAT) was significantly higher in the treatment group (8.5 vs. 7.9, U = 2.34, p < 0.05), fully supporting Hypothesis 3. Heterogeneity analysis indicates a 28% conversion lift among new customers (t = 4.12, p < 0.001), substantially higher than the 8% lift among existing customers (t = 1.56, p = 0.12), highlighting the value of personalization in customer acquisition. This study establishes a “diagnosis-design-validation” framework for digital transformation evaluation, offering a replicable paradigm for data-driven decision-making in retail.
Downloads
References
[1] Dunelm Group PLC, “Annual Report 2023/2024,” 2024. [Online]. Available: https://www.dunelmgroup.com/investors/annual-reports.
[2] Statista, “Global online retail market trends,” 2024. [Online]. Available: https://www.statista.com
[3] A. Pandey and P. Tomar, “Internet advertising and consumer memory: A comparative study,” Journal of Marketing Communications, vol. 28, no. 3, pp. 245–260, 2022.
[4] CNNIC, “The 53rd Statistical Report on China’s Internet Development,” China Internet Network Information Center, Beijing, 2024.
[5] E. Furmaga et al., “Randomization methods in digital experiments,” Journal of Marketing Research, vol. 62, no. 1, pp. 88–104, 2025.
[6] S. Kraus et al., “Digital transformation and supply chain management,” International Journal of Production Economics, vol. 235, p. 108089, 2021.
[7] R. Nitami et al., “Personalized recommendation in retail: Enhancing the shopping experience,” Journal of Consumer Behaviour, vol. 23, no. 1, pp. 45–60, 2024.
[8] D. Soto Setzke et al., “Digital transformation challenges in retail,” Journal of Retail Innovation, vol. 15, no. 2, pp. 111–128, 2023.
[9] Nextplc, “Annual performance report,” 2024. [Online]. Available: https://www.nextplc.co.uk
[10] G. George et al., “Corporate purpose and competitive advantage,” Harvard Business Review, vol. 101, no. 2, pp. 78–92, 2023.
[11] R. Gulati, “Purpose-driven organizations,” MIT Sloan Management Review, vol. 63, no. 3, pp. 45–52, 2022.
[12] Y. Fang, “The causal chain of digital transformation in retail,” Journal of Retail Innovation and Technology, vol. 13, no. 1, pp. 45–60, 2026.
[13] G. J. Stigler, “The economics of information,” Journal of Political Economy, vol. 69, no. 3, pp. 213–225, 1961.
[14] J. R. Hauser, G. Liberali, and G. L. Urban, “Website morphing,” Marketing Science, vol. 28, no. 2, pp. 202–223, 2009.
[15] S. S. Iyengar and M. L. Lepper, “When choice is demotivating: Can one desire too much of a good thing?” Journal of Personality and Social Psychology, vol. 79, no. 6, pp. 995–1006, 2000.
[16] X. Deng, M. Yang, and K. Jin, “Cost-effectiveness of marketing strategies in retail,” Journal of Retail Economics, vol. 18, no. 2, pp. 78–92, 2025.
[17] T. Pham et al., “Key performance indicators for retail digital transformation,” Journal of Retail Strategy, vol. 19, no. 3, pp. 56–72, 2025.
[18] Z. Li, M. Zhang, and R. Aggarwal, “Cross-market validation of retail digital strategies,” Journal of International Retailing, vol. 16, no. 3, pp. 89–105, 2024.
[19] C. Goic, J. Guajardo, and L. Ma, “Long-term effects of digital marketing interventions,” Journal of Digital Business, vol. 17, no. 1, pp. 34–50, 2026.
[20] W. Zhao and Y. Zhou, “Omnichannel integration in retail,” Journal of Retail and Technology, vol. 20, no. 2, pp. 67–83, 2025.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Highlights in Business, Economics and Management
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
