Bundle entropy as an optimized measure of consumers' systematic product choice combinations in mass transactional data

Mansilla, Roberto; Smith, Gavin; Smith, Andrew; Goulding, James

Abstract

Understanding and measuring the predictability of consumer purchasing (basket) behaviour is of significant value. While predictability measures such as entropy have been well studied and leveraged in other sectors, their development and application to very large multi-dimensional data sets present in the retailing sector are less common. While a small number of methods exist, we demonstrate they fail to accord with intuition, leading to the potential for misunderstandings between those who conduct the analysis and those who act on the insights. We delineate the requirements for such a measure in this domain to demonstrate these issues in context. A novel measure is then developed based on entropy to directly measure the predictability of basket composition. The measure is designated as bundle entropy (zero denotes a bundle's total predictability, one the total unpredictability). We empirically compare the proposed bundle entropy against existing measures using two large-scale real-world transactional data sets, each including more than 2,000 households (frequent shoppers) over two years. First, we demonstrate how the proposed measure is the only measure that behaves according to the desired properties. Second, we show empirically that bundle entropy differs noticeably from the other measures. Finally, we consider some use case analyses and discuss the utility of the proposed measure in practice.

Citation

Mansilla, R., Smith, G., Smith, A., & Goulding, J. (2022). Bundle entropy as an optimized measure of consumers' systematic product choice combinations in mass transactional data. In Proceedings 2022 IEEE International Conference on Big Data (1044-1053). https://doi.org/10.1109/BigData55660.2022.10021062

• Presentation Conference Type: Edited Proceedings
• Conference Name: 2022 IEEE International Conference on Big Data (Big Data)
• Start Date: Dec 17, 2022
• End Date: Dec 20, 2022
• Acceptance Date: Nov 15, 2022
• Online Publication Date: Jan 26, 2023
• Publication Date: Dec 17, 2022
• Deposit Date: Mar 8, 2023
• Publicly Available Date: Mar 8, 2023
• Pages: 1044-1053
• Series Title: IEEE International Conference on Big Data
• Book Title: Proceedings 2022 IEEE International Conference on Big Data
• ISBN: 9781665480468
• DOI: https://doi.org/10.1109/BigData55660.2022.10021062
• Public URL: https://nottingham-repository.worktribe.com/output/18230088
• Publisher URL: https://ieeexplore.ieee.org/abstract/document/10021062