Proceedings of TUSUR University / Archive / Issue of Journal № 1, t. 29, 2026 / A Dual-Loop Framework for Selecting and Optimizing Recommendation Algorithms for Business in a Digital Environment

A Dual-Loop Framework for Selecting and Optimizing Recommendation Algorithms for Business in a Digital Environment

DOI: 10.21293/1818-0442-2026-29-1-152-159

Download article in PDF format

JATS xml

Abstract: Relevance. The development of digital platforms reinforces the importance of recommendation algorithms as a tool for managing user choice, personalizing offers, and supporting company business processes. Purpose of the study is to develop a dual-loop framework for selecting and optimizing recommendation algorithms for business in a digital environment, ensuring the alignment of the technical evaluation of algorithms with subsequent verification of their practical effectiveness. Methods. The study employs comparative analysis methods, multi-criteria evaluation, modeling of the algorithm selection procedure, the adjustment of particular indicators into an integral criterion, as well as a conceptual description of the business validation of recommendation solutions. Novelty. The dual-loop framework of selecting and optimizing recommendation algorithms is proposed, which is characterized by dividing the evaluation procedure into an internal loop of a multi-criteria offline analysis and an external loop of verifying the selected solution based on online indicators of user and economic response. Results. It is shown that the choice of a recommendation algorithm should not be based on a single local metric, since the quality of recommendations is determined by a set of multidirectional characteristics. The role of the integral criterion as the basis for preliminary comparison, ranking and tuning of algorithms before their implementation in the real digital environment is substantiated. Practical significance. The proposed approach can be applied by digital platforms, online stores, media services and other companies to make an informed choice of recommendation algorithms, adjust their hyperparameters and then check the impact of recommendations on user behavior and business results.

Keywords: recommendation systems, digital business environment, management decisions, multi-criteria assessment, integral indicator

Funding: The work was carried out within the framework of the state assignment of the Ministry of Education and Science of the Russian Federation; project FEWM-2026–0011.

For citation:
Kulshin R. S., Sidorov A. A. A Dual-Loop Framework for Selecting and Optimizing Recommendation Algorithms for Business in a Digital Environment. Doklady Tomskogo gosudarstvennogo universiteta sistem upravleniya i radioelektroniki, 2026, vol. 29, no. 1, pp. 152–159. DOI: 10.21293/1818-0442-2026-29-1-152-159

Authors and copyright holders:

Kulshin R. S. , Tomsk State University of Control Systems and Radioelectronics (Tomsk, Russia)
Sidorov A. A. , Tomsk State University of Control Systems and Radioelectronics (Tomsk, Russia)

1. Cheng H., Koc L., Harmsen J. Wide & deep learning for recommender systems. Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, Boston, MA, USA, ACM, 2016, pp. 7–10.
2. Ghanem N., Leitner S., Jannach D. Balancing consumer and business value of recommender systems: a simulation-based analysis. Electronic Commerce Research and Applications, 2022, vol. 55, pp. 101195. DOI: 10.1016/j.elerap.2022.101195.
3. De Biasio A., Navarin N., Jannach D. Economic recommender systems – a systematic review. Electronic Commerce Research and Applications, 2024, vol. 63, pp. 101352. DOI: 10.1016/j.elerap.2023.101352.
4. Adomavicius G., Tuzhilin A. Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 2005, vol. 17, no. 6, pp. 734–749. DOI: 10.1109/TKDE.2005.99.
5. Gomez-Uribe C.A, Hunt N. The Netflix recommender system: algorithms, business value, and innovation. ACM Transactions on Management Information Systems, 2015, vol. 6, no. 4, pp. 1–19.
6. Zangerle E., Bauer C. Evaluating recommender systems: survey and framework. ACM Computing Surveys, 2022, vol. 55, no. 8, pp. 1–38. DOI: 10.1145/3556536.
7. Gunawardana A., Shani G., Yogev S. Evaluating recommender systems. Recommender Systems Handbook, 2022, pp. 547–601. DOI: 10.1007/978-1-0716-2197-4_15.
8. Zheng Y., Wang D. A survey of recommender systems with multi-objective optimization. Neurocomputing, 2022, vol. 474, pp. 141–153. DOI: 10.1016/j.neucom.2021.11.041.
9. Jannach D., Abdollahpouri H. A survey on multi-objective recommender systems. Frontiers in Big Data, 2023, vol. 6, pp. 1157899. DOI: 10.3389/fdata.2023.1157899.
10. Jannach D, Jugovac M. Measuring the business value of recommender systems. ACM Transactions on Management Information Systems, 2019, vol. 10, no. 4. p. 1–23. DOI: 10.1145/3370082.
11. Castells P., Moffat A. Offline recommender system evaluation: challenges and new directions. AI Magazine, 2022, vol. 43, no. 2, pp. 225–238. DOI: 10.1002/aaai.12051.
12. Kulshin R.S., Sidorov A.A. An Entropy-Based Composite Indicator for Evaluating the Effectiveness of Recommender System Algorithms. Control Sciences, 2024, vol. 4, pp. 37–51. DOI: 10.25728/cs.2024.4.4.
13. Gilotte A., Calauzènes C., Nedelec T. Offline A/B testing for recommender systems. Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, Marina Del Rey, USA, ACM, 2018, pp. 198–206.
14. Kulshin R.S., Sidorov A.A. [Optimization of hyperparameters of recommendation algorithms through a single evaluation metric]. Software Engineering, 2026, vol. 17, no. 2, pp. 67–82, DOI: 10.17587/prin.17.67-82 (in Russ.).
15. Quin F, Weyns D., Galster M., Silva C. A/B testing: a systematic literature review. Journal of Systems and Software, 2024, vol. 211, pp. 112011, DOI: 10.1016/j.jss.2024.112011.
16. Jeunen O., Baweja S., Pokharna N., Ustimenko A. Powerful A/B-testing metrics and where to find them. Proceedings of the 18th ACM Conference on Recommender Systems, New York, ACM, 2024, pp. 816–818, DOI: 10.1145/3640457.3688036.