Theofilus Yoel Lufung*1, Nur Wening2 and Rianto3
*1Doctoral Student Post Graduate Program of Management, Universitas Teknologi Yogyakarta, Indonesia
2Lecturer of Post Graduate Program of Management, Universitas Teknologi Yogyakarta, Indonesia
3Science Data Program Department of Informatics, Universitas Teknologi Yogyakarta, Indonesia
Theofilus Yoel Lufung
Abstract: Background: COVID-19 has become a severe global public health crisis. The government, public and private institutions face obstacles in handling the issue. So that in solving problems during this pandemic, it is essential to implement a Decision Support System (DSS) in proposing the decision-making to be aligned, efficient, and practical. It includes tracking, reporting, locking down management, and preparing strategies for dealing with covid-19. Objective: The purpose of this study is to review the role of decision support systems in decision-making during the COVID-19 pandemic. Methods: The method is a literature review. It evaluates several previous research related to the role of DSS in handling Covid-19. Scientific articles are compiled to obtain reliable data. The primary data are in the form of national and international journals. It was obtained through electronic-based searches with sources indexed by SINTA, such as Google Scholar and Elsevier. The keywords "role, application, DSS, policy, handling, covid-19" in Indonesian or English are searched through the engine. Journals or articles will then be re-filtered by shorting the results, methods, and text. The papers were collected and analyzed in the period May-June 2022. Result: The benefits in effectiveness were widely felt at the time when DSS was applied. The percentage value is 28%. The percentage in accuracy and efficiency is 25%. DSS also provides a short time to analyze the data to support the decision-making. The percentage value is 22%. Furthermore, a decision support system in an institution can reduce the workload. Conclusion: The decision support system can benefits companies or health facilities during the pandemic. The review results that various DSS methods developed in different parts of the world positively affect the decision-making process in handling COVID-19. Based on this review, the DSS used in decision-making varies widely, ranging from the DSS method used to assist management decisions such as lock down management, information systems for determining vaccine types, vaccine administration priorities, and monitoring the distribution of humanitarian aid. While in the health care sector, DSS plays a role in patient risk assessment, identifying patients with comorbidities, helping to measure vital signs, and supporting laboratory examinations in treating COVID-19.
Keywords: Decision Support System, Covid 19, Policy.
COVID-19 has become a severe global public health crisis. The overload number of patients in hospitals caused a shortage of medical logistics. The government and institutions, either public or private, worked vigorously to develop supporting information technology infrastructure to deal with the increasing number of patients. However, these efforts are not well coordinated between the parties. The rapid spread of COVID-19 requires fast handling as well. This results in asymmetry and fragmentation of information where the information is used only by certain institutions or communities. This becomes a challenge for each party to work hand in hand. So that in solving problems during this pandemic, it is necessary to implement a Decision Support System to be aligned, efficient, and practical (Liu, Y. et al., 2020; Rekso, A. I. et al., 2021).
The concept of a closed decision-making system logically examines all alternatives, ranks them according to the importance of their outcomes, and reveals a rational person to choose the best alternative/maximize the desired outcome. Quantitative decision models are usually closed decision system models. Open decision-making systems assume that decisions are made in a complex and sometimes unknown environment. Decisions are influenced by the environment, and the decision-making process also affects the environment.
Decision makers are not considered logical and completely rational, but exhibit rationality only to the extent implied by context, alternative views, ability to use decision-making models, etc. (Fuglseth, A. M., & Grønhaug, K. 2003). The forces that drive the decision-making process can be dissatisfaction with the current situation, or the expected reward from the new situation. The driving force when there is dissatisfaction is finding the problem. As for expected rewards, the result is a search for opportunities. Some decision-making models place more emphasis on feedback on the outcome of decisions. For example, Rubenstein and Haberstroh suggest the following steps: identification of problem or decision needs, analysis and reporting of alternatives, selection among available alternatives, communication and implementation of decisions, follow-up procedures, and feedback decisions (Utami, S. S. 2011).
A management information system (MIS) as part of a computerized DSS is useful in both closed and open systems. In a closed decision model, the computer acts as a pocket calculator to help calculate the optimal result. In the open model, the computer acts as an aid to human decision makers in calculating, storing, retrieving, and analyzing data. This design allows human decision makers to assign tasks alone or with computers. The limited number of human decision makers in an organization, coupled with the relative efficiency of human decision processing, means that MIS must program as many decisions as possible. Partial programming is possible if decisions cannot be fully programmed. In this case, the rules have been used to some extent before, leaving the final decision to the human decision maker. The MIS should be designed to monitor programmed decisions and identify decisions that appear unrealistic under the decision rules or that do not appear to produce good outcome plans. Programmed decisions are usually unstructured. Where possible, MIS provides a tool that decision makers can use to structure the decision-making process. For problems that are not programmed iteratively, MIS can be designed using a partial structure to speed up human processing of the remaining problems (Al-Tarawneh, H. A. 2012; Janssen, M. et al., 2017; Khakheli, M., & Morchiladze, G. 2015; & Nooraie, M. 2012).
A management information system is a flow processing procedure based on computer data and integrated with other procedures to provide information in a timely and effective manner to support decision-making and other management functions increase. The result is that with the exponential growth in the amount of business data and information today, efficient business practices only exist when the information required is fast, accurate, qualitative, and managed by the right staff. This is also the case if we consider it possible to make decisions on the case is the result of the lack of good management information systems. The rapid development of information technology and the development of telecommunications technology have streamlined all areas of life and human activity (Berisha-Shaqiri, A. 2014). Information has become an essential resource for managing modern organizations. This is because today's business environment is volatile, dynamic and turbulent, increasing the demand for accurate, relevant, complete, timely and timely economic information needed to drive decision-making processes. Increasing highlighting organizational skills in dealing with opportunities and risks (Ghaffarzadeh, S. A. M. 2015).
The positive impact of using DSS in the decision-making process is utilized to assist the handling of covid-19. several previous studies have shown that the use of DSS is used to assist in supervising the distribution of COVID-19 assistance with the VIKOR and AHP method (Mahaputra, P. B. N. et al., 2021; Lengkong, S. P. 2020). The use of DSS during the pandemic is also very helpful regarding the allocation and distribution of vaccines (Shahparvari, S. et al., 2022). In the education sector, especially to assist decision-making in the education sector to prevent a decline in student achievement due to COVID-19, DSS is also used with the Fuzzy Analytical Hierarchy Process (FAHP) method (Rahayu, A., & Gustian, D. 2022).
The method is a literature review. It evaluates several previous research related to the role of DSS in handling Covid-19. Scientific articles are compiled to obtain reliable data. The primary data are in the form of national and international journals. It was obtained through electronic-based searches with sources indexed by SINTA, such as Google Scholar and Elsevier. The keywords "role, application, DSS, policy, handling, covid-19" in Indonesian or English are searched through the engine. Journals or articles will then be re-filtered by shorting the results, methods, and text. The papers were collected and analyzed in the period May-June 2022.
RESULTS AND DISCUSSION:
The decision support system can benefit companies or health facilities during the pandemic. The review results that various DSS methods developed in different parts of the world positively affect the decision-making process in handling COVID-19. Based on this review, the DSS used in decision-making varies widely, ranging from the DSS method used to assist management decisions such as lock down management, information systems for determining vaccine types, vaccine administration priorities, and monitoring the distribution of humanitarian aid. While in the health care sector, DSS plays a role in patient risk assessment for patient triage purposes, identifying patients with comorbidities, helping to measure vital signs, and supporting laboratory examinations in the treatment of COVID-19.
(Majumder, R. et al., 2021) stated that handling Covid-19 in India is carried out using a decision support system. It is MATLAB GUI. MATLAB GUI can help create decisions in the form of Covid-19 predictions, optimal allocations, and lockdown management. (Wibowo, N.et al., 2022) describe the handling of Covid-19 in Kalikatir Village using a decision support system. It can determine the type of Covid-19 vaccine to get accurate results in deciding the type of vaccine. The method used is SMART for selecting new vaccine types. The result is an information system that can determine the selection of new vaccine types more quickly and effectively according to the criteria. (Wu, G. et al., 2020) use DSS to develop and validate the risk assessment and triage for COVID-19 patients in hospital admission. Seven hundred twenty-five patients participated in training and validating the model. This includes a retrospective cohort from Wuhan, China. It is 299 COVID-19 patients.
DSS theory (Almulhim, T. S., & Barahona, I. 2022) is based on fuzzy theory and principles of multi-criteria decision analysis. It needs to be involved in assessing the indicators in developing a post-COVID-19 reopening strategy. These indicators are selected to provide more reliable results. (Liu, Y. et al., 2020) show that COVID-19 has become an urgent and severe global public health crisis. So it is necessary to design, develop, and deploy a mobile-based decision support system for COVID-19 to assist doctors in collecting data, assessing risk, triage, managing, and following up on patients during the COVID-19 outbreak. The tool in the form of DCC19 is a mobile decision support system designed and developed to assist GPS in providing dynamic risk assessments for patients with suspected COVID-19 during an outbreak. The model can accurately predict the level of risk in any scenarios. Decision-making activities become more effective and efficient, especially amid a pandemic and limited resources.
The decision support system in (Lengkong, S. P. 2020) study uses the VIKOR method (VIsekriterijumsko KOMpromisno Rangiranje), a complex linear normalization technique calculations by providing rankings to get the best solution. This method aims to select from various alternatives despite conflicting criteria. In selecting recipients of Covid-19 assistance, the VIKOR method can be used to facilitate decision support and improve the quality of the results of selecting recipients of Covid-19 assistance. This method is supported by (Anam, M. S. 2021) study. His study uses the Naive Bayes method, which can facilitate the determination of prospective social assistance recipients so that the performance of local officers is increased. In addition, this system can store large amounts of data with high security. However, (Strong, P. et al., 2021) use a decision support system with the Bayesian statistical method to evaluate the COVID-19 response strategy for the short, medium, and long-term response strategy. This method involves assessing experts considered vital and a priority, such as hospital admissions, public welfare, and the feasibility of public economists. The assessment is then processed using the Bayesian method to produce priority decisions.
A decision support system is a computer-based system that combines data and decisions as a tool to support creating decisions. Decision makers require a lot of data from various sources, such as age, nutritional intake, vaccination history, diabetes rates, and blood pressure. Shortly, this study uses a multi-criteria support system (MCDSS) that can be applied in aspects such as health care that can assist decision-making (Haqbeen, J. et al., 2021). The Decision Support System (DSS) is an information system used to make decisions regarding various necessary actions related in handling COVID-19. The COVID-19 pandemic in Indonesia from the beginning of 2020 pushed some people to be afraid to visit doctors or hospitals. So to make it easier for someone to monitor their daily health, this study developed an idea. It is wearable sensors in the health sector, especially on Smart Shirts. Smart Shirts are one of the uses of wearable sensors, but users do not need to touch the sensor to use them (hands-free). The use of the Smart Shirt can make it easier for users to monitor health conditions such as pulse sensors, temperature sensors, 3-Axis Accelerometer sensors, fiber optic sensors, and GPS sensors that are used to measure heart rate, body temperature, number of steps, to detect the location of the presence of smart shirt users. This Smart Shirt's existence is expected to help increase Indonesians' awareness of their health conditions. Meanwhile, selling this Smart Shirt can help improve the Indonesian economy, especially during the COVID-19 pandemic and after (Santoso, T. A. P. 2021).
(Pamungkas, T. S. et al., 2020) stated that the decision support system method is used to determine people who are given BLT (cash transfer) during the COVID-19 pandemic. So to find out who deserves it, a decision-making system is needed. When it is applied, BLT deposits from the government can be right on target according to the criteria they set. This system has successfully applied the K-Means and AHP methods for decision-making on receiving cash transfers from the government. This system has a 100% accuracy rate. The most important criteria are monthly income with a weight of 0.394142515. The homeownership is 0.231035138, the number of dependents: 0.190359096, age: 0.081077616, a job: 0.058111736, and education: 0 0.045273898.
(Karthikeyan, A. et al., 2021), India uses a decision support system to assess the clinical utility of machine learning to predict COVID-19 mortality. Different machine learning has been developed based on the support system. The importance of the XGBoost feature provides an interpretation of a model that may be relevant in a clinical setting. The performance metrics obtained instill great confidence in the proposed model. Other possible features with predictive capabilities are identified but will require data from multiple sources to confirm their relevance and improve the model. In Indonesia, (Merliasari, R. P., & Warnars, H. L. 2020) used a decision support system to assess the rating of applications made as an entertainment activity by workers working at home. The results show that the decision-making system can be excellent due to its data. The result depicts 49.2% choosing happy, 20.6% choosing calm, 25.4% choosing bored, and 4.8% choosing stress. This study concludes that some users feel delighted working from home during the Covid-19 pandemic.
Based on (Rekso, A. I. et al., 2021), the Decision Support System is an information system used to make decisions about various necessary actions related to handling COVID-19 in Yogyakarta. DSS was developed by the Ministry of Communication and Informatics and utilized by the COVID-19 Task Force in Yogyakarta. They use DSS as a tool to deal with the COVID-19 pandemic, especially in Yogyakarta. According to the results of (Whetyningtyas, A. 2016), information technology has developed rapidly in terms of hardware and software. This has given birth to a decision support system or DSS, a specific information system aimed at assisting management in making decisions related to semi-structured problems by having facilities to generate various alternatives that users can use interactively. DSS plays a vital role for managers in assisting the decision-making process. DSS is designed with high flexibility and adaptability to adapt to the user's needs quickly. So in the Covid-19 pandemic, DSS is needed because it can increase decision-making effectiveness.
DSS integrates GIS, analytics, and simulation methods to help develop the priority-based distribution of COVID-19 vaccines in large urban environments. Three scenarios of vaccine supply, namely limited, excessive, and disruption, were formulated to operationalize the two-dose vaccination program. The results show that vaccines can be prioritized for the most vulnerable segments of society and distributed using a logistics network. Compared to vaccine distribution plans without priority, the projects generated by the proposed DSS ensure that vaccinations are prioritized for the neediest and most vulnerable populations. The aim is to curb the spread of infection and reduce mortality more effectively. They also achieve population (Shahparvari, S. et al., 2022).
An additive utility assumption-based approach for a multi-criteria decision support system (MCDSS) is proposed empirically using the standard SEIR model approach (Vulnerable, Affected, Infected, and Cured). The results include a comparative analysis in tabular form with existing methods to describe the potential of the proposed system, including parameters such as Precision, Recall, and F-Score. Other advanced parameters such as MCC (Matthews Correlation Coefficient), ROC (Receiver Operating Characteristics), and PRC (Precision Recall) have also been considered for validation, and graphs are illustrated using a Jupyter notebook. Statistical analysis of the top eight most affected states in India was carried out effectively using Weka software tools and IBM Cognos software to correctly predict the outbreak of the Covid-19 pandemic situation (Aggarwal, L. et al., 2021).
The COVID-19 pandemic highlights the need for decision support tools to help cities become more resilient to infectious diseases. Through urban design and planning, non-pharmaceutical interventions can be activated, promote behavior change and facilitate the construction of low-risk buildings and public spaces. Computational tools, including computer simulations, statistical models, and artificial intelligence, with the DSS System (Decision Support System), have been used to support the response to the current pandemic and the spread of previous infectious diseases (Yang, L. 2022).
(Siddiqui, S. Y. et al., 2021) present how to build a multi-attribute decision support system to choose between coping strategies. Such an analysis can evaluate various candidates of the indicators in the field. The following analysis would provide reliable data and eliminate the unsuccessful ones. The methodology is illustrated by showing how the DSS can predict the long and short-term predictions to support the policy.
Based on (Ahmed, F. et al., 2021), the accurate and rapid identification of severe and non-severe COVID-19 patients is necessary. It intends to reduce the risk of hospital overload, effectively utilize hospital resources, and minimize mortality rates. The support system proposed in this paper is to identify critical and non-critical COVID-19 patients in a hospital using only three blood test markers. The conjunctive belief rule-based system achieves accuracy, sensitivity, and specificity are 0.954, 0.923, and 0.959, respectively. Meanwhile, the disjunctive belief rule bases are 0.927, 0.769, and 0.948. Moreover, with a 98.85% AUC value, it is better than four traditional machine learning algorithms: LR, SVM, DT, and ANN. The proposed system will help hospital authorities identify severe and non-severe COVID-19 patients and adopt optimal treatment plans in a pandemic.
Based on 20 reviewed journals, the use of DSS in supporting decisions for institutions such as hospitals to government is then analyzed. The data processing results are in the form of a bar chart shown in Figure 1.
Figure Image is available at PDF file
Figure 1: Benefits of Implementing a Decision Support System Percentage
The data processing results show that the application of the Decision support system provides various benefits for hospitals and the government. It gives the positive effects at the time (28%) followed by accurate and efficient with a percentage value of 25%. Finally, the perceived benefits are fast in making decisions with a percentage value of 22%. The existence of a Decision support system in an institution will be beneficial to facilitating work and reducing workload.
The decision support system benefits many companies or health facilities dealing with the COVID-19 pandemic. The decision support system is used for various purposes in various countries, such as predicting COVID-19 cases, selecting vaccines, assessing the severity of COVID-19 patients, reopening post- lockdown, and selecting COVID-19 aid recipients, evaluating COVID-19 response. The effectiveness of the decision support system is 28% accurate in supporting the decision maker to create better action in facing the pandemic.
The author expresses gratitude to God for being able to publish this paper, the author also thanks those who helped in the completion of this article, especially to the Universitas Teknologi Yogyakarta, which has facilitated and assisted in the preparation and publication of this article. The author hopes that this article will be useful for the readers
Rekso, A. I., Rusdiati, R. D. & Nadi, D. T. (2021). Digital Technology Ecosystem Mapping to Help Handling COVID-19. Journal of Information Systems for Public Health 6(3); 10–27.
Ahmed, F., Hossain, M. S., Islam, R. U., & Andersson, K. (2021). An evolutionary belief rule-based clinical decision support system to predict covid-19 severity under uncertainty. Applied Sciences, 11(13), 5810. https://doi.org/10.3390/app11135810
Almulhim, T. S., & Barahona, I. (2022). Decision support system for ranking relevant indicators for reopening strategies following COVID-19 lockdowns. Quality & quantity, 56(2), 463-491. https://doi.org/10.1007/s11135-021-01129-3
Al-Tarawneh, H. A. (2012). The main factors beyond decision making. Journal of Management Research, 4(1), 1-23. https://doi.org/10.5296/jmr.v4i1.1184
Anam, M. S. (2021). Social Assistance Decision Support System Using the Naive Bayes Method. JTECS: Journal of Electronic Telecommunications Systems Power Control Systems and Computers, 1 (1), 85-92.
Aggarwal, L., Goswami, P., & Sachdeva, S. (2021). Multi-criterion intelligent decision support system for COVID-19. Applied Soft Computing, 101, 107056. doi: 10.1016/j.asoc.2020.107056.
Mahaputra, P. B. N., Sukarsa, I. M., & Wirdiani, N. K. A. (2021). Decision Support System for COVID-19 Direct Target Cash Recipients Using the Analytical Hierarchy Process and Simple Additive Weighting Method. Jurnal Ilmiah Teknologi dan Komputer, 2(2), 407-417.
Berisha-Shaqiri, A. (2014). Management information system and decision-making. Academic Journal of Interdisciplinary Studies, 3(2), 19–24. https://doi.org/10.5901/ajis.2014.v3n2p19X
Fuglseth, A. M., & Grønhaug, K. (2003). Can computerised market models improve strategic decision-making? An exploratory study. The Journal of Socio-Economics, 32(5), 503-520. https://doi.org/10.1016/j.socec.2003.08.007
Ghaffarzadeh, S. A. M. (2015). Decision making based on management information system and decision support system. Journal of Management Research and Analysis, 2(1), 98-107.
Janssen, M., van der Voort, H., & Wahyudi, A. (2017). Factors influencing big data decision-making quality. Journal of business research, 70, 338-345. https://doi.org/10.1016/j.jbusres.2016.08.007
Khakheli, M., & Morchiladze, G. (2015). Factors affecting decision making in an organization. International Journal of Management and Commerce Innovations, 3(1), 425-428.
Lengkong, S. P. (2020). Decision Support System for Covid-19 Aid Recipients using VIKOR method. Jurnal Teknik Informatika, 15(4), 297-302.
Liu, Y., Wang, Z., Ren, J., Tian, Y., Zhou, M., Zhou, T., ... & Li, J. (2020). A COVID-19 risk assessment decision support system for general practitioners: design and development study. Journal of medical Internet research, 22(6), e19786. https://doi.org/10.2196/19786
Nooraie, M. (2012). Factors influencing strategic decision-making processes. International Journal of Academic Research in Business and Social Sciences, 2(7), 405–429.
Rahayu, A., & Gustian, D. (2022). Decision Support System Student Achievement During the Covid-19 Pandemic With Method Fuzzyanalytic Hierarchy Process. Jurnal Teknik Informatika (Jutif), 3(1), 21–28.
Wu, G., Yang, P., Xie, Y., Woodruff, H. C., Rao, X., Guiot, J., ... & Lambin, P. (2020). Development of a clinical decision support system for severity risk prediction and triage of COVID-19 patients at hospital admission: an international multicentre study. European Respiratory Journal, 56(2). https://doi.org/10.1183/13993003.01104-2020
Merliasari, R. P., & Warnars, H. L. (2020). Covid-19 Series: A rule-based decision support system for analysis behavior of people while working from home. In IOP Conference Series: Materials Science and Engineering (Vol. 1007, No. 1, p. 012031). IOP Publishing. doi:10.1088/1757-899X/1007/1/012031
Majumder, R., Bhise, A., Paul, N., Garg, S., & Ghose, D. (2022). Development of Decision Support System for Effective COVID-19 Management. arXiv preprint arXiv:2203.08221. [cs, eess] [Preprint]. Available at: http://arxiv.org/abs/2203.08221 (Accessed: 23 March 2022).
Karthikeyan, A., Garg, A., Vinod, P. K., & Priyakumar, U. D. (2021). Machine learning based clinical decision support system for early COVID-19 mortality prediction. Frontiers in public health, 9, 626697. doi:10.3389/fpubh.2021.626697
Pamungkas, T. S., Nugroho, A. S., Wasiso, I., Anggoro, T., & Kusrini, K. (2020). Decision support system for direct target cash recipients using the AHP and K-Means method. Research: Journal of Computer, Information System & Technology Management, 3(2), 45-54. http://doi.org/10.25273/research.v3i2.7084
Shahparvari, S., Hassanizadeh, B., Mohammadi, A., Kiani, B., Lau, KH, Chhetri, P., & Abbasi, B. (2022). A decision support system for prioritized COVID-19 two-dosage allocation and distribution. Transportation Research Part E: Logistics and Transportation Review, 159 , 102598. doi: 10.1016/j.tre.2021.102598
Haqbeen, J., Sahab, S., Ito, T., & Rizzi, P. (2021). Using decision support system to enable crowd identify neighborhood issues and its solutions for policy makers: An online experiment at kabul municipal level. Sustainability, 13(10), 5453. https://doi.org/10.3390/su13105453
Whetyningtyas, A. (2016). The Role of the Decision Support System (DSS) for Decision Maker Management. Management Analysis, 5(1);102–8.
Santoso, T. A. P. (2021). Smart Shirt untuk Mengukur Tingkat Kesehatan dengan Menggunakan Teknologi Sensor dan Fitur Digital. Jurnal Sistem Cerdas, 4(2), 104-113. https://apic.id/jurnal/index.php/jsc/article/view/94
Siddiqui, S. Y., Abbas, S., Khan, M. A., Naseer, I., Masood, T., Khan, K. M., ... & Almotiri, S. H. (2021). Intelligent decision support system for COVID-19 empowered with deep learning. Comput. Mater. Contin, 66, 1719-1732. doi:10.32604/cmc.2020.012585
Strong, P., Shenvi, A., Yu, X., Papamichail, K. N., Wynn, H. P., & Smith, J. Q. (2021). Building a Bayesian decision support system for evaluating COVID-19 countermeasure strategies. Journal of the Operational Research Society, 1-13. Available at: http://arxiv.org/abs/2101.04774 (Accessed: 23 March 2022).
Utami, S. S. (2011). Peranan Sistem Informasi Manajemen Untuk Pengambilan Keputusan Pengusaha Kecil. Jurnal Ekonomi dan Kewirausahaan, 11(2), 142–150.
Wibowo, N., Zahro, H. Z., & Wahyuni, F. S. (2022). Determination of the Delivery of the Type of Covid-19 Vaccine Using the Simple Multi Attribute Tecnique (SMART) web-based method. JATI (Jurnal Mahasiswa Teknik Informatika), 6(1), 237-244.
Yang, L., Iwami, M., Chen, Y., Wu, M., & van Dam, K. H. (2022). Computational decision-support tools for urban design to improve resilience against COVID-19 and other infectious diseases: A systematic review. Progress in planning, 100657. doi:10.1016/j.progress.2022.100657.