Markov Decision Model of Emergency Medical Supply Scheduling in Public Health Emergencies of Infectious Diseases

Xiaojia Wang; Zhizhen Liang; Keyu Zhu

doi:10.2991/ijcis.d.210222.002

<Previous Article In Issue

Download article (PDF)

Next Article In Issue>

Volume 14, Issue 1, 2021, Pages 1155 - 1169

Markov Decision Model of Emergency Medical Supply Scheduling in Public Health Emergencies of Infectious Diseases

Authors

Xiaojia Wang, Zhizhen Liang, Keyu Zhu^*

Department of Information Management, School of Management, Hefei University of Technology, Hefei, Anhui, 230009, China

^*Corresponding author. Email: zhukeyu@hfut.edu.cn

Corresponding Author

Keyu Zhu

Received 30 September 2020, Accepted 31 January 2021, Available Online 12 March 2021.

DOI: 10.2991/ijcis.d.210222.002 How to use a DOI?
Keywords: Infectious disease public health emergencies; Emergency medical supplies; Material dispatch; Markov decision model
Abstract: In this paper, a Markov decision process (MDP) model was established to study emergency medical material scheduling strategies for public health emergencies such as COVID-19. Within the constraints of dispatchable supplies, the priority of each medical node complicates the problem of deciding which hospital node supplies to respond to. The model assumes that the probability of events in the initial time period is in line with the Poisson distribution and that the location and priority of each hospital node is known when the material demand is initiated. The priority of hospital nodes is divided into four categories: critical, urgent, priority, and routine. There are several patients with different priorities in a hospital node: critical illness, severe illness, and mild illness. The priority of the hospital node is determined by the overall situation of the hospital patients. The MDP model established in this paper gives how to dispatch limited emergency medical supplies in the dispatching center to make the service rate of the whole system the best. The efficiency of the dispatching center in responding to the material needs of the hospital node depends on the constraints of the number and response time of different priority patients at the node. The maximum effect iterative dynamic model was simulated by simulation experiment and compared with the simulation effect under general conditions, so as to observe whether the model improved the system service rate.
Copyright: © 2021 The Authors. Published by Atlantis Press B.V.
Open Access: This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

1. INTRODUCTION

Public health emergencies refer to the sudden occurrence of major infectious diseases, food and occupational poisoning outbreaks, and other events that seriously affect public health. The term “emergency medical supplies” refers to all kinds of medical supplies that the government and society need to take emergency measures to protect lives and carry out rescues in emergencies. Due to the uncertainty of public health emergencies and the technical requirements of emergency medical supplies, the sudden shortage of a large number of specific types of emergency medical supplies often brings great difficulties to epidemic prevention and treatment [1].

Therefore, dispatching and allocating limited emergency medical supplies under sudden public health events is a special emergency material dispatch problem. The dispatch scheme of emergency medical supplies has a significant impact on controlling the development situation of medical events. According to the Chinese government's emergency plan for public health emergencies, the government's rules follow the principle of on-demand dispatch and distribution in the allocation of emergency supplies. This principle is fair but considering that the sudden public health events of infectious diseases such as COVID-19 virus are characterized by rapid spread, a wide range of infections, and great difficulty in prevention and control [2]. This scheduling principle is not effective in controlling the spread of the disease. In addition, the consumption of emergency medical supplies in the early stage of public health emergencies of infectious diseases is substantial. In the distribution of emergency supplies after disasters, there is an imbalance between supply and demand [3] because most emergency supplies are nondurable goods. It is difficult to maintain many stocks for them before the occurrence of low-probability disasters [4]. A timely supply of medical supplies is the key to rescuing patients and controlling the epidemic. In this case, it is imperative to allocate the limited resources reasonably to achieve better results. There are many studies on the allocation of emergency supplies, focusing on different aspects. Overall, the model goals [5–8] generally consider the maximization of the demand satisfaction rate, the minimization of emergency time, and the minimization of cost. The primary purpose is to reduce the cost of rescue as much as possible. However, for public health emergencies, such as infectious diseases, the most important thing is to control the epidemic quickly and ensure people's lives to the greatest extent. Cost is not the core of our concern at this time. Making the best use of limited materials and controlling the epidemic's spread effectively is the most critical problem. Therefore, this paper regards service rate maximization as the realization goal of our model, and the principle of material scheduling is based on the maximization of system efficiency. In the existing research on emergency management, most scholars consider completing one-time scheduling of emergency materials after emergencies [9,10]. This scheduling scheme is reasonable in general material scheduling but unreasonable for emergency medical materials under public health emergencies. Since the demand and material type constantly changes according to the variation in the epidemic situation, this paper applies a Markov decision model to study the continuous scheduling of emergency medical materials under public health emergencies.

Markov decision processes (MDP) are the optimal decision process of stochastic dynamic systems based on Markov process theory. The Markov property refers to the property that the probability law of the future development of a stochastic process has nothing to do with the history before observation. Bandara et al. [11] studied the optimal scheduling strategy in an emergency medical service (EMS) system and developed an MDP model to dispatch ambulances to patients optimally. They use the single-valued method to transform the initially developed continuous-time MDP into the equivalent discrete-time MDP and correctly consider the optimal decision-making strategy of each discrete-time element. The optimal scheduling strategy in EMS systems is studied while paying attention to the urgency of emergency calls. Based on their research, this paper divides the continuous-time Markov decision model into an equivalent continuous discrete-time Markov decision model. According to the existing research experience, adopting a dynamic allocation model has apparent advantages in solving short-term shortages of consumption emergency materials. Keneall et al. [12] developed a MDP model to study the air military medical evacuation dispatch policy in a combat environment. They classified casualties into three priority levels: emergency, priority, and routine. Multiple casualties can occur in one casualty event, and the casualty event with the highest priority determines the priority of the casualty event. Thus, we divided the priority of patients in designated hospitals into three categories: critical illness, severe illness, and mild illness. At the same time, considering the priority of the designated hospital, the number of three types of patients, the required materials, and the position away from the dispatching center determine the hospital's priority. Simultaneously, they take the sum of the probability of Poisson distribution and the service rate of the region as the transition rate of the Markov decision model of the region. Based on them, we make improvements. Since the service rate in this paper is the service rate of the total system, the goal of the model is to maximize the system service rate. When the system service rate is poor, the epidemic situation will be more severe, and the number of hospitals that demand supplies will increase, so that the probability of the event will increase.

Based on the above research, this paper puts forward a Markov scheduling model, providing Markov decision-making scheduling for emergency medical materials under sudden public health events of infectious diseases and divides continuous MDP into equivalent discrete MDP. The model classifies patients and hospitals. By maximizing the system efficiency, the service rate of the system is enhanced. According to previous scholars, the transition probability is improved to be more in line with our research background.

2. LITERATURE REVIEW

In this section, we discuss the three most relevant literatures studied in this paper. They are (1) Emergent public health events, (2) Emergency material dispatch, and (3) Markov decision-making processes.

2.1. Emergent Public Health Event

The studies on public health emergencies are as follows: Zhu L et al. [13] approximately simulated the process of disaster diffusion by using infectious disease model. Aiming at major public health emergencies, Li Nan and others put forward the theory of “public opinion communication” with the government, media, and the public as the main body, and constructed the control mechanism of “public opinion communication” based on susceptible exposed infectious recovered (SEIR) infectious disease model. Liu et al. [15] discussed the location and personnel allocation of integrated temporary facilities for postdisaster humanitarian medical services, and provided an iterative method to obtain Pareto optimization. Yu Jiying et al. [16] gave the layout model of emergency service facilities in infectious public health emergencies. Liu et al. [17] study how to effectively isolate patients when public health events similar to infectious diseases occur, in order to further control the epidemic situation. Garza et al. [18] apply lean thinking and constraint theory in peacetime scenarios to improve the operational efficiency of EMSs. Syahrir and Vanany [19] built a model to predict the number of drugs that hospitals have to provide when public health outbreaks occur in Indonesia in order to ensure the supply of drugs. He and Liu [20] based on the spread characteristics of the epidemic, pay attention to the distribution of single-variety medical materials, predict the medical needs of each region, and apply linear programming method to promote allocation decision-making. Hu Xiaowei [21] in view of the unreasonable dispatching of emergency medical materials and the low transfer efficiency of distribution centers in COVID-19 epidemic prevention and control, Hu Xiaowei redesigned the dispatching and distribution system of urban emergency medical materials under major public health emergencies, and gave the classification method of emergency medical materials.

2.2. Emergency Material Scheduling

Tian Jun [22] describes the demand for emergency materials with the help of triangular fuzzy numbers in fuzzy mathematics, simulates the real dynamic road network traffic condition by using continuous speed time dependence function, and establishes a multi-objective mathematical model for dynamic scheduling of emergency materials distribution. By designing the particle swarm optimization algorithm, using the “discrete-continuous vector hybrid coding\” scheme and the weighted integrated fitness function guidance mechanism, combined with the continuously updated position and speed operation strategy, a fast and efficient algorithm for solving this kind of optimization model with the combination of discrete and continuous variables is established, which provides an effective and reliable method for the dynamic scheduling of material distribution under emergency conditions. Considering the problem of demand allocation and network allocation of emergency materials under fuzzy demand, Wang Hai-jun [23] establishes a network flow model aiming at minimizing the total distribution time. By using the gravity model algorithm and convex combination algorithm based on bilateral constraints, through the interactive iteration of the results of demand allocation and network flow allocation, the optimal demand allocation, path and network flow under the minimum total distribution time are obtained. Zhan Sha-lei et al. [24,25] use Bayesian analysis method to update the demand with historical experience data to establish the optimal material allocation time model. Guo Zixue [26] in order to improve the ability of rapid response to the demand for emergency materials, based on the characteristics of the emergency material scheduling problem, triangular fuzzy numbers are introduced to describe the uncertain attributes of emergency dispatching. The time minimization fuzzy optimization model of emergency material scheduling problem under triangular fuzzy information environment is established, and its equivalent fuzzy chance constrained programming model is given. Explore the deterministic transformation method of the model when the parameter is triangular fuzzy number, and verify the effectiveness of the transformation method through empirical case analysis. He Tilong [27] and others studied the scheduling problem of emergency materials from multiple rescue points to multiple demand points based on three kinds of road damage: feasible, repairable, and impassable. An optimal objective function is established, which takes into account the minimum total loading time and the lowest cost in the process of transportation, and the improved moth flaming swarm intelligent algorithm is used to solve the optimal emergency material scheduling scheme. Wang Hai-jun et al. [28] studied the dynamic supply of emergency materials for multimodal transport under the condition of uncertain supply and demand in the three-level emergency materials distribution network. In order to solve the problem of fairness and efficiency of material distribution in existing models, Zhou et al. [29] proposed a multi-objective dynamic scheduling model of emergency materials with matching supply and demand. Wang and Sun [30] also proposed a multi-stage dynamic scheduling model for emergency materials, but its research is different from the traditional method, using absolute shortage of materials to quantify fairness.

2.3. Markov Decision-Making Process

Zhan Sha-lei et al. [31] used MDP to establish a dynamic distribution model of emergency supplies for the dynamic distribution of emergency supplies in the environment of unbalanced supply and demand under typhoon disaster. According to the existing research experience, the dynamic distribution model has obvious advantages to solve the problem of short time supply shortage of expendable emergency supplies. Zhan Sha-lei et al. [31] for the dynamic allocation of emergency materials under typhoon disasters and the imbalance between supply and demand, the dynamic allocation model of emergency materials is established by using MDP. Regnier [32] successfully applied Markov decision-making method to weather forecast, hurricane path tracking, predisaster evacuation, and so on. Li Haonan [33] uses MDP to solve the problem of route selection in multi-mode traffic networks. Through a comprehensive analysis of the factors that affect travelers' travel choice, a path decision model based on Markov decision method is constructed, an algorithm is designed, and an example is given to verify the feasibility of the proposed model and algorithm. Yang Feng [34] in view of the situation that the demand for emergency rescue materials in urban emergencies changes with the evolution of the accident, the demand for emergency rescue materials is designed as MDP, and a dynamic material allocation strategy is proposed. The decision-making model of rescue material demand is constructed and then optimized by flower pollination algorithm. Deng Xiaoping [35] based on the longitudinal kinematic characteristics of the following vehicle and the pilot vehicle, the MDP of the vehicle following process is established. Combined with the minimum safe distance model, an efficient, comfortable, and safe vehicle following decision algorithm is designed. Liang Feng [36] aims to maximize the profit of hospital inspection equipment, establishes a finite time domain MDP model, and combines the dynamic programming theory to obtain the optimal reservation scheduling strategy of the system. Considering the different quality of service parameters between different mobile terminals, Ning et al. [37] regards the vertical handoff decision problem as a MDP, establishes an incentive function with the goal of maximizing the expected total return and minimizing the average handoff times, evaluates the service quality of each link, and obtains a stable and deterministic handoff decision strategy. Talluri and Van Ryzin [38] and others take customer selection behavior as the research background, construct the MDP model, take the nested allocation strategy as the optimal strategy, and prove that the model and the estimation process are effective. Schütz [39] aiming at the problem of examination resource allocation among different types of patients, a continuous-time MDP, is established to consider the randomness of equipment service time and the unpunctual factors of some patients, and the optimization goal is to obtain the maximum benefit for the hospital, which is solved by the method of approximate dynamic programming. Zhuang and Li [40] uses the MDP model to study how to distribute multiple examination equipment among the three types of patients in order to maximize benefits on the day of service.

3. MDP MODEL OF EMERGENCY MEDICAL SUPPLIES DISPATCHING

In this section, the MDP model and the required parameters and related model components are described. Finally, the optimal equation of the MDP for medical material scheduling in this paper is given.

3.1. Model Description

The materials that can be raised in a certain period under sudden public health events are limited; that is, the materials that can be dispatched by the model in a specific time are limited. According to the time required for dispatching materials, the system time is discretized and divided into segments with time length D, which are represented by Dn. At this time, the material demand in Dn is processed in Dn+1. Nodes that did not respond in the previous time accumulate to respond in the later period.

A specific amount of time is needed for the system to receive and respond to the requirements of a node hospital, which is defined as the response time Ti. The response process can be disassembled into the following key links: ① the scheduling time of personnel and materials D; ② the transportation time of the materials Tij; ③ the potential time delay Tε; and ④ the unloading distribution time Tm.

Ti=D+Tij+Tε+Tm

After the dispatching center responds to the nodes with requirements, the transport team returns to the dispatching center to complete a service. TI Represents service time and Ei return time.

TI=Ti+Ei

3.2. Model Formulation

This section introduces the MDP model formula used to determine the emergency medical material dispatching strategy under the public health emergencies of infectious diseases. The MDP model is designed to determine how to conduct material scheduling in the case of limited materials for requests in a given node network to maximize the response efficiency of the node network. We make it a general dispatching principle that the dispatching center distributes the material needs of nodes according to the order in which material requests are made.

For the model in this article, the following parameters need to be provided.

I=1,2,…,i is the node hospital in the system, where i<∞.

αiDn represents the node hospital that initiates the material request at time Dn in the system, αiDn⊆I.

βiDn represents the node hospital that initiates the material request according to ρi at time Dn in the system, βiDn⊆αiDn.

γiDn represents the node hospital that initiates the material request at time Dn in the system and is responded, γiDn⊆αiDn.

dij stands for the distance between node i and node j.

MiD stands for the total amount of materials required by node i.

MD stands for the materials that can be dispatched by the dispatching center within D period of time.

ρi is the probability of occurrence of node hospital i events in the system, which is obtained by comprehensive consideration of poisson's probability λi and the efficiency of the node.

h represents the classification of patients, h∈1,2,3, h=3 means that the patient is critically ill, h=2 means that the patient is severe case, h=1 means that the patient is mildly ill.

μi is the priority of each node hospital, μ=1,μ=2,μ=3,μ=4, corresponding to level 1, level 2, level 3, and level 4 responses respectively.

gμDn is the proportion of node hospitals with priority μ in hospitals at the moment Dn, so ∑μ=14gμDn=1. The setting of response level is divided by reference to the setting of response level in the emergency plan of public health emergencies published in the region, and the judgment of response level is based on quantitative and qualitative methods.

ψihDn dispatch center is the immediate effect obtained on patients with different priority within node hospital i, hϵ0,1,2,3.

φiDn is the total efficiency of material request response proposed by the dispatching center for node hospital i, which is adjusted by time A. The model will calculate the node total efficiency of the material demand proposed by Dn in the period, and select the node of the expected service.

The model assumes whether the reaction is based on the response level of the node, the material scheduling situation, and the distance between the node's hospital and the dispatching center. We have considered that the assumption of an exponential distribution of node event arrival probability is unreasonable. Computational experiments by Jarvis [41] show that the behavior of the system we are modeling is relatively insensitive to the shape of the service time distribution. Gross and Harris [42] also provide well-known insensitive results. McLay and Mayorga [43] performed simulation analysis to compare the use of exponentially distributed service times with more realistic service times. They found that the assumption of index service time did not significantly affect the optimal strategy. Considering that our model is quite different from theirs, our model has a feedback mechanism; that is, the response effect in the last period affect the event arrival probability of the system in the next period, so we include the response effect, namely, the service efficiency, in the calculation of the event arrival probability. The service rate refers to the ratio between the efficiency obtained by the system in Dn and the maximum efficiency of the system at this time. Then, the optimal efficiency is the ratio between the efficiency obtained by the model in Dn and the maximum efficiency of the system at this time, and the general efficiency is the ratio between the efficiency obtained in Dn according to the general scheduling principle and the maximum efficiency of the system at this time.

The MDP model components are described as follows:

State-space: S=S1×S2×⋯×Sm, which represents the state space of the system, and SiD represents the state of node ai at time D. Considering that a hospital with response level 4 is responded to, we obtain its state, SiD=4,1.

State-space table:

State	Setting
μ	{1,2,3,4}

Action space: The action of whether to respond is indicated by νi, ν=1 corresponding to a response and ν=0 means lack of response. The decision made by the current system is to decide which node hospitals to respond to after receiving a request from a node hospital in the network within the period Dn. Let BsiDn represent the set of nodes i that are in state S in period Dn and propose material requirements.

The model allows nodes that have not been processed in the previous period to join in this stage. aiDn+1 indicates the node to be responded to within the period; then, aiDn+1=αiDn+1+βiDn. We set the response probability of the hospital with the highest priority at 1.

State transition probability matrix

Pi is the transition probability matrix related to νi and the size is 4×4. Pis represents the transition probability of the node from state s to each state in state-space S under given conditions, whether the node hospital responds to it at priority μ, and the transition to the probability of other priorities is a 1×4 vector.

Pi((m|s,νi)) is an element in Pi, which represents the transition probability of the node hospital from state s to state m under action νi.

The Ph patient transfers to another priority state transition matrix after being rescued, and the size is 4×4. Our model assumes that the patient's condition does not deteriorate after receiving assistance and may remain as it is or shift to a lower priority.

Phh is the transition probability of the patient from priority h to other priorities when the node responds, and it is a 1×4 vector. Ph((H|h) is an element in Pi, representing the transition probability of the patient from priority h to priority H.

Ph=Ph0Ph1Ph2Ph3 =Ph((0|0)Ph((1|0)Ph((0|1)Pi((1|1)Ph((2|0)Pi((3|0)Pi((2|1)Pi((3|1)Pi((0|2)Pi((1|2)Pi((0|3)Pi((1|3)Pi((2|2)Pi((3|2)Pi((2|3)Pi((3|3)

Priority: The hospital response level μi is obtained by providing a comprehensive evaluation.

μi=fuQ,uh+12

uQ is the priority of expert evaluation, uQ=∑μiDn/Q. We take the number of different types of patients in the hospital, the distribution of materials, the distance between the hospital and the Red Cross Society, and other related statistical data as a reference to make an expert score table for Q experts to review and each expert gives the response level μiDn of the node in time Dn, and then carries out a weighted average to get the priority level of each hospital. uh=ghh, gh represents the proportion of different types of patients in the whole node hospital, h is the classification of patients, and h∈0,1,2,3, fuQ,uh is a function that combines and weights uQ and uh.

Efficiency: When the dispatching center responds to the material request put forward by node αi with response level μi in the network within a period Dn, the system obtains a service effect φihDn. The effect depends on the number of patients in different categories, the distribution of materials, and the distance between the hospital and the Red Cross Society.

When the material demand proposed by node ai is responded to by the dispatching center, an effect is immediately obtained for patients with different priority h, defined as ψh, ∑h=13ψh=1. We define ψihDn as the immediate effect of node i in patients of various priorities.

ψihDn=∑h=13cμqighψh

We define ψk as the effect of treating a single patient, kϵ1,2,3,4,5,6, which represent the effect of converting h from 3 to 2, h from 2 to 1, h from 1 to 0, h from 3 to 1, h from 2 to 0, h from 3 to 0, so ∑k=16ψk=1ψ4=ψ1+ψ2,ψ5=ψ2+ψ3,ψ6=1.

We add a penalty item to the effect of each node to ensure that some node hospitals in the system do not turn into worse situations. When the total time from node ai's request to the network's response exceeds A, we add a penalty factor to punish the efficiency generated by the dispatching center's response to node ai's request. At this time, the efficiency that the network can obtain at node ai is also the maximum efficiency φiDn that the system can obtain at ai.

φiDn=ψihDn−σLTi>A,σ<∞

LTi≤Ai is an indicator variable. When the condition Ti>A is achieved, or when the response time exceeds A, the value is 1. When the response time is within A, the value is 0. σ is a penalty factor for efficiency and is a sufficiently large positive number.

Transition state: ρi is the event arrival probability of node i in the network. In the general model, the probability of event arrival obeys a Poisson distribution; that is, the probability of event arrival at node ai is the Poisson probability λi. Considering that our model has a feedback mechanism, the service rate has a significant impact on the development of events. Therefore, we revise the utilization service rate φ of λi and obtain the revised ρi. Of course, when the system response effect is good, the service rate is high, and the probability of system events in the next period should be reduced.

ρi=λD=1λi/φD>1

Optimality equation

JDn=ρigμDnmaxφiDn+∑∑k=161ψhψkPhPiJDn+1

φiDn=∑h=13ψihDn−σLTi>A,σ<∞

i∈γiDn

∑Mi≤MD

4. DATA SIMULATION EXPERIMENT

In this section, we apply the MDP model developed in the previous section to Wuhan city under lockdown management due to the COVID-19 epidemic.

4.1. Model Parameters

We set up an application scenario for the model and provided an emergency medical supply scheduling scheme for the designated hospitals that treated patients in Wuhan, closed during the novel coronavirus pneumonia epidemic. City closure management means that to do an excellent job in preventing and controlling pneumonia in novel coronavirus and effectively cut off the route of virus transmission. Since 10:00 on January 23, the city bus, subway, ferry, and long-distance passenger transport in Wuhan have been suspended. Without special reasons, citizens cannot leave Wuhan, and there is a temporary closure of the airport and railway station from Han. According to the government documents of Wuhan Municipal Government and the epidemic prevention and control department, during the epidemic prevention and control period, designated hospitals in Wuhan mainly treated patients with novel coronavirus were divided into five batches, among which the fourth and fifth batches were specially treated for suspected cases transferred from the previous three batches of designated hospitals. Therefore, the system only carries out simulation on 24 hospitals in the first three batches. On January 27, the press conference of the Wuhan epidemic situation said that fundraising was unified and centralized, and donations were only accepted through provincial and municipal Red Cross Societies. Therefore, the Wuhan Red Cross Society was taken as the systematic material dispatching center.

Twenty-four designated hospitals are numbered, and the numbering sequence is listed in Table 1 below.

Hospital	Wuhan Hankou hospital	Wuhan red cross hospital	Wuhan Seventh Hospital	Wuhan No.4 hospital west yard area	Wuhan Ninth Hospital	Wuhan Wuchang hospital
i	1	2	3	4	5	6
Hospital	Wuhan No.5 Hospital	Central hospital of wuhan Houhu Campus	Wuhan No.3 Hospital Guanggu Campus	Wuhan WISCO Second Hospital	Huazhong University of Science and Technology Affiliated tongji hospital Sino-French New City Campus	Wuhan union medical college hospital west area
i	7	8	9	10	11	12
Hospital	Hubei provincial people's hospital east yard	Hubei Provincial Hospital of Integrated Traditional Chinese and Western Medicine	Tianyou Hospital Affiliated to Wuhan University of Science and Technology	Wuhan No.6 Hospital	Wuhan traditional Chinese medicine hospital hanyang branch	Wuhan Zijing hospital
i	13	14	15	16	17	18
Hospital	Hubei liuqier combination of Chinese traditional and western medicine orthopedics hospital	Wuhan Xinzhou district traditional Chinese medicine hospital	Wuhan caidian district Maternal and Child Health Hospital	Wuhan huangpi district traditional Chinese medicine hospital	Wuhan qiaoya boai recovery hospital	Wuhan hannan district traditional Chinese medicine hospital
i	19	20	21	22	23	24

Table 1

Hospital number table.

According to Baidu Map navigation, the shortest driving distance between each designated hospital and the Red Cross Society and the time required for driving at this time are obtained, as displayed in Table 2. As the Wuhan municipal government has carried out road control in the city, it is not affected by traffic factors (traffic lights and jams) in general, so we take the highest driving speed in the shortest route as the simulated average driving speed.

i	1	2	3	4	5	6	7	8	9	10	11	12
dij/km	5.7	5.3	9.4	5.1	11.9	7.7	7.7	7.6	19	24.7	20.8	19.6
Tij/min	18	18	22	27	27	18	22	24	47	61	49	53
v	0.32	0.29	0.43	0.19	0.32	0.43	0.35	0.317	0.4	0.4	0.42449	0.37
MinTij/min	9	8	14	8	18	12	12	11.97	28	36	31	29

	13	14	15	16	17	18	19	20	21	22	23	24

dij/km	27.8	4.8	11.6	3.3	13.9	5.9	12.9	65.3	30.1	37.4	31.9	43.5
Tij/min	63	17	28	11	34	14	35	95	65	56	72	68
v	0.44	0.2824	0.32	0.32	0.41	0.42	0.37	0.69	0.32	0.67	0.44	0.64
MinTij/min	41	6.9565	17	5	21	9	19	95	44	55	79	63

Table 2

Hospital distance table.

According to official statistics, as of 24:00 on February 10, 2020, Hubei Province reported 31,728 cases of pneumonia in COVID-19, including 18,454 cases in Wuhan, and 974 cases died in the province, with a fatality rate of 3.07%, including 748 cases in Wuhan with a fatality rate of 4.05%. According to the relevant news reports during the epidemic, when all sectors of society donate materials to the Wuhan Red Cross Society, the unloading time of all kinds of materials is usually computed by tons: the unloading time per unit of materials is accumulated, and the unloading time per unit of materials is 10 minutes. The error time is controlled within 0–60 minutes, and random error is made for each point by Python. As the number of patients and materials mentioned above are too large and the scales are not uniform, the data are uniformly processed, the number of people is equivalent to 0–50, and the number of materials is equivalent to a number in m. The materials needed by each designated hospital are within 0–10 m, and the number of materials that can be dispatched within d is 50 m. Scheduling time d is set to 360 minutes (6 hours). Each designated hospital's priority calculation process is not detailed in this section but is directly given in the table.

State transition matrix: The priority of the state transition matrix of the Pi data set is made by three batches of designated hospitals in Wuhan city of Wuhan city that have been reported in the receiving medical supplies of the Red Cross and the social from all walks of life aid after the treatment of the related news, and Wuhan municipal government published daily by different hospital patient data given by the comprehensive analysis of data.

Designated hospital state transition matrix priority:

Pi=Pi1Pi2Pi3Pi4=0.950.05000.150.850.0500.050.150.750.050.050.10.250.7

The patient state transition matrix priority:

Ph=Ph0Ph1Ph2Ph3=00000.70.3000.20.50.300.10.20.40.3

Different priorities of patient treatment immediately affect the following equation:

ψ1=0.2,ψ2=0.3,ψ3=0.5

Different types of patient treatment effects:

ψ1=0.5,ψ2=0.3,ψ3=0.2,ψ4=0.8,ψ5=0.5,ψ6=1

Effect of different priorities:

c1=1(μ=1),c2=2μ=2,c3=3μ=3,c4=4μ=4

The penalty time in the system is set to 1590 minutes (26.5 hours). From the state transition matrix, the probability of maintaining the original priority is the highest when the hospital that puts forward the urgent medical supplies demand within D is responded, thus, the situation does not noticeably deteriorate if it is not responded within D. Considering the extremely infectious characteristics of the novel coronavirus, and according to relevant news reports, during the epidemic prevention and control period, many hospitals were infected due to the lack of emergency medical materials such as masks and protective clothing, greatly reducing the rescue efficiency of the hospital. We assume that the emergency medical supply-demand of a designated hospital in two consecutive days has never been met, and the treatment situation becomes terrible. The calculation of specific data is based on the calculation equation of response time:

Ti=D+Tij+Tε+Tm

Scheduling time is 360 minutes, maximum driving time is 95 minutes, maximum error time is 60 minutes, and maximum unloading time is 100 minutes, so the longest response time of two consecutive periods is 1230 minutes (28.5 hours), and the demands put forward in the previous stage is processed in the next stage, so A = 1590 minutes (26.5 hours).

4.2. Simulation Results and Optimal Strategy

This section only presents the simulation data of four time periods D and the scheduling results of the model. The number of designated hospitals that put forward material demand in period D1 is calculated by using ρi after being randomly selected four times by Python. The specifically designated hospitals and the order in which they put forward material requirements are randomly selected.

i	8	22	2	1	7	4	21	14	15	6
order	1	2	3	4	5	6	7	8	9	10
μi	3	2	2	1	3	2	1	3	4	2
Mi	5	9	4	2	7	5	3	8	9	4
h = 1	43	29	31	11	6	19	20	10	19	15
h = 2	25	5	11	7	9	35	16	16	45	16
h = 3	7	24	7	3	44	2	2	32	37	21

Table 3

Simulation data of D1.

The rescue efficiency, total expected efficiency, and expected response time of 10 designated hospitals on different patients were calculated. As shown in Table 4, the red mark is the expected effect of the designated hospital that has not responded under the maximum efficiency. In contrast, the blue mark is the designated hospital's expected efficiency that has not responded under general efficiency. The service rate under optimal efficiency is defined as φin, and the service rate for general efficiency is φn:

i	8	22	2	1	7	4	21	14	15	6
μi	3	2	2	1	3	2	1	3	4	2
φi	58.8	38.6	26	5.8	77.7	30.6	9.8	68.4	143	36.6
φe	46.5	32.7	20.6	4.67	70	24.7	7.72	60.36	124	31.5
φ	105	71.3	46.6	10.5	148	55.3	17.5	128.8	267	68.1
φm	21.05	7.92	11.66	5.24	21.10	11.05	5.84	16.10	29.66	17.03

Table 4

Efficiency of D1.

Based on the optimality equation:

JDn=ρigμDnmax∑φihDn+∑1ψhψkPhPiJDn+1

In general, the efficiency is 582.89, the materials used and 47 m, the remaining 3 m, φ1=0.635.

Choice for designated hospitals: 8, 22, 2, 1, 7, 4, 6, 14, 21

The system had a maximum efficiency of 917.93, an optimal efficiency of 860.81, φi1=0.938, and the following materials were employed:

System choice for designated hospitals, numbers 4, 6, 7, 8, 14, 15, 21, and 22

Two kinds of schemes under the expected response time are calculated, and the response time limit after A comparison shows that the designated hospital's response time was not beyond A. Thus, the system does not penalize the expected efficiency achieved at each designated hospital. The efficiency obtained at this time is the optimal efficiency of the system.

i	8	22	2	1	7	4	21	14	15	6
D	360	360	360	360	360	360	360	360	360	360
Tij	12	55	8	9	12	8	44	7	17	12
Tε	53	17	26	13	36	55	46	28	14	35
Tm	50	20	70	20	70	50	30	80	90	40
Ti	475	452	464	402	478	473	480	475	481	447

Table 5

Response time of D1.

D2 simulation data, because of the D1's φ1 and φi1 are 0.635 and 0.938, respectively. According to ρi=λiD=1λi/φD>1 calculate D2 phase, under the general efficiency, the number of designated hospitals that demand materials is 12, while under the optimal efficiency, the number of designated hospitals that demand materials is 9. The data shown in Tables 6 and 7 are for other related simulations. We abbreviate the optimal and general efficiency as optimal Efficiency (QE) and general Efficiency (GE), respectively.

i	15	10	18	17	3	11	14	5	19	7	1	6	16
order	1	1	2	3	4	5	6	7	8	9	10	11	12
μi	4	2	1	3	2	2	4	4	2	3	3	1	2
Mi	9	6	2	6	6	2	9	5	2	9	8	3	9
h = 1	19	26	22	2	15	12	49	44	13	14	45	18	46
h = 2	45	5	35	10	41	28	26	28	39	1	28	39	28
h = 3	37	37	9	44	25	29	39	43	17	42	26	4	17

Table 6

GE Simulation data of D2.

i	2	1	3	22	4	13	10	15	8	12	16
order	1	1	1	2	3	4	5	6	7	8	9
μi	2	1	3	4	2	2	1	2	1	2	1
Mi	4	2	6	9	6	7	3	4	5	3	3
h = 1	31	11	42	48	10	22	15	31	12	41	24
h = 2	11	7	15	29	11	17	10	9	16	13	2
h = 3	7	3	48	44	45	40	26	38	35	15	12

Table 7

OE Simulation data of D2.

Points that have not responded under different internal efficiencies of D1 are incorporated into the above table, in which the red marks are designated hospitals that have not responded under the optimal efficiency of the system, and the blue marks are designated hospitals that have not responded under the general efficiency of D1. By default, the demand arrival order of these three hospitals is the first in D2.

i	15	10	18	17	3	11	14	5	19	7	1	6	16
μi	4	2	1	3	2	2	4	4	2	3	3	1	2
φi	143	50.4	19.4	76.2	55.6	50.6	148	154.8	45.6	72.3	91.2	17.3	52.2
φe	123.72	43.82	15.97	69.06	47.70	44.04	125.20	131.76	38.78	64.59	75.78	14.11	42.52
φ	266.92	94.22	35.37	145.26	103.30	94.64	273.60	286.56	84.38	136.89	166.98	31.41	94.72
φm	29.66	15.70	17.69	24.21	17.22	47.32	30.40	57.31	42.19	15.21	20.87	10.47	10.52

Table 8

GE of D2.

The current material in D2 is 53 m, and the remaining material in D1 is 3 m, so the general efficiency of D2 is 1415.66. At this time, the maximum efficiency of the system is 1814.25. At 50 m, φ2=0.78. The materials used are 50 m and the remaining 3 m.

The designated hospitals are numbered 15, 10, 18, 17, 3, 11, 14, 5, 19, and 6.

i	2	1	3	22	4	13	10	15	8	12	16
μi	2	1	3	4	2	2	1	2	1	2	1
φi	26	5.8	111	161	55.6	59	19	55.8	24.7	39.2	11.4
φe	20.62	4.67	95.13	136.84	49.70	51.46	16.56	48.14	21.78	31.78	9.38
φ	46.62	10.47	205.83	298.04	105.30	110.46	35.56	103.94	46.48	70.98	20.78
φm	11.66	5.24	34.31	33.12	17.55	15.78	11.85	25.99	9.30	23.66	6.93

Table 9

OE of D2.

The maximum efficiency of the system in D2 is 1054.46, and the optimal efficiency is 1043.99, φi2=0.99. At this time, all materials are used. The designated hospitals selected by the service system are numbered 2, 3, 22, 4, 13, 10, 15, 8, 12, and 16.

i	15	10	18	17	3	11	14	5	19	7	1	6	16
D	720	360	360	360	360	360	360	360	360	360	360	360	360
Tij	17	36	9	21	14	31	7	18	19	12	9	12	5
Tε	90	60	20	60	20	20	50	50	10	90	80	30	30
Tm	30	37	9	44	25	29	39	43	17	42	26	14	17
Ti	857	493	398	485	419	440	456	471	406	504	475	416	412

Table 10

GE response time of D2.

i	2	1	3	22	4	13	10	15	8	12	16
D	720	720	360	360	360	360	360	360	360	360	360
Tij	8	9	31	55	18	29	12	36	9	29	12
Tε	70	20	60	30	10	70	30	40	50	90	40
Tm	5	12	14	8	8	41	36	17	12	15	5
Ti	803	761	465	453	396	500	438	453	431	494	417

Table 11

OE response time of D2.

The expected response time of the two schemes is not more than A, and the peak efficiency that the system can obtain is the optimal efficiency of the system.

D3 simulation data According to the transition rate ρi adjusted by φ2=0.78, φi2=0.99, the number of designated hospitals proposing material requirements under the general efficiency is 11, and the number of designated hospitals proposing material requirements under the optimal efficiency is nine. Other relevant data obtained by simulation are as follows.

i	7	1	16	6	14	13	12	9	4	21	18	8	20	19
order	1	1	1	1	2	3	4	5	6	7	8	9	10	11
μi	3	3	2	4	4	1	2	3	1	2	1	4	3	2
Mi	9	8	9	4	5	2	4	4	1	4	3	6	6	3
h = 1	14	45	46	2	5	10	26	17	6	40	8	33	11	26
h = 2	1	28	28	50	40	6	18	30	10	7	15	25	49	43
h = 3	42	26	17	41	50	8	42	42	14	9	12	50	30	11

Table 12

GE Simulation data of D3.

The points that have not responded under different efficiencies of D2 are incorporated into Table 13, and it is assumed that the demand arrival order of these four hospitals ranks first in D2.

i	1	4	9	16	10	3	2	12	17	7
order	1	1	2	3	4	5	6	7	8	9
μi	1	2	4	4	3	1	2	4	3	1
Mi	2	7	9	8	5	2	5	8	6	3
h = 1	11	31	25	38	38	25	20	19	36	2
h = 2	7	10	39	33	13	3	41	30	7	7
h = 3	3	27	47	41	39	6	11	48	42	13

Table 13

OE Simulation data of D3.

i	7	1	16	6	14	13	12	9	4	21	18	8	20	19
μi	3	3	2	4	4	1	2	3	1	2	1	4	3	2
φi	72.3	91.2	52.2	144	152	7.8	63.2	100.2	11.2	29.2	12.1	156	95.7	47.2
φe	64.59	75.78	42.52	126.56	134.80	6.58	54.92	87.60	9.78	22.98	10.39	135.48	82.77	38.90
φ	136.89	166.98	94.72	270.16	286.80	14.38	118.12	187.80	20.98	52.18	22.49	291.88	178.47	86.10
φm	15.21	20.87	10.52	67.54	57.36	7.19	29.53	46.95	20.98	13.05	7.50	48.65	29.75	28.70

Table 14

GE of D3.

Materials in D3 are 53 m, including 3 m materials in D1 balance, so the general efficiency of D3 is 1371.15. At this time, the maximum efficiency of the system is 1814.25, φ3=0.728 and 53 m materials are utilized.

The designated hospitals selected for service are numbered 7, 1, 16, 6, 14, 13, 12, 9, 4, 21, and 18.

i	1	4	9	16	10	3	2	12	17	7
μi	1	2	4	4	3	1	2	4	3	1
φi	5.8	45.4	161	152	93	8.9	43.6	147.2	90.9	9
φe	4.67	38.52	139.48	129.72	79.53	7.01	36.22	128.96	78.33	8.01
φ	10.47	83.92	300.28	281.72	172.53	15.91	79.82	276.16	169.23	17.01
φm	5.24	11.99	33.36	35.22	34.51	7.96	15.96	34.52	28.21	5.67

Table 15

OE of D3.

In D3, the maximum efficiency of the system is 1407.05, the optimal efficiency is 1380.02, and φi3=0.98, when all materials are applied.

The designated hospitals selected by the service system are numbered 4, 9, 16, 10, 3, 2, 12, 17, and 7, respectively.

i	7	1	16	6	14	13	12	9	4	21	18	8	20	19
D	720	720	720	360	360	360	360	360	360	360	360	360	360	360
Tij	12	9	5	12	7	5	29	28	8	44	9	12	95	19
Tε	90	80	30	90	90	20	80	40	50	40	30	60	60	70
Tm	38	27	25	55	6	29	26	38	49	42	19	43	28	53
Ti	860	836	780	517	463	414	495	466	467	486	418	475	543	502

Table 16

GE response time of D3.

i	1	4	9	16	10	3	2	12	17	7
D	1080	360	360	360	360	360	360	360	360	360
Tij	9	8	28	5	36	14	8	29	21	12
Tε	20	70	90	80	50	20	50	80	60	30
Tm	26	27	27	41	39	6	11	48	31	13
Ti	1135	465	505	486	485	400	429	517	472	415

Table 17

OE response time of D3.

The expected response time of the two schemes is not more than A, and the ideal efficiency that the system can obtain is the optimal efficiency of the system.

According to the transition rate ρi adjusted by φ3=0.728, φi3=0.98. D4 computed that the number of designated hospitals that proposed material requirements under general efficiency is 11, and the number of designated hospitals that proposed material requirements under optimal efficiency is 8. The relevant data are described as follows.

i	8	20	19	5	23	10	9	14	1	17	21	18	13	4
order	1	1	1	1	2	3	4	5	6	7	8	9	10	11
μi	4	3	2	1	2	2	2	2	4	2	3	2	2	1
Mi	6	6	3	3	4	4	5	4	8	2	8	4	5	3
h = 1	33	11	26	6	30	32	15	28	32	3	42	50	8	7
h = 2	25	49	43	20	13	23	22	25	45	28	11	11	34	37
h = 3	50	30	11	3	14	30	19	8	47	23	37	1	18	2

Table 18

GE Simulation data of D4.

However, the unresponsive points in D3 with different efficiencies are incorporated into Table 19, and by default, the demand arrival order of these five hospitals ranks first in D4.

i	1	7	8	12	4	18	17	3	9	21
order	1	1	1	2	3	4	5	6	7	8
μi	1	1	3	3	3	2	1	2	4	3
Mi	2	3	6	5	6	4	4	5	9	8
h = 1	11	2	28	26	50	3	9	28	20	21
h = 2	7	7	22	34	17	12	8	17	32	33
h = 3	3	13	30	28	20	34	11	37	41	39

Table 19

OE Simulation data of D4.

i	8	20	19	5	23	10	9	14	1	17	21	18	13	4
μi	4	3	2	1	2	2	2	2	4	2	3	2	2	1
φi	156	95.7	47.2	8.7	33.8	56.6	38.2	34.2	174	41	90.6	27.6	41.6	13.5
φe	135.48	82.77	38.90	7.22	27.78	48.06	32.68	27.70	149.40	36.00	76.95	20.42	35.80	11.15
φ	291.88	178.47	86.10	15.92	61.58	104.66	70.88	61.90	323.00	77.00	167.55	48.02	77.40	24.65
φm	48.65	29.75	28.70	5.31	15.40	26.17	14.18	15.48	40.38	38.50	20.94	12.01	15.48	8.22

Table 20

GE of D4.

The general efficiency of D4 is 1319.41. At this time, the maximum efficiency of the system is 1589.01, φ4=0.830, and the material usage is 49 m.

The designated hospitals selected for service are numbered 8, 20, 19, 5, 23, 10, 9, 14, 1, 17, and 18.

i	1	7	8	12	4	18	17	3	9	21
μi	1	1	3	3	3	2	1	2	4	3
φi	5.8	9	81.6	88.2	75.3	42.4	9.7	58.4	136	101
φe	4.67	8.01	69.66	75.06	61.35	38.12	8.32	50.38	118.64	87.39
φ	10.47	17.01	151.26	163.26	136.65	80.52	18.02	108.78	255.04	188.19
φm	5.24	5.67	25.21	32.65	22.78	20.13	4.51	21.76	28.34	23.52

Table 21

OE of D4.

The maximum efficiency of the system at D4 is 1129.20, and according to the principle of maximizing efficiency, the optimal efficiency is 1118.73. At this time, all materials were used, but the expected response time of unresponsive designated hospital 1 at this time was 1495 minutes. If it is decided whether to respond until D5, according to the optimal equation, the model penalizes the system efficiency and affects the overall effectiveness. Therefore, we respond to it at this stage. At this time, the optimal efficiency of the system is 1112.19, φi4=0.985, and 49 m of materials are used at this time, with a balance of 1 m to D5.

The designated hospitals selected by the service system are numbered 1, 8, 12, 4, 18, 17, 3, 9, and 21.

i	8	20	19	5	23	10	9	14	1	17	21	18	13	4
D	720	720	720	360	360	360	360	360	360	360	360	360	360	360
Tij	12	95	19	18	79	36	28	7	9	21	44	9	41	8
Tε	60	60	70	30	40	40	50	40	80	20	80	40	50	30
Tm	25	37	45	9	52	37	21	38	39	37	4	12	58	8
Ti	817	912	854	417	531	473	459	445	488	438	488	421	509	406

Table 22

GE response time of D4.

i	1	7	8	12	4	18	17	3	9	21
D	1440	720	360	360	360	360	360	360	360	360
Tij	9	12	28	5	21	14	8	29	21	12
Tε	20	30	60	50	60	40	40	50	90	80
Tm	26	35	10	30	44	12	33	42	36	7
Ti	1495	797	458	445	485	426	441	481	507	459

Table 23

OE response time of D4.

4.3. Discussion of Simulation Result

The result analysis shows that when the optimal strategy is applied to the medical material scheduling system, the response to the material request of the designated hospital is mainly determined by the priority of the designated hospital and the efficiency of the material unit. By comparing the simulation data of the four stages, it can be seen that when the optimal strategy is applied to the medical material scheduling system, the service rate of the system in the four stages is much higher than that in general. Under the optimal strategy, the service rate of the system in four stages is close to one, which basically meets the overall material demand of the system at the current stage, while the maximum service rate under the general strategy is only 0.83, as shown in Figure 1. Although there are designated hospitals under the optimal strategy, their response time is much longer than other designated hospitals. The model added a time constraint based on optimal efficiency to avoid the rapid deterioration of the epidemic situation in such designated hospitals due to the lack of timely services. In the fourth stage of the simulation experiment, we had this situation, and our optimal strategy solved it well.

We found that even though the optimal point strategy was adjusted to meet the time constraints, the service rate of the adjusted system did not decrease much before the adjustment, still reaching 0.985, proving. our model as a whole can significantly improve the service efficiency of the system.

Figures 2–4 refers to the incident probability under service rate adjustment. For each stage of the system with two different material scheduling strategies, the number of designated hospitals that the system needs to respond to is αi, the number of designated hospitals that present material requirements is βi, and the number of designated hospitals that the final system finally responds to under the optimal strategy and general strategy is γi. In Figures 5 and 6, the changing trend of the number of patients with three different priorities. Taken together, the system reduces the number of designated hospitals that demand medical supplies in the next phase, and the total number of patients in each phase in three priority categories is significantly reduced. The reduction of the total number of patients means that the development trend of the epidemic has been well controlled, which indicates that our model has a good effect on improving the service rate of the system and controlling the epidemic.

5. CONCLUSIONS

In recent years, large-scale public health emergencies such as infectious diseases have frequently occurred worldwide, seriously threatening people's lives and severely challenging the emergency management systems of all countries in the world. The timely supply of medical materials is essential to control the epidemic situation, among which emergency medical materials are the most important. Therefore, this paper proposes a Markov decision model for emergency medical material scheduling under public health emergencies such as infectious diseases, which discretizes the time of material scheduling continuously, describes the efficiency of medical material scheduling in a single period in detail, and finally obtains an optimality equation that maximizes the material efficiency of the whole system through iteration.

We applied the model to the scene of epidemic prevention and control in Wuhan and simulated the emergency medical material dispatching data of 24 designated hospitals in Wuhan under the condition of city closure management. According to the experimental data changes and results, our Markov decision model in the four stages dramatically improves the service rate of the emergency medical material scheduling system compared with the general material scheduling strategy. The total number of patients with three different priorities in each stage decreased significantly. Moreover, the number of designated hospitals that put forward material demand has also been considerably reduced, meaning our Markov decision model is effective in controlling the development of the epidemic in Wuhan and reducing the scale of infected people.

In the early stage of the epidemic outbreak, Wuhan closed the city promptly and controlled urban road traffic. City closure management has a remarkable effect in cutting off the virus's spread and curbing the epidemic's proliferation. Traffic control has brought great convenience to the government to allocate materials uniformly, reflected in the early stage of simulation data experiments. Our driving time is substantially lowered in the experiment, and there is no need to consider various traffic obstacles. This result shows that when similar public health emergencies such as infectious diseases occur, the government can take similar measures to curb the epidemic's spread. From the experimental results, the general material scheduling strategy cannot control the development of the epidemic situation in Wuhan, which shows that the material scheduling in such cases does not apply the principle of distribution according to needs but should adopt a dynamic scheduling strategy similar to the Markov decision model according to the development of events.

The main contributions of this paper are described as follows: we transform continuous MDP into equivalent continuous-discrete MDP. Our model prioritizes patients and hospitals, considers the state transitions of patients and hospitals at different stages, and gives state transition matrices, making the efficiency iteration of our model more detailed and reasonable. Based on previous scholars and the research content of this paper, we adjusted the utilization rate of service in the event occurrence probability (transition rate). There are also many shortcomings in this paper. First, the designated hospitals in Wuhan are put forward in five batches, and the situations in other provinces and cities are given in batches in accordance with those in Wuhan. This paper does not consider the problems of different batches between such hospitals, and the default system initially has the nodes of the first three batches of hospitals. We did not give the basis for the priority evaluation of the response of designated hospitals in detail but simply considered the number, distance, materials, and other factors of three patient types. Our simulation experiment is only a simple numerical simulation, and the simulation time stage is also less, and the amount of data is not large enough; We did not discuss the different kinds of medical supplies that were dispatched. These shortcomings provide directions for our future research, such as combining deep learning methods in the simulation stage of the model to make the results more accurate. Combined with the problem of hospitals opening in batches, we consider adding a certain number of designated hospitals in different stages of efficiency iteration. In the application of the model, it is used to dispatch and distribute complex medical materials. In summary, this paper proposes the Markov decision model of emergency medical supplies under the public health emergencies of infectious diseases, which is suitable for the outbreak of public emergencies of infectious diseases, without considering the restrictions of personnel circulation and vehicle traffic in a closed environment. This method can make more efficient use of emergency medical supplies, improve the system's service rate, and effectively control the development of the epidemic.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.

AUTHORS' CONTRIBUTIONS

Xiaojia Wang, Zhizhen Liang, Keyu Zhu contributed to the conception of the study; Zhizhen Liang performed simulation experiment; Xiaojia Wang contributed significantly to the model; Xiaojia Wang, Zhizhen Liang performed the data analyses and wrote the manuscript; Keyu Zhu helped perform the analysis with constructive discussions.

ACKNOWLEDGMENTS

This work was supported by a grant from the Key Disease of Diabetes Mellitus Study Center at the National Chinese Medicine Clinical Research Base, the China Scholarship Council, the National Natural Science Foundation of China Grant No. U2001201, 61876055, 71101041, and the National Statistics Research Projects Grant No. 2013LZ07, and National Steering Committee for Graduate Education of Chinese Medicine and Traditional Chinese Medicine Grant No. 20190723-FJ-B39.

REFERENCES

1.J. Yingmei and C. Minsheng, Discussion on supply chain of emergency medical supplies in public health emergencies, Jiangsu Health Adm., Vol. 531, 2020, pp. 1139-1143.

2.C. Huang, Y. Wang, and X. Li, Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China, Lancet, Vol. 10223, 2020, pp. 497-506.

3.J.B. Sheu, Post-disaster relief-service centralized logistics distribution with survivor resilience maximization, Transp. Res. Part B, Vol. 68, 2014, pp. 288-314.

4.W. Yi and L. Özdamar, A dynamic logistics coordination model for evacuation and support in disaster response activities, Eur. J. Oper. Res., Vol. 179, 2007, pp. 1177-1193.

5.S. Ying-hua, G. Yan, and D. Li-jing, Optimization of emergency materials allocation plan considering vehicle waiting, Control Decis., Vol. 34, 2019, pp. 2229-2236.

6.W. Li, Z. Xian-cheng, and Z. Zhi-xue, Integrated decision making of emergency vehicle allocation and emergency material distribution, J. Central South Univ. Sci. Technol., Vol. 49, 2018, pp. 2766-2775.

7.Z. Zhi-xia, T. Jun, and X. Han, Multi-objective robust optimization of emergency resource scheduling for large scale emergency, Ind. Saf. Environ. Protect., Vol. 43, 2017, pp. 1-4.

8.W. Yan-yan and S. Bai-qing, Dynamic multi-stage allocation model of emergency materials for multiple disaster sites, Chinese J. Manag. Sci., Vol. 27, 2019, pp. 138-147.

9.X. Xiong, F. Zhao, and Y. Wang, Research on the model and algorithm for multimodal distribution of emergency supplies after earthquake in the perspective of fairness, Math. Probl. Eng., Vol. 2019, 2019, pp. 1-12.

10.H. Xi-hao and Z. Ling, Research on robust optimization of emergency resource allocation in post-earthquake rescue network, J. Wuhan Univ. Technol. Inf. Manag. Eng., Vol. 41, 2019, pp. 560-566.

11.D. Bandara, M.E. Mayorga, and L.A. McLay, Optimal dispatching strategies for emergency vehicles to increase patient survivability, Int. J. Oper. Res., Vol. 15, 2012, pp. 195-214.

12.K. Keneally Sean, J. Robbins Matthew, and J. Lunday Brian, A Markov decision process model for the optimal dispatch of military medical evacuation assets, Health Care Manag. Sci., Vol. 19, 2016, pp. 111-129.

13.L. Zhu and J. Cao, Emergency resources allocation optimization under disaster spreading with fuzzy demand, J. Syst. Sci. Math. Sci., Vol. 34, 2014, pp. 663-673.

14.L. Nan and Y. Yang, Research on public opinion communication in the context of major public health emergencies retrospective and construction of control mechanism, Manag. Modernization, Vol. 5, 2020, pp. 95-98.

15.Y. Liu, N. Cui, and J. Zhang, Integrated temporary facility location and casualty allocation planning for post-disaster humanitarian medical service, Transp. Res. Part E Log. Transp. Rev., Vol. 128, 2019, pp. 1-16.

16.Y. Ying-ying, Analysis on allocation of emergency material of public health emergency with infection, Math. Pract. Theory, Vol. 45, 2015, pp. 77-82.

17.M. Liu, X. Xu, and J. Cao, Integrated planning for public health emergencies: a modified model for controlling H1N1 pandemic, J. Oper. Res. Soc., Vol. 71, 2019, pp. 748-761.

18.J.A. Garza-Reyes, B. Villarreal, and V. Kumar, A lean-TOC approach for improving Emergency Medical Services (EMS) transport and logistics operations, Int. J. Logist. Res. Appl., Vol. 22, 2019, pp. 253-272.

19.I. Syahrir and I. Vanany, Drug supplies planning in hospital for epidemic attack using SEIR model, J. Phys. Conf. Ser., Vol. 1179, 2019. 012150

20.Y. He and N. Liu, Methodology of emergency medical logistics for public health emergencies, Transp. Res. Part E Logist. Transp. Rev., Vol. 79, 2015, pp. 178-200.

21.H. Xiaowei, S. Lang, Y. Binyi, and W. Jian, Study on optimal dispatching of urban emergency medical materials under major public health emergencies, China J. Highway, Vol. 33, 2020, pp. 1-11.

22.J. Tian, W.Z. Ma, and Y.L. Wang, Emergency supplies distributing and vehicle routes programming based on particle swarm optimization, Syst. Eng. Theory Pract., Vol. 31, 2011, pp. 898-906.

23.H.J. Wang, B.H. Li, and K.K. Liu, Demand allocationand network flow assignment under emergency rescue circumstance, Syst. Eng. Theory Pract., Vol. 35, 2015, pp. 1457-1464.

24.S.L. Zhan, N. Liu, and S.F. Chen, Coordination between efficiency and equity in relief allocation problem via demand updates, Control Decis., Vol. 29, 2014, pp. 686-690.

25.S. Zhan and N. Liu, Determining the optimal decision time of relief allocation in response to disaster via relief demand updates, Int. J. Syst. Sci., Vol. 47, 2016, pp. 509-520.

26.G. Zixue, G. Liang, Z. Pei, and Y. Xiaohui, Fuzzy optimization model for minimizing the time of emergency material scheduling, Chinese J. Saf. Sci., Vol. 25, 2015, pp. 172-176.

27.H. Tilong and L. Wengao, Based on the improved moth flaming algorithm to solve the emergency material scheduling with multiple demand points, Minicomput. Syst., Vol. 541, 2020, pp. 1334-1300.

28.W. Hai-jun, W. Jing, and M. Shi-hua, Decision-making for emergency materials dynamic dispatching based on fuzzy demand and supply, Chinese J. Manag. Sci., Vol. 22, 2014, pp. 55-64.

29.Y. Zhou, J. Liu, and Y. Zhang, A multi-objective evolutionary algorithm for multi-period dynamic emergency resource scheduling problems, Transp. Res. Part E. Logist. Transp. Rev., Vol. 99, 2017, pp. 77-95.

30.Y. Wang and B. Sun, A multi-objective allocation model for emergency resources that balance efficiency and fairness, Math. Probl. Eng., Vol. 2018, 2018, pp. 1-8.

31.Z. Sha-lei, F. Pei-hua, and L. Xiu-lin, Dynamic programming approach for relief goods allocation based on Markov decision, Control Decis., Vol. 33, 2018, pp. 1312-1318.

32.E. Regnier, Public evacuation decisions and hurricane track uncertainty, Manag. Sci., Vol. 54, 2008, pp. 16-28.

33.Y. Feng, Y. Chunming, and G. Dashuang, Rescue demand decision-making model based on accident evolution in urban emergencies and its optimal solution, Oper. Manag., Vol. 29, 2020, pp. 79-88.

34.D. Xiaoping, H. Jin, T. Guanghong, W. Binyang, and C. Tingting, Multi-objective vehicle following decision algorithm based on reinforcement learning, Control Decis., 2020, pp. 1-7.

35.L. Feng and X. Ping, Research on scheduling optimization method of medical examination appointment based on MDP and dynamic programming, Oper. Manag., Vol. 29, 2020, pp. 17-25.

36.L. Haonan, Research on Path Decision-making of Multi-mode Urban Traffic Network Optimization Based on Markov Decision Process, Beijing Jiaotong University, No.3 Shangyuan Village, Haidian District, Beijing, 2019.

37.Z. Ning, Q. Song, and Y. Liu, Markov-based vertical handoff decision algorithms in heterogeneous wireless networks, Comput. Electr. Eng., Vol. 40, 2014, pp. 456-472.

38.K. Talluri and G. Van Ryzin, Revenue management under a general discrete choice model of consumer behavior, Manag. Sci., Vol. 50, 2004, pp. 15-33.

39.H.J. Schütz and R. Kolisch, Approximate dynamic programming for capacity allocation in the service industry, Eur. J. Oper. Res., Vol. 218, 2012, pp. 239-250.

40.W.F. Zhuang and M.Z.F. Li, A new method of proving structural properties for certain class of stochastic dynamic control problems, Oper. Res. Lett., Vol. 38, 2010, pp. 462-467.

41.J.P Jarvis, JP Approximating the equilibrium behavior of multi-server loss systems, Manag Sci., Vol. 31, 1985, pp. 235-239.

42.C. Harris and D. Gross, Fundamentals of queueing theory, 3rd, Wiley, New York, 1998.

43.L.A. McLay, Mayorga ME A model for optimally dis-patching ambulances to emergency calls with classification errorsin patient priorities, IIE Trans., Vol. 45, 2013b, pp. 1-24.

<Previous Article In Issue

Download article (PDF)

Next Article In Issue>

Journal: International Journal of Computational Intelligence Systems
Volume-Issue: 14 - 1
Pages: 1155 - 1169
Publication Date: 2021/03/12
ISSN (Online): 1875-6883
ISSN (Print): 1875-6891
DOI: 10.2991/ijcis.d.210222.002 How to use a DOI?
Open Access: This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

ris enw bib

TY  - JOUR
AU  - Xiaojia Wang
AU  - Zhizhen Liang
AU  - Keyu Zhu
PY  - 2021
DA  - 2021/03/12
TI  - Markov Decision Model of Emergency Medical Supply Scheduling in Public Health Emergencies of Infectious Diseases
JO  - International Journal of Computational Intelligence Systems
SP  - 1155
EP  - 1169
VL  - 14
IS  - 1
SN  - 1875-6883
UR  - https://doi.org/10.2991/ijcis.d.210222.002
DO  - 10.2991/ijcis.d.210222.002
ID  - Wang2021
ER  -

download .riscopy to clipboard

i	8	22	2	1	7	4	21	14	15	6
D	360	360	360	360	360	360	360	360	360	360
Tij	12	55	8	9	12	8	44	7	17	12
Tε	53	17	26	13	36	55	46	28	14	35
Tm	50	20	70	20	70	50	30	80	90	40
Ti	475	452	464	402	478	473	480	475	481	447

i	15	10	18	17	3	11	14	5	19	7	1	6	16
D	720	360	360	360	360	360	360	360	360	360	360	360	360
Tij	17	36	9	21	14	31	7	18	19	12	9	12	5
Tε	90	60	20	60	20	20	50	50	10	90	80	30	30
Tm	30	37	9	44	25	29	39	43	17	42	26	14	17
Ti	857	493	398	485	419	440	456	471	406	504	475	416	412

i	2	1	3	22	4	13	10	15	8	12	16
D	720	720	360	360	360	360	360	360	360	360	360
Tij	8	9	31	55	18	29	12	36	9	29	12
Tε	70	20	60	30	10	70	30	40	50	90	40
Tm	5	12	14	8	8	41	36	17	12	15	5
Ti	803	761	465	453	396	500	438	453	431	494	417

i	7	1	16	6	14	13	12	9	4	21	18	8	20	19
D	720	720	720	360	360	360	360	360	360	360	360	360	360	360
Tij	12	9	5	12	7	5	29	28	8	44	9	12	95	19
Tε	90	80	30	90	90	20	80	40	50	40	30	60	60	70
Tm	38	27	25	55	6	29	26	38	49	42	19	43	28	53
Ti	860	836	780	517	463	414	495	466	467	486	418	475	543	502

i	8	20	19	5	23	10	9	14	1	17	21	18	13	4
D	720	720	720	360	360	360	360	360	360	360	360	360	360	360
Tij	12	95	19	18	79	36	28	7	9	21	44	9	41	8
Tε	60	60	70	30	40	40	50	40	80	20	80	40	50	30
Tm	25	37	45	9	52	37	21	38	39	37	4	12	58	8
Ti	817	912	854	417	531	473	459	445	488	438	488	421	509	406

i	8	22	2	1	7	4	21	14	15	6
D	360	360	360	360	360	360	360	360	360	360
Tij	12	55	8	9	12	8	44	7	17	12
Tε	53	17	26	13	36	55	46	28	14	35
Tm	50	20	70	20	70	50	30	80	90	40
Ti	475	452	464	402	478	473	480	475	481	447

i	15	10	18	17	3	11	14	5	19	7	1	6	16
D	720	360	360	360	360	360	360	360	360	360	360	360	360
Tij	17	36	9	21	14	31	7	18	19	12	9	12	5
Tε	90	60	20	60	20	20	50	50	10	90	80	30	30
Tm	30	37	9	44	25	29	39	43	17	42	26	14	17
Ti	857	493	398	485	419	440	456	471	406	504	475	416	412

i	2	1	3	22	4	13	10	15	8	12	16
D	720	720	360	360	360	360	360	360	360	360	360
Tij	8	9	31	55	18	29	12	36	9	29	12
Tε	70	20	60	30	10	70	30	40	50	90	40
Tm	5	12	14	8	8	41	36	17	12	15	5
Ti	803	761	465	453	396	500	438	453	431	494	417

i	7	1	16	6	14	13	12	9	4	21	18	8	20	19
D	720	720	720	360	360	360	360	360	360	360	360	360	360	360
Tij	12	9	5	12	7	5	29	28	8	44	9	12	95	19
Tε	90	80	30	90	90	20	80	40	50	40	30	60	60	70
Tm	38	27	25	55	6	29	26	38	49	42	19	43	28	53
Ti	860	836	780	517	463	414	495	466	467	486	418	475	543	502

i	8	20	19	5	23	10	9	14	1	17	21	18	13	4
D	720	720	720	360	360	360	360	360	360	360	360	360	360	360
Tij	12	95	19	18	79	36	28	7	9	21	44	9	41	8
Tε	60	60	70	30	40	40	50	40	80	20	80	40	50	30
Tm	25	37	45	9	52	37	21	38	39	37	4	12	58	8
Ti	817	912	854	417	531	473	459	445	488	438	488	421	509	406

i	8	22	2	1	7	4	21	14	15	6
D	360	360	360	360	360	360	360	360	360	360
Tij	12	55	8	9	12	8	44	7	17	12
Tε	53	17	26	13	36	55	46	28	14	35
Tm	50	20	70	20	70	50	30	80	90	40
Ti	475	452	464	402	478	473	480	475	481	447

i	15	10	18	17	3	11	14	5	19	7	1	6	16
D	720	360	360	360	360	360	360	360	360	360	360	360	360
Tij	17	36	9	21	14	31	7	18	19	12	9	12	5
Tε	90	60	20	60	20	20	50	50	10	90	80	30	30
Tm	30	37	9	44	25	29	39	43	17	42	26	14	17
Ti	857	493	398	485	419	440	456	471	406	504	475	416	412

i	2	1	3	22	4	13	10	15	8	12	16
D	720	720	360	360	360	360	360	360	360	360	360
Tij	8	9	31	55	18	29	12	36	9	29	12
Tε	70	20	60	30	10	70	30	40	50	90	40
Tm	5	12	14	8	8	41	36	17	12	15	5
Ti	803	761	465	453	396	500	438	453	431	494	417

i	7	1	16	6	14	13	12	9	4	21	18	8	20	19
D	720	720	720	360	360	360	360	360	360	360	360	360	360	360
Tij	12	9	5	12	7	5	29	28	8	44	9	12	95	19
Tε	90	80	30	90	90	20	80	40	50	40	30	60	60	70
Tm	38	27	25	55	6	29	26	38	49	42	19	43	28	53
Ti	860	836	780	517	463	414	495	466	467	486	418	475	543	502

i	8	20	19	5	23	10	9	14	1	17	21	18	13	4
D	720	720	720	360	360	360	360	360	360	360	360	360	360	360
Tij	12	95	19	18	79	36	28	7	9	21	44	9	41	8
Tε	60	60	70	30	40	40	50	40	80	20	80	40	50	30
Tm	25	37	45	9	52	37	21	38	39	37	4	12	58	8
Ti	817	912	854	417	531	473	459	445	488	438	488	421	509	406