Markov Processes in Blockchain Systems

In this paper, we develop a more general framework of block-structured Markov processes in the queueing study of blockchain systems, which can provide analysis both for the stationary performance measures and for the sojourn times of any transaction and block. Note that an original aim of this paper is to generalize the two-stage batch-service queueing model studied in Li et al. \cite{Li:2018} both ``from exponential to phase-type"service times and ``from Poisson to MAP"transaction arrivals. In general, the MAP transaction arrivals and the two stages of PH service times make our blockchain queue more suitable to various practical conditions of blockchain systems with crucial random factors, for example, the mining processes, the block-generations, the blockchain-building and so forth. For such a more general blockchain queueing model, we focus on two basic research aspects: (1) By using the matrix-geometric solution, we first obtain a sufficient stable condition of the blockchain system. Then we provide simple expressions for the average number of transactions in the queueing waiting room, and the average number of transactions in the block. (2) However, comparing with Li et al. \cite{Li:2018}, analysis of the transaction-confirmation time becomes very difficult and challenging due to the complicated blockchain structure. To overcome the difficulties, we develop a computational technique of the first passage times by means of both the PH distributions of infinite sizes and the $RG$-factorizations. Finally, we hope that the methodology and results given in this paper will open a new avenue to queueing analysis of more general blockchain systems in practice, and can motivate a series of promising future research on development of lockchain technologies.


Introduction
Background and Motivation. Blockchain is one of the most popular issues discussed extensively in recent years, and it has already changed people's lifestyle in some real areas due to its great impact on finance, business, industry, transportation, heathcare and so forth. Since the introduction of Bitcoin by Nakamoto [65], blockchain technologies has obtained many important advances in basic theory and real applications up to now. Readers may refer to, for example, excellent books by Wattenhofer [95], Prusty [79], Drescher [29], Bashir [9] and Parker [73]; and survey papers by Zheng et al. [100], Constantinides et al. [22], Yli-Huumo et al. [98], Lindman et al. [58] and Risius and Spohrer [83].
Based on the above discussions, whether it is theoretical research or real applications, we always hope to know how performance of the blockchain system are obtained, and wheth there is still some room to be able to further improve performance of the blockchain system. Based on this, it is a key to find solution of such a performance issue in the study of blockchain systems. Thus we need to provide mathemtical modeling and analysis for blockchain performance evaluation by means of, for example, Markov processes, Markov decision processes, queueing networks, Petri networks, game models and so on. Unfortunately, so far only a little work has been on performance modeling of blockchain systems.
Therefore, this motivates us in this paper to develop Markov processes and queueing models for a more general blockchain system. We hope that the methodology and results given in this paper will open a new avenue to Markov processes of blockchain systems, and can motivate a series of promising future research on development of blockchain technologies.
Related Work. Now, we provide several different classes of related work for Markov processes in blockchain systems, for example, queueing models, Markov processes, Markov decision processes, random walks, fluid limit and so on.
Queueing models: To use queueing theory to model a blockchain system, we need to observer some key factors, for example, transaction arrivals, block-generation, block size, transaction fee, mining pools, mining reward, solving difficulty of crypto mathematical puzzle, blockchain-building, throughput and so forth. As sketched in Figure 1, we design a two stage, Service-In-Random-Order and batch service queueing system by means of two stages of asynchronous processes: Block-generation and blockchain-building. Li et al. [56] is the first one to provide a detailed analysis for such a blockchain queue by means of the matrix-geometric solution. Kasahara and Kawahara [47] and Kawase and Kasahara [48] discussed the blockchain queue with general service times through an incompletely solving idea for dealing with an interesting open problem. In addition, they also gave some useful numerical experiments for performance observation. Ricci et al. [82] proposed a framework encompassing machine learning and a queueing model, which is used to identify which transactions will be confirmed, and to characterize the confirmation time of confirmed transactions. Memon et al. [60] proposed a simulation model for the blockchain systems by means of queuing theory.
Bowden et al. [15] discussed time-inhomogeneous behavior of the block arrivals in the bitcoin blockchain because the block-generation process is influenced by multiple key factors such as the solving difficulty level of crypto mathematical puzzle, transaction fee, mining reward, mining pools and so on. Papadis et al. [72] applied the timeinhomogeneous block arrivals to set up some Markov processes to study evolution and dynamics of blockchain networks, and discussed key blockchain characteristics such as the number of miners, the hashing power (block completion rates), block dissemination delays, and block confirmation rules. Further, Jourdan et al. [45] proposed a probabilistic model of the bitcoin blockchain by means of a transaction and block graph, formulated some conditional dependencies induced by the bitcoin protocol at the block level. Based on this, it is clear that when the block-generation arrivals are a time-inhomogeneous Poisson process, we believe that the blockchain queue analyzed in this paper will become very difficult and challenging, thus it will be an interesting topic in our future study.
Markov processes: To evaluate performance of a blockchain system, Markov processes are a basic mathematical tool, e.g., see Bolch et al. [14] for more details. As an early key work to apply Markov processes to blockchain performance issues, Eyal and Sirer [31] established a simple Markov process to analyze the vulnerability of Nakamoto Protocols through studying the block-forking behavior of blockchain. Note that some selfish miners may get higher payoffs by violating the information propagation protocols and postponing their mined blocks so that such selfish miners exploits the inherent block forking phenomenon of Nakamoto protocols. Nayak et al. [66] extended the work by Eyal and Sirer [31] through introducing a new mining strategy: Stubborn mining strategy. They used three improved Markov processes to further study the stubborn mining strategy and two extensions: the Equal-Fork Stubborn (EFS) and the Trail Stubborn (TS) mining strategies. Carlsten [17] used the Markov process to study the impact of transaction fees on the selfish mining strategies in the bitcoin network. Göbel et al. [36] further considered the mining competition between a selfish mining pool and the honest community by means of a two-dimensional Markov process, in which they extended the Markov model of selfish mining by considering the propagation delay between the selfish mining pool and the honest community.
Kiffer and Rajaraman [50] provided a simple framework of Markov processes for analyzing consistency properties of the blockchain protocols, and used some numerical experiments to check the consensus bounds for network delay parameters and adversarial computing percentages. Huang et al. [41] set up a Markov process with an absorbing state to analyze performance measures of the Raft consensus algorithm for a private blockchain.
Markov decision processes: Note that the selfish miner may adopt different mining policies to release some blocks under the longest-chain rule for controling the block-forking structure, thus it is interesting to find an optimal mining policy in the blockchain system.
To do this, Sapirshtein et al. [85], Sompolinsky and Zohar [86] and Gervais et al. [35] applied the Markov decision processes to find the optimal selfish-mining strategy, in which four actions: adopt, override, match and wait, are introduced in order to control the state transitions of the Markov decision process.
Random walks: Goffard [37] proposed a random walk method to study the doublespending attack problems in the blockchin system, and focused on how to evaluate the probability of the double-spending attack ever being successful. Jang and Lee [43] discussed profitability of the double-spending attacks in the blockchain systems through using the random walk of two independent Poisson counting processes.
Fluid limit: Frolkova and Mandjes [34] considered a bitcoin-inspired infinite-server model with a random fluid limit. King [52] developed the fluid limit of a random graph model to discuss the shared ledger and the distributed ledger technologies in the blockchain systems.
Contributions. The main contributions of this paper are twofold. The first contribution is to develop a more general framework of block-structured Markov processes in the study of blockchain systems. We design a two stage, Service-In-Random-Order and batch service queueing system, whose original aim is to generalize the blockchain queue studied in Li et al. [56] both "from exponential to phase-type" service times and "from Poisson to MAP" transaction arrivals. Note that the transaction MAP arrivals and two stages of PH service times make our new blockchain queueing model more suitable to various practical conditions of blockchain systems. By using the matrix-geometric solution, we obtain a sufficient stable condition of the more general blockchain system, and provide simple expressions for two key performance measures: The average stationary number of transactions in the queueing waiting room, the average stationary number of transactions in the block.
The second contribution of this paper is to provide an effective method for computing the average transaction-confirmation time of any transaction in a more general blockchain system. In general, it is always very difficult and challenging to analyze the transactionconfirmation time in the blockchain system with MAP inputs and PH service times, because the service discipline of the blockchain system is new from two key points: (1) The "block service" is a class of batch service; and (2) some transactions are chosen into a block through Service-In-Random-Order. In addition, the MAP inputs and PH service times also make analysis of the blockchain queue more complicated. To study the transactionconfirmation time, we set up a Markov process with an absorbing state (see Figure 4) according to the blockchain system (see Figures 1 and 2). Based on this, we show that the transaction-confirmation time of any transaction is the first passage time of the Markov process with an absorbing state, hence we can discuss the transaction-confirmation time (or the first passage time) by means of both the PH distributions of infinite sizes and the RG-factorizations. Further, we propose an effective algorithm for computing the average transaction-confirmation time of any transaction. We hope that our approach given in this paper can be applicable to deal with the transaction-confirmation times in more general blockchain systems.
The structure of this paper is organized as follows. Section 2 describes a two stage, Service-In-Random-Order and batch service queueing system, where the transactions arrive at the blockchain system according to a Markovian arrival process (MAP), the blockgeneration and blockchain-building times are all of phase type (PH). Section 3 establishes a continuous-time Markov process of GI/M/1 type, derives a sufficient stable condition of In this section, from a more general point of view of blockchain, we design an interesting and practical blockchain queueing system, where the transactions arrive at the blockchain system according to a Markovian arrival process (MAP), while the block-generation and blockchain-building times are all of phase type (PH).
From a more practical background of blockchain, it is necessary to extend and generalize the blockchain queueing model, given in Li et al. [56], to a more general case not only with non-Poisson transaction inputs but also with non-exponential block-generation and blockchain-building times. Based on this, we further abstract the block-generation and blockchain-building processes as a queue of batch service, Service-In-Random-Order input and two different service stages by means of the MAP and the PH distribution. Such a blockchain queueing system is depicted in Figure 1.   (C + D) e = 0, and e is a column vector of suitable size in which each element is one.
Obviously, the Markov process C + D with finite states is irreducible and positive recurrent. Let ω be the stationary probability vector of the Markov process C + D, it is clear that ω (C + D) = 0 and ωe = 1. Also, the stationary arrival rate of the MAP is given by In addition, we assume that each arriving transaction must first enter a queueing waiting room of infinite size. See the lower left part corner of Figure 1.
A block-generation process: Each arriving transaction first needs to queue in a waiting room. Then it is possibly chosen into a block of the maximal size b. This is regarded as the first stage of service, called block-generation process. Note that the arriving transactions will be continually chosen into the block until the block-generation process is over under which a nonce is appended to the block by a mining winner. See the lower middle part of Figure 1 for more details.
The block-generation time begins the initial epoch of a mining process until a nonce of the block is found (i.e., the cryptographic mathematical puzzle is solved for sending a nonce to the block), then the mining process is terminated immediately. We assume that all the block-generation times are i.i.d., and are of phase type with an irreducible representation (β, S) of order m 2 , where βe = 1, the expected blockchain-building time is The block-generation discipline: A block can consist of some transactions but at most b transactions. Once the mining process begins, the transactions are chosen into a block, in which they are not completely based on the First Come First Service (FCFS) from the order of transaction arrivals. In this case, several transactions in the back of this queue may also be preferentially chosen into the block. When the block is formed, it will not receive any new arriving transaction. See the lower middle part of Figure 1.
A blockchain-building process: Once the mining process is over, the block with a group of transactions will be pegged to a blockchain. This is regarded as the second stage of service due to the network latency, called blockchain-building process, see the lower right corner of Figure  Remark 2 In the blockchain system, there are some key factors including the maximum block size, mining reward, transaction fee, mining strategy, security of blockchain and so on. Based on this, we may develop reward queueing models, decision queueing models, and game queueing models in the study of blockchain systems. Therefore, analysis for the key factors will be not only theoretically necessary but also practically important in development of blockchain technologies.

A Markov Process of GI/M/1 Type
In this section, to analyze the blockchain queueing system, we first establish a continuoustime Markov process of GI/M/1 type. Then we derive a system stable condition and express the stationary probability vector of the blockchain queueing system by means of the matrix-geometric solution.
Let N 1 (t) , N 2 (t) , I (t) , J 1 (t) and J 2 (t) be the number of transactions in the queueing waiting room, the number of transactions in the block, the phase of the MAP, the phase of a blockchain-building PH time, and the phase of a block-generation PH time at time Then it is easy to see that X is a continuous-time Markov process with block structure whose state space is given by From Figure 1, it is easy to set up the state transition relation of the Markov process X, see Figure 2 for more details. It is a key in understanding of Figure 2 that there is a different transition between State (k, 0) for the block-generation and State (k, l) for the blockchain-building with 1 ≤ l ≤ b because the block-generation and blockchain-building processes can not simultaneously exist at a time, and specifically, a block must first be generated, then it can enter the blockchain-building process.  Figure 2, the infinitesimal generator of the Markov process X is given by where ⊗ and ⊕ are the Kronecker product and the Kronecker sum of two matrices, respectively, Clearly, the continuous-time Markov process X is of GI/M/1 type. Now, we use the mean drift method to discuss the system stable condition of the continuous-time Markov process X of GI/M/1 type. Note that the mean drift method for checking system stability is given a detailed introduction in Chapter 3 of Li [54].
From Chapter 1 of Neuts [67] or Chapter 3 of Li [54], for the Markov process of GI/M/1 type, we write Clearly, the matrix A is the infinitesimal generator of an irreducible, aperiodic and positive recurrent Markov process with two levels (i.e., Levels 0 and b), together with b − 1 instantaneous levels (i.e., Levels 1, 2, ..., b − 1) which will vanish as the time t goes to infinity. On the other hand, such a special Markov process A will not influence applications of the matrix-geometric solution because it is only related to the mean drift method for establishing system stable conditions.
The following theorem discusses the invariant measure θ of the Markov process A, that is, the vector θ satisfies the system of linear equations θA = 0 and θe = 1.
Theorem 1 There exists the unique invariant measure θ = (θ 0 , 0, . . . , 0, θ b ) of the Markov process A, where (θ 0 , θ b ) is the stationary probability vector of the irreducible positiverecurrent Markov process whose infinitesimal generator Proof: It follows from θA = 0 that For Equation (3) It follows from (2) and (4) Thus we have Then the matrix ℜ is the infinitesimal generator of an irreducible positive-recurrent Markov process. Thus the Markov process ℜ exists the stationary probability vector (θ 0 , θ b ), that is, there exists the unique solution to the system of linear equations: (θ 0 , θ b ) ℜ = 0 and θ 0 e + θ b e = 1. This completes the proof.
The following theorem provides a necessary and sufficient conditions under which the Markov process Q is positive recurrence.
Note that and bθA b+1 e = bθ 0 I ⊗ S 0 α e, thus we obtain This completes the proof.
It is necessary to consider a special case in which the transaction inputs are Poisson with arrival rate λ, and the blockchain-building and block-generation times are exponential with service rates µ 1 and µ 2 , respectively. Note that this special case was studied in Li et al. [56], here we only restate the stable condition as the following corollary.

Corollary 3 The Markov process Q of GI/M/1 type is positive recurrent if and only if
By observing (9), it is easy to see that 1/ (bµ 1 ) + 1/ (bµ 2 ) < 1/λ, that is, the complicated service speed of transactions is faster than the transaction arrival speed, under which the Markov process Q of GI/M/1 type is positive recurrent. However, it is not easy to understood from (5).
If (θ 0 + θ b ) (D ⊗ I) e < bθ 0 I ⊗ S 0 α e, then the Markv process Q of GI/M/1 type is irreducible and positive recurrent. Thus the Markov process Q exists a unique stationary probability vector, which is also matrix-geometric. Thus, to express the matrix-geometric stationary probability vector, we need to first obtain the rate matrix R, which is the minimal nonnegative solution to the following nonlinear matrix equation In general, it is very complicated to solve this nonlinear matrix equation (10) due to the term R b+1 A b+1 of size b + 1. In fact, for the blockchain queueing system, here we can not provide an explicit expression for the rate matrix R yet. In this case, we can use some iterative algorithms, given in Neuts [67], to give its numerical solution. For example, an effective iterative algorithm given in Neuts [67] is described as Note that this algorithm is fast convergent, that is, after a finite number of iterative steps, we can numerically obtain a solution of higher precision which is used to approximate the rate matrix R.
The following theorem directly comes from Theorem 1.2.1 of Chapter 1 in Neuts [67].
Here, we restate it without a proof.

The Stationary Transaction Numbers
In this section, we discuss two key performance measures: The average stationary numbers of transactions both in the queueing waiting room and in the block, and give their simple expressions by means of the vectors π 0 and π 1 , and the rate matrix R. Finally, we use numerical examples to verify computability of our theoretical results, and show how the performance measures depend on the main parameters of this system.
If (θ 0 + θ b ) (D ⊗ I) e < bθ 0 I ⊗ S 0 α e, then the blockchain system is stable. In this case, we write that w.p.1, where N 1 (t) and N 2 (t) are the numbers of transactions in the queueing waiting room and of transactions in the block at time t ≥ 0, respectively.
(a) The average stationary number of transactions in the queueing waiting room It follows from (11) and (12) that Note that the above three vectors e has different sizes, for example, the size of the first one is m 0 × m 2 for l = 0 and m 0 × m 1 for 1 ≤ l ≤ b; while the sizes of the second and third are m 0 × (m 2 + bm 1 ). For simplicity of description, here we use only a vector e whose size can easily be inferred by the context.

(b) The average stationary number of transactions in the block
Let h = (0, e, 2e, . . . , be) T . Then In the remainder of this section, we provide some numerical examples to verify computability of our theoretical results, and to analyze how the two performance measures  Remark 3 It is necessary to explain the absorbing rates in the below part of Figure 4.  When a transaction arrives at the queueing waiting room, it can observes the states of the blockchain system having two different cases: Case one: State (k, 0; i, r) for k ≥ 1; 1 ≤ i ≤ m 0 and 1 ≤ r ≤ m 2 . In this case, with the initial probability π (i,r) k,0 , the transaction-confirmation time ℑ is the first passage time ξ |(k,0;i,r) of the Markov process with an absorbing state, whose state transition relation given in Figure 4.
In this case, with the initial probability π (i,j) k,l , the transaction-confirmation time ℑ is decomposed into the sum of the random variable Γ s and the first passage time ξ |(k,0;i,r) given in Case one. It is easy to see Figure 4 that there exists a stochastic decomposition: For the above analysis, it is easy to see that computation of the first passage time ξ |(k,0;i,r) is a key in analyzing the transaction-confirmation time.
Based on the state transition relation given in Figure 4, now we write the infinitesimal generator of the Markov process {Y (t) : t ≥ 0} as where If the blockchain system is stable, then the probability that a transaction observes State (0, 0; i) only after arrived at the instant is π (i,r) 1,0 ; for 1 ≤ l ≤ b, the probability that a transaction observes State (0, l; i, j) only after arrived at the instant is π (i,j) 1,l ; for k ≥ 2, the probability that a transaction observes State (k − 1, 0; i, r) only after arrived at the instant is π (i,r) k,0 ; for k ≥ 2, 1 ≤ l ≤ b, the probability that a transaction observes State (k − 1, l; i, j) only after arrived at the instant is π (i,j) k,l . Obviously, for 0 ≤ l ≤ b, States (0, 0; i) and (0, l; i, j) will not be encountered by the transaction only after arrived at the instant, thus the stationary probabilities π To emphasize on the event that the transaction observes State (k − 1, 0; i, r) only after arrived at the instant, we introduce a new initial probability vector where for k ≥ 1 ϕ k = (γ k,0 , 0, 0, . . . , 0) .
Also, we have This completes the proof.
Based on Theorem 5, now we extend the first passage time ξ |(k,0;i,r) to ξ |(0,ϕ) , which is the first passage time of the Markov process H with an initial probability vector (0, ϕ).
The following corollary shows that ξ |(0,ϕ) is PH distribution of infinite size, while its proof is easy and is omitted here.
Corollary 6 If the blockchain system is stable, then the first passage time ξ |(0,ϕ) is a PH distribution of infinite size with an irreducible representation ((0, ϕ) , H), and The following theorem provides a simple expression for the average transaction-confirmation time E[ℑ] by means of Corollary 6.
Theorem 7 If the blockchain queueing system is stable, then the average transactionconfirmation time E[ℑ] is given by where Γ s is the stationary life time of the PH blockchain-building time with an irreducible representation (α, T ). Further, we have where ̟ is the stationary probability vector of the Markov process T + T 0 α.
Proof: We first introduce two basic events Θ = {The transaction observes States (0, 0; i) and (k, 0; i, r) only after arrived at the instant} and Θ c = {The transaction observes States (k, l; i, j) only after arrived at the instant} .
It is easy to see that Θ ∪ Θ c = Ω. Thus the two events is complementary according to the fact that the transaction can observe all the states of the Markov process Q only after arrived at the instant. If the blockchain system is stable, then it is easy to compute the probabilities of the two events as follows: By using the law of total probability, we obtain The proof is completed.
As shown in Theorem 7, it is a key in the study of PH distributions of infinite sizes whether or not we can compute the inverse matrix H −1 of infinite size. To this end, we need to use the RG-factorizations, given in Li [54], to provide such a computable path.
In what follows we provide only a simple interpretation on the computation, while some detailed discussions will be left in our another paper in the future.
In fact, it is often very difficult and challenging to compute the inverse of a matrix of infinite size only except for the triangular matrices. Fortunately, by using the RG- to define and compute the R-, U -and G-measures as follows.
The R-measure. Let R k for k ≥ 0 be the minimal nonnegative solution to the system of nonlinear matrix equations: . . .
The U -measure. Based on the R-measure R k for k ≥ 0, we have . . .
The G-measure. Based on the R-measure R k for k ≥ 0 and the U -measure U k for k ≥ 0, we have . . .
and for k ≥ 3 Based on the R-, U -and G-measures, we provide the UL-type RG-factorization of the infinitesimal generator H as follows: and Based on the UL-type RG-factorization H = (I − R U ) U (I − G L ), we obtain Once the inverse of matrix H of infinite size is given, the PH distribution of infinite size can be constructed under a computable and feasible framework. In fact, this is very important in the study of stochastic models. Also see Li et al. [55] and Takine [89] for more details.

Remark 4
In general, it is always very difficult and challenging to discuss the transactionconfirmation time of any transaction in a blockchain system due to two key points: The block service is a class of batch service, and some transactions are chosen into a block by means of the Service-In-Random-Order. For a more general blockchain system, this paper sets up a Markov process with an absorbing state, and shows that the transactionconfirmation time is the first passage time of the Markov process with an absorbing state.
Therefore, this paper can discuss the transaction-confirmation time by means of the PH distribution of infinite size (corresponding to the first passage time), and provides an effective algorithm for computing the average transaction-confirmation time by using the RG-factorizations of block-structured Markov processes of infinite levels. We believe that the RG-factorizations of block-structured Markov processes will play a key role in the queueing study of blockchain systems.

Concluding Remarks
In this paper, we develop a more general framework of block-structured Markov processes in the queueing study of blockchain systems. To do this, we design a two stage, Service-In-Random-Order and batch service queueing system with MAP transaction arrivals and two-stages of PH service times, and discuss some key performance measures such as the stationary average number of transactions in the queueing waiting room, the stationary average number of transactions in the block, and the average transaction-confirmation time of any transaction. Note that the study of performance measures is a key to improve blockchain technologies sufficiently. On the other hand, an original aim of this paper is to generalize the two-stage batch-service queueing model studied in Li et al. [56] both "from exponential to phase-type" service times and "from Poisson to MAP" transaction arrivals.
Note that the MAP transaction arrivals and the two stages of PH service times make our queueing model more suitable to various practical conditions of blockchain systems with key factors, for example, the mining processes, the reward incentive, the consensus mechanism, the block-generation, the blockchain-building and so forth.
By using the matrix-geometric solution, we first obtain a sufficient stable condition of the blockchain system. Then we provide simple expressions for two key performance In addition, we use numerical examples to verify computability of our theoretical results.
Along these lines, we will continue our future research on several interesting directions as follows: -Developing effective algorithms for computing the average transaction-confirmation times in terms of the RG-factorizations.
-Analyzing multiple classes of transactions in the blockchain systems, in which the transactions are processed in the block-generation and blockchain-building processes according to a priority service discipline.
-When the arrivals of transactions are a renewal process, and/or the block-generation times and/or the blockchain-building times follow general probability distributions, an interesting future research is to focus on fluid and diffusion approximations of blockchain systems.
-Setting up reward function with respect to cost structures, transaction fees, mining reward, consensus mechanism, security and so forth. It is very interesting in our future study to develop stochastic optimization, Markov decision processes and stochastic game models in the study of blockchain systems.