text:
ÇëÊäÈëÄÚÈÝ£º ¡¡
µ±Ç°Î»ÖÃ£º ÎÄµµÏÂÔØ > ËùÓÐ·ÖÀà > Scaling of avalanche queues in directed dissipative sandpiles
Ãâ·ÑÏÂÔØ´ËÎÄµµ

# Scaling of avalanche queues in directed dissipative sandpiles

We simulate queues of activity in a directed sandpile automaton in 1+1 dimensions by adding grains at the top row with driving rate $0 < r \leq 1$. The duration of elementary avalanches is exactly described by the distribution \$P_1(t) \sim t^{-3/2}\exp{(-1

a r X i v :c o n d -m a t /0006246v 1 [c o n d -m a t .s t a t -m e c h ] 15 J u n 2000Scaling of avalanche queues in directed dissipative sandpiles Bosiljka Tadi´c 1,?and Vyatcheslav Priezzhev 2,??1Jo¡¦z ef Stefan Institute,P.O.Box 3000,1001Ljubljana,Slovenia 2Bogolubov Laboratory of Theoretical Physics,Joint Institute of Nuclear Research,141980Dubna,Russia Using numerical simulations and analytical methods we study a two-dimensional directed sandpile automaton with nonconservative random defects (concentration c )and varying driving rate r .The automaton is driven only at the top row and driving rate is measured by the number of added particles per time step of avalanche evolution.The probability distribution of duration of elementary avalanches at zero driving rate is exactly given by P 1(t,c )=t ?3/2exp [t ln(1?c )].For driving rates in the interval 0<r ¡Ü1the avalanches are queuing one after another,making increase the periods of non-interrupted activity of the automaton.Recognizing the probability P 1as a distribution of service time of jobs arriving at a server with frequency r ,the model represents a new example of the E,1,GI/¡Þ/1 server queue in the queue theory.We study scaling properties of the busy period and dissipated energy of sequences of non-interrupted activity.In the limit c ¡ú0and varying linear system size L ?1/c we ?nd that at driving rates r ¡ÜL ?1/2the distributions of duration and energy of the avalanche queues are characterized by a multifractal scaling and we determine th

e corresponding spectral functions.For L ?1/c increasing of the driving rate somewhat compensates the energy losses at defects above the line r ?¡Ì


text:
# Prove that a language is not regular with process of elimination [duplicate]

When deterministic automaton, I need to prove that you can't implement the language in it, because the language is not regular.

Easiest way to prove that a language is regular, is just by making an automaton for it. But if the automaton is not regular, you can't just say that it's not regular because you couldn't build an automaton for it.

So I am wondering, how to prove that a language is not regular the simplest way?

Given language:

$$L = \{a^n b^n | n > 0\}$$

Definition: The language accepts series of a's and b's where the amount of as equals to the amount of bs. This is not a regular language.

How can you prove that it's not a regular language?

From what I remember (I didn't understand it much), you had to take a sub-langauge of that language, and mark it as W

So $$W = \{a^i\}$$

Then select 2 words from W and mark them as w1 and w2

$$w1 = a^i$$ $$w2 = a^j$$

where i != j

And then I don't really remember what to do.

## marked as duplicate by David Richerby, Yuval Filmus, Raphael♦Feb 4 '15 at 13:54

A simple technique is using the pumping lemma:

For any regular language $L$ there exists an integer $n$ such that for all $x \in L$, with $|x| \geq n$ there exists $u,v,w \in \Sigma^*$, such that,

• $x = uvw$
• $|uv| \leq n$
• $|v| \geq 1$
• for all $i \geq 0: uv^iw \in L$

Assumption: Suppose your language is regular.

Consider $s = a^nb^n$. Clearly $s \in L$ and $|s| \geq n$. We get that $uvw = a^nb^n$ and we know that $|uv| \leq n$, i.e $v$ can only contain $a$'s and since $|v| \geq 1$ it must contain at least one $a$. According to the last condition of the pumping lemma, $uv^0w \in L$ but this is clearly not the case. We now have a contradiction as we assumed the language to be regular:

The assumption must be wrong implying non-regularity of the language

• Thanks for taking the time to answer! Note that some questions pop up over and over again, and sometimes the exact exercise problem has even been posted before. We have compiled a list of reference questions for FAQs which you may want to check out before spending time on redundant answers. – Raphael Feb 4 '15 at 13:55
• Yeah I noticed that - but thanks. – User Feb 4 '15 at 17:44

What you are suggesting is the Myhill–Nerode criterion, which is more natural, and in many cases simpler, than the standard pumping lemma which for some reason has become the undergraduate hallmark of the subject.

Consider your two words $a^i$ and $a^j$. Suppose we are given some DFA for the language. Can the DFA be at the same state after reading $a^i$ and $a^j$? It cannot, since if it were, the DFA would behave the same after seeing the word $b^i$, say reaching some state $q$; but since $a^ib^i \in L$, $q$ must be accepting, while since $a^jb^i \notin L$, $q$ must be rejecting! We conclude that the DFA must be at different states after reading $a^i$ and $a^j$.

We can consider the infinitely many words $a^1,a^2,a^3,\ldots$. The argument above shows that any DFA for $L$ must reach a different state after reading them. So it must have infinitely many states. Contradiction.

In contrast to the pumping lemma, which doesn't always "work" (i.e., there are non-regular languages which satisfying the pumping lemma), the Myhill–Nerode criterion is a complete characterization: if a language is not regular, then you can always find an infinite list of mutually inequivalent words (two words $w_1,w_2$ are inequivalent if for some word $x$, either $w_1x \in L$ while $w_2x \notin L$, or $w_1x \notin L$ while $w_2x \in L$), which proves that the language is not regular.

format: plain


text:
# What is the domain of the Black-Scholes operator?

By the Black-Scholes operator I mean the following.

$$L_{BS}u(x) = \frac{1}{2}\sigma^2x^2\frac{\partial^2}{\partial x^2}u(x) + rx\frac{\partial}{\partial x}u(x) - ru(x)$$

Obviously, the domain of $L_{BS}$ must be a subset of twice continuously differentiable functions on some interval but making a definitive statement about the domain requires something specific about the application at hand (option pricing in this instance) and knowledge about the origin of the operator. So what is what I am looking for.

Feynman-Kac says something about which solutions of the Black-Scholes PDE satisfy the pricing formula, that is in the form of conditional expectation. The solution class is the set of twice continuously differentiable functions that are continuous at the boundary and whose growth is bounded by a function of the form $e^{\alpha x^2}$.

There is also the uniqueness class of the Black-Scholes PDE, which has a similar growth characteristic.

This paper mentions the Schwartz space as the domain of Black-Scholes operator but I have never seen this anywhere else.

I am looking for an answer that summarizes all the considerations that go into this, preferably with references. Thank you.


text:
# Hierarchical Species Sampling Models

This paper introduces a general class of hierarchical nonparametric prior distributions. The random probability measures are constructed by a hierarchy of generalized species sampling processes with possibly non-diffuse base measures. The proposed framework provides a general probabilistic foundation for hierarchical random measures with either atomic or mixed base measures and allows for studying their properties, such as the distribution of the marginal and total number of clusters. We show that hierarchical species sampling models have a Chinese Restaurants Franchise representation and can be used as prior distributions to undertake Bayesian nonparametric inference. We provide a method to sample from the posterior distribution together with some numerical illustrations. Our class of priors includes some new hierarchical mixture priors such as the hierarchical Gnedin measures, and other well-known prior distributions such as the hierarchical Pitman-Yor and the hierarchical normalized random measures.

## Authors

• 4 publications
• 5 publications
• 10 publications
08/02/2016

### Modelling and computation using NCoRM mixtures for density regression

Normalized compound random measures are flexible nonparametric priors fo...
10/19/2021

### BNPdensity: Bayesian nonparametric mixture modeling in R

Robust statistical data modelling under potential model mis-specificatio...
03/27/2020

### Enriched Pitman-Yor processes

In Bayesian nonparametrics there exists a rich variety of discrete prior...
11/03/2021

### On Johnson's "sufficientness" postulates for features-sampling models

In the 1920's, the English philosopher W.E. Johnson introduced a charact...
08/26/2021

### Contaminated Gibbs-type priors

Gibbs-type priors are widely used as key components in several Bayesian ...
02/06/2021

### Video-based Hierarchical Species Classification for Longline Fishing Monitoring

The goal of electronic monitoring (EM) of longline fishing is to monitor...
03/11/2022

### Bayesian Nonparametric Inference for "Species-sampling" Problems

"Species-sampling" problems (SSPs) refer to a broad class of statistical...
##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Cluster structures in multiple groups of observations can be modelled by means of hierarchical random probability measures or hierarchical processes that allow for heterogenous clustering effects across groups and for sharing clusters among the groups. As an effect of the heterogeneity, in these models the number of clusters in each group (marginal number of clusters) can differ, and due to cluster sharing, the number of clusters in the entire sample (total number of clusters) can be smaller than the sum of the marginal number of clusters. An important example of hierarchical random measure is the Hierarchical Dirichlet Process (HDP), introduced in the seminal paper of Teh et al. (2006)

. The HDP involves a simple Bayesian hierarchy where the common base measure for a set of Dirichlet processes is itself distributed according to a Dirichlet process. This means that the joint law of a vector of random probability measures

is

 pi|p0iid∼DP(θ1,p0),i=1,…,I,p0|H0∼DP(θ0,H0), (1)

where denotes the Dirichlet process with base measure and concentration parameter . Once the joint law of has been specified, observations are assumed to be conditionally independent given with

 ξi,j|(p1,…,pI)ind∼pi,i=1,…,I and j≥1.

If the observations take values in a Polish space, then this is equivalent to partial exchangeability of the array (de Finetti’s representation Theorem). Hierarchical processes are widely used as prior distributions in Bayesian nonparametric inference (see Teh and Jordan (2010) and reference therein), by assuming are hidden variables describing the clustering structures of the data and the observations in the -th group, , are conditionally independent given with

 Yi,j|ξi,jind∼f(⋅|ξi,j),

where is a suitable kernel density.

In this paper we provide a new class of hierarchical random probability measures constructed by a hierarchy of generalized species sampling sequences, and call them Hierarchical Species Sampling Models (HSSM). A species sampling sequence is an exchangeable sequence whose directing measure is a discrete random probability

 p=∑j≥1δZjqj, (2)

where and are stochastically independent, are i.i.d. with common distribution and the non-negative weights sum to one almost surely. By Kingman’s theory on exchangeable partitions, any random sequence of positive weights such that can be associated to an exchangeable random partition of integers . Moreover, the law of an exchangeable random partition is completely described by the so-called exchangeable probability partition function (EPPF). Hence, the law of the above defined random probability measure , turns out to be parametrized by an EPPF and by a base measure , which is diffuse in species sampling sequences, and possibly non-diffuse in our generalized species sampling construction.

A vector of random measures in the HSSM class can be described as follows. Denote by the law of the random probability measure defined in (2) and let be a vector of random probably measures such that

 pi|p0iid∼SSrp(q,p0),i=1,…,I,p0∼SSrp(q0,H0), (3)

where is a base measure and and are two EPPFs. A HSSM is an array of observations conditionally independent given with , and .

The proposed framework is general enough to provide a probabilistic foundation of both existing and novel hierarchical random measures, also allowing for non diffuse base measures. Our HSSM class includes the HDP, its generalization given by the Hierarchical Pitman–Yor process (HPYP), see Teh (2006); Du et al. (2010); Lim et al. (2016) and the hierarchical normalized random measures with independent increments (HNRMI), recently studied in Camerlenghi et al. (2018). Among the novel measures, we study the hierarchical generalization of the Gnedin process (Gnedin (2010)) and of finite mixtures (e.g., Miller and Harrison (2017)) and the asymmetric hierarchical constructions with and of different type (Du et al. (2010); Buntine and Mishra (2014)). Also, we consider new HSSM with non-diffuse base measure in the spike-and-slab class of prior introduced by George and McCulloch (1993) and now widely studied in Bayesian parametric (e.g., Castillo et al. (2015), Rockova and George (2017) and Rockova (2018)) and nonparametric (e.g., Canale et al. (2017)) inference. Finally, note that non-diffuse base measures are also used in other random measures models (e.g., see Prunster and Ruggiero (2013)), although these models are not in our class, the hierarchical species sampling specification can be used to generalize them.

By exploiting the properties of hierarchical species sampling sequences, we are able to provide the finite sample distribution of the number of clusters for each group of observations and the total number of clusters. Moreover, we provide some new asymptotic results when the number of observations goes to infinity, thus extending the asymptotic approximations for species sampling given in Pitman (2006)) and for hierarchical normalized random measures given in Camerlenghi et al. (2018).

We show that the measures in the proposed class have a Chinese Restaurant Franchise representation, that is appealing for the applications to Bayesian nonparametrics, since it sheds light on the clustering mechanism of the processes and suggests a simple sampling algorithm for posterior computations whenever the EPPFs and are known explicitly. In the Chinese Restaurant Franchise metaphor, observations are attributed to “customers”, identified by the indexes , denote the restaurants (groups) and the customers are clustered according to “tables”. Hence, the first step of the clustering process (from now on, bottom level) is the restaurant-specific sitting plan. Tables are then clustered, in an higher level of the hierarchy (top level), by means of “dishes” served to the tables. In a nutshell, observations driven by HSSM can be described as follows:

 ξi,j=ϕdi,ci,j,ϕniid∼H0,[di,c,ci,j]∼Q, (4)

where is the (random) table at which the -th “customer” of “restaurant” sits, is the (random) index of the “dish” served at table in restaurant and the are the ”dishes” drawn from the base probability measure . The distribution of the observation process is completely described once the law of the process is specified. We will see that the process plays a role similar to the one of the random partition associated with exchangeable species sampling sequences.

The paper is organized as follows. Section 2 reviews exchangeable random partitions, generalized species sampling sequences and species sampling random probability measures. Some examples are discussed and new results obtained under the assumption of non-diffuse base measure. Section 3 introduces hierarchical species sampling models, presents some special cases and shows some properties such as the Chinese restaurant franchise representation, which are useful for applications to Bayesian nonparametric inference. Section 4 provides the finite-sample and the asymptotic distribution of the marginal and total number of clusters under both assumptions of diffuse and non-diffuse base measure. A Gibbs sampler for hierarchical species sampling mixtures is established in Section 5. Section 6 presents some simulation studies and a real data application.

## 2 Background material

Exchangeable random partitions provide an important probabilistic tool for a wide range of theoretical and applied problems. They have been used in various fields such as population genetics Ewens (1972); Kingman (1980); Donnelly (1986); Hoppe (1984), combinatorics, algebra and number theory Donnelly and Grimmett (1993); Diaconis and Ram (2012); Arratia et al. (2003)

Teh (2006); Wood et al. (2009), psychology Navarro et al. (2006), and model-based clustering Lau and Green (2007); Müller and Quintana (2010). In Bayesian nonparametrics they are used to describe the latent clustering structure of infinite mixture models, see e.g. Hjort et al. (2010) and the references therein. For a comprehensive review on exhangeable random partitions from a probabilistic perspective see Pitman (2006).

Our HSSM build on exchangeable random partitions and related processes, such as species sampling sequences and species sampling random probability measures. We present their definitions and some properties which will be used in this paper.

### 2.1 Exchangeable partitions

A (set) partition of is an unordered collection of disjoint non-empty subsets (blocks) of such that . A partition has blocks (with ) and we denote by , the number of elements of the block . We denote by the collection of all partitions of and, given a partition, we list its blocks in ascending order of their smallest element. In other words, a partition is coded with elements in order of appearance.

A sequence of random partitions, , is called random partition of if for each

takes values in and, for , the restriction of to is (consistency property). A random partition of is said to be exchangeable if for every the distribution of is invariant under the action of all permutations (acting on in the natural way).

Exchangeable random partitions are characterized by the fact that their distribution depends on only through its block sizes. In point of fact, a random partition on is exchangeable if and only if its distribution can be expressed by an exchangeable partition probability function (EPPF). An EPPF is a family111 An EPPF can be seen as a family of symmetric functions defined on . To lighten the notation we simply write in place of . Alternatively, one can think that is a function on . of symmetric functions defined on the integers , with , that satisfy the additions rule

 q(n1,…,nk)=k∑j=1q(n1,…,nj+1,…,nk)+q(n1,…,nk,1),

(see Pitman (2006)). In particular, if is an exchangeable random partition of , there exists an EPPF such that for every and

 P{Πn=πn}=q(|π1,n|,…,|πk,n|) (5)

where . In other words, corresponds to the probability that is equal to any particular partition of having distinct blocks with frequencies .

Given an EPPF , one deduces the corresponding sequence of predictive distributions. Starting with , given (with ), the conditional probability of adding a new block (containing ) to is

 νn(|π1,n|,…,|πk,n|):=q(|π1,n|,…,|πk,n|,1)q(|π1,n|,…,|πk,n|); (6)

while the conditional probability of adding to the -th block of (for ) is

 ωn,c(|π1,n|,…,|πk,n|):=q(|π1,n|,…,|πc,n|+1,…,|πk,n|)q(|π1,n|,…,|πk,n|). (7)

An important class of exchangeable random partitions is the Gibbs-type partitions, introduced in Gnedin and Pitman (2005) and characterized by an EPPF with a product form, that is

 q(n1,…,nk):=Vn,kk∏c=1(1−σ)nc−1,

where is the rising factorial (or Pochhammer polynomial), and are positive real numbers such that and

 (n−σk)Vn+1,k+Vn+1,k+1=Vn,k (8)

for every and . Hereafter, we report some important examples of Gibbs-type random partitions.

###### Example 1 (Pitman-Yor two-parameter distribution).

A noteworthy example of Gibbs-type EPPF is the so-called Pitman-Yor two-parameters family, . It is defined by

 q(n1,…,nk):=∏k−1i=1(θ+iσ)(θ+1)n−1k∏c=1(1−σ)nc−1,

where and ; or and for some integer , see Pitman (1995); Pitman and Yor (1997). This leads to the following predictive rules

 νn(|π1,n|,…,|πk,n|)=θ+σkθ+nandωn,c(|π1,n|,…,|πk,n|)=|πc,n|−σθ+n.

The Pitman-Yor two-parameters family generalizes the Ewens distribution Ewens (1972), which is obtained for

 q(n1,…,nk):=θk(θ)nk∏i=1(ni−1)!

If and , then for , which means that the maximum number of blocks in a random partition of length is with probability one. It is possible to show that these random partitions can be obtained by sampling individuals from a population composed by different species with proportions distributed according to a symmetric Dirichlet distribution of parameter , see Gnedin and Pitman (2005).

###### Example 2 (Partitions induced by Mixtures of Finite Mixtures).

In Gnedin and Pitman (2005), it has been proved that any Gibbs-type EPPF with is a mixture of partitions with respect to , with mixing probability measure on the positive integers. In this case , where and

 Vn,k=|σ|k−1∑m≥kΓ(m)Γ(|σ|m+1)Γ(m−k+1)Γ(|σ|m+n)ρm. (9)

These Gibbs type EPPFs can also be obtained by considering the random partitions induced by the so-called Mixture of Finite Mixtures (MFM), see Miller and Harrison (2017). When , Gnedin (2010) shows a distribution on for which has closed-form and this special case will be described in the following example.

###### Example 3 (Gnedin model).

Gnedin (2010) introduced a sequence of exchangeable partitions with explicit predictive weights

 ωn,c(|π1,n|,…,|πk,n|) =(|πc|+1)(n−k+γ)n2+γn+ζ, (10) νn(|π1,n|,…,|πk,n|) =k2−γk+ζn2+γn+ζ, (11)

where the parameter must be chosen such that and is (i) either (strictly) positive for all or (ii) positive for and has a root at . In point of fact, the Gnedin model, denoted with GN(, can be deduced as special case of Gibbs-type EPPF with negative described in the previous example. As shown in Theorem 1 by Gnedin (2010), these random partitions have representation (9) with

 ρm=Γ(z1+1)Γ(z2+1)Γ(γ)∏m−1l=1(l2−γl+ζ)m!(m−1)!, (12)

where are complex root for the equation , that is . See also Cerquetti (2013).

###### Example 4 (Poisson-Kingman partitions).

Using the ranked random discrete distribution derived from an inhomogeneous Poisson point process, Pitman (2003) introduced a very broad class of EPPF, the so-called Poisson-Kingman exchangeable partition probability functions,

 q(n1,…,nk)=θkΓ(n)∫R+λn−1L(λ)k∏j=1∫R+ynje−λyη(y)dydλ, (13)

with Lévy density , where and is a measure on (absolutely continuous with respect to the Lebesgue measure) such that

 ∫R+(1−e−λy)η(y)dy<+∞,for allλ>0andη(R+)=+∞, (14)

and . This EPPF is related to normalized homogeneous completely random measures of James et al. (2009) (see Example 7 and Appendix A for details).

### 2.2 Species Sampling Models

Kingman’s theory of random partitions sets up a one-one correspondence (Kingman’s correspondence) between EPPFs and distributions for decreasing sequences of random variables with and a.s., by using the notion of random partition induced by a sequence of random variables.

A sequence of random variables induces a random partition on by equivalence classes if and only if . Note that if is exchangeable then the induced random partition is also exchangeable.

Kingman’s correspondence theorem states that for any exchangeable random partition with EPPF , it exists a random decreasing sequence with and , such that if are conditionally independent allocation variables with

 P{Ii=j|(~q↓k)k}=⎧⎪ ⎪ ⎪⎨⎪ ⎪ ⎪⎩~q↓jj≥1,1−∑k~q↓kj=−i,0otherwise (15)

the partition induced by has the same law of . See Kingman (1978) and Aldous (1985). When is such that a.s., Kingman’s correspondence can be made more explicit by the following result: let be the EPPF of a random partition built following the above construction and let be any (possibly random) permutation of , then

 q(n1,…,nk)=∑j1,…,jkE[k∏i=1~qniji], (16)

where ranges over all ordered -tuples of distinct positive integers, see equation (2.14) in Pitman (2006).

We call Species Sampling random probability of parameter and , , a random distribution , where are i.i.d. with common distribution (not necessarily diffuse) and the EPPF defined by via (16) is . Such random probability measures are sometimes called species sampling models.

Given an EPPF and a diffuse probability measure (i.e., for every ) on a Polish space , an exchangeable sequence taking values on is a Species Sampling Sequence, , if the law of is characterized by the predictive system:

• ;

• the conditional distribution of given is

 P{ξn+1∈dx|ξ1,…,ξn}=k∑c=1ωn,cδξ∗c(dx)+νnH(dx), (17)

where is the vector of distinct observations in order of appearance, , , , is the random partition induced by and and are related with the by (6)-(7).

See Pitman (1996).

In point of fact, as shown in Proposition 11 of Pitman (1996), is a if and only if the are conditionally i.i.d. given , with common distribution

 p=∑j≥1δZj~qj+(1−∑j≥1~qj)H, (18)

where a.s., and are stochastically independent and are i.i.d. with common diffuse distribution . The random probability measure in (18) is said to be proper if a.s. In this case, is a and the EPPF of the exchangeable partition induced by is , where and are related by (16). For more details see Pitman (1996).

The name species sampling sequences is due to the following interpretation: think to as an infinite population of individuals belonging to possibly infinite different species. The number of partition blocks takes on the interpretation of the number of distinct species in the sample , the are the observed distinct species types and the are the corresponding species frequencies. In this species metaphor, is the probability of observing a new species at the -th sample, while is the probability of observing an already sampled species of type .

### 2.3 Generalized species sampling sequences

Usually a species sampling sequence is defined by the predictive system (17) assuming that the measure is diffuse (i.e. non-atomic). While this assumption is essential for the results recalled above to be valid, an exchangeable sequence can be defined by sampling from a for any measure . More precisely, we say that a sequence is a generalized species sampling sequence, , if the variables are conditionally i.i.d. (given ) with law or equivalently if the directing measure of is . From the previous discussion, it should be clear that if is a with diffuse, then it is a , see Proposition 13 in Pitman (1996). When is not diffuse, the relationship between the random partition induced by the sequence and the EPPF is not as simple as in the non-atomic case. Understanding this relation and the partition structure of the is of paramount importance in order to define and study hierarchical models of type (3), since the random measure in the hierarchical specification (3) is almost surely discrete (i.e. atomic). Moreover, properties of these sequences are also relevant for studying non-parametric prior distributions with mixed base measure, such as the Pitman-Yor process with spike-and-slab base measure introduced in Canale et al. (2017).

Given a random partition , let be the random index of the block containing , that is

 Cj(Π)=c if j∈Πc,j (19)

or equivalently if for some (and hence all) . In the rest of the paper, if is a partition of and is any EPPF, we will write in place of .

###### Proposition 1.

Let , with proper and not necessarily diffuse. Let be the allocation sequence defined in (15) by taking the weights of in decreasing order. Assume that and (in the definition of ) are independent. Finally, let be a random partition, with EPPF equal to , also independent of . We define, for every ,

 ξn:=ZInandξ′n:=ZCn(Π).

Then:

• the sequence is a ;

• the sequences and have the same law;

• for any in the Borel -field of

 P{ξ1∈A1,⋯,ξn∈An}=∑πn∈Pnq(πn)|πn|∏c=1H(∩j∈πc,nAj),
###### Remark 1.

If is a and is not diffuse, then is not necessarily the EPPF induced by . To see this, take , and . Let be the random partition induced by and a random partition with EPPF . Using Proposition 1, one gets , if , which shows that the EPPF of is not .

When the base measure is not diffuse, the representation in Proposition 1 can be used to derive the EPPF of the partition induced by any . Since this property is not used in the rest of the paper we leave it for further research. Here we focus on the distribution of the number of distinct observations in , i.e. , for any base measure . We specialize the result for the spike-and-slab type of base measures, which have been used by Suarez and Ghosal (2016) in DP and by Canale et al. (2017) in PY processes.

###### Corollary 1.

Let the assumptions of Proposition 1 hold.

• If (for ) is the probability of observing exactly distinct values in the vector and let be the random partition induced by then,

 P{|~Πn|=d}=n∑k=dH∗(d|k)P{|Πn|=k}.
• If the base measure is in the spike-and-slab class, i.e. where , is a point of and is a diffuse measure on , then

 P{|~Πn|=d}=(1−a)dP{|Πn|=d}+n∑k=d(kd−1)ak+1−d(1−a)d−1P{|Πn|=k}.
###### Corollary 2.

Under the assumptions of Proposition 1, for every and one has with and

 P{ξ′n+1∈dx|ξ′1,…,ξ′n,Πn}=|Πn|∑c=1ωn,c(Πn)δZc(dx)+νn(Πn)H(dx) (20)

where and are related to by (6)-(7),

###### Remark 2.

Equation (20) in Corollary 2 differs substantially from the predictive system in (17) due to the conditioning on the latent partition . Nevertheless, if is diffuse then a.s., conditioning on is the same as conditioning on and is equal to the -th distinct observation in order of appearance. Hence, in this case, (20) reduces to (17).

Hereafter, we discuss some examples of and which will be used in our hierarchical constructions.

###### Example 5 (Pitman-Yor and Dirichlet processes).

If is the two-parameter Pitman-Yor distribution and , then is a Pitman-Yor process (PYP), denoted with where and are the discount and concentration parameters, respectively, and is the base measure (see Pitman (1996)). To see this equivalence, recall the description of the PYP in terms of its stick-breaking construction

 p=∑k≥1qkδZk, (21)

where is an i.i.d. sequence of random variables with law and , with independent random variables. From (21), it is clear that a is a . Moreover, it is well-known that the EPPF associated to the weights defined above is the Pitman-Yor EPPF of Example 1 (see Pitman (1995, 1996); Pitman and Yor (1997)). As a special case, if is the Ewens distribution, , and , then is a Dirichlet process (DP) denoted with . Note that this is true even if have atoms.

###### Example 6 (Mixture of finite mixture processes).

If is the distribution described in Example 2 and , then is a mixture of finite mixture process denoted with and can be written as

 p=K∑k=1qkδZk, (22)

where , is a p.m. on , , and

 (q1,…,qK)∣K∼DirichletK(σ,…,σ), (23)

see Miller and Harrison (2017). For and given by (12), an analytical expression for is available (see Example 3) and the process is called Gnedin process (GP) denoted with .

###### Example 7 (Normalized completely random measures).

Assume and let be a measure satisfying (14), a Poisson-Kingman EPPF defined in (13) and a probability measure (possibly not diffuse) on . If , then is a normalized homogeneous completely random measure, , of parameters . See Appendix A for the definition. The sequence obtained by sampling from , i.e. a , is a sequence from a . All these facts are well known when is a non-atomic measure (see James et al. (2009)). The results for general measures and are implicitly contained, although not explicitly stated, in Sangalli (2006). A detailed proof of the general case is given in Proposition 13 of Appendix A.

## 3 Hierarchical Species Sampling Models

In this section we introduce hierarchical species sampling models (HSSMs) and study the relationship between HSSMs and a general class of hierarchical random measures which contains some well-known random measures (e.g., the HDP of Teh et al. (2006)). Some examples of HSSM are provided and some relevant properties of the HSSMs are given, such as the clustering structure and the induced random partitions.

### 3.1 HSSM definition, properties and examples

In the following definition a hierarchy of exchangeable random partitions is used to build hierarchical species sampling models.

###### Definition 3.

Let and be two EPPFs and

. We define an array as a Hierarchical Species Sampling model, , of parameters , if for every vector of integer numbers and every collection of Borel sets it holds

 P{ξi,j∈Ai,ji=1,…,I,j=1,…,ni}=∑π(1)n1∈Pn1,…,π(I)nI∈PnII∏i=1q(π(i)ni)E⎡⎢ ⎢⎣I∏i=1|π(i)ni|∏c=1~p(∩j∈π(i)c,niAi,j)⎤⎥ ⎥⎦, (24)

with . Moreover, the directing random measures of the array are called Hierarchical Species Sampling random measures, .

The next result states a relationship between hierarchies of and hierarchies of random probabilities, which are widely used in Bayesian nonparametrics, thus motivating the choice of name Hierarchical Species Sampling Model (HSSM) for the stochastic representation in Definition 3.

###### Proposition 2.

Let be a vector of random probably measures such that

 pi|p0iid∼SSrp(q,p0),i=1,…,I,p0∼SSrp(q0,H0),

where is a base measure. Let be conditionally independent given with , where and . Then, is a .

Proposition 2 provides a probabilistic foundation to a wide class of hierarchical random measures. It is worth noticing that the base measure is not necessarily diffuse and, thanks to the properties of the SSrp and of the gSSS (see Proposition 1), the hierarchical random measures in Proposition 2 are well defined also for non-diffuse (e.g., atomic or mixed) probability measures . Our result is general enough to be valid for many of the existing hierarchical random measures (e.g., Teh et al. (2006), Teh (2006), Bacallado et al. (2017)). As with species sampling sequences, HSSMs enjoy some exchangeability and clustering properties stated in the following proposition, where, recalling (19), denotes the random index of the block of the random partition that contains .

###### Proposition 3.

Let be i.i.d. exchangeable partitions with EPPF and a random probability measure independent of . If are conditionally i.i.d. with law given , then the random variables

 ξi,j:=ζi,Cj(Π(i)), where i=1,…,I and j≥1 (25)

are partially exchangeable and satisfy Eq. (24). Furthermore, if is a then the sequence is a .

The stochastic representation given in Proposition 3 allows us to find a simple representation of the HSSM clustering structure (see Section 3.2). In Bayesian nonparametric inference, such representation turns out to be very useful because it leads to a generative interpretation of the nonparametric-priors in the HSSM class, and also makes possible to design general procedures for approximated posterior inference (see Section 5).

The definition of Hierarchical Species Sampling models includes some well known hierarchical processes and allows for the definition of new processes, as showed in the following set of examples.

###### Example 8 (Hierarchical Pitman-Yor process).

Let and denote PY and DP processes, respectively, given in Example 5. A vector of dependent random measures , with law characterized by the following hierarchical structure

 pi|p0iid∼PYP(σ1,θ1,p0),i=1,…,I,p0|H0∼PYP(σ0,θ0,H0) (26)

is called Hierarchical Pitman-Yor Process, , of parameters


text:
Page 1 of 1

### Writing Equation using LaTeX had never been easier!

Posted: Fri Dec 10, 2010 3:41 pm
Probably, you already have noticed the cool equation writing feature of this website. In this post we shall discuss the visual equation editor that is integrated into the forum. To learn more about LaTeX code read: How to use LaTeX.

In this forum we have integrated WYSIWYM (What You See Is What You Mean) Equation Editor in this forum. So from now on, you can write equation using LaTeX with out knowing any code! Yes, you just need to know how to use your mouse.

To check it out, find any post (example this one) and click reply. Then the message editor will open and the right most button is ; just click on it, and you'll see the Equation Editor in a new window! (like the one below)

Now write equations by clicking the buttons of the editor and type simple text using keyboard. Now after you are done, click on the button, and Voilà! You get your equation code in the message body.

Remember, you don't need to close the equation editor and for each equation in a post; you an just move to cursor to the correct place, write an equation in the Equation Editor, and click .

Features:

1. If you want inline equation, I mean smaller equation inside a line check (tick) inline, compressed from the options. It generates a code within dollar sign.
2. If you want just one equation in a single line, don't check (tick) those options. Then you'll get the codes like this

Code: Select all

``\[ your equation code \]``
.

### Re: Writing Equation using LaTeX had never been easier!

Posted: Thu Jan 06, 2011 1:14 pm
ভাইয়া ফ্লোর, সিলিং কেমনে লেখবো???

### Re: Writing Equation using LaTeX had never been easier!

Posted: Thu Jan 06, 2011 3:08 pm
\$\lfloor x \rfloor\$ = \lfloor x \rfloor
\$\lceil x \rceil\$ = \lceil x \rceil

http://www.artofproblemsolving.com/Wiki ... eX:Symbols

আর একটা কথা... latex কোড দিয়ে লেখা কোন কিছুর উপর ডাবল ক্লিক করলেই কোডটা দেখতে পাবা...

### Re: Writing Equation using LaTeX had never been easier!

Posted: Sat Oct 22, 2011 12:07 am
Can i write latex on mobile? How?

### Re: Writing Equation using LaTeX had never been easier!

Posted: Fri Nov 11, 2011 11:29 pm
you can write all the codes for latex in a mobile.after writing properly ,submit it, it will turn into latex equations.

### Re: Writing Equation using LaTeX had never been easier!

Posted: Fri Feb 21, 2014 5:03 pm
কোন লেখা Hide করে রাখা যায় কিভাবে । যেমন আমি যদি কোন প্রবলেম এর ক্লু দিতে চাই তাইলে তো আমাকে প্রথমে অইটা হাইড করে রাখতে হবে । কারো যদি ক্লুএর প্রয়োজন হয় তাইলে সে Show বাটন চাপ দিবে । এই সিস্টেম করে কিভাবে ?

### Re: Writing Equation using LaTeX had never been easier!

Posted: Sat Feb 22, 2014 12:38 am
@Kiriti
See the code given below.

Code: Select all

``````[hide] whatever you want to write [/hide]
``````
The result will be this-

### Re: Writing Equation using LaTeX had never been easier!

Posted: Sun Jun 25, 2017 5:48 pm


text:
# Calculate the electric field produced by the radiation coming from a $100\: W$ bulb at a distance of $3\: m$. Assume that the efficiency of the bulb is $2.5\ %$ and it is a point source.

$\begin {array} {1 1} (a)\;29 V/m & \quad (b)\;4.07 V/m \\ (c)\;2.9 V/m & \quad (d)\;40.7 V/m \end {array}$

The bulb, as a point source, radiates light in all directions uniformly. At a distance of $3\: m$, the surface area of the surrounding sphere is
$A = 4 \pi^2 = 4\pi(3)^2 = 113 m^2$
The intensity at this distance is
$I = \large\frac{power}{area}$$= \large\frac{100 \times 2.5\%}{113}$$ = 0.022 W/m^2$
Half of this intensity is provided by the electric field and half by the magnetic field
$\large\frac{1}{2}$$I =\large\frac{1}{2}$$ (\in_oE^2_{rms}c)$
$\Rightarrow \large\frac{1}{2}$$(0.022 W/m^2 ) = \large\frac{1}{2}$$ (\in_oE^2_{rms}c)$
$\Rightarrow E_{rms} = \sqrt{ \bigg[ \large\frac{(0.022)}{(8.85 \times 10^{-12} \times 3 \times 10^8 )} \bigg]}$$V/m = 2.9 V/m$
Since the electric field is sinusoidal, the peak electric field $E_o$ is
$E_o = \sqrt{2} E_{rms}= \sqrt{2} \times 2.9 V/m = 4.07 V/m$
Ans : (b)


text:
FirstOrder - Maple Help

Student[ODEs][Solve]

 FirstOrder
 Solve a first order ODE

 Calling Sequence FirstOrder(ODE, y(x))

Parameters

 ODE - a first order ordinary differential equation y - name; the dependent variable x - name; the independent variable

Description

 • The FirstOrder(ODE, y(x)) command finds the solution of a first order ODE.

Examples

 > $\mathrm{with}\left(\mathrm{Student}\left[\mathrm{ODEs}\right]\left[\mathrm{Solve}\right]\right):$
 > $\mathrm{ode1}≔{t}^{2}\left(z\left(t\right)+1\right)+{z\left(t\right)}^{2}\left(t-1\right)\mathrm{diff}\left(z\left(t\right),t\right)=0$
 ${\mathrm{ode1}}{≔}{{t}}^{{2}}{}\left({z}{}\left({t}\right){+}{1}\right){+}{{z}{}\left({t}\right)}^{{2}}{}\left({t}{-}{1}\right){}\left(\frac{{ⅆ}}{{ⅆ}{t}}\phantom{\rule[-0.0ex]{0.4em}{0.0ex}}{z}{}\left({t}\right)\right){=}{0}$ (1)
 > $\mathrm{FirstOrder}\left(\mathrm{ode1},z\left(t\right)\right)$
 $\frac{{{z}{}\left({t}\right)}^{{2}}}{{2}}{-}{z}{}\left({t}\right){+}{\mathrm{ln}}{}\left({z}{}\left({t}\right){+}{1}\right){=}{-}\frac{{{t}}^{{2}}}{{2}}{-}{t}{-}{\mathrm{ln}}{}\left({t}{-}{1}\right){+}{\mathrm{_C1}}$ (2)

Compatibility

 • The Student[ODEs][Solve][FirstOrder] command was introduced in Maple 2021.


text:
# Math Help - A proof involving the Law of Excluded Middle

1. ## A proof involving the Law of Excluded Middle

Good evening everyone,

With regards to ((A & B) V nA) V nB, I found this sentence to be a tautology, where the LEM applies. I am trying to create a formal proof in Fitch to show that it is indeed a tautology, but don't know which direction to go in.

Any help is greatly appreciated. Thank you

---
& represents the conjunctive
V the disjunctive
n is negation

2. ## Re: A proof involving the Law of Excluded Middle

If you are allowed to use the LEM, then this is easy. Do a disjunction elimination on A V ~A and then on B V ~B. In each of the four possibilities it is easy to derive the necessary conclusion.

3. ## Re: A proof involving the Law of Excluded Middle

Hi emakarov,

Thank you for that hint. I can obtain the same conclusions from the A and nA subproofs, as well as the B and nB ones, but I can't figure out how to link them together properly.

Here is where I am stuck:

I'm sure it's something really simple, I'm just at a block!

Thank you

4. ## Re: A proof involving the Law of Excluded Middle

You need to do \/E on B \/ ~B inside the \/E on A \/ ~A.

Code:
 1.  A \/ ~A                     LEM
2.    A                         Hyp
3.    B \/ ~B                   LEM
4.      B                       Hyp
5.      A /\ B                  /\I: 2, 4
6.      (A /\ B) \/ ~A          \/I: 5
7.      ((A /\ B) \/ ~A) \/ ~B  \/I: 6
8.      ~B                      Hyp
9.      ((A /\ B) \/ ~A) \/ ~B  \/I: 8
10.    ((A /\ B) \/ ~A) \/ ~B    \/E: 4-7, 8-9, 3
11.    ~A                        Hyp
12.    (A /\ B) \/ ~A            \/I: 11
13.    ((A /\ B) \/ ~A) \/ ~B    \/I: 12
14.  ((A /\ B) \/ ~A) \/ ~B      \/E: 2-10, 11-13, 1


text:
# Ubuntu – How to make INTEX 3.5G (aka 普创E003) Wireless USB Modem work on Ubuntu

modemusb

How to install and work with INTEX 3.5G 7.2 Mbps (aka 普创E003) Wireless USB Datacard Modem on Ubuntu?

Please assist me as a Linux beginner!
Thank You

• ## Step 1

Before plugging in your modem, check the entries in your /dev directory.

ls /dev

You'll find an entry named sr0 if you have an Optical Disk drive(CD, DVD, Blu-Ray, etc.). If you have no optical drive (as in the case of NetBooks and newer machines), then the dongle's storage will appear as sr0, and IntexModem binary'll work for you. If that's the case, you don't need to follow these steps. Else, plug in your modem, wait a few seconds and then repeat the ls /dev command to look again in /dev directory.

## Step 2

Inside /dev directory, you'll find a new entry /sr1 along with previous /sr0. This is a Read Only partition present on your modem, which must be ejected everytime you plug in the dongle, for switching the modem from removable storage state to modem state.

eject /sr1

In case you had no optical drive, you'll have to run eject /sr0 instead. Wait a few seconds and then check your Modem's USB ID:

lsusb

If everything worked as expected, it'd have turned to 20a6:1105 from the previous ID 20a6:f00e.

## Step 3

Now you have to load the kernel module to make this device recognised as a USB modem:

sudo modprobe usbserial vendor=0x20a6 product=0x1105

Now that you have succesfully loaded the module, the modem will be residing at /dev/ttyUSB0, /dev/ttyUSB1, /dev/ttyUSB2. To check, do:

ls /dev | grep ttyUSB

If nothing turns out, then wait. It can take anywhere from 30 seconds to a whole minute. Once you are sure there are these ttyUSB entries in your /dev, just use your default connection client to connect. For example in Ubuntu, the Network Manager or the connection icon on top panel.

## Background/History:

IntexModem binary didn't work for me and showed strange behaviors such as ejecting the Optical Drive. So I contacted the author of the IntexModem binary - Prateek "GunJack" Sharma, who explained to me how the binary works. It's his reply I formulated as this answer.

Newer usb-modeswitch versions contain 20a6:1105 and 20a6:f00e config files by default and so you don't need to do the above steps to get it working. As a result, if you're running a recent Ubuntu release, the modem'll work 'out of box' without any extra configuration.


text:
Display Abstract
 Title Transient growth phenomenon in a parabolic-elliptic chemotaxis system
 Name Michael Winkler Country Germany Email michael.winkler@math.upb.de Co-Author(s) Submit Time 2014-02-26 13:08:17 Session Special Session 60: Recent advances in evolutionary equations
 Contents We consider variants of the Keller-Segel system of chemotaxis which contain logistic-type source terms and thereby account for proliferation and death of cells. We briefly review results and open problems with regard to the fundamental question whether solutions exist globally in time or blow up. The primary focus will then be on the prototypical parabolic-elliptic system $\begin{array}{l} u_t=\varepsilon u_{xx} - (uv_x)_x + ru - \mu u^2, 0= v_{xx}-v+u, \end{array} \right.$ in bounded real intervals. The corresponding Neumann initial-boundary value problem, though known to possess global bounded solutions for any reasonably smooth initial data, is shown to have the property that the so-called {\em carrying capacity} $\frac{r}{\mu}$ can be exceeded dynamically to an arbitrary extent during evolution in an appropriate sense, provided that $\mu0$ is sufficiently small. This is in stark contrast to the case of the corresponding Fisher-type equation obtained upon dropping the term $-(uv_x)_x$, and hence reflects a drastic peculiarity of destabilizing action due to chemotactic cross-diffusion, observable even in the simple spatially one-dimensional setting. Numerical simulations underline the challenge in the analytical derivation of this result by indicating that the phenomenon in question occurs at intermediate time scales only, and disappears in the large time asymptotics.


text:
Bodhee Prep-Online CAT Coaching | Online CAT Preparation | CAT Online Courses

15% OFF on all CAT Courses. Discount code: BODHEE015. Valid till 7th Feb Enroll Now
| Best Online CAT PreparationFor Enquiry CALL @ +91-95189-40261

# CAT 2020 Quant Question [Slot 1] with Solution 26

Question

A solid right circular cone of height 27 cm is cut into two pieces along a plane parallel to its base at a height of 18 cm from the base. If the difference in volume of the two pieces is 225 cc, the volume, in cc, of the original cone is

1. 232
2. 256
3. 264
4. 243
Option: 4
Solution:

As the cone is cut at one-third height from the top (the vertex), the total volume is proportional to the cubes of the heights of the two parts.

Ratio of volumes two parts $= {\left( {\frac{1}{3}} \right)^3}:{1^3} = 1:27$

Hence the bottom part will have volume of $27 - 1$ i.e., 26 parts.

Given $(26 - 1)$ i.e., 25 parts -225 cc.

Hence the required answer is 27 parts $= \frac{{27 \times 225}}{{25}}$ =243 cc.

CAT Online Course @ INR 11999 only

## CAT 2020 Quant questions with Solutions

##### CAT Online Courses
FREE CAT prep Whatsapp Group


text:
# QuTiP: Independent verification of Q-CTRL optimization solutions

### Using QuTiP to simulate controls optimized with the Q-CTRL Python Package

The Q-CTRL Python Package includes functionality for creating optimized control pulses and simulating quantum systems. Performing simulations of optimized control pulses can then provide additional information about the properties of the controls, such as noise robustness. Another use for simulation is to verify that the optimized controls perform as well as predicted by the optimizer. For this purpose, using an entirely separate, independent tool provides a more meaningful validation. In this notebook, we use the QuTiP library to verify the performance of an optimized Q-CTRL pulse implementing a robust $X$ gate.

### Imports and initialization

All usage of the Q-CTRL Python Package begins by importing the qctrl package and starting a session.

# Essential imports
import numpy as np
from qctrl import Qctrl

# QuTiP imports
import qutip

# Predefined pulse imports
from qctrlopencontrols import new_predefined_driven_control

# Plotting imports
import matplotlib.pyplot as plt
from matplotlib import cm
from qctrlvisualizer import plot_controls

# Starting a session with the API
qctrl = Qctrl()

## Creating an optimized pulse

As an example, we will consider a single-qubit $X$ gate in a system obeying the following time-dependent Hamiltonian:

$$H(t) = \frac{1 + \beta_\Omega (t)}{2} \left[ \Omega(t) \sigma_- + \Omega^* (t) \sigma_+ \right] + \frac{1}{2} \Delta(t) \sigma_z + \frac{1}{2} \eta(t) \sigma_z,$$

where the Rabi frequency $\Omega(t)$ and the detuning $\Delta(t)$ are the two controls that will be optimized. These controls will be limited by a constant upper bound to their modulus, $\Delta_\mathrm{max}=\Omega_\mathrm{max}$, which we are assuming to be the same, for simplicity.

The functions $\beta_\Omega(t)$ and $\eta(t)$ represent multiplicative and additive noises, respectively, against which we will optimize our pulses.

### Obtaining robust controls

To set up a system using the Q-CTRL Python Package, we just need to follow the instructions in the Setup feature guide. In this example, we will create two versions of this system: one undergoing a robust optimized pulse, and another one just undergoing the primitive $X$ gate. These two systems will be used to compare the robustness of the two pulses. Details of how a robust optimized pulse can be calculated are found in the Optimization feature guide.

# Defining system constants
omega_max = (2.*np.pi) * 2.e6  # Hz
delta_max = omega_max
gate_duration = 1.0e-6  # s

# Defining optimization parameters
number_of_segments = 256

# Standard matrices
identity = np.identity(2, dtype=np.complex)
sigma_x = np.array([[0., 1.], [1., 0.]], dtype=np.complex)
sigma_z = np.array([[1., 0.], [0., -1.]], dtype=np.complex)
sigma_m = np.array([[0., 0.], [1., 0.]], dtype=np.complex)

# List of types of pulses to be tested and compared
schemes = {'primitive': {},
'robust': {}}

for scheme, saved_objects in schemes.items():
# Defining the system
saved_objects['system'] = qctrl.factories.systems.create(
name=scheme,
hilbert_space_dimension=2
)

# Defining controls
saved_objects['drive'] = qctrl.factories.drive_controls.create(
name=r'$\Omega$',
operator=.5*sigma_m,
system=saved_objects['system'],
)

saved_objects['shift'] = qctrl.factories.shift_controls.create(
name=r'$\Delta$',
operator=.5*sigma_z,
system=saved_objects['system'],
)

# Defining noises
saved_objects['amplitude_noise'] = qctrl.factories.control_noises.create(
control=saved_objects['drive'],
name='Amplitude noise',
)

name='Dephasing noise',
operator=.5*sigma_z,
system=saved_objects['system'],
)

# Defining primitive pulse
pulse = new_predefined_driven_control(
rabi_rotation=np.pi,
azimuthal_angle=0.,
maximum_rabi_rate=omega_max,
scheme='primitive',
name='primitive',
)

qctrl.factories.custom_pulses.create(
control=schemes['primitive']['drive'],
segments=[{'duration': d, 'value': v}
for d, v in zip(pulse.durations,
pulse.rabi_rates * np.exp(1.j*pulse.azimuthal_angles))],
)

qctrl.factories.custom_pulses.create(
control=schemes['primitive']['shift'],
segments=[{'duration': d, 'value': v}
for d, v in zip(pulse.durations, pulse.detunings)],
)

schemes['primitive']['duration'] = np.cumsum(pulse.durations)[0]

# Defining robust pulses
qctrl.factories.optimum_pulses.create(
control=schemes['robust']['drive'],
duration=gate_duration,
fixed_modulus=False,
segment_count=number_of_segments,
upper_bound=omega_max,
maximum_slew_rate=omega_max/5.,
)

qctrl.factories.optimum_pulses.create(
control=schemes['robust']['shift'],
duration=gate_duration,
fixed_modulus=False,
segment_count=number_of_segments,
upper_bound=delta_max,
maximum_slew_rate=delta_max/10.,
)

schemes['robust']['duration'] = gate_duration

# Defining the target
for saved_objects in schemes.values():
qctrl.factories.targets.create(
projection_operator=identity,
system=saved_objects['system'],
unitary_operator=sigma_x,
)

# Calculating the robust pulse
qctrl.services.robust_optimization.run(system=schemes['robust']['system'])

for scheme in schemes.values():
scheme['system'].refresh()

100%|██████████| 23/23 [04:18<00:00, 11.26s/it, running=0]

# Plotting controls using the Q-CTRL Visualizer Python Package
for scheme, saved_objects in schemes.items():
saved_objects['controls'] = {control.name: control.pulse.segments
for control in saved_objects['system'].controls
if hasattr(control.pulse, 'segments')}

fig = plt.figure()
fig.suptitle(scheme.capitalize() + " controls")
plot_controls(fig, saved_objects['controls'])

Above we show a visual representation of what the controls look like, using the Q-CTRL Visualizer Python Package. Here, $\Omega(t)$ represents the Rabi rate and $\Delta(t)$ is the detuning, according to the Hamiltonian:

$$H(t) = \frac{1 + \beta_\Omega (t)}{2} \left[ \Omega(t) \sigma_- + \Omega^* (t) \sigma_+ \right] + \frac{1}{2} \Delta(t) \sigma_z + \frac{1}{2} \eta(t) \sigma_z.$$

## Independent verification with QuTiP

QuTiP is an open-source library for simulating open quantum systems in Python. We will use it to simulate the evolution of a system undergoing our optimized pulse. By calculating the fidelity of the gate at the end of the simulation, we are able to characterize the robustness of the gate against different kinds of errors. As QuTiP has tools to perform the simulation of open systems, we will use it to characterize our pulses in the presence of a loss process. We will also compare their scans of quasi-static noise with those generated with the Q-CTRL Python Package for the case of non-Markovian noise, to check that they match.

### Converting the controls into QuTiP format and simulating them

The first step to perform the independent verification is to make sure that QuTiP can understand the controls that we are applying to the system. Once these controls have been adapted into QuTiP format, we can simulate their evolution using both packages, and compare the results. Starting with a qubit initially in the state $\left| 0 \right\rangle$, the expected evolution with the $X$ gate is to flip it to $\left| 1 \right\rangle$.

By setting the noise functions to zero in the system Hamiltonian, we obtain the evolution of the pulses in the ideal case. Both the Q-CTRL Python Package and QuTiP are capable of calculating these noiseless simulations, and a comparison of them is a first test of whether the pulses are being treated the same by both packages.

# Transforming the piecewise constant controls into functions of time
segment = lambda segments, duration, t: min(segments-1, int(np.floor(segments*t/duration)))
Delta = lambda controls, duration, t, args: controls[r'$\Delta$'][segment(len(controls[r'$\Delta$']),
duration, t)]['value']
Omega = lambda controls, duration, t, args: controls[r'$\Omega$'][segment(len(controls[r'$\Omega$']),
duration, t)]['value']

# Defining the time-dependent Hamiltonian in a a format that QuTiP can parse
for scheme, saved_objects in schemes.items():
saved_objects['H'] = [[.5*qutip.sigmam(), (lambda t, args: Omega(saved_objects['controls'],
saved_objects['duration'], t, args))],
[.5*qutip.sigmap(), (lambda t, args: Omega(saved_objects['controls'],
saved_objects['duration'], t, args).conj())],
[.5*qutip.sigmaz(), (lambda t, args: Delta(saved_objects['controls'],
saved_objects['duration'], t, args))]]

# Defining initial state and range of times
ket_0 = np.array([1., 0.], dtype=np.complex)
point_times = np.linspace(0., gate_duration, 101)

# Defining and calculating simulation with the Q-CTRL Python Package
for saved_objects in schemes.values():
qctrl.factories.coherent_simulations.create(
initial_state_vector=ket_0,
point_times=point_times[point_times <= saved_objects['duration']],
system=saved_objects['system'],
)

qctrl.services.coherent_simulations.run(saved_objects['system'])

saved_objects['system'].refresh()

100%|██████████| 2/2 [00:11<00:00,  5.58s/it, running=0]
100%|██████████| 2/2 [00:10<00:00,  5.38s/it, running=0]

qutip_ket_0 = qutip.basis(2, 0)
qutip_ket_1 = qutip.basis(2, 1)

# Calculating the coherent evolutions using QuTiP
qutip_noiseless_simulations = {}
for scheme, saved_objects in schemes.items():
qutip_noiseless_simulations[scheme] = qutip.sesolve(
saved_objects['H'], qutip_ket_0, point_times[point_times <= saved_objects['duration']], [])

for scheme, saved_objects in schemes.items():
population_0 = [np.abs(frame['state_vector'][0])**2.
for frame in saved_objects['system'].simulations[0].trajectories[0].frames]
population_1 = [np.abs(frame['state_vector'][1])**2.
for frame in saved_objects['system'].simulations[0].trajectories[0].frames]

qutip_population_0 = [np.abs(ket[0][0][0])**2.
for ket in qutip_noiseless_simulations[scheme].states]
qutip_population_1 = [np.abs(ket[1][0][0])**2.
for ket in qutip_noiseless_simulations[scheme].states]

fig = plt.figure()
fig.suptitle(scheme.capitalize() + " controls")
plt.plot(point_times[point_times <= saved_objects['duration']]/1e-6,
qutip_population_0, label="Population |0> (QuTiP)", color="#BF04DC", alpha=0.6)
plt.plot(point_times[point_times <= saved_objects['duration']]/1e-6,
qutip_population_1, label="Population |1> (QuTiP)", color="#680CE9", alpha=0.6)
plt.plot(point_times[point_times <= saved_objects['duration']]/1e-6,
population_0, ':', label="Population |0> (Q-CTRL)", color="#BF04DC")
plt.plot(point_times[point_times <= saved_objects['duration']]/1e-6,
population_1, ':', label="Population |1> (Q-CTRL)", color="#680CE9")
plt.xlim(0., saved_objects['duration']/1e-6)
plt.ylim(0., 1.)
plt.xlabel("Time ($\\mu$s)")
plt.ylabel("Probability")
plt.legend(loc=3, bbox_to_anchor=(1., 0.7))

The graphs above show the evolution of the state populations during the primitive and the robust X gate. Both reach the same final state in this noiseless simulation, but the robust case has a more intricate trajectory. The simulations obtained via the Q-CTRL Python Package (dotted) and QuTiP (solid) are so close to the point of being indistinguishable.

### Comparing quasi-static scans of amplitude errors

A quasi-static scan can help us characterize the robustness of a set of controls against a particular kind of noise. To obtain them, we give a constant value to the noise throughout the application of the gate, and then observe how varying this static value affects the final infidelity of the system. In the case of amplitude errors, our quasi-static scan will consist in varying a parameter that multiplies the Rabi frequency of the system.

The Q-CTRL Python Package provides built-in functions that calculate this, as detailed in the Quasi-static scans feature guide. For QuTiP, we will manually provide Hamiltonians with each of the static values of the noise parameter, and calculate the final infidelity. As we are interested in the operational fidelity, in QuTiP we will calculate the evolution of two qubits in a Bell state, only one of which evolves according to the Hamiltonian, and then calculate their final state fidelity. The Q-CTRL Python Package already outputs the results in terms of the operational infidelity, so the intermediary step of calculating the entanglement infidelity is not necessary.

# Range of the scan
beta_values = np.linspace(-.5, .5, 101)

# Calculating the scan with the Q-CTRL Python Package
for saved_objects in schemes.values():
amplitude_profile = qctrl.factories.quasi_static_functions.create(
x_noise=saved_objects['amplitude_noise'],
x_coefficients=beta_values,
)

beta_scan = qctrl.services.quasi_static_functions.calculate(
amplitude_profile)

saved_objects['beta_infidelities'] = [point['infidelity']
for point in beta_scan.sampled_points]

100%|██████████| 2/2 [00:11<00:00,  5.89s/it, running=1]
100%|██████████| 2/2 [00:22<00:00, 11.11s/it, running=1]

# Defining the Bell state and its ideal evolution
bell_state = qutip.bell_state()
ideal_final_state = qutip.tensor(
qutip.sigmax(), qutip.identity(2)) * bell_state

# Calculating the scan with QuTiP
for saved_objects in schemes.values():
saved_objects['qutip_beta_infidelities'] = []
for beta in beta_values:
Hamiltonian = [[.5*(1.+beta)*qutip.sigmam(),
(lambda t, args: Omega(saved_objects['controls'],
saved_objects['duration'], t, args))],
[.5*(1.+beta)*qutip.sigmap(),
(lambda t, args: Omega(saved_objects['controls'],
saved_objects['duration'], t, args).conj())],
[.5*qutip.sigmaz(),
(lambda t, args: Delta(saved_objects['controls'],
saved_objects['duration'], t, args))]]

for term in Hamiltonian:
term[0] = qutip.tensor(term[0], qutip.identity(2))

qutip_simulation = qutip.sesolve(
Hamiltonian, bell_state, point_times[point_times <= saved_objects['duration']], [])
qutip_final_state = qutip_simulation.states[-1]

saved_objects['qutip_beta_infidelities'].append(
1.-qutip.fidelity(ideal_final_state, qutip_final_state)**2.)

# Plotting the results
for scheme, saved_objects in schemes.items():
fig = plt.figure()
fig.suptitle(scheme.capitalize() + " controls")
plt.xlabel(r"Static amplitude noise, $\beta_\Omega$")
plt.ylabel("Infidelity")
plt.xlim(-.5, .5)
plt.ylim(0., .5)
plt.plot(beta_values, saved_objects['qutip_beta_infidelities'], label="QuTiP", color="#BF04DC")
plt.plot(beta_values, saved_objects['beta_infidelities'], ':', label="Q-CTRL", color="#680CE9")
plt.legend()

In the graphs above we can see how a static amplitude noise represented by $\beta_\Omega$ can affect the fidelity of the system. This parameter $\beta_\Omega$ represents a multiplicative error in the amplitude of $\Omega(t)$, as we had defined in the Hamiltonian of the system:

$$H(t) = \frac{1 + \beta_\Omega}{2} \left[ \Omega(t) \sigma_- + \Omega^* (t) \sigma_+ \right] + \frac{1}{2} \Delta(t) \sigma_z.$$

Notice that for the purposes of this robustness characterization we are not taking into account dephasing noise.

The quasi-static scan of the robust pulse presents a much flatter profile around the origin than the primitive pulse, showing that the robust controls are less sensitive to amplitude noise. We also see that quasi-static scans obtained by QuTiP and the Q-CTRL Python Package match closely.

### Comparing quasi-static scans of dephasing noise

The procedure for generating a quasi-static scan for dephasing noise is similar to the process we used above for amplitude errors, the difference being that now we will add a static term containing $\sigma_z$ to the Hamiltonian, instead of changing a multiplicative factor. Once again, we can compare the robustness of the optimized pulse against the primitive, and also compare the results obtained via the Q-CTRL Python Package with those from QuTiP.

# Range of the scan
eta_values = delta_max*np.linspace(-.5, .5, 101)

# Defining and calculating the scan
for saved_objects in schemes.values():
dephasing_profile = qctrl.factories.quasi_static_functions.create(
x_noise=saved_objects['dephasing_noise'],
x_coefficients=eta_values,
)

eta_scan = qctrl.services.quasi_static_functions.calculate(dephasing_profile)

saved_objects['eta_infidelities'] = [point['infidelity'] for point in eta_scan.sampled_points]

100%|██████████| 2/2 [00:17<00:00,  8.86s/it, running=1]
100%|██████████| 2/2 [00:27<00:00, 13.92s/it, running=0]

# Calculating the scan with QuTiP
for saved_objects in schemes.values():
saved_objects['qutip_eta_infidelities'] = []
for eta in eta_values:
Hamiltonian = ([[qutip.tensor(term[0], qutip.identity(2)), term[1]]
for term in saved_objects['H']]
+[.5*eta*qutip.tensor(qutip.sigmaz(), qutip.identity(2))])

qutip_simulation = qutip.sesolve(
Hamiltonian, bell_state, point_times[point_times <= saved_objects['duration']], [])
qutip_final_state = qutip_simulation.states[-1]

saved_objects['qutip_eta_infidelities'].append(1.-qutip.fidelity(ideal_final_state, qutip_final_state)**2.)

# Plotting the results
for scheme, saved_objects in schemes.items():
fig = plt.figure()
fig.suptitle(scheme.capitalize() + " controls")
plt.xlabel(r"Static dephasing noise, $\eta/\Delta_\mathrm{max}$")
plt.ylabel("Infidelity")
plt.xlim(-.5, .5)
plt.ylim(0., .5)
plt.plot(eta_values/delta_max, saved_objects['qutip_eta_infidelities'], color="#BF04DC", label="QuTiP")
plt.plot(eta_values/delta_max, saved_objects['eta_infidelities'], ':', color="#680CE9", label="Q-CTRL")
plt.legend()

In the graphs above, we are seeing the infidelity of the primitive and optimized controls at the end of the gate, when subjected to static dephasing noise represented by the parameter $\eta$, as defined in the Hamiltonian:

$$H(t) = \frac{1}{2} \left[ \Omega(t) \sigma_- + \Omega^* (t) \sigma_+ \right] + \frac{1}{2} \Delta(t) \sigma_z + \frac{1}{2} \eta \sigma_z.$$

Notice that for the purposes of this characterization we are assuming there is no amplitude noise.

Here we once again see a flatter profile of the quasi-static scan around the origin for the case of the optimized pulse, representing less sensitivity to dephasing noise. Again, the results obtained with QuTiP and the Q-CTRL Python Package are indistinguishably close.

### Obtaining a quasi-static scan of dephasing noise with a loss process

Besides these static coherent noises, QuTiP also allows us to calculate the evolution of an open system. In particular, we can model the same quasi-static evolution used in the dephasing scans above, but this time including a finite value for the time $T_1$, which characterizes a simultaneous loss process.

These open-system dynamics can be simulated in QuTiP as the solution for a Lindblad equation, in which we include extra Lindblad operators $L_i$ to the original Schrödinger equation:

$$\frac{\mathrm{d}}{\mathrm{d}t} \rho(t) = - i \left[ H(t), \rho(t) \right] + \sum_i \left[ L_i \rho(t) L_i^\dagger - \frac{1}{2} \left\{ \rho(t), L_i^\dagger L_i \right\} \right].$$

Above, $\rho(t)$ is the density matrix and $H(t)$ is the original Hamiltonian.

For the loss process, the Lindblad operator to be included is $\sigma_-/\sqrt{T_1}$, which, in isolation, reduces the population in the $\left| 0 \right\rangle$ state by $e^{-t/T_1}$. Its joint dynamics together with a static term representing the dephasing process will be simulated below with QuTiP, the results then used to generate quasi-static scans that represent a situation with both finite $T_1$ and $T_2$.

In this simulation, $T_1$ will be represented by a Markovian loss process, while $T_2$ will be represented by a static detuning in place of a non-Markovian dephasing noise. We choose our $T_1$ as being 100 times longer than our robust pulse, which means $T_1 = 100 \mu \mathrm{s}$. This number is of the same order of magnitude as the $T_1$ in current superconducting devices.

t1 = 100*gate_duration

# Using QuTiP to calculate the scan
for saved_objects in schemes.values():
saved_objects['qutip_eta_t1_infidelities'] = []
for eta in eta_values:

Hamiltonian = ([[qutip.tensor(term[0], qutip.identity(2)), term[1]]
for term in saved_objects['H']]
+[.5*eta*qutip.tensor(qutip.sigmaz(), qutip.identity(2))])

qutip_simulation = qutip.mesolve(
Hamiltonian, bell_state, point_times[point_times <= saved_objects['duration']], lindbladian)
qutip_final_state = qutip_simulation.states[-1]

saved_objects['qutip_eta_t1_infidelities'].append(
1.-qutip.fidelity(ideal_final_state, qutip_final_state)**2.)

# Plotting the results
for scheme, saved_objects in schemes.items():
fig = plt.figure()
fig.suptitle(scheme.capitalize() + " controls")
plt.xlabel(r"Static dephasing noise ($\eta/\Delta_\mathrm{max}$)")
plt.ylabel("Infidelity")
plt.xlim(-.5, .5)
plt.ylim(0., .5)
plt.plot(eta_values/delta_max, saved_objects['qutip_eta_t1_infidelities'], color="#BF04DC")

In the graphs above, we perform a scan of static values of $\eta$, which are constant detuning terms added to the Lindblad equation:

$$\frac{\mathrm{d}}{\mathrm{d}t} \rho(t) = - i \left[ \frac{1}{2} \left[ \Omega(t) \sigma_- + \Omega^* (t) \sigma_+ \right] + \frac{1}{2} \Delta(t) \sigma_z + \frac{1}{2} \eta \sigma_z, \rho(t) \right] + \frac{1}{T_1} \left[ \sigma_- \rho(t) \sigma_+ - \frac{1}{2} \left\{ \rho(t), \sigma_+ \sigma_- \right\} \right],$$

As we are only interested in errors caused by the finite-time $T_1$ and $T_2$, we will be ignoring the effect of amplitude noise. The loss process, however, is still present in the form of the Lindbladian term in the equation above.

The graphs above still show a flatter surface for the robust controls, when compared to the primitive pulse. However, the presence of the loss process causes the robust curve to be shifted up, due to the fact that the robust pulse is four times longer than the primitive one.

### Obtaining simultaneous scans of dephasing noise and a loss process

Once we have the ability to include both dephasing terms and a loss process in our simulations, we can perform scans of the sensitivity of our noise to these two sources of error. Now, we go beyond the previous section, where we only changed the static dephasing term that represents $T_2$, and also vary the value of $T_1$.

# Using QuTiP to calculate the scan
t1_values = np.linspace(1*gate_duration, 500*gate_duration, 101)

for saved_objects in schemes.values():
saved_objects['qutip_t1_t2_infidelities'] = []
for t1 in t1_values:
saved_objects['qutip_t1_t2_infidelities'].append([])
for eta in eta_values:

Hamiltonian = ([[qutip.tensor(term[0], qutip.identity(2)), term[1]]
for term in saved_objects['H']]
+ [.5*eta*qutip.tensor(qutip.sigmaz(), qutip.identity(2))])

qutip_simulation = qutip.mesolve(
Hamiltonian, bell_state, point_times[point_times <= saved_objects['duration']], lindbladian)
qutip_final_state = qutip_simulation.states[-1]

saved_objects['qutip_t1_t2_infidelities'][-1].append(
1.-qutip.fidelity(ideal_final_state, qutip_final_state)**2.)

# Plotting the results
X, Y = np.meshgrid(np.arange(len(eta_values)), np.arange(len(t1_values)))

for scheme, saved_objects in schemes.items():
Z = np.array(saved_objects['qutip_t1_t2_infidelities'])
fig = plt.figure(figsize=(7, 5))
plt.title(scheme.capitalize() + " controls")
contours = plt.contour(X, Y, Z, levels=[0.005, .01, .05, .1], colors='#680CEA')
plt.clabel(contours, inline=True, fontsize=8)
cmap_reversed = plt.cm.get_cmap('gray').reversed()
plt.imshow(Z, cmap=cmap_reversed, vmin=0, vmax=0.5)
plt.xticks(np.linspace(0, len(eta_values), 5),
[str(np.around(value, 2))
for value in np.linspace(min(eta_values/omega_max), max(eta_values/omega_max), 5)])
length = len(t1_values)
plt.yticks(np.concatenate((np.array([0]), np.linspace(length/5, length, 5)-1./length)),
['1.0'] + [str(np.around(value, 0))
for value in np.linspace(max(t1_values/1e-6)/5, max(t1_values/1e-6), 5)])
plt.xlabel(r"Static dephasing, $\eta/\Omega_\mathrm{max}$")
plt.ylabel(r"$T_1$ ($\mu$s)")
cbar = plt.colorbar()
cbar.set_ticks(np.linspace(0, 0.5, 6))
cbar.set_ticklabels(['0', '0.1', '0.2', '0.3', '0.4', '0.5'])
cbar.set_label('Infidelity')
plt.tight_layout()

The heat maps above show how the infidelity at the end of the gate changes when we vary a static dephasing term (related to the $T_2$ noise) and the loss rate (the inverse of $T_1$). Auxiliary contours in purple show where specific values of the infidelity occur.

We see again that, for a fixed value of $T_1$, the optimized controls present a wider region around the origin where the infidelity remains low, showing that they are more robust against dephasing noise. The longer duration of the robust pulse also makes it more sensitive to a decrease in $T_1$, which shifts further up the base value of its infidelity. This is visible in how the contour lines at the top of the graph close at a lower point for the robust controls than for the primitive ones.


text:
# Acids and Bases

## Key Questions

A compound with a proton ${\text{H}}^{+}$ that is easily dissociated (removed) into solution. Another way of looking at this is by considering the pH. A pH < 3-4 at ${25}^{\circ} \text{C}$ is considered strongly acidic.
Generally a compound that loses a proton ${\text{H}}^{+}$ in an acid-base reaction which forms a weak (stable) conjugate base in turn is considered a strong acid.


text:
The inclusive jet cross section in $\bar{p}p$ collisions at $\sqrt{s} = 1.8$ TeV

The collaboration
Phys.Rev.Lett. 82 (1999) 2451-2456, 1999.

Abstract (data abstract)
FERMILAB-COLLIDER. Precise measurement of the central inclusive jet cross section in PBAR P collisions at a centre-of-mass energy of 1.8 TeV. Data comes from sample of integrated luminosity 92 pb-1.


text:
# Solution to this equation

by zetafunction
Tags: equation, solution
 P: 399 given any function (or maybe distribution) f(x) and g(x) so $$f(x+i\epsilon ) - f(x-i\epsilon ) = g(x)$$ if we know f(x) could we obtain g(x) from the difference above ?? if we knew g(x) could we solve the equation to get f(x) ?? here $$\epsilon \rightarrow 0$$


text:
📖 You can now grab a copy of our new Deep Learning in Production Book 📖

# Speech synthesis: A review of the best text to speech architectures with Deep Learning

Sergios Karagiannakoson2021-05-13·14 mins
Audio

Speech synthesis is the task of generating speech from some other modality like text, lip movements, etc. In most applications, text is chosen as the preliminary form because of the rapid advance of natural language systems. A Text To Speech (TTS) system aims to convert natural language into speech.

Over the years there have been many different approaches, with the most prominent being concatenation synthesis and parametric synthesis.

## Concatenation synthesis

Concatenation synthesis, as the name suggests, is based on the concatenation of pre-recorded speech segments. The segments can be full sentences, words, syllables, diphones, or even individual phones. They are usually stored in the form of waveforms or spectrograms.

We acquire the segments with the help of a speech recognition system and we then label them based on their acoustic properties (e.g. their fundamental frequency). At run time, the desired sequence is created by determining the best chain of candidate units from the database (unit selection).

## Statistical Parametric Synthesis

Parametric synthesis utilizes recorded human voices as well. The difference is that we use a function and a set of parameters to modify the voice. Let’s break that down:

In statistical parametric synthesis, we generally have two parts. The training and the synthesis. During training, we extract a set of parameters that characterize the audio sample such as the frequency spectrum (vocal tract), fundamental frequency (voice source), and duration (prosody) of speech. We then try to estimate those parameters using a statistical model. The one that has been proven to provide the best results historically is the Hidden Markov Model (HMM).

During synthesis, HMMs generate a set of parameters from our target text sequence. The parameters are used to synthesize the final speech waveforms.

• No need to store audio sample in a database

• Language independence

• Flexibility in voice characteristics

However, in most cases, the quality of the synthesized speech is not ideal. This is where Deep Learning based methods come into play.

But before that, I would like to open a small parenthesis and discuss how we evaluate speech synthesis models.

## Speech synthesis evaluation

Mean Opinion Score (MOS) is the most frequently used method to evaluate the quality of the generated speech. MOS has a range from 0 to 5 where real human speech is between 4.5 to 4.8

MOS comes from the telecommunications field and is defined as the arithmetic mean over single ratings performed by human subjects for a given stimulus in a subjective quality evaluation test. This historically means that a group of people sits in a quiet room, listens to the generated sample, and gives it a score. MOS is nothing more than the average of all “people’s opinion”

Today’s benchmarks are performed over different speech synthesis datasets in English, Chinese, and other popular languages. You can find such benchmarks in paperswithcode.com

## Speech synthesis with Deep Learning

Before we start analyzing the various architectures, let’s explore how we can mathematically formulate TTS.

Given an input text sequence $\mathbf{Y}$ , the target speech $\mathbf{X}$ can be derived by:

$X=\arg \max P(X|Y,\theta )$

where $\theta$ is the model’s parameters.

In most models, we first pass the input text to an acoustic feature generator, which produces a set of acoustic features such as the fundamental frequency or spectrogram.

To generate the final speech segment, a Neural vocoder is typically used.

A traditional vocoder is a category of voice codec which encrypts and compresses the audio signal and vice versa. This was traditionally accomplished through digital signal processing techniques. A neural vocoder achieves the encoding/decoding using a neural network.

## WaveNet

WaveNet was the first model that successfully modeled the raw waveform of the audio signal instead of the acoustic features. It is able to generate new speech-like waveforms at 16,000 samples per second.

WaveNet in its core is an autoregressive model where each sample depends on the previous ones. Mathematically, this can be expressed as :

$p_{\theta }(\mathbf {x} )=\prod _{t=1}^{T}p(x_{t}|x_{1},...,x_{t-1})$

In essence, we factorize the joining probability of the waveform as a product of conditional probabilities of the previous time steps.

To build such autoregressive models, the authored used a fully convolutional neural network with dilated convolutions. WaveNet was inspired by PixelCNN and PixelRNN, which are able to generate very complex natural images.

As we can see in the image above, each convolutional layer has a dilation factor. They used real waveforms recorded from human speakers during training. After training, the final waveform is produced by sampling from the network. How the sampling is performed?

The autoregressive model computes the probability distribution $p_{\theta }(\mathbf {x} )$. At each timestep:

• We sample a value from the distribution

• We feed the value back to the input, and the model generates the new prediction

• We continue this procedure one step at a time to generate the entire speech waveform.

This is the main shortcoming of WaveNet. Because we need to perform this for every simple sample, inferences can become very slow and computationally expensive

The first version of WaveNet managed to has a MOS of 4.21 in the English language where for previous state of art models, MOS was between 3.67 and 3.86.

### Fast WaveNet

Fast WaveNet managed to reduce the complexity of the original WaveNet from $O( 2^L)$ to $O(L)$ where $L$ is the number of layers in the network. This was achieved by introducing a caching system that stored previous calculations. That way no redundant convolutions were ever be calculated.

The caching scheme of Fast WaveNet. Source: Fast WaveNet Generation Algorithm

## Deep Voice

Deep Voice by Baidu laid the foundation for the later advancements on end-to-end speech synthesis. It consists of 4 different neural networks that together form an end-to-pipeline.

1. A segmentation model that locates boundaries between phonemes. It is a hybrid CNN and RNN network that is trained to predict the alignment between vocal sounds and the target phonemes using the CTC loss.

2. A model that converts graphemes to phonemes. A multi-layer encoder-decoder model with GRU cells was chosen for this task.

3. A model to predict phonemes duration and the fundamental frequencies. Two fully connected layers followed by two unidirectional GRU layers and another fully connected layer, were trained to learn both tasks simultaneously

4. A model to synthesize the final audio. Here the authors implemented a modified WaveNet. The WaveNet consists of a conditioning network that upsamples linguistic features to the desired frequency, and an autoregressive network, which generates a probability distribution P over discretized audio samples

System diagram depicting (a) training procedure and (b) inference procedure of DeepVoice. Source: Deep Voice: Real-time Neural Text-to-Speech

They also managed to achieve real-time inference by constructing highly optimized CPU and GPU kernels to speed up the inference. It received a MOS of 2.67 in US English.

## Tacotron

Tacotron was released by Google in 2017 as an end-to-end system. It is basically a sequence to sequence model that follows the familiar encoder-decoder architecture. An attention mechanism was also utilized.

Let’s break down the above diagram.

The model takes as input characters and outputs the raw spectrogram of the final speech, which is then converted to waveform.

### The CBHG module

You might wonder what is this CBHG. CBHG stands for: 1-D convolution bank + highway network + bidirectional GRU. The CBHG module is used to extract representations from sequences, and it was originally developed for neural machine translation. The below diagram will give you a better understanding:

The CBHG module. Source: Tacotron: Towards End-to-End Speech Synthesis

Back to Tacotron. The encoder’s goal is to extract robust sequential representations of text. It receives a character sequence represented as one-hot encoding and through a stack of PreNets and CHBG modules, it outputs the final representation. PreNet is used to describe the non-linear transformations applied to each embedding.

Content-based attention is used to pass the representation to the decoder, where a recurrent layer produces the attention query at each time step. The query is concatenated with the context vector and passed to a stack of GRU cells with residual connections. The output of the decoder is converted to the end waveform with a separate post-processing network, containing a CBHG module.

Tacotron achieved a MOS of 3.82 on an US English evaluation set.

## Deep Voice 2

Deep Voice 2 came as an improvement of the original Deep Voice architecture. While the main pipeline was quite similar, each model was created from scratch to enhance its performance. Another big enhancement was the addition of multi-speaker support.

Key points of the architecture:

• Separation of the phoneme duration and fundamental frequency models

• Speaker embeddings were introduced on each model to achieve multiple-speaker capabilities. The speaker embeddings hold the unique information per speaker and are used to produce recurrent neural network (RNN) initial states, nonlinearity biases, and multiplicative gating factors, used throughout the networks.

• Batch normalization and residual connections were applied to the basic models

Segmentation, duration and frequency models of DeepVoice 2. Source: Deep Voice 2: Multi-Speaker Neural Text-to-Speech

A surprising fact is that the authors showed, in the same paper, that we can also enhance Tacotron to support multi-speakers using similar techniques. Moreover, they replace Tacotron’s spectrogram-to-waveform Model with their own WaveNet-based neural vocoder and the results were very promising

DeepVoice 2 with an 80-layer WaveNet, as the sound synthesizer model, achieved a MOS of 3.53

## Deep Voice 3

Deep Voice 3 is a complete redesign of the previous versions. Here we have a single model instead of four different ones. More specifically, the authors proposed a fully-convolutional character-to-spectrogram architecture which is ideal for parallel computation. As opposed to RNN-based models. They were also experimenting with different waveform synthesis methods with the WaveNet achieving the best results once again.

As you can see, Deep Voice 3 is an encoder-decoder architecture and is able to produce a variety of textual features(character, phonemes, etc.) to a variety of vocoder parameters.

The encoder is a fully-convolutional neural network that transforms textual features into a compact representation. The decoder is another fully-convolutional network that converts the learned representation into a low-dimensional audio representation. This is achieved using a multi-hop convolutional attention mechanism.

The convolution block comprises 1-D convolutions followed by a GRU cell and a residual connection.

The convolution block of Deep Voice 3. Source: Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

The attention mechanism uses a query vector (the hidden states of the decoder) and the per-timestep key vectors from the encoder to compute attention weights. It then outputs a context vector as the weighted average of the value vectors.

Attention block of Deep Voice 3. Source: Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

Deep Voice 3 with WaveNet achieved a MOS of 3.78 at the time of publishing

## Parallel WaveNet

Parallel WaveNet aims to solve the complexity and performance issues of the original WaveNet, which relies on sequential generation of the audio, one sample at a time.

They introduced a concept called Probability Density Distillation that tries to marry Inverse autoregressive flows with efficient WaveNet training methods.

Inverse autoregressive flows (IAFs) represent a kind of dual formulation of deep autoregressive modelling, in which sampling can be performed in parallel. IAFs are stochastic generative models whose latent variables are arranged so that all elements of a high dimensional observable sample can be generated in parallel

Let’s break that down and explain it in simple terms:

Because each sample depends on the previous ones, we can’t simple parallelized this process and compute them in parallel. Instead, we start out from simple white noise and apply changes over time until it morphs to the desired output waveform. These changes are applied to the entire signal in a parallel fashion. How?

We use a teacher-student relationship. The teacher is the original Network that holds the ground truth but it is quite slow. The student is the new network that tries to mimic the teacher but in a more efficient way.

According to the authors: “To stress the fact that we are dealing with normalized density models, we refer to this process as Probability Density Distillation (in contrast to Probability Density Estimation). The basic idea is for the student to attempt to match the probability of its own samples under the distribution learned by the teacher”

Overview of Probability Density Distillation. Source: Parallel WaveNet: Fast High-Fidelity Speech Synthesis

Parallel WaveNet is 1000 times faster than the original networks and can produce 20 seconds of audio in 1 second.

Also, note that similar techniques with IAFs to parallelize wave generation have also been used by other architectures such as ClariNet

## Tacotron 2

Tacotron 2 improves and simplifies the original architecture. While there are no major differences, let’s see its key points:

• The encoder now consists of 3 convolutional layers and a bidirectional LSTM replacing PreNets and CHBG modules

• Location sensitive attention improved the original additive attention mechanism

• The decoder is now an autoregressive RNN formed by a Pre-Net, 2 uni-directional LSTMs, and a 5-layer Convolutional Post-Net

• A modified WaveNet is used as the Vocoder that follows PixelCNN++ and Parallel WaveNet

• Mel spectrograms are generated and passed to the Vocoder as opposed to Linear-scale spectrograms

• WaveNet replaced the Griffin-Lin algorithm used in Tacotron 1

Tacotron 2. Source: Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions

Tacotron 2 received an impressive MOS of 4.53.

## Global Style Tokens (GST)

Global Style Tokens is a new idea to augment Tacotron-based architectures . The authors proposed a bank of embeddings that can be trained jointly with Tacotron in an unsupervised manner (also referred as GST-Tacotron). The embeddings represent the acoustic expressiveness of different speakers and are trained with no explicit labels. In other words, they aim to model different speaking styles.

During training, a reference encoder is used to extract a fixed-length vector which encodes the information about the speaking style (also known as prosody). This is then passed to the “style token layer”, an attention layer that calculates the contribution of each token to the resulting style embedding.

During inference, a reference audio sequence can be used to produce a style embedding or we can manually control the speech style.

## TTS with Transformers

Transformers are dominating the Natural Language field for a while now, so it was inevitable that they will gradually enter the TTS field. Transformers-based models aim to tackle two problems of previous TTS methods such as Tacotron2:

• Low efficiency during training and inference

• Difficulty in modeling long dependencies using RNNs

The first transformers-based architecture introduced in 2018 and replaced RNNs with multi-head attention mechanisms that can be trained in parallel.

As you can see above, the proposed architecture resembles the Transformer proposed in the famous “Attention is all you need” paper. In more details we have:

• A Text-to-Phoneme Converter: converts text to phonemes

• Scaled positional encoding: they use a sinusoidal form that captures information about the position of phonemes

• An Encoder Pre-Net: a 3-layer CNN similar to Tacotron 2, which learns the phonemes embeddings

• A Decoder Pre-Net: consumes a mel spectogram and projects it into the same subspace as phoneme embeddings

• The Encoder: The bi-directional RNN is replaced with a Transformer Encoder with multi-head attention

• The Decoder: The 2-layer RNN with location-sensitive attention is replaced by a Transformer decoder with multi-head self-attention

• Mel Liner and Stop Linear: Two different linear projections are used to predict the mel spectrogram and the stop token respectively

The Transformer-based system achieved a MOS of 4.39.

### FastSpeech

A similar approach with Transformers is followed by FastSpeech. FastSpeech managed to speed up the aforementioned architecture by 38x. In short, this was accomplished by the following 3 things:

• Parallel mel-specogram generation

• Hard alignment between phonemes and their mel-spectograms in contrast to soft attention alignments in the previous model

• A length regulator that can easily adjust voice speed by lengthening or shortening the phoneme duration to determine the length of the generated mel spectrograms,

In the same direction, Fast Speech 2 and FastPitch came later and improved upon the original idea.

## Flow-based TTS

Before we examine flow-based TTS, let’s explain what flow-based models are. Contrary with GANs and VAEs which approximate the probability density function of our data $p(x)$, Flow-based models do exactly that with the help of normalizing flows.

Normalizing Flows are a method for constructing complex distributions by transforming a probability density through a series of invertible mappings. By repeatedly applying a predefined rule for change of variables, the initial density ‘flows’ through the sequence of invertible mappings. At the end of this sequence, we obtain a valid probability distribution and hence this type of flow is referred to as a normalizing flow. For more details, check out the original paper

A lot of models have been proposed based om that idea with the most popular being RealNVP, NICE and Glow. You can have a look at this excellent article by Lillian Weng to get a more complete understanding.

So as you may have guessed, Flow-based TTS models take advantage of this idea and apply it on speech synthesis.

### WaveGlow

WaveGlow by Nvidia is one of the most popular flow-based TTS models. It essentially tries to combine insights from Glow and WaveNet in order to achieve fast and efficient audio synthesis without utilizing auto-regression. Note that WaveGlow is used strictly to generated speech from mel spectograms replacing WaveNets. They are not end-to-end TTS systems.

Waveglow. Source: WaveGlow: A Flow-based Generative Network for Speech Synthesis

The model is trained by minimizing the negative log-likelihood function of the data. To achieve that, we need to use Invertible Neural Networks because otherwise, the function is intractable. I won’t go into many details because we would need a separate article to explain everything but here are a few things to remember:

• Invertible neural networks are usually constructed using coupling layers. In this case, the authors used affine coupling layers

• They also used 1x1 invertible convolutions following the Glow paradigm

Once the model is trained, the inference is simply a matter of randomly sampling values and run them through the network

Similar models include Glow-TTS and Flow-TTS. Flowtron, on the other hand, uses an Autoregressive Flow-based Generative Network to generate speech. So we can see that there are research works in both areas of flow-based models.

## GAN-based TTS and EATS

Finally, I’d like to close with one of the most recent and impactful works. End-to-End Adversarial Text-to-Speech by Deepmind.

EATS falls into the category of GAN-based TTS and is inspired by a previous work called GAN-TTS

EATS takes advantage of the adversarial training paradigm used in Generative Adversarial Networks. It operates on pure text or raw phoneme sequences and produces raw waveforms as outputs. EATS consists of two basic submodules: the aligner and the decoder.

The aligner receives the raw input sequence and produces low-frequency aligned features in an abstract feature space. The aligner’s job is to map the unaligned input sequence to a representation that is aligned with the output. The decoder takes the features and upsamples them using 1D convolutions to produce audio waveforms. The whole system is trained as a whole entity in an adversarial manner

A few key things worth mentioning are:

• The generator is a feed-forward neural network that uses a differentiable alignment scheme based on token length prediction

• To allow the model to capture temporal variation in the generated audio, soft dynamic time warping is also employed.

EATS achieved a MOS of 4.083

You can also find a great explanation of this architecture by Yannic Kilcher, on his Youtube channel.

## Conclusion

Text to speech is an area of research with a lot of novel ideas. It is evident that the field has come a long way over the past few years. Take a look at smart devices such as Google assistant, Amazon’s Alexa and Microsoft’s Cortana.

If you want to experiment with some of the above models, all you have to do is go into Pytorch’s or TensorFlow model hub, find your model and play around with it. Another great resource is the following repo by Mozilla: TTS: Text-to-Speech for all. If you also want us to explore a different architecture, feel free to ping us and we can include it here as well.

## Deep Learning in Production Book 📖

#### Learn how to build, train, deploy, scale and maintain deep learning models. Understand ML infrastructure and MLOps using hands-on examples.

* Disclosure: Please note that some of the links above might be affiliate links, and at no additional cost to you, we will earn a commission if you decide to make a purchase after clicking through.