text:
# Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series

Recent advances in nonlinear Independent Component Analysis (ICA) provide a principled framework for unsupervised feature learning and disentanglement. The central idea in such works is that the latent components are assumed to be independent conditional on some observed auxiliary variables, such as the time-segment index. This requires manual segmentation of data into non-stationary segments which is computationally expensive, inaccurate and often impossible. These models are thus not fully unsupervised. We remedy these limitations by combining nonlinear ICA with a Hidden Markov Model, resulting in a model where a latent state acts in place of the observed segment-index. We prove identifiability of the proposed model for a general mixing nonlinearity, such as a neural network. We also show how maximum likelihood estimation of the model can be done using the expectation-maximization algorithm. Thus, we achieve a new nonlinear ICA framework which is unsupervised, more efficient, as well as able to model underlying temporal dynamics.

## Authors

• 3 publications
• 36 publications
• ### Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning

Nonlinear ICA is a fundamental problem for unsupervised representation l...
05/22/2018 ∙ by Aapo Hyvärinen, et al. ∙ 0

• ### Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA

We introduce a new general identifiable framework for principled disenta...
06/17/2021 ∙ by Hermanni Hälvä, et al. ∙ 0

• ### Robust Nonlinear Component Estimation with Tikhonov Regularization

Learning reduced component representations of data using nonlinear trans...
07/15/2019 ∙ by Reuben Feinman, et al. ∙ 2

• ### Nonlinear ISA with Auxiliary Variables for Learning Speech Representations

This paper extends recent work on nonlinear Independent Component Analys...
07/25/2020 ∙ by Amrith Setlur, et al. ∙ 7

• ### Independent innovation analysis for nonlinear vector autoregressive process

The nonlinear vector autoregressive (NVAR) model provides an appealing f...
06/19/2020 ∙ by Hiroshi Morioka, et al. ∙ 0

• ### The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICA

We consider the problem of recovering a common latent source with indepe...
05/16/2019 ∙ by Luigi Gresele, et al. ∙ 9

• ### Asymptotic risks of Viterbi segmentation

We consider the maximum likelihood (Viterbi) alignment of a hidden Marko...
02/18/2010 ∙ by Kristi Kuljus, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Representation learning – the task of finding useful features from data – is one of the main challenges in unsupervised learning. Recent theoretical and practical advances in Nonlinear ICA provide a principled approach to this problem (Hyvärinen and Morioka, 2016, 2017; Hyvärinen et al., 2019; Khemakhem et al., 2020; Sorrenson et al., 2020). These works frame Nonlinear ICA as deep generative models, which allows them to harness deep neural networks to recover latent independent components from observed data. Identifiability of the latent components can be guaranteed by explicitly defining probabilistic generative models with appropriate conditional independence structures. A general framework was proposed recently by Hyvärinen et al. (2019), who assumed that the components are independent given some other observed auxiliary variable. For example, in time-series data this can be the time-index or segment-index if the data is non-stationary, as was earlier assumed in Time-Contrastive Learning or TCL (Hyvärinen and Morioka, 2016). Non-stationarity is a fundamental property of many applications, since for example, video, audio, and most neuroscience data are non-stationary.

A crucial limitation of all of the above nonlinear ICA models is that the conditioning auxiliary variable is always assumed observable. In some sense, these models are therefore not fully unsupervised. If, for instance, we wish to exploit the nonstationary temporal structure optimally in estimating independent components, TCL would require segment indices that correspond to the different latent data generative states. In practice we don’t observe such states so the default approach is to manually segment the data.

In general, it is unrealistic to assume that we can infer from observed data alone the exact time-points at which the latent data distribution changes. In fact, such change-points may not exist at all. Segmenting data manually is also infeasible for large datasets. The default approach is therefore to segment data at equal intervals, however, this is problematic for various reasons. Consider, for example, a situation where the true latent state switches between five different states. By segmenting the data at equal intervals we will end up with an unnecessarily large number of states where just a few would have done the job. This is computationally expensive, inaccurate and will completely miss out on temporal dynamics in situations where the latent states repeat over time.

In fact, often a reasonable assumption is that non-stationarity can be succinctly summarized using a limited number of segment indices or latent states, and properly modelling such state switching is likely to improve learning. Notice that even if ground-truth nonstationary information was available, the existing methods lack the machinery to perform inference on latent temporal dynamics. In many applications, for example brain imaging, describing the dynamics in terms of latent states could be very useful in its own right.

The points above highlight the need for a nonlinear ICA method that is able to cluster observations and learn latent states and their temporal dynamics in an unsupervised fashion. A well-known approach to modelling hidden latent states in time series is to use a Hidden Markov Model (HMM). HMMs can be viewed as probabilistic mixture models where the discrete latent states, which specify the data generating distribution, are time-dependent with Markov dynamics. HMMs are especially well suited for modelling non-stationary data as they automatically allow for a representation of the time series in terms of a discrete number of states.

In this work, we therefore resolve the above limitations by combining Nonlinear ICA with a HMM. This idea has been proposed earlier for linear ICA (Penny et al., 2000; Zhou and Zhang, 2008) but their identifiability and estimation results do not directly extend to the nonlinear case. In our model, we achieve this by having the latent state act in place of the conditioning auxiliary variable in the framework of Hyvärinen et al. (2019). Importantly, we are able to prove that Hidden Markov Nonlinear ICAs are identifiable. Attaining identifiability has been a major research focus for both Nonlinear ICA (Hyvärinen and Morioka, 2017; Hyvärinen et al., 2019) and HMMs (Allman et al., 2009; Gassiat et al., 2016), and therefore much of our paper is devoted to combining these two research strands. To the best of our knowledge, this is the first fully unsupervised non-linear ICA, in the sense that the model’s identifiability comes from an unobserved conditioning variable which is inferred from the time series as a part of learning. We further show how the structure of the model allows us to use the Expectation-Maximization (EM) algorithm for parameter estimation. In practice the Hidden Markov Nonlinear ICA is endowed with rich representation learning capabilities that allow it to simultaneously extract independent components and to learn the dynamics of the latent state that drives non-stationarity data, as illustrated by our simulations.

## 2 Background

We start by giving an overview of the problem of unidentifiability in both nonlinear ICA and HMMs, and recently proposed solutions. For both types of models, identifiability arises as a consequence of appropriate temporal structures which suggests a natural synthesis between the two.

### 2.1 Nonlinear Ica and Identifiability

Consider a parametric model of observed data

with marginal likelihood . This model is identifiable if it fulfils below:

 pθ(x)=pθ′(x)⇒θ=θ′:∀(θ,θ′) (1)

In the context of a latent variable model, this is closely connected to the idea of being able to recover the original latent variables, as discussed by Khemakhem et al. (2020).

Assume we observe -dimensional data at discrete time-steps, . Simple nonlinear ICA can be defined as the task of estimating an unobserved

-dimensional independent component vector

such that , as well as the inverse of a mixing function , that has generated the observed data:

 x(t)=f(s(t)) (2)

Unfortunately, without a temporal structure, that is if are i.i.d over the time-index, and if there are no constraints on , then this model is unidentifiable (Hyvärinen and Pajunen, 1999). In fact, the authors show that there are infinite potential nonlinear transformations and independent components that would satisfy the model, with no criterion for choosing one of them over the others.

In order to make the model identifiable, constraints are thus needed. For time-series data, this comes naturally by placing restrictions on the temporal structure of the model. For linear ICA this approach has been shown to yield identifiable models (Belouchrani et al., 1997; Tong et al., 1991), and extensions to the nonlinear case have been also been proposed in earlier work (Harmeling et al., 2003; Sprekeler et al., 2014). The first fully rigorous proof of an identifiable nonlinear ICA model, along with an estimation algorithm (Time-Contrastive Learning or TCL), was given by Hyvärinen and Morioka (2016). The constraint imposed in that work is that of a non-stationary data generative process such that independent component vectors within different time-segments have different distributional parameters. Specifically, the model assumes that each independent component has an exponential family distribution, where the time segment index modulates the natural parameters (denoted as ):

 pτ(si)=qi(si)Z(λi)exp{⟨λi(τ),T(si)⟩} (3)

where is the base measure and are the sufficient statistics. TCL then assumes that the independent components in all the segments are transformed into observed variables by some mixing function (2

). The authors prove identifiability up to a linear transformation of pointwise functions of the components:

 T(s(t))=Ah(x(t);θ)+b (4)

By learning to contrast between the different segments, the TCL algorithm learns the inverse of the mixing function and the independent components.

This seminal work has inspired other frameworks for identifiable nonlinear ICA estimation. Permutation Contrastive Learning (Hyvärinen and Morioka, 2017), for instance, exploits temporal dependencies, rather than non-stationarity, to identify independent components. The unifying tenet of these identifiable nonlinear ICA algorithms is that independent components are conditionally independent given some observed auxiliary variable. This general idea was formalized in Hyvärinen et al. (2019), of which both the TCL (segment index as auxiliary variable) and the PCL (past data as auxiliary variable) are special cases.

These identifiable nonlinear ICA models provide a principled approach to finding meaningful data representations. This is in contrast to the majority of recent deep generative models used for representation learning, such as VAEs (Kingma and Welling, 2014) and GANs (Goodfellow et al., 2014), which are all malaised by unidentifiability. In fact, any generative latent variable model with an unconditional prior is unidentifiable. This issue is portrayed in depth by Khemakhem et al. (2020) who resolve it by introducing identifiable VAE (iVAE). Like regular VAE, this model estimates a full generative model , but with a factorial conditional prior . As in Hyvärinen et al. (2019), u is some auxiliary variable, and iVAE provides a novel algorithm to estimate nonlinear independent components in the same identifiable framework. iVAE however suffers from the same problems as TCL, as its auxiliary variable u has to be observed.

### 2.2 Hidden Markov Models and Identifiability

In order to define HMMs, let

be an observed random variable from a time series with a discrete time index

. In a standard hidden Markov model, distribution of the observations depends conditionally on a discrete latent state random variable as per ; we refer to this as the emission distribution. The latent state undergoes first-order Markov process governed by a

transition-probability matrix

. is used to denote the probability of transitioning from state to , and the starting-state probabilities. The likelihood of a typical HMM is hence given by:

 p(x(1),…,x(T);A)= ∑c(1),…,c(T)π(c(1))p(x(1)|c(1))T∏t=2Ac(t−1),c(t)p(x(t)|c(t)) (5)

HMMs can be viewed as mixture models where the latent state is coupled across time by a Markov process. This observation raises the question of identifiability since mixture models with non-parametric emission distributions are generally unidentifiable, though many commonly used parametric forms are identifiable (Allman et al., 2009). Recently, however, Gassiat et al. (2016) have proven a major result that nonparametric HMMs are in general identifiable under some mild assumptions. We will use this result later and thus reproduce it here (notice that their nonparametric result subsumes parametric HMMs):

###### Theorem 1.

(Gassiat et al., 2016)  Assume the number of latent states, , is known. Use

to denote nonparametric probability distributions of the

emission distributions. Also assume that the transition-matrix is full rank. Then the parameters and are identifiable given the distribution, , of at least 3 consecutive observations , up to label swapping of the hidden states, that is: if is a transition matrix, if is a stationary distribution of with , and if are probability distributions on that verify the equality of the HMM distribution functions , then there exists a permutation of the set such that for all we have and .

Much like for TCL, identifiability in nonparametric HMMs is a result of temporal structure, namely observations across time are independent conditionally on the latent state—this is in contrast to simple (i.i.d.) mixture models for which such general identifiability results are not available. We show below that this temporal structure of the HMM’s, together with nonstationarity similar to TCL, combine to identify the resulting Hidden Markov Nonlinear ICA model.

## 3 Identifiable Nonlinear Ica for Nonstationary Data

In this section, we propose a combination of a hidden Markov model and nonlinear ICA. Specifically, we propose an HMM which has nonlinear ICA as its emission model, and show how to estimate it by Expectation-Maximization.

### 3.1 Model Definition

To incorporate nonlinear ICA into the standard HMM of (5) we define the emission distribution as a deep latent variable model. First, the latent independent component variables are generated from a factorial exponential family prior, given the hidden state , as

 p(s(t)|c(t);λc(t))=N∏i=1p(s(t)i|c(t);λi,c(t)) =N∏i=1h(s(t)i)Z(λi,c(t))exp{⟨λi,c(t),Ti(si)⟩} (6)

where are the base measures, the normalizing constants, and the sufficient statistics. Second, the observed data is generated by a nonlinear mixing function as in Eq. (2).

For remainder of the paper we assume that the exponential family is in minimal representation form so that the sufficient statistics are linearly independent. The corresponding -dimensional parameter vectors are denoted by . The subscripts indicate that the parameters of the different components are modulated directly, and independently, by the HMM latent state. Indeed, it is precisely this conditional dependence of the parameters on the discrete latent state that seeps through our model and generates non-stationary observed data. Note that the parameters themselves are time-homogeneous, that is they are constant over time; instead, the latent state evolves over time and determines which set of parameters is active a point in time. In other words, non-stationary arises purely from the dynamics of the latent state . The full set of parameters for the independent components can hence be captured by a matrix (plus the transition probabilities of the hidden states).

The nonlinear mixing function in Eq. (2) is assumed to be bijective with inverse given by . It follows that in our model the conditional emission distribution for observed data is:

 p(x(t)|c(t);f,λc(t))= |Jg(x(t))|H(g(x(t)))Z(λc(t))exp{⟨λc(t),T(g(x(t)))⟩} (7)

where is short-hand notation for the absolute value of the determinant of the Jacobian of the inverse (demixing) function, and . We have also simplified notation by stacking the vectors for different components and .

We allow to be any arbitrary but bijective function. In practice, it can be learned as a neural network. The model can therefore be viewed as a deep generative model for non-stationary data. Finally, using and our hidden Markov nonlinear ICA model’s data-likelihood is given as:

 p(x(1),…,x(T);Θ)= ∑c(1),…,c(T)(π(c(1))p(x(1)|c(1),θc(1))× T∏t=2Ac(t−1),c(t)p(x(t)|c(t);θc(t))) (8)

where the emission distributions in Eq. (7) should be plugged in.

### 3.2 Estimation

Assume we have a sequence of observed data generated by (8). In order to estimate the model parameters in practice we will choose the factorial prior in (6) from a well-known family such that the normalizing constant is tractable, such as a Gaussian location-scale family. Intractable normalizing constant would make estimation very difficult, even by approximate inference methods such as Variational Bayes or VAEs. However, notice that the choice of distribution for the latent prior does not severely limit the type of data that can be modelled since the non-linear mixing function can be any arbitrary function.

Tractable exponential families also make it easy to estimate the model parameters by maximizing the likelihood in (8) by the EM algorithm. The ”free-energy” EM lower bound for our model is given by:

 L(q(c),Θ):= Eq(c)[logp(c,x(1),…,x(T);Θ)]−Eq(c)[logq(c)] (9)

where , such that the first RHS terms is the complete-data likelihood under some distribution . In the E-step one finds which is the standard result for HMMs and can be easily computed using the forward-backward (Baum-Welch) algorithm. In the M-step we aim to find , which reduces to maximizing:

 ~L(q(c),Θ):=Eq(c⋆)[logp(c,x(1),…,x(T);Θ)]= +T∑t=2Eq(c(t−1)⋆,c(t)⋆)[logAc(t−1),c(t)] (10)

where we have left out the initial-state probability term as we can assume a stationary process and infer them directly from

as its left eigenvector. The M-step updates for

are standard:

 A⋆i,j←∑Tt=2q(c(t−1)⋆=i,c(t)⋆=j)∑Tt=1q(c(t)⋆) (11)

M-step updates for the parameters also follow from standard EM results for exponential families:

 ∇λkT∑t=1Eq(c(t)⋆)[logp(x(t)|c(t);θc(t))] =∇λkT∑t=1Eq(c(t)⋆)[⟨λk,T(g(x(t)))⟩−logZ(λk)] =T∑t=1q(c(t)⋆=k)[T(g(x(t)))−∇λkZ(λk)Z(λk)]=0 ⇒∇λkZ(λk)Z(λk)=∑Tt=1q(c(t)⋆=k)T(g(x(t)))∑Tt=1q(c(t)⋆=k) (12)

where LHS can be rewritten as:

 1Z(λk)∇λk∫(|Jg(x(t))|H(g(x(t))) ×exp{⟨λc(t),T(g(x(t)))⟩}) =Ep(x(t)|c(t);θc(t))[T(g(x(t)))] (13)

Thus the M-step updates for are the ones that solve:

 =∑Tt=1q(c(t)⋆=k)T(g(x(t)))∑Tt=1q(c(t)⋆=k) (14)

In practice, (14

) has closed-form updates for many usual exponential family members. As an example, if we were to use a Gaussian distribution, the updates for mean and variance vectors would be:

 μ⋆k←∑Tt=1q(c(t)⋆=k)g(x(t))∑Tt=1q(c(t)⋆=k) σ2⋆k←diag⎛⎝∑Tt=1q(c(t)⋆=k)y⋆ky⋆,Tk∑Tt=1q(c(t)⋆=k)⎞⎠ (15)

where

Next, the demixing function is estimated by parameterizing it as a deep neural network but for notational simplicity we will not write these parameters explicitly and instead subsume them in . Since an exact M-step is not possible, a gradient ascent step on the lower bound is taken instead, where the gradient is given by:

 ∇g~L(q(c),Θ)=∇gT∑t=1Eq(c(t)⋆)[logp(x(t)|c(t);θc(t))] =∇gT∑t=1log|Jg| +∇gT∑t=1Eq(c(t)⋆)[logH(g)+⟨λc(t),T(g)⟩] (16)

where we have used for brevity. The parameters are then updated as:

 gnew←gold+η∇g~L(q(c),Θ) (17)

See Appendix A for a discussion on the convergence of our algorithm.

The gradient term with respect to the determinant of the Jacobian deserves special attention. It is widely considered difficult to compute, and therefore, normalizing flows models are often used in literature in order to make the Jacobians more tractable. The problem with this approach is that, to our best knowledge, none of such flow models has universal function approximation capabilities (despite some being universal distribution approximators). This would restrict the possible set of nonlinear mixing functions that can be estimated, and is thus not practical for our purposes. Fortunately modern autograd packages such as JAX make it possible to calculate gradients of the log determinant Jacobian efficiently up to moderate dimensions (see Appendix B) – this is the approach we take. Very recent, promising, alternative for computing the log-determinant is the relative gradient (Gresele et al., 2020) which could easily be implemented in our framework. Finally, notice that the second term (16) is easy to evaluate since the expectation is just a discrete sum over the posteriors that we get from the E-step.

### 3.3 Comment on Estimation for Long Time Sequences

The above estimation method may be impractical for very long time sequences since the forward-backward algorithm has computational complexity of . In such situations we can adapt the subchain sampling approach of Foti et al. (2014). This involves splitting up the full dataset into shorter time sequences and then forming minibatches over time. The resulting gradient updates would be biased and therefore a scaling term will be applied to them. The forward-backward algorithm applied to the subchains is also only approximate due to loss of information at the ends of the chains but the authors describe a technique to buffer the chains with extra observations to reduce this effect.

### 3.4 Comment on Dimension Reduction

An important problem in applying our method on real data is dimension reduction. While in the theory above, we assumed that the number of independent components is equal to the number of observed variables, in many practical cases, we would like to have a smaller number of components than observed variables. We propose here two solutions for this problem.

The first solution, which is widely used in the linear ICA case, is to first reduce the dimension of the data by PCA, and then do ICA in that reduced space with the same dimensions of components and observed variables. In the nonlinear case, a number of nonlinear PCA methods, also called manifold learning methods, has been proposed and could be used for such a two-stage method. In particular, dimension reduction is achieved by even the very simplest autoencoders; recent work has developed the theory further in various directions

(Maaten and Hinton, 2008; Vincent et al., 2010). This approach has the advantage of reducing the noise in the data, which is a well-known property of PCA, and allows us to separate the problem of dimension reduction from the problem of developing ICA algorithms. A possible drawback is that such dimension reduction may not be optimal from the viewpoint of estimating independent components.

The second solution is to build an explicit noise model into the nonlinear ICA model, following Khemakhem et al. (2020). Denote by a random vector of Gaussian noise which is white both temporally and spatially and of variance . Instead of the Eq. (2), we would define a mixing model as

 x(t)=f(s(t))+n(t) (18)

where the model of the components is unchanged. We could then combine the variational estimation method presented by Khemakhem et al. (2020) with the HMM inference procedure presented here. However, we leave the details for future work.

## 4 Identifiability Theory

In this section we provide identifiability theory for the model discussed in the previous section. As was discussed above, many deep latent variable models are non-identifiable. In other words, an estimation method such as the EM proposed above might not have a unique solution, or even a small number of solutions which are indistinguishable for any practical purposes.

Fortunately, we are able to combine previous nonlinear ICA theory with the identifiability of Hidden Markov Models to prove the identifiability of our combined model. Albeit our model being different from (Hyvärinen and Morioka, 2017), (Hyvärinen et al., 2019) and (Khemakhem et al., 2020), the identifiability we reach is very similar. We also show that in the case of Gaussian independent components we can get exact identifiability up to linear transformation of the components.

### 4.1 Definitions

In order to illustrate the relationship of our model’s identifiability to earlier works in the area, we introduce the following definitions from Khemakhem et al. (2020)

###### Definition 1.

Let be the equivalence relation on . (8) is said to be identifiable up to if

 p(x(1),…,x(T);Θ)=p(x(1),…,x(T);^Θ)⇒Θ∼^Θ (19)
###### Definition 2.

Let be the binary relation on defined by:

 (f,λ)∼(^f,^λ)↔ ∃W,b∣T(g(x(t)))=WT(^g(x(t)))+b (20)

where is an matrix and is an vector.

If is invertible, the above relation is denote by , and if is a block permutation matrix, it is denoted by . In block permutation, each block linearly transforms into with each corresponding to one, and only one, .

### 4.2 General Result

Now we present our most general Theorem on identifiability. It will be followed by stronger results in the Gaussian case below.

###### Theorem 2.

Assume observed data is generated by a Hidden Markov Nonlinear ICA according to (5) - (8). Also, assume:

1. [label=()]

2. The time-homogeneous transition matrix has full rank and induces an irreducible 111all states can be reached from every stateMarkov chain with a unique stationary state distribution

3. The number of latent states, , is known and

4. There exists an square matrix of the different states’ parameters with respect to a pivot state

 ˜L=⎛⎜ ⎜⎝(λc=1−λc=0)T⋮(λc=NV−λc=0)T⎞⎟ ⎟⎠ (21)

which is invertible.

5. The emission distributions for the different latent states are linearly independent functions of

6. The non-linear mixing function is bijective

Then the model parameters are identifiable.

###### Proof.

Suppose we have

 p(x(1),…,x(T);Θ)=p(x(1),…,x(T);^Θ) (22)

Using assumptions 1-4, we can invoke Theorem 1 and apply it to our model to get:

 ^Ak,l=Aσ(k),σ(l) (23) p(x|k;^θk)=p(x|σ(k);θσ(k)) (24)

where superscript is dropped for convenience. For notational simplicity, and without loss of generality, we assume the components are ordered such that . Substituting in we have:

 |J^g(x)|H(^g(x))Z(^λk)exp{⟨^λk,T(^g(x))⟩} =|Jg(x)|H(g(x))Z(λk)exp{⟨λk,T(g(x))⟩} (25)

for some latent state . Recall from assumption 3 that . We can thus take states and assign one of them, say as a pivot states. Taking logs of (25) for all the other states with respect to the pivot state gives equations of below form:

 ⟨(λk−λ1),T(gi(x))⟩+logZ(λ1)−logZ(λk) =⟨(^λk−^λ1),T(^gi(x))⟩+logZ(^λ1)−logZ(^λc) (26)

Collecting all the such equations, we can stack them into a linear system :

 ˜LT(g(x))=ˆ˜LT(^g(x))+β (27)

where is the invertible square matrix defined in assumption 3, and the elements of are defined similarly, but no assumption about its invertibility is made. The constants that result from the sums of the log-normalizers are stacked to form vector . Multiplying both sides by results in our desired form:

 T(g(x))=˜L−1ˆ˜LT(^g(x))+˜L−1β T(s)=WT(^g(x))+b (28)

Recall that we defined the exponential families to be in minimal representation in Section 3.1. It follows that we can find an arbitrary number of points such that the vectors formed by the sufficient statistic functions of each independent component , are linearly independent. This can be done separately for each . Additionally, as and can be changed independently, we can find for then and are linearly independent for all . Therefore, all elements of the vector are linearly independent which implies that the square matrix in (28) is invertible. ∎

#### 4.2.1 Comments on the assumptions of Theorem 1

The assumptions 1, 2 are standard HMM assumptions. The assumption of a full rank transition matrix is non-standard but crucial here. Intuitively speaking, it allows the latent states to be distinguished from each other, while the irreducibility assumptions ensures that there is a single unique stationary state distribution.222technically one aperiodic state is also required. An aperiodic state is one which can be returned to after an irregular number of steps Notice that these assumptions necessarily hold, for example, when the transition matrix is close to identity, as in a case where the states are strongly persistent.

The assumption that the real number of latent components is known, is valid in certain applications, and if not it could be estimated for instance be increasing the number of latent states between each estimation and then detecting the point at which increases in likelihood become marginal (the elbow method). Assumption 3 is valid in practice as long as the parameters are generated randomly - in that case it almost surely holds as singular solutions will lie in a submanifold of lower dimension. The validity of assumption is less obvious 4, however, we will below prove that it holds, for instance, in the case of Gaussian independent components.

### 4.3 Identifiability With Gaussian Independent Components

In this section, we first provide two lemmas which we use to prove the claim, already alluded to above, that assumption 4 of Theorem 2 is satisfied for Gaussian components. Then, we prove that in this case a stronger form of identifiability can be reached as a special case of above results, namely that we get exact identification of components up to linear transformation. Together these results make a strong case for using Gaussian latent components in practical applications.

We begin by stating two Lemmas (proofs in Appendix C):

###### Lemma 1.

Assumption 4 of Theorem 1 requires the emission distributions defined by (7) to be linearly independent. A sufficient, and necessary, condition is that the conditional source distributions defined by (6) are linearly independent.

###### Lemma 2.

Assume probability density functions of random variables , and that each factorizes across the variables: . If the factorial density functions are linearly independent for some , then the joint-density functions are linearly independent.

Based on these Lemmas, we can prove the following Theorem (proof in Appendix C):

###### Theorem 3.

Assume that distributions of the independent components conditional on the latent state, as defined by (6), are Gaussian parameterised by mean and variance. Assume also that the means of the density functions are all different. Then the emission distributions, defined by (7), are linearly independent, thus satisfying assumption 4 in Theorem 2.

Finally, we have the following Theorem which proves a stronger form of identifiability—essentially recovering the components with minimum indeterminacy—of our Hidden Markov Nonlinear ICA model in the Gaussian case (proof in Appendix C)

###### Theorem 4.

Assume that the latent independent components have a conditionally Gaussian distributions, and assume hypotheses 1, 2,3 and 5 of Theorem 2 hold, as well as the assumptions of Theorem 3. Additionally assume that the mixing function has all of it second-order cross derivatives, then the components in our Hidden Markov Nonlinear ICA model are exactly identified up to linear transformation.

Notice that this proof and the identifiability result is similar to that in (Sorrenson et al., 2020), although our models are entirely different. These authors also prove a general version for other distributions with different sufficient statistics.

## 5 Experiments

In this section we present results from our simulations on artificial non-stationary data. Code, written in JAX, is available at github.com/HHalva/hmnlica.

Dataset: We generated a synthetic dataset from the model defined in Section 3.1. More specifically, the independent components are created from non-stationary Gaussian emission distributions of an HMM with discrete states – the latent state determines the means and the variances of the independent components at each time point. The transition matrix was defined so that at each time-step there was a 99% probability that the state didn’t change and a 1% probability the latent state switches to another state.333for context, the probability of staying in the same state for over 100 time steps with these numbers is around 37% If it switches to another state, it will always go to the next one ‘in line’, where we define a circular ordering for the states. That is, we defined a circular repeating path for the latent state where transitions could only happen to two states such that the transition matrix is close to identity (Figure 1 illustrates this). These settings were chosen to ensure the HMM assumptions of Theorem 2 hold, as well as to reflect a situation where a relatively small number of states repeats over time with some interesting, non-random, temporal dynamics, including persistence to stay in the same state. The mean and variance parameters were chosen at random for each latent state before data generation so that assumption 3 of Theorem 2 holds. Similarly to Hyvärinen et al. (2019), the mixing function (2) was simulated with a randomly initialized, invertible 444invertibility achieved by having all layers units wide and utilizing leaky ReLUs

multi-layer perceptron (MLP) – this produced the observed data for our experiments. The sequences that were created are 100,000 time steps long. The number of latent states was set such that

, which ensures that the assumptions 2 and 3 were fulfilled.

Model estimation: We estimate the (inverse) mixing function and distribution parameters of our Hidden Markov nonlinear-ICA model using the EM algorithm described in Section 3.2. Mean and variance parameter estimates for the independent components are initialized randomly at the start of the EM algorithm. The inverse mixing function is parameterized with an MLP where the number hidden layers is set to match the number of data generating mixing layers. The gradient M-step are taken with the Adam optimizer (Kingma and Ba, 2017). Random restarts were used to avoid inferior local maxima. Further, we found that a stochastic version of our algorithm (see Section 3.3) converged faster – thus, the experiments here have been run with 100 time-step long sub-sequences in minibatches of 64.

Results – independent component recovery: After estimating the model parameters and independent components, a linear sum assignment problem is solved to optimally match each of the estimated components to one of the real ones. This is necessary as the ordering of the components is arbitrary. Mean absolute correlation coefficients over the resulting pairs of true and estimated components are then used to measure how well original components were recovered. This is the methodology taken in previous nonlinear ICA works (Hyvärinen and Morioka, 2016).

Figure 2 shows the mean correlation between the estimated components for our model in comparison to TCL, which is the only other nonlinear ICA model for non-stationary data. For TCL, the data was split into 500 time-step long segments; 500 steps provided best performance relative to other computationally feasible options (100, 250, 750, 1000). We can see that our model outperforms TCL for all levels of nonlinearity. This validates our theoretical arguments that the TCL framework struggles with non-stationary data in which latent states (often a relatively small number) repeat over time since the segments it has access to don’t correspond well with the true data generating process.

Results – temporal dynamics Unlike previous models, Hidden Markov Nonlinear ICA is able to perform unsupervised clustering of latent states and to take into account the learned temporal dynamics in doing so. To estimate this ability, we ran the well-know Viterbi algorithm (Viterbi, 1967) which finds the most likely path of latent states based on our estimated model. The results show that on average for and our model reaches near perfect classification accuracy in the linear ICA case, mean accuracy of around 80% for , and 68% for , thus clearly outperforming random chance level (figure in Appendix E).

## 6 Conclusion

We proposed a framework nonlinear ICA based on a Hidden Markov Model of the temporal dynamics. This improves on existing nonlinear ICA methods in several ways. First, it removes the need for any arbitrary segmentation of the data as in TCL, which is likely to improve the estimation of the demixing function. Second, it leverages the fact that the nonstationary structure is often repeating with a limited number of hidden states, which not only reduces the computation by limiting the number classes, but again is likely to improve estimation of the demixing function. Third, our method estimates the underlying latent temporal dynamics, which are often interesting in their own right. We believe this in an important advance in order to apply nonlinear ICA methods on real data.

#### Acknowledgements

The authors would like to thank Elisabeth Gassiat, Ilyes Khemakhem and Ricardo Pio Monti for helpful discussion. I.K.’s help with the experiments is also much appreciated. A.H. was supported by a Fellowship from CIFAR, and from the DATAIA convergence institute as part of the “Programme d’Investissement d’Avenir” (ANR-17-CONV-0003) operated by Inria.

## References

• E. S. Allman, C. Matias, and J. A. Rhodes (2009) Identifiability of parameters in latent structure models with many observed variables. The Annals of Statistics 37 (6A), pp. 3099–3132 (en). Note: arXiv: 0809.5032 External Links: ISSN 0090-5364, Link, Document Cited by: §1, §2.2.
• A. Belouchrani, K. Abed-Meraim, J.-F. Cardoso, and E. Moulines (1997) A blind source separation technique using second-order statistics. IEEE Transactions on Signal Processing 45 (2), pp. 434–444 (en). External Links: ISSN 1053587X, Link, Document Cited by: §2.1.
• A. P. Dempster, N. M. Laird, and D. B. Rubin (1977) Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 39 (1), pp. 1–38. Cited by: Appendix A.
• N. Foti, J. Xu, D. Laird, and E. Fox (2014) Stochastic variational inference for hidden Markov models. In Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.), pp. 3599–3607. External Links: Link Cited by: §3.3.
• E. Gassiat, A. Cleynen, and S. Robin (2016) Inference in finite state space non parametric Hidden Markov Models and applications. Statistics and Computing 26 (1-2), pp. 61–71 (en). External Links: ISSN 0960-3174, 1573-1375, Link, Document Cited by: §1, §2.2, Theorem 1.
• I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio (2014) Generative adversarial nets. In Advances in neural information processing systems, pp. 2672–2680. Cited by: §2.1.
• L. Gresele, G. Fissore, A. Javaloy, B. Schölkopf, and A. Hyvärinen (2020)

Relative gradient optimization of the jacobian term in unsupervised deep learning

.
Submitted. Cited by: §3.2.
• S. Harmeling, A. Ziehe, M. Kawanabe, and K. Müller (2003) Kernel-Based Nonlinear Blind Source Separation. Neural Computation 15 (5), pp. 1089–1124 (en). External Links: ISSN 0899-7667, 1530-888X, Link, Document Cited by: §2.1.
• A. Hyvärinen and H. Morioka (2016)

Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA

.
arXiv:1605.06336 [cs, stat] (en). Note: arXiv: 1605.06336 External Links: Link Cited by: §1, §2.1, §5.
• A. Hyvärinen and H. Morioka (2017) Nonlinear ICA of Temporally Dependent Stationary Sources.

Proceedings of the 20 th International Con- ference on Artificial Intelligence and Statistics (AISTATS)

, pp. 14 (en).
Cited by: §1, §1, §2.1, §4.
• A. Hyvärinen and P. Pajunen (1999) Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks 12 (3), pp. 429–439 (en). External Links: ISSN 08936080, Link, Document Cited by: §2.1.
• A. Hyvärinen, H. Sasaki, and R. E. Turner (2019) Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning. arXiv:1805.08651 [cs, stat] (en). Note: arXiv: 1805.08651 External Links: Link Cited by: §1, §1, §2.1, §2.1, §4, §5.
• I. Khemakhem, D. P. Kingma, R. P. Monti, and A. Hyvärinen (2020) Variational Autoencoders and Nonlinear ICA: A Unifying Framework. arXiv:1907.04809 [cs, stat] (en). Note: arXiv: 1907.04809 External Links: Link Cited by: Appendix C, §1, §2.1, §2.1, §3.4, §4.1, §4.
• D. P. Kingma and J. Ba (2017) Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs] (en). Note: arXiv: 1412.6980 External Links: Link Cited by: §5.
• D. P. Kingma and M. Welling (2014) Auto-Encoding Variational Bayes. arXiv:1312.6114 [cs, stat] (en). Note: arXiv: 1312.6114 External Links: Link Cited by: §2.1.
• L. v. d. Maaten and G. Hinton (2008) Visualizing data using t-SNE.

Journal of machine learning research

9 (Nov), pp. 2579–2605.
Cited by: §3.4.
• W.D. Penny, E. R.M., and R. S.J. (2000) Hidden markov independent component analysis. In Advances in Independent Component Analysis, Cited by: §1.
• P. Sorrenson, C. Rother, and U. Köthe (2020) Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN). arXiv:2001.04872 [cs, stat] (en). Note: arXiv: 2001.04872 External Links: Link Cited by: §1, §4.3.
• H. Sprekeler, T. Zito, and L. Wiskott (2014) An extension of slow feature analysis for nonlinear blind source separation. J.\ of Machine Learning Research 15 (1), pp. 921–947. Cited by: §2.1.
• L. Tong, R.-w. Liu, V.C. Soon, and Y.-F. Huang (1991) Indeterminacy and identifiability of blind identification. IEEE Transactions on Circuits and Systems 38 (5), pp. 499–509 (en). External Links: ISSN 00984094, Link, Document Cited by: §2.1.
• P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P. Manzagol (2010)

Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion

.
journal of machine learning research 11 (dec), pp. 3371–3408. Cited by: §3.4.
• A. J. Viterbi (1967) Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transactions on Information Theory 13, pp. 260–267. Cited by: §5.
• J. Zhou and X. Zhang (2008) An ica mixture hidden markov model for video content analysis. IEEE Transactions on Circuits and Systems for Video Technology 18 (11), pp. 1576–1586. Cited by: §1.

## Appendix A Note on the convergence of our estimation algorithm

Standard theory (Dempster et al., 1977) shows that each EM iteration increases the likelihood, unless parameters are already at a zero-gradient point. Further, maxima of free-energy and likelihood coincide. This also holds under the gradient M-steps in our algorithm (with classical assumption of sufficiently small step size). Under suitable regularity conditions, theoretical limit of infinite data and universal approximation of the nonlinear transformation, combined with our identifiability proof, MLE guarantees convergence to correct parameters up to the equivalence class identified in our Theorem 2. In practice, however, these assumptions may not be satisfied — for instance, parameters may approach a boundary point and likelihood tend to infinity. Random restarts and regularisation are common strategies to avoid these problems.

## Appendix B Note on the compute time of the gradients of the logdet Jacobian

We estimate the non-linear mixing function in our model using a multi-layer perceptron without any restrictions on it. As a consequence of the change of variable formula for probability densities, we have to calculate the gradient of the log-determinant of the Jacobian as part of our parameter updates. JAX, a new machine learning package that utilizes autograd, has the ability to calculate the Jacobian in just a single forward pass thus making the computations efficient for typical data dimensions. For our model, we can see that the compute time required for the log-determinant of the Jacobian starts to dominate as we approach 100 dimensions and above.

## Appendix C Proofs

Proof for Lemma 1

###### Proof.

Assume that we have linear independence of the conditional source distributions as defined by (6). Then we have that:

 a1pS(s|1;λ1)+⋯+aCpS(s|C;λC)≡0⇒a=0 (29)

Above holds if we multiply it through by the Jacobian determinant of the mixing function as we have assumed bijectivity, that is:

 a1|Jg(x)|pS(s|1;λ1)+… +aC|Jg(x)|pS(s|C;λC)≡0⇒a=0 (30)

which is equivalent to:

 a1|Jg(x)|pS(g(x)|1;λ1)+… +aC|Jg(x)|pS(g(x)|C;λC)≡0⇒a=0 (31)

And therefore by (7) we have:

 a1pX(x|1;f,λ1)+… +aCpX(x|C;f,λC)≡0⇒a=0 (32)

So the emission distributions are linearly independent if the densities for the independence components are linearly independent across the different latent states. Necessity follows easily by the reverse of above argumentation. ∎

Proof for Lemma 2

###### Proof.

Assume linear independence of the joint-density functions for some subset of variables , where and , that is:

 w1p1(zi,…,zi+n)+⋯+wKpK(zi,…,zi+n) =w1i+n∏j=ip(j)1(zj)+⋯+wKi+n∏j=ip(j)K(zj)≡0 ⇒w=0 (33)

Now the linear independence for joint of requires:

 w1p1(zi,…,zi+n,zi+n+1)+ …+wKpK(zi,…,zi+n,zi+n+1)≡0⇒w=0

Using the factorial form of the joint, we can rewrite this as:

 w1p(i+n+1)1(zi+n+1)p1(zi,…,zi+n)+… +wKp(i+n+1)K(zi+n+1)pK(zi,…,zi+n) ≡0⇒w=0 (34)

If this didn’t hold we could define constants such that:

 v1p1(zi,…,zi+n)+⋯+vKpK(zi,…,zi+n)≡0 (35)

where the constants are not all zero which would contradict our original assumption. Thus it is sufficient to prove linear independence of , say for , without loss of generality, and then apply the above induction step to guarantee linear independence of the joint-density functions . ∎

Proof for Theorem 3

###### Proof.

By Lemma 1 it is sufficient to prove the linear independence of the different conditional independent component density functions, rather than emission densities. And by Lemma 2 it suffices to prove this only for any one of the different independent components. In exponential family form, the density is written as (see Appendix D):

 p(si|c)=1√2πexp{ηi,c,1si−ηi,c,2s2i}Zi,c (36)

where . We drop subscript for convenience. Consider:

 w11√2πexp{η1,1s−η1,2s2}Z1+… +wC1√2πexp{ηC,1s−ηC,2s2}ZC=0 (37)

First assume, all the are distinct. Also we can assume, without loss of generality, that the latent states are ordered such that . We can divide all the terms with the first density to give:

 w1+w2Z1Z2exp{(η1,1−η2,1)s+(η1,2−η2,2)s2}+… +wCZ1ZCexp{(η1,1−ηC,1)s+(η1,2−ηC,2)s2}=0 (38)

taking of above gives . Repeatedly performing this process for remaining terms eventually gives . Consider now the opposite case in which all the are equal. Then we have:

 w11√2πexp{η1,1}Z1+⋯+wC1√2πexp{ηC,1}ZC=0 (39)

If we re-order the terms such that is the largest (recall we assumed that the means are different). We can again divide everything by this term, take , and establish and repeat the process to get . In the the final case where more than one component has the highest variance, but rest are unequal, (only the equality of the largest variances of the remaining terms matters), we can first perform the variance division followed by division by the largest mean, repeatedly until . ∎

Proof for Theorem 4

###### Proof.

By Theorem 3, the above assumptions suffice for Theorem 2 to hold. Next, note that the sufficient statistics of a Gaussian distribution are twice differentiable. This, combined with the assumption about the existence of ’s cross-derivatives fulfils the conditions of Theorem 2 of Khemakhem et al. (2020) and thus our model’s parameters are identifiable (as per Definition 2). We therefore have:

 (sis2i)=Wj(gj(x)gj(x)2)+bi (40)

for some . Hence, we have

 (w11gj(x)+w12gj(x)2+b11)2 =w21gj(x)+w22gj(x)2+b21 w212z4+2w11w12z3+(w211−w22)z2−w21z+b=0 (41)

Above has to hold for all values of . The trivial solution of all is impossible as would not be invertible. Therefore, it must be that and . Thus we have exact identification (up to linear transformation) for some constants . ∎

## Appendix D Model with Gaussian independent components

 p(si|c)= 1√2πσ2i,cexp{−12σ2i,c(si−μi,c)2} (42) = 1√2πσ2i,cexp{−12σ2i,c(s2i−2siμi,c+μ2i,c)} (43) = 1√2πσ2i,cexp{siμi,cσ2i,c−s2i12σ2i,c−μ2i,c2σ2i,c} (44) = Z−1i,cexp{siμi,cσ2i,c−s2i12σ2i,c} (45)

Therefore by independence of components:

 p(s|c)= exp{N∑i=1(siμi,cσ2i,c−s2i12σ2i,c)}N∏i=1Z−1i,c (46) = exp{N∑i=1(siμ


text:
$$\require{cancel}$$

# 27.3: Guide for writing a proposal

## Summary and Goal

Write a few short sentences briefly summarizing the aim of your experiment, how it will be conducted, and how precise of a result you expect to obtain.

## Method and equipment

Clearly describe, in as much detail as required, the method/procedure that you will use to carry out your experiment, and how you will analyze the results. Justify the choices that you made (no need to say you chose to use a ruler because you will need to measure a distance, but perhaps say why you need to measure a given distance, or that you chose to measure something in a particular way as it would reduce the corresponding uncertainty). Provide a list of the equipment that you will need. Also, propose a method of assessing whether or not your project was successful.

Consider the following questions:

• What theory are you testing and through what model?
• How precisely do you estimate that you will be able to make your measurement? Estimate the uncertainty that you will obtain with the proposed experiment. Use this in guiding the design of your experiment.
• What materials, equipment and/or tools are necessary in making your measurements?
• What are the cost of these materials? Can they be easily obtained?
• Where should this experiment be conducted?
• Are there any safety concerns?
• How will you make your measurements? How many times will you make them?
• How will you record your measurements?
• How will you maximize the precision of your experiments?
• How will you determine uncertainties?
• How will you analyze the data?
• What issues could arise in your experiment? How do you plan to resolve these issues?

## Timeline and Team

Provide the names of team members, and assign relevant duties to each member. Give a rough outline of the timeline to conduct the experiment, to analyze the data, and to report on the results.

format: plain


text:
### Top 10 Arxiv Papers Today in Pattern Formation And Solitons

##### #1. Variational approximations using Gaussian ansatz, false instability, and its remedy in nonlinear Schrödinger lattices
###### Rahmi Rusin, Rudy Kusdiantara, Hadi Susanto
We study the fundamental lattice solitons of the discrete nonlinear Schr\"{o}dinger (DNLS) equation and their stability via a variational method. Using a Gaussian ansatz and comparing the results with numerical computations, we report a novel observation of false instabilities. Comparing with established results and using Vakhitov-Kolokolov criterion, we deduce that the instabilities are due to the ansatz. In the context of using the same type of ansatzs, we provide a remedy by employing multiple Gaussian functions. The results show that the higher the number of Gaussian function used, the better the solution approximation.
more | pdf | html
None.
###### Tweets
SciencePapers: Variational approximations using Gaussian ansatz, false instability, and its remedy in nonlinear Schr\"{o}dinger lattices. https://t.co/eZRImQtQiH
None.
None.
###### Other stats
Sample Sizes : None.
Authors: 3
Total Words: 5181
Unqiue Words: 1549

##### #2. Large-amplitude periodic atomic vibrations in diamond
###### George Chechin, Denis Ryabov, Stepan Shcherbinin
Symmetry-determined nonlinear normal modes, which describe large-amplitude vibrations of diamond lattice, are studied by specific group-theoretical methods. For two of these modes, corresponding to vibrational state with and without multiplication of the unit cell, we present their atomic patterns and dynamical properties obtained with the aid of the ab initio simulations based on the density functional theory.
more | pdf | html
###### Tweets
SciencePapers: Large-amplitude periodic atomic vibrations in diamond. https://t.co/Spever3l02
CondMatPhys: Large-amplitude periodic atomic vibrations in diamond https://t.co/s3yAGwjWgJ
None.
None.
###### Other stats
Sample Sizes : None.
Authors: 3
Total Words: 6583
Unqiue Words: 1720

##### #3. Pattern formation aspects of electrically charged tri-stable media with implications to bulk heterojunction in organic photovoltaics
###### Alon Z. Shapira, Nir Gavish, Arik Yochelis
A common thread in designing electrochemically-based renewable energy devices comprises materials that exploit nano-scale morphologies, e.g., supercapacitors, batteries, fuel cells, and bulk heterojunction organic photovoltaics. In these devices, however, Coulomb forces often influence the fine nano-details of the morphological structure of active layers leading to a notorious decrease in performance. By focusing on bulk heterojunction organic photovoltaics as a case model, a self-consistent mean-field framework that combines binary (bi-stable) and ternary (tri-stable) morphologies with electrokinetics is presented and analyzed, i.e., undertaking the coupling between the spatiotemporal evolution of the material and charge dynamics along with charge transfer at the device electrodes. Particularly, it is shown that tri-stable composition may stabilize stripe morphology that is ideal bulk heterojuction. Moreover, since the results rely on generic principles they are expected to be applicable to a broad range of electrically...
more | pdf | html
None.
None.
###### Other stats
Sample Sizes : None.
Authors: 3
Total Words: 4606
Unqiue Words: 1776

##### #4. Charged solitons in branched conducting polymers
###### D. Babajanov, H. Matyoqubov, D. Matrasulov
We consider dynamics of charged solitons in branched conducting polymers. An effective model based on the sine-Gordon equation on metric graphs is used for computing the charge transport and scattering of charge carriers at the polymer branching points. Condition for the ballistic charge carrier transport is revealed.
more | pdf | html
###### Tweets
SciencePapers: Charged solitons in branched conducting polymers. https://t.co/SOiAa3mgKR
None.
None.
###### Other stats
Sample Sizes : None.
Authors: 3
Total Words: 3562
Unqiue Words: 1301

##### #5. Soliton dynamics in finite nonlocal media with cylindrical symmetry
###### Emmanuel Garza, Servando Lopez-Aguayo, Julio C. Gutiérrez-Vega
The effect of finite boundaries in the propagation of spatial nonlocal solitons in media with cylindrical symmetry is analyzed. Using Ehrenfest's theorem together with the Green's function of the nonlinear refractive index equation, we derive an analytical expression for the force exerted on the soliton by the boundaries, verifying its validity by full numerical propagation. We show that the dynamics of the soliton are determined not only by the degree of nonlocality, but also by the boundary conditions for the refractive index. In particular, we report that a supercritical pitchfork bifurcation appears when the boundary condition exceed a certain threshold value.
more | pdf | html
###### Tweets
SciencePapers: Soliton dynamics in finite nonlocal media with cylindrical symmetry. https://t.co/d8T3ePSzG1
None.
None.
###### Other stats
Sample Sizes : None.
Authors: 3
Total Words: 4522
Unqiue Words: 1396

##### #6. Molding the asymmetry of localized frequency-locking waves by a generalized forcing and implications to the inner ear
###### Yuval Edri, Dolores Bozovic, Ehud Meron, Arik Yochelis
Frequency locking to an external forcing frequency is a {well} known phenomenon. In the auditory system, it results in a localized traveling wave, the shape of which is essential for efficient discrimination between incoming frequencies. An amplitude equation approach is used to show that the shape of the localized traveling wave depends crucially on the relative strength of additive vs. parametric forcing components; the stronger the parametric forcing the more asymmetric the response profile and the sharper the traveling-wave front. The analysis captures the empirically observed regions of linear and nonlinear responses and highlights the significance of parametric forcing mechanisms in shaping the resonant response in the inner ear.
more | pdf | html
None.
None.
###### Other stats
Sample Sizes : None.
Authors: 4
Total Words: 4123
Unqiue Words: 1534

##### #7. Modified method of simplest equation and exact traveling wave solutions of a hyperbolic reaction-diffusion equation
###### I. P. Jordanov, Nikolay K. Vitanov
We discuss a class of hyperbolic reaction-diffusion equations and apply the modified method of simplest equation in order to obtain an exact solution of an equation of this class (namely the equation that contains polynomial nonlinearity of fourth order). We use the equation of Bernoulli as a simplest equation and obtain traveling wave solution of a kink kind for the studied nonlinear reaction-diffusion equation.
more | pdf | html
None.
None.
None.
###### Other stats
Sample Sizes : None.
Authors: 2
Total Words: 4555
Unqiue Words: 1443

##### #8. Spectral and dynamical analysis of a single vortex ring in anisotropic harmonically trapped three-dimensional Bose-Einstein condensates
###### Christopher Ticknor, Wenlong Wang, P. G. Kevrekidis
In the present work, motivated by numerous recent experimental developments we revisit the dynamics of a single vortex ring in anisotropic harmonic traps. At the theoretical level, we start from a general Lagrangian dynamically capturing the evolution of a vortex ring and not only consider its spectrum of linearized excitations, but also explore the full nonlinear dynamical evolution of the ring as a vortical filament. The theory predicts that the ring is stable for $1 \leq \lambda \leq 2$, where $\lambda=\omega_z/\omega_r$ is the ratio of the trapping frequencies along the $z$ and $r$ axes, i.e., for spherical to slightly oblate condensates. We compare this prediction with direct numerical simulations of the full 3D Gross-Pitaevskii equation (GPE) capturing the linearization spectrum of the ring for different values of the chemical potential and as a function of the anisotropy parameter $\lambda$. We identify this result as being only asymptotically valid as the chemical potential $\mu \rightarrow \infty$, revealing how the...
more | pdf | html
None.
None.
###### Other stats
Sample Sizes : None.
Authors: 3
Total Words: 5889
Unqiue Words: 1892

##### #9. Universality of the amplitude shift in fast two-pulse collisions in weakly perturbed linear physical systems
###### Quan M. Nguyen, Toan T. Huynh, Avner Peleg
We demonstrate that the amplitude shifts in fast two-pulse collisions in perturbed linear physical systems with weak nonlinear dissipation exhibit universal soliton-like behavior. The behavior is demonstrated for linear optical waveguides with weak cubic loss and for systems described by linear diffusion-advection models with weak quadratic loss. We show that in both systems, the expressions for the collision-induced amplitude shifts due to the nonlinear loss have the same form as the expression obtained for a fast collision between two solitons of the nonlinear Schr\"odinger equation in the presence of weak cubic loss. Furthermore, we show that the expressions for the amplitude shifts are universal in the sense that they are independent of the exact details of the initial pulse shapes. We demonstrate the universal soliton-like behavior of the collision-induced amplitude shifts by carrying out numerical simulations with the two perturbed coupled linear evolution models with three different initial conditions corresponding to...
more | pdf | html
None.
###### Tweets
SciencePapers: Universality of the amplitude shift in fast two-pulse collisions in weakly perturbed linear physical systems. https://t.co/J34DiylmHs
None.
None.
###### Other stats
Sample Sizes : None.
Authors: 3
Total Words: 12910
Unqiue Words: 1831

##### #10. Novel nonlinear wave equation: regulated rogue waves and accelerated soliton solutions
###### Abhik Mukherjee, Anjan Kundu
A new exactly solvable (1+1)-dimensional complex nonlinear wave equation exhibiting rich ana- lytic properties has been introduced. A rogue wave (RW), localized in space-time like Peregrine RW solution, though richer due to the presence of free parameters is discovered. This freedom allows to regulate amplitude and width of the RW as needed. The proposed equation allows also an intriguing topology changing accelerated dark soliton solution in spite of constant coefficients in the equation.
more | pdf | html
None.
None.
###### Other stats
Sample Sizes : None.
Authors: 2
Total Words: 2953
Unqiue Words: 1041

Assert is a website where the best academic papers on arXiv (computer science, math, physics), bioRxiv (biology), BITSS (reproducibility), EarthArXiv (earth science), engrXiv (engineering), LawArXiv (law), PsyArXiv (psychology), SocArXiv (social science), and SportRxiv (sport research) bubble to the top each day.

Papers are scored (in real-time) based on how verifiable they are (as determined by their Github repos) and how interesting they are (based on Twitter).

To see top papers, follow us on twitter @assertpub_ (arXiv), @assert_pub (bioRxiv), and @assertpub_dev (everything else).

To see beautiful figures extracted from papers, follow us on Instagram.

Tracking 58,338 papers.

###### Search
Sort results based on if they are interesting or reproducible.
Interesting
Reproducible
Online
###### Stats
Tracking 58,338 papers.


text:
# ibis.expr.api.IntervalValue.hours¶

IntervalValue.hours

Extract the number of hours from an IntervalValue expression.

Returns: IntegerValue The number of hours in the expression


text:
# Writer

class Writer < Object

Abstract writer.

Typical use is to use the Writer factory function to instantiate an object of an appropriate derived class based on the file extension of the given path:

auto writer <- Writer(path);

A write of a single buffer can then be performed with:

writer.write(buffer);

A write of a sequence of buffers can be performed with:

writer.startSequence();
writer.write(buffer1);
writer.write(buffer2);
writer.write(buffer3);
writer.endSequence();

which is useful for not keeping the entire contents of the file, to be written, in memory.

Finally, close the file:

writer.close();

A file may not be valid until the writer is closed, depending on the file format.

### Factory Functions

Name Description
Writer Create a writer for a file.

### Member Functions

Name Description
open Open a file.
write Write the entire contents of the file.
flush Flush accumulated writes to the file.
close Close the file.
startMapping Start a mapping.
endMapping End a mapping.
startSequence Start a sequence.
endSequence End a sequence.

### Factory Function Details

function Writer(path:String) -> Writer

Create a writer for a file.

• path: Path of the file.

Returns: the writer.

The file extension of path is used to determine the precise type of the returned object. Supported file extension are .json and .yml.

### Member Function Details

function close()

Close the file.

#### endMapping

function endMapping()

End a mapping.

#### endSequence

function endSequence()

End a sequence.

#### flush

function flush()

Flush accumulated writes to the file.

#### open

function open(path:String)

Open a file.

• path : Path of the file.

#### startMapping

function startMapping()

Start a mapping.

#### startSequence

function startSequence()

Start a sequence.

#### write

function write(buffer:MemoryBuffer)

Write the entire contents of the file.

• buffer: Buffer to write.


text:
Skip to main content

# 5: Eigenvalue problems

## Contributors and Attributions

• Was this article helpful?


text:
## Quantum Principal Component analysis by Seth Lloyd

3

0

I am currently reading the paper quantum principal component analysis from Seth Lloyd's article Quantum Principal Component Analysis There is the following equation stated.

Suppose that on is presented with n copies of $$\rho$$. A simple trick allows one to apply the unitary transformation $$e^{-i\rho t}$$ to any density matrix $$\sigma$$ up to $$n$$th order in $$t$$. Note that \begin{align}\label{eq1}\tag{1} \text{tr}_pe^{-iS\Delta t}\rho\otimes\sigma e^{iS\Delta t} &= \left(\cos^2\Delta t\right)\sigma + \left(\sin^2 \Delta t\right)\rho - i\sin\Delta t\left[\rho, \sigma\right] \\ &= \sigma - i\Delta t\left[\rho, \sigma\right] + \mathcal O\left(\Delta t^2\right) \end{align} Here $$\text{tr}_p$$ is the partial trace over the first variable and $$S$$ is the swap operator. $$S$$ is a sparse matrix and so $$e^{-iS\Delta t}$$ can be performed efficiently [6-9]. Repeated application of \eqref{eq1}

I know from the qiskit website, that we can express $$\mathrm{e}^{i\gamma B}$$ as $$\cos(\gamma)I + i\sin(\gamma)B$$ with $$\gamma$$ being some real number and $$B$$ is an involutory matrix. Can someone explain me why there is only a single $$\sigma$$ and no $$\rho$$ in $$(\cos^2\triangle t)\sigma$$ and a single $$\rho$$ and no $$\sigma$$ in $$(\sin^2\triangle t)\rho$$? Is it because of the partial trace?

Any intuition or approach is welcome. Many thanks in advance.

As you say, start by expanding $$e^{-iS\Delta t}=\cos(\Delta t)I-i\sin(\Delta t)S$$, so you'd be calculating $$(\cos(\Delta t)I-i\sin(\Delta t)S)\rho\otimes\sigma(\cos(\Delta t)I+i\sin(\Delta t)S).$$ If you multiply out all the terms, then the $$\cos^2(\Delta t)$$ comes from the two $$I$$ terms, leaving you with $$\rho\otimes\sigma$$. If you trace out the first term, you're left with $$\sigma$$.
Similarly, $$S(\rho\otimes\sigma)S=\sigma\otimes\rho$$, so if you trace out the first system, you're left with $$\rho$$.
It's actually the cross-terms that are the trickier ones. Take, for example, $$S(\rho\otimes\sigma)$$. If I try to trace out the first system, I have \begin{align} \sum_{i,j,k}|j\rangle\langle k|\langle i,j|S\rho\otimes\sigma|i,k\rangle&=\sum_{i,j,k}|j\rangle\langle k|\langle j,i|\rho\otimes\sigma|i,k\rangle \\ &=\sum_{i,j,k}|j\rangle\langle k|\langle j|\rho|i\rangle\langle i|\sigma|k\rangle \\ &=\sum_{j,k}|j\rangle\langle k|\langle j|\rho\sigma|k\rangle \\ &=\rho\sigma. \end{align}


text:
# zbMATH — the first resource for mathematics

Homotopy Batalin-Vilkovisky algebras. (English) Zbl 1258.18005
Recall that a Batalin-Vilkovisky algebra is a graded commutative algebra $$A$$ equipped with a unary operator $$\Delta: A\rightarrow A$$, homogeneous of degree $$1$$, satisfying $$\Delta^2 = 0$$, and so that the operation $\lambda(x_1,x_2) = \Delta(x_1 x_2) - (\Delta(x_1) x_2 + x_1\Delta(x_2)){(*)}$ satisfies the relations of an odd Poisson bracket (a Gerstenhaber bracket) on $$A$$. The operad of Batalin-Vilkovisky algebras $$\mathcal{BV}$$ is defined by generators and relations, with a two-ary generating operation $$\mu\in\mathcal{BV}(2)$$ representing the product underlying a Batalin-Vilkovisky structure, a second two-ary generating operation $$\lambda\in\mathcal{BV}(2)$$ representing the Gerstenhaber bracket, and a unary generating operation $$\Delta\in\mathcal{BV}(1)$$ which represents the Batalin-Vilkovisky operator.
The Koszul resolution of an operad, in the sense of V. Ginzburg and M. Kapranov [Duke Math. J. 76, No. 1, 203–272 (1994; Zbl 0855.18006)], is a quasi-free operad $$\mathcal P_{\infty} = (\mathcal F(s^{-1}\mathcal P^{\text{<}}),d_2)$$ whose differential $$d_2$$ is quadratic on generators $$\mathcal P^{\text{<}}$$. The definition of the operad $$\mathcal{BV}$$ involves a relation (*) which is not homogeneous quadratic with respect to the composition of generating operations, since this equation involves the Gerstenhaber bracket as a linear term. The existence of this inhomogeneous relation gives an obstruction for the construction of a Koszul resolution in the above sense. The authors therefore consider a graded operad $$q\mathcal{BV}$$, with the same generating operations as in the operad $$\mathcal{BV}$$, but where this linear term in (*) is canceled. The crux of their construction relies on the observation that this operad $$q\mathcal{BV}$$ is Koszul, and that a linear term can be added to the differential of the Koszul resolution of this graded operad to get a quasi-free operad $$\mathcal{BV}_{\infty} = (\mathcal F(s^{-1}q\mathcal{BV}^{\text{<}}),d_1+d_2)$$ defining a resolution of the operad $$\mathcal{BV}$$.
The authors considers the category of algebras associated to the operad $$\mathcal{BV}_{\infty}$$ to get an explicit definition of the notion of a homotopy $$\mathcal{BV}$$-algebra. The authors notably use this effective approach to prove that any non-negatively graded topological vertex operator algebra in the sense of B. H. Lian and G. J. Zuckerman [Commun. Math. Phys 154, 613–646 (1993; Zbl 0780.17029)] inherits a homotopy $$\mathcal{BV}$$-algebra structure.


text:
ISSN 0439-755X
CN 11-1911/B

Acta Psychologica Sinica ›› 2014, Vol. 46 ›› Issue (2): 216-226.

### Temporal-Spatial Metaphor in Conceptual Representation: Can Spatial Information be Activated When Processing the Temporal Reference Which Implied in the Changing Shape of Objects?

SONG Yiqi;ZHANG Jijia

1. (Department of Psychology, Renmin University of China, Beijing 100872, China) (Center for Psychological Application, South China Normal University, Guangzhou 510631, China)
• Received:2013-05-13 Published:2014-02-25 Online:2014-02-25
• Contact: ZHANG Jijia

Abstract:

Although time is abstract, unbodied and intangible, human also can feel it in some way. The changing shapes of objects show the existence of time. As time goes on, the shape of objects will change or be changed. The objects in a forward position of time series refer to “earlier”, while the objects in a afterward position of time series refer to “later”. It has been demonstrated that English participants were faster responding to “earlier” objects presented on the left of the screen and to “later” objects presented on the right of the screen. Both the perceptual symbol theory and the propositional symbol theory assure that this finding can be explained by the opinion of the reading/writing directionality affecting the representation of temporal sequences. Therefore, the former studies haven’t solved the disputes of the above two theories. The substantial difference between two theories is whether the information, activated during the concept processing, is perceptual. In present research, two experiments were conducted to investigate whether the perceptual traits (the degree of shape change) affected congruency effects between temporal reference and spatial relation. In each experiment, pairs of words or pictures of objects, which were displayed on the left and right of the screen respectively, were used as targets. It is called metaphor-consistent condition, if the “earlier” objects in this condition were on the left of screen and the “later” ones were on the right; by contrast, it is called metaphor-inconsistent condition. Pairs of objects were divided into two groups. In the one group, the shapes of objects in pairs have changed gradually as time goes on. There are still a lot of same traits between two objects; in the other group, the shapes of objects in pairs have changed abruptly. There is little resemblance between two objects. In Experiment 1, Participants were instructed to carry out semantic relatedness judgment to pairs of object names with go/no-go task. The results showed semantic processing was inhibited in metaphor-inconsistent condition rather than metaphor-consistent condition. Participants were faster responding when the “earlier” objects in the left of the screen and the “later” objects in the right of the screen. It indicated that the temporal reference which implied in the changing shape of objects was activated. Furthermore, there were more significant temporal-spatial congruency effects in the shapes-changing-abruptly group. This result showed that perceptual traits were activated when processing the concept. Experiment 2, which was same as Experiment 1 except changing words into pictures, got the same temporal-spatial congruency effects as Experiment 1. Furthermore, Experiment 2 got a very important and different result. Perceptual traits affected the semantic relatedness judgment to pairs of object pictures which did not observe in Experiment 1. Participants were faster responding to the objects of gradually changing shapes than the ones of changing abruptly shapes. In short, it is claimed that spatial information can be activated when processing temporal reference which implied in the changing shape of objects. Combining the results of two experiments, it suggested that there were both semantic processing and perceptual simulation in the conceptual processing, which further proved the perspectives of dual-processing theory.


text:
# Nov 8, 2018 - Higher-order variations of the $p$-Dirichlet energy

In this post, we would like to discuss some combinatorial aspects of the $p$-Laplacian. Namely, let $\int_\Omega |\nabla u|^p \, dx$ be the $p$-Dirichlet energy, where $u \in W^{1,p}(\Omega)$ and $p>1$. Its first variation is given by

$D^1 \left(\int_\Omega |\nabla u|^p \, dx\right) (\xi_1) = p \int_\Omega |\nabla u|^{p-2} (\nabla u, \nabla \xi_1) \, dx,$

where $\xi_1 \in W^{1,p}(\Omega)$.

The second variation (if exists) is also easy to compute:

\begin{align} \notag D^2 \left(\int_\Omega |\nabla u|^p \, dx\right) (\xi_1, \xi_2) &= p(p-2) \int_\Omega |\nabla u|^{p-4} (\nabla u, \nabla \xi_1) (\nabla u, \nabla \xi_2) \, dx \newline \notag &+ p \int_\Omega |\nabla u|^{p-2} (\nabla \xi_1, \nabla \xi_2) \, dx, \end{align}

where $\xi_2 \in W^{1,p}(\Omega)$.

Let us make some effort to calculate the third variation:

\begin{align} \notag D^3 \left(\int_\Omega |\nabla u|^p \, dx\right) (\xi_1, \xi_2, \xi_3) &= p(p-2)(p-4) \int_\Omega |\nabla u|^{p-6} (\nabla u, \nabla \xi_1) (\nabla u, \nabla \xi_2) (\nabla u, \nabla \xi_3) \, dx \newline \notag &+ p(p-2) \int_\Omega |\nabla u|^{p-4} (\nabla u, \nabla \xi_1) (\nabla \xi_2, \nabla \xi_3) \, dx \newline \notag &+ p(p-2) \int_\Omega |\nabla u|^{p-4} (\nabla u, \nabla \xi_2) (\nabla \xi_1, \nabla \xi_3) \, dx \newline \notag &+ p(p-2) \int_\Omega |\nabla u|^{p-4} (\nabla u, \nabla \xi_3) (\nabla \xi_1, \nabla \xi_2) \, dx, \end{align}

where $\xi_3 \in W^{1,p}(\Omega)$.

We already start seeing some structure. So, let us now try to derive a general formula for the $n$-th variation of the energy functional. Our main result is the following one.

Proposition. Let $u \in W^{1,p}(\Omega)$. If for a natural $n \geq 1$ there exists $n$-th variation of the $p$-Direchlet energy of $u$ in direction $(\xi_1,\dots,\xi_n) \in (W^{1,p}(\Omega))^n$, then

\begin{align} \notag &D^n \left(\int_\Omega |\nabla u|^p \, dx\right) (\xi_1, \dots, \xi_n) \newline \notag &= \int_\Omega \left( \sum\limits_{i=0}^{\lfloor \frac{n}{2} \rfloor} |\nabla u|^{p-2(n-i)} \prod\limits_{j=0}^{n-i-1} (p-2j) \left[\sum\limits_{\sigma \in B(n,n-2i)} \prod\limits_{k=1}^{n-2i} (\nabla u, \nabla \xi_{\sigma(k)}) \left(\sum\limits_{\omega \in P(n,\sigma)} \prod\limits_{l=1}^{i} (\nabla \xi_{\omega(l,1)}, \nabla \xi_{\omega(l,2)}) \right) \right] \right) dx, \end{align}

where

• $B(n,n-2i)$ is the set of all possible $(n-2i)$-combinations of $\{1,2,\dots,n\}$ such that the ordering inside each $\sigma \in B(n,n-2i)$ is immaterial. Evidently, the cardinality of $B(n,n-2i)$ is ${n \choose n-2i}$. In particular, if $i=0$, then $card(B(n,n-2i)) = 1$.

• $P(n,\sigma)$ is the set of all possible partitions of the set $\{1,2,\dots,n\} \setminus \sigma$ into pairs such that the ordering of pairs and inside a pair is immaterial. Note that $card(\sigma)=n-2i$, and hence the number of pairs in each $\omega \in P(\sigma)$ is $i$. It is not hard to see that the cardinality of $P(\sigma)$ is $\frac{(2i)!}{2^i i!}$. We represent $\omega$ as a $i \times 2$-matrix $(\omega(s,t))_{s=1..i,~t=1,2}$. For instance, if $n=6$ and $\sigma = \{1,2\}$, then

$P(\sigma) = \left\{ \begin{pmatrix} 3 & 5 \newline 4 & 6 \end{pmatrix}, \begin{pmatrix} 3 & 4 \newline 6 & 5 \end{pmatrix}, \begin{pmatrix} 3 & 4 \newline 4 & 6 \end{pmatrix} \right\}.$

# Oct 19, 2018 - Sobolev-type inequality with product of norms

Lemma. Let $\Omega \subset \mathbb{R}^N$ be a bounded open set, $N \geq 1$. Let $1 \leq q \leq p \leq \gamma$, and $\gamma \leq p^*$ if $N>p$ and $\gamma<\infty$ if $N < p$. Then there exists $C=C(\Omega,q,p,\gamma)$ such that for any $u \in W_0^{1,p}(\Omega)$ the following inequality is satisfied:

$\left(\int_\Omega |u|^q \, dx \right)^{\gamma-p} \left(\int_\Omega |u|^\gamma \, dx \right)^{p-q} \leq C \left(\int_\Omega |\nabla u|^p \, dx \right)^{\gamma-q}.$

In fact, this inequality easily follows from the general Sobolev inequality just by applying the latter one to each term on the lhs of the former one. (I’m almost sure there could be some investigation of an inequality of this type in the literature, but I was not able to find it.)

1) It is just beautiful that the exponents are permuted)

2) If one considers the optimization problem for the constant $C$, then it is possible to see that a corresponding minimizer of

$\inf_{u \in W_0^{1,p}(\Omega) \setminus \{0\}} \frac{\left(\int_\Omega |\nabla u|^p \, dx \right)^{\gamma-q}}{\left(\int_\Omega |u|^q \, dx \right)^{\gamma-p} \left(\int_\Omega |u|^\gamma \, dx \right)^{p-q}}$

exists and satisfies the convex-concave equation

$-\Delta_p u = \lambda |u|^{q-2} u + |u|^{\gamma-2} u.$

In fact, such a minimization problem was explicitly considered in connection with the above equation in [1].

# Bibliography

1. Il’yasov, Y. (2005). On nonlocal existence results for elliptic equations with convex–concave nonlinearities. Nonlinear Analysis: Theory, Methods & Applications, 61(1-2), 211-236.

# Sep 25, 2018 - Growth rate of divisor function for odd numbers

Consider the standard divisor function $\sigma_\alpha(n)$ defined as

$\sigma_\alpha(n) = \Sigma_{d | n} d^\alpha.$

Assume that $\alpha > 1$. It was proved at least by Gronwall in [1] (yeah, this is the same Gronwall who obtained Gronwall’s inequality) that $\sigma_\alpha(n)$ satisfies

$\sigma_\alpha(n) < \zeta(\alpha) n^\alpha$

for all $n \in \mathbb{N}$, and this upper bound is achieved. Here $\zeta(\alpha)$ is the Riemann Zeta function.

Recently, thinking with Falko Baustian on one problem, we were partially interested in the asymptotic behaviour of $\sigma_\alpha(n)$ for odd numbers $n$. The natural question is whether we can improve the upper bound for odd $n$’s?

Proposition. Let $\alpha>1$, and let $n \in \mathbb{N}$ be odd. Then

$\sigma_\alpha(n) < \left(1-\frac{1}{2^\alpha}\right) \zeta(\alpha) n^\alpha,$

and this upper bound is achieved.

Proof. We will argue sketchely and along the same lines as in [1], pp. 114-116. First, we decompose an arbitrary $n$ into prime factors as

$n = p_{\lambda_1}^{\nu_1} \cdot \ldots \cdot p_{\lambda_k}^{\nu_k},$

where $p_i$ is a prime, $\lambda_i < \lambda_{i+1}$, and $\nu_i > 0$. Note that since $n$ is odd, we have $p_i \neq 2$.

As in formula (5) of [1], we have

$$\label{eq:sigma} \sigma_\alpha(n) = n^\alpha \Pi_{i=1}^k \frac{1-\frac{1}{p_{\lambda_i}^{\alpha(\nu_i+1)}}}{1-\frac{1}{p_{\lambda_i}^\alpha}}.$$

Since the numerator and denominator in \eqref{eq:sigma} are less than one and since $n$ is odd, we can estimate $\sigma_\alpha(n)$ as

$\sigma_\alpha(n) < n^\alpha \Pi_{i=1}^k \frac{1}{1-\frac{1}{p_{\lambda_i}^\alpha}} < n^\alpha \Pi_{i=2}^\infty \frac{1}{1-\frac{1}{p_{i}^\alpha}} = n^\alpha \left(1-\frac{1}{2^\alpha}\right) \Pi_{i=1}^\infty \frac{1}{1-\frac{1}{p_{i}^\alpha}} = n^\alpha \left(1-\frac{1}{2^\alpha}\right) \zeta(\alpha),$

where $p_i$ are consequtive primes $p_1 = 2$, $p_2=3$, etc. Therefore, the desired upper bound is obtained.

1. Gronwall, T. H. (1913). Some asymptotic expressions in the theory of numbers. Transactions of the American Mathematical Society, 14(1), 113-122.  2 3

# May 15, 2018 - Eigenvalues of cylinders

Consider a standard 3D cylinder $C_{R,a} := B_R \times [0,a]$, where $B_R \subset \mathbb{R}^2$ is a disk of radius $R>0$, and $a>0$ is the height of the cylinder. It is not hard to see (we will discuss it below) that the spectrum of the Laplace operator on $C_{R,a}$ under zero Dirichlet conditions consists of eigenvalues

$\lambda_{m,n,k}(R,a) = \frac{j_{m,n}^2}{R^2} + \frac{k^2 \pi^2}{a^2}, \quad m \in \mathbb{N} \cup \{0\}, ~ n,k \in \mathbb{N},$

where $j_{m,n}$ is the $n$-th positive zero of the Bessel function $J_{m}$ of order $m$. That is, the first term corresponds to the spectrum of $B_R$, and the second term - to the spectrum of $[0,a]$.

Recalling the properties of the spectrum of $B_R$, we know that the multiplicity of any $\lambda_{m,n,k}(R,a)$ can be, at least, one or two. Let us give the following more detailed result whose proof follows directly from the expression of $\lambda_{m,n,k}(R,a)$.

Lemma. Let $R,a>0$. If there exist $m_i,n_i,k_i$, $i=1,2$, such that $\lambda_{m_1,n_1,k_1}(R,a) = \lambda_{m_2,n_2,k_2}(R,a)$, then

$a = \pi R \left(\frac{k_1^2-k_2^2}{j_{m_2,n_2}^2-j_{m_1,n_1}^2}\right)^{1/2}.$

In particular, if $R>0$ is fixed, then the set of $a$’s for which the multiplicity of some $\lambda_{m,n,k}(R,a)$ exceeds two is at most countable. The same holds true if we fix $a>0$ and vary $R>0$. Moreover, clearly, analogous assertions can be stated for higher-dimensional cylinders $B_R \times [0,a]$, where a ball $B_R \subset \mathbb{R}^N$.

# May 3, 2018 - Can the multiplicity be purely geometric?

Yesterday, during a discussion with P. Drabek, I have learned about one nice fact (see Lemma below) on the multiplicity of eigenvalues for the following one-dimensional (nonlinear) eigenvalue problem for the weighted $p$-Laplacian:

\left\{ \begin{aligned} -(a(x)|u'|^{p-2} u')' &= \lambda b(x) |u|^{p-2}u \quad {\rm in}\ (0,1), \\\ u(0) &= u(1) = 0, \end{aligned} \right. \tag{1}\label{1}

where $p \geq 2$, $a,b \in C^1(a,b)$, $a,b > 0$ on $(0,1)$. Denote by $\{\lambda_k\}$ a set of eigenvalues of \eqref{1}. (In fact, $\{\lambda_k\}$ is countable, see below.)

Unlike the linear case $p=2$, in the general nonlinear case $p \neq 2$ it is necessary to specify what one means under the multiplicity of an eigenvalue $\lambda_k$. Naturally, there are two types of the multiplicity:

• Algebraic multiplicity, say $m$, means that there exist $m$ eigenvalues $\lambda_{l}, \dots, \lambda_{l+m-1}$ with $l \leq k \leq l+m-1$ such that
$\lambda_{l-1} < \lambda_{l} = \dots = \lambda_{k} = \dots = \lambda_{l+m-1} < \lambda_{l+m}.$
• Geometric multiplicity, say $m$, means that there exist $m$ distinct eigenfunctions associated with $\lambda_k$. In fact, it is not entirely clear what it should be meant under “distinct”. Definitely, each eigenfunction has to be defined uniquely up to scaling. Moreover, in the linear case $p=2$, the eigenfunctions have to be linearly independent. However, in the case $p \neq 2$, it seems to be unnecessary to demand the linear independency from eigenfuctions. That is, if $u_1$ and $u_2$ are linearly independent and both associated with $\lambda_k$, and, by some chance, $c_1 u_1 + c_2 u_2$ is again an eigenfunction associated with $\lambda_k$ for some $c_1,c_2 \neq 0$, then all three eigenfunctions $u_1$, $u_2$, $c_1 u_1 + c_2 u_2$ can be considered as distinct.

Just by definition, the algebraic multiplicity implies the geometric one. The natural (and seems to be open) question is the converse assertion: Does the geometric multiplicity imply the algebraic one? Surely, this question is worth for a higher-dimensional problem, too.

In this direction, the following interesting result was proved by J. Nečas (see Appendix V, Theorme 1.1, p. 233 in [1]):

Lemma. All eingevalues of \eqref{1} form a countable set $0 < \lambda_1 < \lambda_2 < \ldots$ such that $\lim\limits_{k\to\infty} \lambda_k = \infty$. All (normed) eigenfunctions are isolated. Moreover, each $\lambda_k$ has only a finite number of (normed) eigenfunctions.

1. Fučik, S., Nečas, J., Souček, J., & Souček, V. (1973). Spectral analysis of nonlinear operators. Springer.

# Apr 6, 2018 - A wrong integral criterion for Bourget's hypothesis

Recently, I accidentally stumbled upon the old article [1] which claimed that the Bessel functions $J_{\nu}$ and $J_{\mu}$ have no common positive zero for any positive real orders $\nu, \mu \in \mathbb{R}^+$.

Recall that the same assertion for $\nu, \mu \in \mathbb{N}$ holds true and usually referred as Bourget’s hypothesis, see p.485 in [2].

However, it is quite easy to see that Bourget’s hypothesis fails, in general, if $\nu, \mu \in \mathbb{R}^+$. Thus, the result of [1] is already incorrect. The authors of [1] provided, as a main tool, the following integral criterion for Bourget’s hypothesis:

$J_{\nu}(\lambda) = J_{\mu}(\lambda) = 0$ for some $\lambda, \nu, \mu \in \mathbb{R}^+$ if and only if $\int_0^\lambda \frac{1}{t} J_{\nu}(t) J_{\mu}(t) \, dt = 0$.

This result would be an interesting observation if it were not false. In fact, it is ok in one direction. However, if the integral vanishes, it doesn’t imply the vanishing of $J_\nu$ and $J_\mu$. This can be easily shown numerically by considering, say, $\nu=1$, $\mu=3.2$, and looking for the value of the integral and a common zero (which doesn’t exist) in the interval $(14,15)$.

P.S. Already in 1964 the authors speak about Bourget’s hypothesis as an “old fashion problem”. I feel a bit of shame that, after almost 60 years from [1], I’m still interested in this very outdated problem, especially in connection with multiplicity of Dirichlet eigenvalues in annuli, see the previous post :-)

# Bibliography

1. Kitamura, T., & Tani, T. (1964). On the Bourget’s hypothesis and the non-degeneracy of the zeros of the Bessel functions. Bulletin of the Faculty of Arts and Sciences, Ibaraki University. Natural science, 15, 1-4. http://hdl.handle.net/10109/5360  2 3 4

2. Watson, G. N. (1944). A treatise on the theory of Bessel functions. Cambridge: The University Press.

# Mar 29, 2018 - Asymptotic relation for zeros of cross-product of Bessel functions

Recently, I’ve submitted a preprint to arXiv, where I studied some properties of zeros of the cross-product of Bessel functions

$$\label{eq:cp} J_\nu(R z) Y_\nu(z) - J_\nu(z) Y_\nu(R z), \quad \nu\geq 0, ~R>1.$$

This research originated from the previous post, where I studied the explicit expression (and value) of the Pleijel constant for the disk; see also this preprint. Recall that the expression for $Pl(B)$ crucially relies on the result of Elbert & Laforgia [1] (see also [2]) on the asymptotic $j_{kx,k}$ as $k\to \infty$ for the zero $j_{kx,k}$ of the Bessel function $J_{kx}$. More precisely, Elbert & Laforgia [1] shown that

$\lim_{k \to \infty} \frac{j_{kx,x}}{k} = \iota(x), \quad x > -1,$

where $\iota(x)$ is a unique solution of the intial value problem

$\frac{dy}{dx} = \frac{\arccos\left(\frac{x}{y}\right)}{\sqrt{1 - \left(\frac{x}{y}\right)^2}}, \qquad y(0) = \pi.$

In fact, $\iota(x)$ admits a closed-form representation in terms of a solution of a transcendental equation (2.1) in [1]; see also Section 1.5 in [2].

However, if one would like to obtain an explicit expression for the Pleijel constant of annuli in a similar fashion as for the disk, then a similar asymptotic relation as of Elbert & Laforgia but for zeros of \eqref{eq:cp} is needed. (Let me recall that eigenvalues of the Dirichlet Laplace operator on the annulus $A_{R} = \{x \in \mathbb{R}^2:~ 1 < |x| < R\}$ are given by $\lambda_{\nu,k} = a_{\nu,k}^2$, where $a_{\nu,k}$ is the $k$-th positive zero of \eqref{eq:cp}, $\nu \in \mathbb{N}$.) Unfortunately, I was not able to find any appropriate result for $a_{\nu,k}$ in the literature. Moreover, it really seems that the theory of zeros of \eqref{eq:cp} is much much less developed than the theory of zeros of $J_{\nu}$, and only few bounds and asymptotic results are known. (But, from eigenbusiness point of view, zeros of \eqref{eq:cp} are no less important than zeros of $J_\nu$.)

1. Elbert, Á., & Laforgia, A. (1994). An asymptotic relation for the zeros of Bessel functions. Journal of Mathematical Analysis and Applications, 98(2), 502-511. DOI:10.1016/0022-247X(84)90265-8  2 3

2. Elbert, Á. (2001). Some recent results on the zeros of Bessel functions and orthogonal polynomials. Journal of computational and applied mathematics, 133(1-2), 65-83. DOI:10.1016/S0377-0427(00)00635-X  2

# Feb 14, 2018 - Asymptotic multiplicity of eigenvalues

Let $\Omega \subset \mathbb{R}^2$ be a bounded domain with the smooth boundary $\partial \Omega$. Consider the eigenvalues $\lambda_k$ of the Laplace operator on $\Omega$ under zero Dirichlet boundary conditions. Let us denote by $m(\lambda_k)$ the multiplicity of $\lambda_k$.

It was proved by Hoffmann-Ostenhof, Michor and Nadirashvili 1 that $m(\lambda_k) \leq 2k-3$ for any $k \geq 3$. Here, I would like to provide the following Pleijel-type remark about asymptotic behavior of $m(\lambda_k)$ as $k \to \infty$.

Remark. The following inequality is satisfied:

$$\label{eq:1} \limsup_{k \to \infty} \frac{m(\lambda_k)}{k} < \frac{8}{j_0^2} - 6 \cdot 10^{-9} = 1.383320546…,$$

where $j_0$ is the first zero of the Bessel function $J_0$.

Moreover, $m(\lambda_k) \leq 2k-5$ provided

$$\label{2} k > \frac{32 \pi j_0^4 |\Omega|}{(j_0^2-4)^2} \left(\inf \left\{\epsilon:~ |{x \in \Omega: d(x,\partial \Omega) < \epsilon}| \geq \frac{(j_0^2-4)|\Omega|}{2j_0^2}\right\}\right)^{-2}.$$

The inequality \eqref{eq:1} follows from Theorem B of 1 in combination with the result of Pleijel 2 and Bourgain 3.

The estimate \eqref{2} follows from Theorem B of 1 in combination with Theorem 1 (iii) of van den Berg & Gittins 4.

# Bibliography

1. Hoffmann-Ostenhof, T., Michor, P. W., & Nadirashvili, N. (1999). Bounds on the multiplicity of eigenvalues for fixed membranes. Geometric & Functional Analysis GAFA, 9(6), 1169-1188. arXiv:9801090  2 3

2. Pleijel, Å. (1956). Remarks on Courant’s nodal line theorem. Communications on pure and applied mathematics, 9(3), 543-550. DOI:10.1002/cpa.3160090324

3. Bourgain, J. (2015). On Pleijel’s nodal domain theorem. International Mathematics Research Notices, 2015(6), 1601–1612. DOI:10.1093/imrn/rnt241 arXiv:1308.4422

4. Berg, M. V. D., & Gittins, K. (2016). On the number of Courant-sharp Dirichlet eigenvalues. Journal of Spectral Theory, 6(4), 735-x745. arXiv:1602.08376


text:
### Home > CALC > Chapter 7 > Lesson 7.3.5 > Problem7-158

7-158.

Explain why a differential equation has infinitely many solutions. Homework Help ✎

Discuss the meaning of the constant of integration, C. Be careful, not every C is a vertical shift.


text:
# eccentricity application

• May 23rd 2009, 11:40 PM
vance
eccentricity application
can someone help me with this problem ( I'm quite new with some terms)

A satellite is to be placed in an elliptical orbit, with the center of the earth as one focus. The satellite's maximum distance from the surface of the earth is to be 22380 km , and its minimum distance is to be 6540 km . Assume that the radius of the earth is 6400 km , and find the eccentricity of the satellite's orbit.

• May 24th 2009, 03:38 AM
yeongil
I'll take a crack at this...

If you have an ellipse with equation
$\frac{(x - h)^2}{a^2} + \frac{(y - k)^2}{b^2} = 1$
or
$\frac{(y - k)^2}{b^2} + \frac{(x - h)^2}{a^2}= 1$,
then the eccentricity e is
$e = \frac{c}{a}$,
where
$c = \sqrt{a^2 - b^2}$.
The eccentricity relates to the shape of the ellipse, and it tells us how far off-center the foci are. In ellipses, $0 \leq e < 1$. If e is close to 0 then the foci are close to the center, and if e is close to 1 then the foci are close to the vertices.

In an elliptical orbit, the perihelion (minimum distance) is a - c, and the aphelion (maximum distance) is a + c. Since you mention distances "from the surface of the earth," we'll have to add on the radius of the earth.

Minimum distance:
a - c = 6540 + 6400
a - c = 12940

Maximum distance:
a + c = 22380 + 6400
a + c = 28780

You have two equations and two unknowns. Solve for a and c.
a - c = 12940
a + c = 28780
2a = 41720
a = 20860
20860 + c = 28780
c = 7920

The eccentricity e, therefore, is
$e = \frac{c}{a} = \frac{7920}{20860} \approx 0.380$.

Edit: Fixed typo.

01
• May 24th 2009, 04:17 AM
vance
thanks , but according with my book , the answer should be 0.38
• May 24th 2009, 04:33 AM
yeongil
Oops, I plugged in the wrong value for a at the end. I fixed it, and you're right, e = 0.380.

01


text:
# Symplectic reduction of 4-manifolds with circle actions

Let $(M,\omega)$ be a $4$-dimensional closed symplectic manifold. Assume there exists a Hamiltonian $S^1$-action on $M$, let $\mu:M \to \mathbb{R}^*$ be its moment map and let $M_{\text{red}}=\mu^{-1}(r)/S^1$ denote the symplectic reduction/symplectic quotient, where $r$ is a regular value of $\mu$ and where we assume that $S^1$ acts freely on $\mu^{-1}(r)$. For dimensional reasons, $M_{\text{red}}$ is a closed manifold with $\dim M_{\text{red}}=2$, i.e. a closed surface.

Q: (How) can one determine the genus of $M_{\text{red}}$ from the topology of $M$ and the $S^1$-action?

By a result of Hui Li https://arxiv.org/abs/math/0605133, the fundamental groups of $M_{red}$ and $M$ are isomorphic, so one can deduce the genus of $M_{red}$ from the fundamental group of $M$.


text:
#### ${\mathit m}_{{{\mathit \Sigma}_{{c}}{(2455)}^{0}}}–{\mathit m}_{{{\mathit \Lambda}_{{c}}^{+}}}$

VALUE (MeV) EVTS DOCUMENT ID TECN  COMMENT
 $\bf{ 167.290 \pm0.017}$ OUR FIT
 $\bf{ 167.290 \pm0.022}$ OUR AVERAGE
$167.29$ $\pm0.01$ $\pm0.02$ 32k
 2014
BELL ${{\mathit e}^{+}}{{\mathit e}^{-}}$ at ${{\mathit \Upsilon}{(4S)}}$
$167.28$ $\pm0.03$ $\pm0.12$ 15.9k
 2011 H
CDF ${{\mathit p}}{{\overline{\mathit p}}}$ at 1.96 TeV
$167.2$ $\pm0.1$ $\pm0.2$ 2k
 2002
CLE2 ${{\mathit e}^{+}}{{\mathit e}^{-}}$ $\approx{}{{\mathit \Upsilon}{(4S)}}$
$167.38$ $\pm0.21$ $\pm0.13$ 362
 2000 C
FOCS ${{\mathit \gamma}}$ A, ${{\overline{\mathit E}}}_{\gamma }$ 180 GeV
$167.38$ $\pm0.29$ $\pm0.15$ 143
 1996 B
E791 ${{\mathit \pi}^{-}}{{\mathit N}}$ , 500 GeV
$167.8$ $\pm0.6$ $\pm0.2$
 1996
SPEC ${{\mathit n}}$ nucleus, 50 GeV/$\mathit c$
$166.6$ $\pm0.5$ $\pm0.6$ 69
 1996
E687 ${{\mathit \gamma}}{}^{}\mathrm {Be}$, ${{\overline{\mathit E}}}_{\gamma }\approx{}$ 220 GeV
$167.1$ $\pm0.3$ $\pm0.2$ 124
 1993
CLE2 ${{\mathit e}^{+}}{{\mathit e}^{-}}$ $\approx{}{{\mathit \Upsilon}{(4S)}}$
$168.4$ $\pm1.0$ $\pm0.3$ 14
 1989 D
E691 ${{\mathit \gamma}}{}^{}\mathrm {Be}$ 90$-$260 GeV
• • We do not use the following data for averages, fits, limits, etc. • •
$167.9$ $\pm0.5$ $\pm0.3$ 48 1
 1989
CLEO ${{\mathit e}^{+}}{{\mathit e}^{-}}$ 10 GeV
$167.0$ $\pm0.5$ $\pm1.6$ 70 1
 1988 D
ARG ${{\mathit e}^{+}}{{\mathit e}^{-}}$ 10 GeV
$178.2$ $\pm0.4$ $\pm2.0$ 85 2
 1987
SPEC ${{\mathit n}}{}^{}\mathrm {A}$ $\sim{}$600 GeV
$163$ $\pm2$ 1
 1986
EMUL ${{\mathit \nu}}{}^{}\mathrm {A}$
 1 This result enters the fit through ${\mathit m}_{{{\mathit \Sigma}_{{c}}^{++}}}–{\mathit m}_{{{\mathit \Sigma}_{{c}}^{0}}}$ given below.
 2 See the note on DIESBURG 1987 in the ${\mathit m}_{{{\mathit \Sigma}_{{c}}^{++}}}–{\mathit m}_{{{\mathit \Sigma}_{{c}}^{0}}}$ section below.
References:
 LEE 2014
PR D89 091102 Measurements of the Masses and Widths of the and Baryons
 AALTONEN 2011H
PR D84 012003 Measurements of the Properties of ${{\mathit \Lambda}_{{c}}{(2595)}}$ , ${{\mathit \Lambda}_{{c}}{(2625)}}$ , ${{\mathit \Sigma}_{{c}}{(2455)}}$ , and ${{\mathit \Sigma}_{{c}}{(2520)}}$ Baryons
 ARTUSO 2002
PR D65 071101 Measurement of the Masses and Widths of the ${{\mathit \Sigma}_{{c}}^{++}}$ and ${{\mathit \Sigma}_{{c}}^{0}}$ Charmed Baryons
PL B488 218 Measurements of the ${{\mathit \Sigma}_{{c}}^{0}}$ and ${{\mathit \Sigma}_{{c}}^{++}}$ Mass Splittings
 AITALA 1996B
PL B379 292 Mass Splitting and Production of ${{\mathit \Sigma}_{{c}}^{0}}$ and ${{\mathit \Sigma}_{{c}}^{++}}$ Measured in 500 GeV ${{\mathit \pi}^{-}}{{\mathit N}}$ Interactions
 ALEEV 1996
JINRRC 3-77 31 Observation ${{\mathit \Sigma}_{{c}}^{0}}$ Charmed Baryon in the Experiment EXCHARM
 FRABETTI 1996
PL B365 461 Study of Higher Mass Charm Baryons Decaying to ${{\mathit \Lambda}_{{c}}^{+}}$
 CRAWFORD 1993
PRL 71 3259 Observation of the Charmed Baryon ${{\mathit \Sigma}_{{c}}^{+}}$ and Measurement of the Isospin Mass Splitting of the ${{\mathit \Sigma}_{{c}}}$
 ANJOS 1989D
PRL 62 1721 Observation of ${{\mathit \Sigma}_{{c}}{(2455)}^{0}}$ $\rightarrow$ ${{\mathit \Lambda}_{{c}}^{+}}{{\mathit \pi}^{-}}$ Decays
 BOWCOCK 1989
PRL 62 1240 ${{\mathit \Sigma}_{{c}}^{++}}$ and ${{\mathit \Sigma}_{{c}}^{0}}$ Production from ${{\mathit e}^{+}}{{\mathit e}^{-}}$ Annihilation in the ${{\mathit \Upsilon}}$ Energy Region
 ALBRECHT 1988D
PL B211 489 Observation of the Charmed Baryon ${{\mathit \Sigma}_{{c}}}$ in ${{\mathit e}^{+}}{{\mathit e}^{-}}$ Annihilations
 DIESBURG 1987
PRL 59 2711 Measurement of the ${{\mathit \Sigma}_{{c}}^{0}}$ $−$ ${{\mathit \Lambda}_{{c}}^{+}}$ and ${{\mathit \Sigma}_{{c}}^{++}}$ $−$ ${{\mathit \Lambda}_{{c}}^{+}}$ Mass Differences
 AMMAR 1986
JETPL 43 515 Observation in Nuclear Emulsion of a Charmed ${{\mathit \Sigma}_{{c}}^{0}}$ Baryon Decay to ${{\mathit \Lambda}_{{c}}^{+}}$ ${{\mathit \pi}^{-}}$ with a Subsequent ${{\mathit \Lambda}_{{c}}^{+}}$ Decay to ${{\mathit \Sigma}^{+}}$ ${{\mathit \pi}^{-}}$ ${{\mathit \pi}^{+}}$