text:
Department of

# Mathematics

Seminar Calendar
for Model Theory events the next 12 months of Sunday, January 1, 2017.

.
events for the
events containing

More information on this calendar program is available.
Questions regarding events or the calendar should be directed to Tori Corkery.
    December 2016           January 2017          February 2017
Su Mo Tu We Th Fr Sa   Su Mo Tu We Th Fr Sa   Su Mo Tu We Th Fr Sa
1  2  3    1  2  3  4  5  6  7             1  2  3  4
4  5  6  7  8  9 10    8  9 10 11 12 13 14    5  6  7  8  9 10 11
11 12 13 14 15 16 17   15 16 17 18 19 20 21   12 13 14 15 16 17 18
18 19 20 21 22 23 24   22 23 24 25 26 27 28   19 20 21 22 23 24 25
25 26 27 28 29 30 31   29 30 31               26 27 28

Friday, January 27, 2017

4:00 pm in 345 Altgeld Hall,Friday, January 27, 2017

#### On "Structurable equivalence relations" by R. Chen and A. Kechris: Introduction

###### Anush Tserunyan (UIUC Math)

Abstract: For a class $\mathcal{K}$ of countable relational structures, a countable Borel equivalence relation $E$ is said to be $\mathcal{K}$-structurable if there is a Borel way to put a structure from $\mathcal{K}$ on each $E$-equivalence class. The paper of Chen and Kechris [arXiv link] studies the global structure (including Borel homomorphisms and reductions) of the classes of $\mathcal{K}$-structurable equivalence relations for various $\mathcal{K}$. In this introductory talk, we will give some background and survey the main results of the paper.

Friday, February 3, 2017

4:00 pm in 345 Altgeld Hall,Friday, February 3, 2017

#### On "Structurable equivalence relations" by R. Chen and A. Kechris: Universal equivalence relations (2nd talk)

###### Anush Tserunyan (UIUC Math)

Abstract: In our previous talk, we stated the first main result of the paper: a characterization of the elementary classes of countable equivalence relations. In this second talk, we prove that every elementary class admits a $\sqsubseteq_B^i$-universal equivalence relation. This implies one direction of the aforementioned characterization.

Friday, February 10, 2017

4:00 pm in 345 Altgeld Hall,Friday, February 10, 2017

#### On "Structurable equivalence relations" by R. Chen and A. Kechris: Characterization of elementary classes (3rd talk)

###### Anush Tserunyan (UIUC Math)

Abstract: In our previous talk, we proved that any elementary class of equivalence relations admits an invariantly injective universal element. This completes one direction of the characterization of elementary classes. In this third talk, we will prove the other direction of the characterization, as well as discuss other results of the paper if time permits.

Friday, February 17, 2017

4:00 pm in 345 Altgeld Hall,Friday, February 17, 2017

#### "Strong theories of ordered abelian groups" by A. Dolich and J. Goodrick: Introduction

###### Erik Walsberg (UIUC Math)

Friday, February 24, 2017

4:00 pm in 345 Altgeld Hall,Friday, February 24, 2017

#### On "Structurable equivalence relations" by R. Chen and A. Kechris: Structurability by structures with TDC (4th talk)

###### Aristotelis Panagiotopoulos (UIUC Math)

Abstract: In this talk, we prove a theorem of A. Marks included in the current paper. It says that every aperiodic countable Borel equivalence relation can be $\mathcal{A}$-structured for any countable structure $\mathcal{A}$ with trivial definable closure (TDC). Examples include the rationals, the random graph, and the rational Urysohn sphere.

Friday, March 10, 2017

4:00 pm in 345 Altgeld Hall,Friday, March 10, 2017

#### Strong Theories of Ordered Abelian groups

###### Travis Nell (UIUC Math)

Abstract: We will continue discussing the paper "Strong Theories of Ordered Abelian Groups" by A. Dolich and J. Goodrick

Tuesday, March 14, 2017

1:00 pm in Altgeld Hall,Tuesday, March 14, 2017

#### Model theory and Painleve equations

###### James Freitag (UIC Math)

Abstract: We will discuss how to use model theory to prove some transcendence results for solutions of Painleve equations.

Tuesday, April 4, 2017

1:00 pm in UIC SEO 636,Tuesday, April 4, 2017

#### MidWest Model Theory Day at UIC

Friday, April 7, 2017

4:00 pm in 245 Altgeld Hall,Friday, April 7, 2017

#### Model Theory as a Geography of Mathematics

###### Lou van den Dries (UIUC)

Abstract: This is a dry run for the first talk in the Tarski lectures I am giving the week after in Berkeley. This first talk is for a rather general audience of mathematicians, logicians, and philosophers. I like to think of model theory as a {\em geography of mathematics \}, especially of its tame'' side. Here {\em tame\/} roughly corresponds to {\em geometric\/} as opposed to {\em combinatorial-arithmetic}. In this connection I will discuss Tarski's work on the real field, and the notion of o-minimality that it suggested. A structure $M$ carries its own mathematical territory with it, via interpretability: its own posets, groups, fields,and so on. Understanding this world according to $M$'' can be rewarding. Stability-like properties of $M$ forbid certain combinatorial patterns, thus providing highly intrinsic and robust information about this world.

Wednesday, April 12, 2017

3:00 pm in 243 Altgeld Hall,Wednesday, April 12, 2017

#### First order logic and Sub-riemannian spheres.

###### Erik Walsberg (UIUC)

Abstract: I will discuss a connection between sub-riemannian geometry and first order model theory. Researchers in sub-riemannian geometry have essentially been working on the following: are sub-riemannian spheres definable in an o-minimal expansion of the ordered field of real numbers? (O-minimality is an important and popular topic in model theory developed in large part by UIUC's own Lou van den Dries). It also seems that some model theorists have been trying to construct the kind of structure that the geometers are looking for. As far as I can tell neither side was really aware of what the other was doing until now. I will try to explain some of this. No knowledge of logic or sub-riemannian geometry is necessary.

Friday, April 21, 2017

4:00 pm in 345 Altgeld Hall,Friday, April 21, 2017

#### The logical complexity of finitely generated commutative rings

###### Matthias Aschenbrenner (UCLA)

Abstract: Since the work of G\"odel we know that the theory of the ring $\mathbb Z$ of integers is very complicated. Using the coding techniques introduced by him, every finitely generated commutative ring can be interpreted in $\mathbb Z$ and therefore has a theory which is no more complicated than that of $\mathbb Z$. It has also been long known that conversely, every infinite finitely generated commutative ring interprets the integers, and hence its theory is at least as complex as that of $\mathbb Z$. However, this mutual interpretability does not fully describe the class of definable sets in such rings. The correct point of view is provided by the concept of bi-interpretability, an equivalence relation on the class of first-order structures which captures what it means for two structures to essentially have the same categories of definable sets and maps. We characterize algebraically those finitely generated rings which are bi-interpretable with $\mathbb Z$. (Joint work with Anatole Kh\'elif, Eudes Naziazeno, and Thomas Scanlon.)

Friday, April 28, 2017

4:00 pm in 345 Altgeld Hall,Friday, April 28, 2017

#### On definability and interpretability in model theory

###### Ward Henson (UIUC, UC Berkeley)

Abstract: This will be an expository talk. Topics covered/mentioned will include: extension by definition and expansion/extension by interpretation (the eq-construction), Beth's Theorem (characterizing definability), and Makkai's Theorem (characterizing the eq-expansion). The setting will be model theory of classical (discrete) structures and then of metric (real-valued) structures. In the metric setting, definability of sets/relations has some subtleties that do not arise in discrete structures.

Friday, May 12, 2017

4:00 pm in 345 Altgeld Hall,Friday, May 12, 2017

#### O-minimality and the Sub-Riemannian Sphere

###### Erik Walsberg

Abstract: I will discuss an interesting connection I recently found between Sub-riemannian geometry and o-minimality. In short: sub-riemannian geometers believe that the spheres in real analytic sub-riemannian metrics are definable in some o-minimal expansion of the real field. Furthermore, people in o-minimality are trying to construct the kind of o-minimal expansion that the sub-riemannian geometry are looking for, It seems that the o-minimality community was entirely unaware of this until now. No knowledge of sub-riemannian geometry will be assumed in this talk. Basic knowledge of model theory will be assumed.

Friday, September 1, 2017

4:00 pm in 345 Altgeld Hall,Friday, September 1, 2017

#### Organizational meeting

Friday, September 8, 2017

1:00 pm in 141 Altgeld Hall,Friday, September 8, 2017

#### Keisler Measures, ctd.

###### Travis Nell   [email]

Abstract: We continue reading chapter 7 of Simon's A Guide to NIP Theories.

4:00 pm in 345 Altgeld Hall,Friday, September 8, 2017

#### "Borel circle squaring" by Marks and Unger: Part 1

###### Anton Bernshteyn (Illinois Math)

Abstract: This is the first in a series of talks based on a recent paper of A.S. Marks and S.T. Unger. In 1925, Tarski asked if it is possible to decompose a disk in the plane into finitely many pieces and then rearrange them to form a square of the same area. Due to the apparent similarity between this problem and the Banach--Tarski paradox, it might appear that any such "circle squaring" must rely on the Axiom of Choice; and indeed, in 1990 Laczkovich answered Tarski's question in the affirmative using a non-constructive approach. However, Marks and Unger show in their paper that, somewhat surprisingly, it is possible to perform a "circle squaring" using only Borel pieces. In this talk I will go over the history of the problem and sketch some of the main ingredients that go into Marks and Unger's proof.

Friday, September 15, 2017

4:00 pm in 345 Altgeld Hall,Friday, September 15, 2017

#### "Borel circle squaring" by Marks and Unger: Part 2

###### Dakota Ihli (Illinois Math)

Abstract: This is the second part in a series of talks based on a recent paper of A.S. Marks and S.T. Unger. The main result of the paper gives sufficient conditions for two Borel subsets of $\mathbb{R}^k$ to be equidecomposable by translations with Borel pieces. In the first part, flows on graphs were introduced as the primary means to prove this result. In this part I will outline in more detail how these flows are constructed.

Friday, September 22, 2017

1:00 pm in 141 Altgeld Hall,Friday, September 22, 2017
###### Travis Nell

Abstract: We continue reading Simon's "A guide to NIP Theories", chapter 7.

4:00 pm in 345 Altgeld Hall,Friday, September 22, 2017

#### Constructing Analyzable Types in Differentially Closed Fields with Logarithmic Derivatives

###### Ruizhang Jin (U Waterloo Math)

Abstract: We generalize the well-known fact that the equation $\delta(\log \delta x) = 0$ is analyzable in but not internal to the constants. We use the logarithmic derivative as a building block to construct analyzable types with a unique analysis of minimal length (up to interalgebraicity). We also look for criteria for a given definable set such that its pre-image under the logarithmic derivative is analyzable in but not internal to the constants.

Friday, September 29, 2017

4:00 pm in 345 Altgeld Hall,Friday, September 29, 2017

#### "Borel circle squaring" by Marks and Unger: Part 3

###### Anush Tserunyan (Illinois Math)

Abstract: In the previous talk of this series, we built a real valued Borel flow by taking the average of the "relative" matchings over each connected component. In the current talk, I'll describe how to obtain an integer valued Borel flow out of a real valued one, which is the main difficulty of the whole proof.

Friday, October 6, 2017

4:00 pm in 345 Altgeld Hall,Friday, October 6, 2017

#### "Borel circle squaring" by Marks and Unger: Part 4 (last)

###### Anush Tserunyan (Illinois Math)

Abstract: In the previous talk of this series, we discussed how to obtain an integral Borel flow out of a real valued one. In this last talk of the series, we will put all the ingredients together and show how to obtain a Borel matching from an integral Borel $f$-flow for the specific $f := \mathbb{1}_A - \mathbb{1}_B$.

Friday, October 20, 2017

4:00 pm in 345 Altgeld Hall,Friday, October 20, 2017

#### Distal and non-distal ordered abelian groups

###### Allen Gehret (UCLA Math)

Abstract: I will discuss various things we know about distal and non-distal ordered abelian groups. This is joint work with Matthias Aschenbrenner and Artem Chernikov.


text:
# Settings¶

Settings contain your environment variables and are open to being defined and extended with additional fields being added to the settings struct.

Below is an example of configuring settings:

def mySettings():

settings= {}

settings['markets']  = ['CASH','F_AD', 'F_BO', 'F_BP', 'F_C', 'F_CC', 'F_CD',
'F_CL', 'F_CT', 'F_DX', 'F_EC', 'F_ED', 'F_ES', 'F_FC','F_FV', 'F_GC',
'F_HG', 'F_HO', 'F_JY', 'F_KC', 'F_LB', 'F_LC', 'F_LN', 'F_MD', 'F_MP',
'F_NG', 'F_NQ', 'F_NR', 'F_O', 'F_OJ', 'F_PA', 'F_PL', 'F_RB', 'F_RU',
'F_S','F_SB', 'F_SF', 'F_SI', 'F_SM', 'F_TU', 'F_TY', 'F_US','F_W', 'F_XX',
'F_YM']
settings['beginInSample'] = '20120506'
settings['endInSample'] = '20150506'
settings['lookback']= 504
settings['budget']= 10**6
settings['slippage']= 0.05

return settings

## Default Settings¶

Your algorithm will always be called with the following settings by default unless you define them otherwise.

settings['beginInSample'] = '19900101'
settings['lookback']      = 504
settings['budget']        = 1000000
settings['slippage']      = 0.05

Note that you aren’t required to define any settings field, but for most trading systems lookback will likely need to be changed from 1 if your trading system needs any historical market data. Leaving lookback at the default value while attempting to access market prices further back than the previous day will result in an error.

## Markets¶

Currently Quantiacs trades in both Stock and Futures markets. A Future is a contract to deliver a specified amount of the underlying product at a certain date. You can learn more about quant trading with Futures from the video below.

Trading in both Stocks and Futures allows for a great degree of diversification and helps you to generate a trading system that is successful in many market environments. You can find the full list of available markets on Quantiacs’ Markets page.

In general, you can find the contract unit of any futures by looking at the Futures Contracts Specifications. And finding the contract unit allows you to relate a futures contract to their commonly found prices such as on Yahoo Finance.

Another unique aspect of futures contracts that doesn’t apply to stocks, is an expiration date on each contract. The only real difference in your trading data is that the Quantiacs toolbox automatically implements a roll over (see Roll Overs (R & RINFO) section below) whenever a futures contract expires, to make the pricing data the latest and most accurate.

## Market Data¶

Note that Quantiacs market data is not updated in realtime, this means you won’t be able to do live trading using the toolbox. If a market is not defined for some time period, its values are set to not a number (NaN). If data is missing within the time series (for example due to a holiday), the missing data is replaced by the last known value with zero volume.

If you want to backtest across every available market, you can get a text file with a string for all of the markets here. This makes it easy to copy paste all of the available markets into your settings struct.

Here is an example of what the data looks like for a futures contract:

DATE OPEN HIGH LOW CLOSE VOL OI P R RINFO
19900102 77300 77400 77020 77020 125 2559 0 0 0
19900103 76890 77030 76700 76740 1495 3215 0 0 0
19900104 77080 77610 77000 77490 932 3122 0 0 0
19900105 77050 77280 76980 76980 272 2542 0 0 0
19900108 77280 77300 77090 77200 177 2439 0 0 0
19900109 77289 77370 77210 77290 106 2379 0 0 0

And here is what the data fields look like for a stock:

AAPL (Apple)
DATE OPEN HIGH LOW CLOSE VOL P
20010102 1.067 1.0893 1.0446 1.0625 1.12E+08 0
20010103 1.0357 1.192 1.0313 1.1696 2.02E+08 0
20010104 1.2946 1.3125 1.2054 1.2188 1.84E+08 0
20010105 1.2098 1.2411 1.1473 1.1696 1.02E+08 0
20010108 1.2098 1.2098 1.1384 1.183 92568000 0
20010109 1.2009 1.2589 1.183 1.2277 1.44E+08 0

The P column is for backwards compatibility to support the Quantiacs 1.X Toolbox versions. OI represents open interest for futures contracts, and R and RINFO both provide information about futures contracts roll overs (see Roll Overs (R & RINFO) section below).

Whenever you run runts, it will automatically download the necessary market data. When backtesting across new markets, or a new sample size, the toolbox will automatically download the corresponding market data if it hasn’t been downloaded before.

To manually initiate this process, you can use the command loadData. You can find a full breakdown of loadData under Reference section. The main arguments loadData needs are a list of markets you want downloaded, and the time period (sample size). An example use case would be:

quantiacsToolbox.loadData(marketList = allMarkets, dataToLoad = ['DATE','OPEN','CLOSE'], beginInSample = '20150101')

## Roll Overs (R & RINFO)¶

Futures, as opposed to Stocks, come in single contracts with an expiration (delivery) date. This requires that we treat futures contracts slightly different than stocks in the backtester. Since there is an expiration to the contract, we have to sell the contract before the expiry and buy a different contract (of the same underlying) that expires further in the future (this is called ‘rolling’ a contract). There are extra costs and uncertainties associated with this.

Rolling explains why the plot of the prices of the time series (as shown on the website) is not necessarily what you get when you buy and hold that commodity. The differences between the price plot and the trading result are higher for commodities and lower for financial futures, since the cost of carry for a Stock Index Future or a Government Bond is usually very low.

In the market data files (found in the tickerData folder of the toolbox), R and RINFO columns address roll overs. The data column R contains the roll announcement - the contract maturity of the new contract (i.e. the contract we’re rolling into) in the format yyyymm. RINFO is the roll difference in the time series data. At a roll we back-adjust the data in the lookback window by RINFO to keep the time series data steady. We also adjust the performance by the roll amount since the price difference between the two contracts at the same time is not a win or a loss that can be traded. So our raw data are not continuous contracts, but single contracts.

Here is an example of rollover data from F_AD.txt:

DATE OPEN CLOSE R RINFO
20150902 70120.0000 70250.0000 0 0.0000
20150903 70360.0000 70100.0000 0 0.0000
20150904 70080.0000 69230.0000 201512 0.0000
20150908 68820.0000 69930.0000 0 -290.0000
20150909 69840.0000 69840.0000 0 0.0000
20150910 69500.0000 70480.0000 0 0.0000

Roll overs are all done automatically in runts, and because of this on-the-fly rolling method you always get:

1. The true Dollar value of the commodity at that point in time - at least for the last data point, i.e. the last row of the CLOSE matrix.
2. A steady course with no disruptions/gaps because of rolls.

## Why Only Daily Data¶

Quantiacs only supports daily historical market data for several reasons. The first is that our investors want scalable strategies that can manage hundreds of millions rather than just hundreds of thousands. As limit orders can only be filled during those times of the session, in which the market trades below the limit, we’d only have a fraction of the session to execute these orders. Naturally this leads to a much lower capacity of the trading strategy. Additionally, if we’d allow limit orders we would have to account for partial fills in the backtest, which could make the backtest results no longer representative in extreme cases.

Secondly, we are a Commodity Trading Advisor registered with the NFA, and we have to comply with the rules of our regulators. We have to protect our institutional clients from front-running, arbitrage and other potentially criminal activities. It’s impossible to protect investors trading third party strategies on 1 minute bars. On end of day data we can ensure their protection from criminal activities.

We have to separate the strategic part of the trading system (its logic of when to buy what) strictly from the actual order execution and risk management, that’s handled by us (and might actually involve the use of leverage, limit orders, stop loss orders etc.).

## Sample Size¶

By default, the system will load market data for all dates available, so the backtest will run across the entire 25+ years of historical market data. Alternatively, you have the ability to define the specific start and end dates for your backtests through beginInSample and endInSample respectively. Both fields follow the format of YYYYMMDD.

## Budget¶

Although you can change your budget to any size, it is good to test it at $1 million because that would provide it with the proper scale to effectively trade futures in the real world. Moreover, good trading strategies will show similar results whether they’re traded at$1 million or $10 million. Our backtester, no matter the budget allocated, assumes the ability to purchase non-discrete or fractional amounts of contracts. In reality this is not possible, however, it allows the trading strategy to be evaluated without significant deviation caused by budgets. Since futures generally have a very large contract size, there would be a big difference between real and intended allocations at lower capital sizes. For example, if you attempted to manage your algorithm with 500k and had the following target allocation: Market Allocation Cash in market Price of 1 contract F_ES 0.5 0.5 * 500k = 250k 104k F_SI 0.2 0.2 * 500k = 250k 79k F_GC 0.1 0.1 * 500k = 50k 118k F_TY 0.1 0.1 * 500k = 50k 126k F_FV 0.1 0.1 * 500k = 50k 119k Again because of the large contract sizes of futures (and the fact that it is impossible to buy half contracts) a naïve discrete representation would give you 2 contracts F_ES, 1 contract F_SI, and ignore the rest. Thus the real exposure would be: Market Allocation Price of 1 contract F_ES 2 * 104 / 500 = 0.416 104k F_SI 1 * 79 / 500 = 0.158 79k F_GC 0 118k F_TY 0 126k F_FV 0 119k CASH 0.426 Realistically, any institution would put down at least$1 million to trade futures with. So our non-discrete trading positions turn out to be a better representation of real life trading situations.

In our backtesting toolbox we use a very simple yet conservative approach to estimate slippage and commissions: We take 5% of the daily range as the trading costs. This computes as (HIGH - LOW) * 0.05. This covers the assumption, that you’ll have more slippage on days with larger market moves, than on days with smaller. This approximation might overestimate the real trading costs. In this case, it is better to overestimate than underestimate.

### Slippage¶

settings['anotherField'] =  some_value

format: plain


text:
# Q. The circle passing through the point (-1,0) and touching the Y-axis at (0, 2) also passes through the point

IIT JEEIIT JEE 2011Conic Sections

Solution:

## Equation of circle passing through a point $(x_1, y_1)$ and touching the straight line L, is given by $( x - x_1 )^2 + ( y - y_1)^2 + \lambdaL = 0$ $\therefore$ Equation of circle passing through (0, 2) and touching x=0 $\Rightarrow \, \, \, \, \, \, \, ( x -0)^2+ (y -2 )^2+ \lambdax = 0\hspace15mm ...(i)$ Also, it passes through $(-1 ,0 ) \Rightarrow 1 + 4 - \lambda = 0$ $\therefore$ \lambda=5 Eq. (i) becomes, $\hspace22mm x^2 + y^2 - 4y + 4 + 5x = 0$ $\Rightarrow\hspace15mm x^2 + y^2 - 5x - 4y+4 =0$, For x-intercept put $y = 0 \Rightarrow x^2 + 5x + 4 = 0,$ $(x + 1) (x+4)=0$ $\therefore$ $\hspace20mm$x=-1, -4 Hence, (d) option (-4, 0).

You must select option to get answer and solution

Determinants

Determinants

AIEEE 2002 Sets

## 2. For $r = 0, 1, ... , 10,$ if $A_r,B_r$ and $C_r$ denote respectively the coefficient of $x^r$ in the expansions of $(1 + x)^{10}, (1 + x)^{20}$ and $(1 + x)^{30}$. Then, $\displaystyle \sum A_r(B_{10}B_r-C_{10}A_r)$ is equal to

IIT JEE 2010 Binomial Theorem

## 3. Let $\omega \ne 1$ be a cube root of unity and S be the set of all non-singular matrices of the form $\begin {bmatrix} 1 & a & b \\ \omega & 1 & c \\ \omega^2 & \omega & 1 \end {bmatrix}$ where each of a , b and c is either $\omega \ or \ omega^2$ Then, the number of distinct matrices in the set S is

IIT JEE 2011 Determinants

## 4. A ray of light along $x+\sqrt 3 \, y=\sqrt 3$ gets reflected upon reaching X-axis, the equation of the reflected ray is

JEE Main 2013 Straight Lines

## 5. A ray of light along $x+\sqrt 3 \, y=\sqrt 3$ gets reflected upon reaching X-axis, the equation of the reflected ray is

JEE Main 2013 Straight Lines

## 6. A ray of light along $x+\sqrt 3 \, y=\sqrt 3$ gets reflected upon reaching X-axis, the equation of the reflected ray is

JEE Main 2013 Straight Lines

## 7. Perpendicular are drawn from points on the line $\dfrac{x+2}{2}=\dfrac{y+1}{-1}= \dfrac{z}{3}$ to the plane $x+ y + z = 3$. The feet of perpendiculars lie on the line

JEE Advanced 2013 Introduction to Three Dimensional Geometry

## 8. Let AB be a chord of the circle $x^2 + y^2 = r^2$ subtending a right angle at the centre. Then, the locus of the centroid of the $\Delta PAB$ as P moves on the circle, is

IIT JEE 2001 Conic Sections

## 9. The centre of the circle passing through the point (0, 1)and touching the curve $y = x^2 at (2,4)$ is

IIT JEE 1983 Conic Sections


text:
3 Tutor System
Starting just at 265/hour

3.Work out the following divisions. (i) $$(10x – 25) ÷ 5$$ (ii) $$(10x – 25) ÷ (2x – 5)$$ (iii) $$10y (6y + 21) ÷ 5 (2y + 7)$$ (iv) $$9x^2y^2 (3z – 24) ÷ 27xy (z – 8)$$ (v) $$96abc (3a – 12) (56 – 30) ÷ 144 (a – 4) (b – 6)$$

(i)Given:$$(10x – 25) ÷ 5$$

$$=\frac{10x-25}5=\frac{5\times (2x-5)}5$$

$$=2x-5$$

(ii)Given:$$(10x – 25) ÷ (2x – 5)$$

$$=\frac{10x-25}{2x – 5}=\frac{5\times (2x-5)}{2x – 5}$$

$$=5$$

(iii)Given:$$10y (6y + 21) ÷ 5 (2y + 7)$$

$$=\frac{10y (6y + 21)}{ 5 (2y + 7)}=\frac{5\times 2\times y\times 3\times (2y + 7)}{ 5\times (2y + 7)}$$

$$=\frac{2\times y \times 3}1=6y$$

(iv)Given: $$9x^2y^2 (3z – 24) ÷ 27xy (z – 8)$$

$$=\frac{9x^2y^2 (3z – 24)}{ 27xy (z – 8)}=\frac{9\times x^2y^2 \times 3\times (z-8)}{ 9\times 3\times xy \times (z -8)}$$

$$=\frac{xy}1=xy$$

(v)Given: $$96abc (3a – 12) (56 – 30) ÷ 144 (a – 4) (b – 6)$$
$$=\frac{96abc (3a – 12) (56 – 30)}{ 144 (a – 4) (b – 6)}=\frac{48\times 2 \times abc \times 3\times (a-4)\times 5 \times (b-6)}{48\times 3\times (a-4) \times (b-6)}$$

$$=\frac{2 \times abc \times 5}1=10abc$$


text:
### determinism's blog

By determinism, 6 years ago,

I could't solve this problem, then I've read the editorial; but, I still can't understand how we convert dp states into matrices. Can someone, who knows the solution, elaborate?

PS: I know how to compute Nth power of matrix (size M*M) in O(M^3 * log N). I couldn't understand the values we are storing in the matrix and how kth matrix is used to get (k+1)th matrix.

UPD: I've coded the solution as sandeepmohanty told me, but I think that my implementation is awful. How can I improve it (especially initialising adjacency matrix) ?

• +9

 » 6 years ago, # |   +3 Consider a state (x1, x2,.. , xn).Here xi denotes the number of times crime i is committed % its multiplicity.So there are a maximum of 123 states.Now, create a graph where there is an edge from (x1,.. xi,.. xn) to (x1,.. (xi + 1) % mi,.. xn), for each i. (Basically from each state there are n edges, each edge means committing a particular crime).Now, you can see that committing k number of crimes means traversing k edges.So, the question now basically reduces to finding out the number of n length paths in this graph from node (0, 0,... 0) to (0, 0, .., 0). This is a pretty standard problem whose answer is simply [0][0] element in the adjacency matrix of this graph exponentiated to n-1.
•  » » 6 years ago, # ^ |   0 How can I hash (x1,x2,x3...,xn) into an integer?
•  » » » 6 years ago, # ^ |   +3 You can generate all the required vectors, sort them and then use a map. But this is a very bad way. :PI hope someone else gives you a better answer.
•  » » » 6 years ago, # ^ |   +8 An alternative form is to treat it like a number base, i.e. hash it asx_1 + mult[1] * (x_2 + mult[2] * (x_3 + ... mult[n-1] * x_n))))))) orx_1 + x_2 * mult[1] + x_3 * mult[1] * mult[2] + x_4 * mult[1] * mult[2] * mult[3] + ...I don't know if it's strictly better than what was suggested, but it's an alternative way: 9453430
 » 4 weeks ago, # |   0 Nikhil ( pa.n.ik ) might be of some help, being a criminal himself. (Rapists like you should just eat poison and die. I'm providing you with link for cyanide.) https://www.amazon.com/Industrial-Test-Systems-486812-Cyanide/dp/B00DIJ9UEM/


text:
# Damped Harmonic Motion

1. Feb 23, 2008

### Vuldoraq

1. The problem statement, all variables and given/known data

Hi all,

A hard boiled egg, with a mass m=51g, moves on the end of a spring, with force constant k=26N/m. It's initial displacement is 0.300m. A damping force F$$^{}x$$=-bv$$^{}x$$ acts on the egg and the amplitude of the motion decreases to 0.106m in a time of 5.45s.

Calculate the magnitude of the damping constant b.

2. Relevant equations

(1) $$\omega$$'=$$\sqrt{k/m-(b/2m)^2}$$ where $$\omega$$' is the damped angular frequency

(2) x(t)=Ae^(-bt/2m)cos($$\omega$$'t) where x(t)=displacement, A=amplitude, t=time, e=natural exponential, w'=as above.

3. The attempt at a solution

I cannot see how you can calculate the damping constant from the data given. Equation (1) above has two unknowns w' and b, while equation (2) has three x(t), b and w'.

I tried to rearrange equation (2) to get w' as the subject, but found I was unable to separate the variables. I also attempted setting w' as equal to $$\sqrt{k/m}$$, but this gave the wrong answer too. I'm really just stabbing in the dark, so some help to understand how to go about it is what I need.

I must be missing something obvious and fundamental, for instance is there another way you can calculate $$\omega$$' from the given data?


text:
# Why exactly do neural networks require i.i.d. data?

In reinforcement learning, successive states (actions and rewards) can be correlated. An experience replay buffer was used, in the DQN architecture, to avoid training the neural network (NN), which represents the $$Q$$ function, with correlated (or non-independent) data. In statistics, the i.i.d. (independently and identically distributed) assumption is often made. See e.g. this question. This is another related question. In the case of humans, if consecutive data points are correlated, we may learn slowly (because the differences between those consecutive data points are not sufficient to infer more about the associated distribution).

Mathematically, why exactly do (feed-forward) neural networks (or multi-layer perceptrons) require i.i.d. data (when being trained)? Is this only because we use back-propagation to train NNs? If yes, why would back-propagation require i.i.d. data? Or is actually the optimisation algorithm (like gradient-descent) which requires i.i.d. data? Back-propagation is just the algorithm used to compute the gradients (which is e.g. used by GD to update the weights), so I think that back-propagation isn't really the problem.

When using recurrent neural networks (RNNs), we apparently do not make this assumption, given that we expect consecutive data points to be highly correlated. So, why do feed-forward NNs required the i.i.d. assumption but not RNNs?

I'm looking for a rigorous answer (ideally, a proof) and not just the intuition behind it. If there is a paper that answers this question, you can simply link us to it.

There is an assumption behind the theory training a neural network, that also applies to many other supervised learning methods, that a training sample is representative of the data set as a whole - that it has been sampled fairly from the population that the learning algorithm has been set up to approximate.

The term i.i.d. stands for "independent and identically distributed". If you pick trajectories from RL then the sampling (per individual* record as used to train with) is not independent. Even if you choose the start of the trajectory randomly, you will have made one random choice then your remaining choices are made according to the trajectory - technically they are chosen by the policy and environment dynamics over a single step, which is usually not enough to make the second, third etc steps from a trajectory fully independent random samples from the training population. To make the sampled observations independent, you should make a new random choice, and to be identically distributed that choice has to be made fairly over the whole dataset.

If a subset of samples drawn for a mini-batch (or as consecutive individual samples) are correlated for an online algorithm like gradient descent, it causes a problem. For the algorithm to converge towards a globally optimal solution, it needs errors and the gradients they generate to be unbiased samples of the "true" function gradients across the loss function. Whilst correlated data from a trajectory does not do this at all - it exhibits a strong sampling bias, causing weight updates to move consistently in the wrong direction.

You can demonstrate this effect trying to learn any simple function like $$y = x^2$$ with $$x$$ from $$-1$$ to $$+1$$. Training a NN with randomly selected $$x,y$$ pairs results in far better results than training it serially with ~2000 records starting $$(-1.000, 1.000) (-0.999, 0.998) (-0.998, 0.996)$$ etc

So, why do feed-forward NNs required the i.i.d. assumption but not RNNs?

RNNs do need i.i.d. data, but the "unit" of sampling here is each sequence.

* By "individually" I mean in terms of how they are used in the training process. A "unit" here is the smallest set of records that leads to a complete measure of loss in your optimisation.

It is ok to draw longer trajectories at random if your smallest unit of measurable error is from a trajectory, as in Monte Carlo or TD($$\lambda$$) for RL, or for RNNs. There can be subtle problems with this - the trajectories still have to be i.i.d. as in they represent fair samples from the assumed population of trajectories.

If you are working at the level of longer trajectories, typically this reduces systemic bias: In RL it reduces bias from initial conditions due to bootstrap process. For RNNs I am less sure what this would mean, but suspect there is some equivalent. Typically, it also increases variance, meaning you may need more training samples. There is a bias/variance trade-off, which is why setting $$\lambda$$ somewhere between 0 and 1 in TD($$\lambda$$) is often the optimal choice. Note in RL this is a different source of bias and variance than discussed when considering number of parameters and regularisation in supervised learning.

• Comments are not for extended discussion; this conversation has been moved to chat. – nbro Nov 3 '20 at 14:23
• What do you mean by piecewise learning method? ReLU network? – nn0p Apr 6 at 1:11
• @nn0p: I mean that it learns from a finite set of samples of a target function that have been provided to it. It does appear that the term is used elsewhere for othe rmeanings, so I will change that sentence. – Neil Slater Apr 6 at 6:58

Suppose that we have some optimization criterion $$J(x)$$, which we aim to optimize (maybe maximize, maybe minimize), which we can compute for a single example $$x$$.

In an "ideal world", where we have no restrictions on computation time and memory, we would generally want to run training algorithms on the complete "ground truth" population. For example, if we're training a model (may be a DNN, but may also be some other kind of Machine Learning model), we'd ideally train it on the complete population of all "real-world" images that have ever been produced, ever will be produced, or ever could be produced (and, of course, all with accurate labels). We could write our full optimization criterion as:

$$\sum_{x \in \mathcal{P}} J(x),$$

where $$\mathcal{P}$$ denotes the complete population. If we have a model that we would like to train using gradient descent (such as a Neural Network), this means we have to compute the gradient with respect to some trainable parameters $$\theta$$:

$$\nabla_{\theta} \sum_{x \in \mathcal{P}} J(x).$$

In practice, we do not have this ideal scenario, we do not have access to the complete population $$\mathcal{P}$$. Often, we find ourselves approximating the population with a (hopefully rather large) dataset $$\mathcal{D}$$. This could be a collection of images like ImageNet, or in (Deep) Reinforcement Learning it could be a large experience replay buffer. If the dataset $$\mathcal{D}$$ is an accurate representation of the complete population's distribution, we can estimate the gradient of the objective we ultimately care about (computed over the complete population) by the gradient computed over the dataset $$\mathcal{D}$$:

$$\nabla_{\theta} \sum_{x \in \mathcal{P}} J(x) \approx \nabla_{\theta} \sum_{x \in \mathcal{D}} J(x).$$

Collecting such a dataset rather than the complete population is often actually feasible in practice. If we can afford to compute the objective/gradient over such a complete dataset, that's great. Note that in such a case, the data can already be viewed as being i.i.d.: it's the best approximation we have of a single, complete distribution (the population distribution).

However, computing the objective / gradient over a large complete dataset is often still prohibitively expensive in terms of computation time. This is one of the reasons (not the only one) why we often use minibatches $$B$$. Then, we approximate the gradient of the objective of the dataset (which is itself an approximation of the gradient for the complete population) by computing it only over a minibatch $$B$$:

$$\nabla_{\theta} \sum_{x \in \mathcal{P}} J(x) \approx \nabla_{\theta} \sum_{x \in \mathcal{D}} J(x) \approx \nabla_{\theta} \sum_{x \in \mathcal{B}} J(x).$$

If we're repeatedly (over many, many training iterations) going to use such an approximation to take gradient descent steps, it is crucial that the gradient computed over this minibatch is actually an accurate approximation of the "true" gradient over the complete population; if it's not an accurate approximation, we're optimizing the wrong objective! In my opinion, this means that we have an even stronger requirement for our minibatch than just wanting it to be identically distributed. It's not sufficient for our minibatch to be sampled from any arbitrary identical distribution. We want the instances in our minibatch to be sampled from one very specific identical distribution; the dataset/population distribution! If they're not all sampled from that particular distribution, they're an unreliable approximation of the objective we truly care about.

Now, you may wonder if it wouldn't be possible for multiple biased, "unrepresentative" minibatches with "opposite" biases to "cancel each other out" if they're used in different, subsequent gradient descent steps. Couldn't we first run a bunch of gradient descent steps on minibatches that only contain images of dogs, and afterwards "cancel out" any errors by also running a bunch of updates on minibatches containing only images of cats?

If you're lucky, it might work sometimes, but it's unreliable. One problem is that the first iterations using only dog images may cause you to end up in a poor area of the "parameter space", which you would never have ended up in if had used a correct mix of cat and dog images all along. Escaping that poor area may be much more difficult than simply ensuring you never reach it. Another problem is that it would be extremely difficult to find the "correct" number of gradient descent updates to run with cat images. Run too few, you're still stuck recognising only dogs. Run too many, and you may forget how to recognise dogs at all and only start recognising cats. Things like momentum in more sophisticated optimizers will exacerbate this issue.

Note that I don't think the requirement for i.i.d. batches is necessarily unique to gradient descent. Other learning techniques may have the same requirement (maybe for similar reasons, maybe for different reasons).

• Comments are not for extended discussion; this conversation has been moved to chat. – nbro Nov 3 '20 at 14:23

The reason is very simple. It is due to the cost function. For instance, for making the binary cross-entropy cost function, we discard the joint probability by considering the i.i.d condition. So we will have products of marginal probabilities. The final result will be the sum that we have in the binary cross-entropy cost function after applying the logarithm.

Update due to the request:

In a dataset, suppose we have n examples. Our goal is to maximize the probability of examples to happen. Consequently, we demand to maximize the following probability:

$$P(x_1, x_2, ..., x_n)$$

As it can be observed, the joint probability is hard to be interpreted. The point is that it is hard to be calculated. Suppose each example is binary. Then the probability distribution will need $$2^n$$ number of values to be defined properly.

If we sample the examples i.i.d the above distribution becomes the product of marginal probabilities:

$$P(x_1, ..., x_n)=\prod_{i}P(x_i)$$

The above probability is far easier to compute. We tend to maximize this probability. Due to the fact that logarithm is a strictly increasing function, we can take the logarithm of the above formula for numerical stability, because while multiplying many numbers, the outcome can become very small or big soon.

On the other hand, we try to minimize the negative of the maximization problem. It is more common among optimization practitioners.

$$log P(x_1, ..., x_n)=\sum_i{log(P(x_i))}$$

$$Cost=-\sum_i{log(P(x_i))}$$

This is our goal to maximize the likelihood.

The probability distribution is a Bernoulli distribution for binary classification problems and in neural networks, we demand to find $$P(y=1|x)$$ for each example:

$$P(y=1|x)=\hat{p}^y(1-\hat{p})^{(1-y)}$$

where $$\hat{p}$$ is the output of the network. Our goal is to maximize the probability of all the dataset. Like above, we minimize the cost function which will be like the following:

$$cost=-\sum_i{y^{(i)}log(\hat{y}^{(i)}) + (1-y^{(i)})log(1-\hat{y}^{(i)})}$$

• Ok, how does this answer my question that neural networks require i.i.d. data? At most, that answers the question "why is the independence assumption?" useful. I think you can make this answer clearer by just emphasizing this point, i.e. "By making the independence assumption, some of the computations can be simpler. So, in this answer, I am not focusing on how learning proceeds when the i.i.d. assumption is made, but how computations are simplified. Moreover, note that this is just about the independence assumption and not the other i (identically distributed) in i.i.d.". – nbro Nov 16 '20 at 12:31
• Something like this sentence should be your first sentence. – nbro Nov 16 '20 at 12:33
• @nbro unfortunately your voice is aggressive and hostile. By the way, this is my answer. And as a side note, the cost function will be useful to minimize if we sample our data in a way that we assumed to make the cost function. Your answer was exactly in the first paragraph: "we discard the joint probability by considering the i.i.d condition", and your question: "Why exactly do neural networks require i.i.d. data?" – Media Nov 16 '20 at 12:54
• What I was saying is that this answer can be improved significantly (from an explanation/pedagogical point of view), although what you say (about the simplifications of computations) is true. I will upvote it if you improve it as I suggested. For example, you say "As it can be observed, the joint probability is hard to be interpreted". The main problem is not "interpretability", but computation. In fact, that joint probability, without the IID assumption, is computed as a product of conditional probabilities (as opposed to just marginals). That's the main point. – nbro Nov 16 '20 at 17:47
• Moreover, as I said, the initial explanation, at the top of your answer, is very unclear (e.g. "we discard the joint probability by considering the i.i.d condition": that's not what we do: we do not discard anything, but we assume the IID, then the joint can be written as a product of marginals; maybe this is just a language problem), though your update clarifies it. Also, your answer applies to any model, in the sense that the IID assumption is not just useful for neural networks, but also e.g. for naive Bayes. – nbro Nov 16 '20 at 17:48


text:
Which of the following is the smallest fraction?
$\frac{8}{25}, \frac{7}{23}, \frac{11}{23}, \frac{14}{53}$
[A] $\frac{11}{23}$
[B] $\frac{14}{53}$
[C] $\frac{8}{25}$
[D] $\frac{7}{23}$


text:
# ATLAS Papers

Последно добавени:
2015-09-01
16:24
 Search for a high-mass Higgs boson decaying to a $W$ boson pair in $pp$ collisions at $\sqrt{s} = 8$ TeV with the ATLAS detector / , A search for a high-mass Higgs boson $H$ is performed in the $H\to WW\to\ell\nu\ell\nu$ and $H\to WW\to \ell\nu{qq}$ decay channels using $pp$ collision data corresponding to an integrated luminosity of 20.3 fb$^{-1}$ collected at $\sqrt{s}=$ 8 TeV by the ATLAS detector at the Large Hadron Collider. [...] CERN-PH-EP-2015-185 ; arXiv:1509.00389. - 2015. - 51 p. Previous draft version - Full text - Full text

2015-08-31
20:14
 Search for single top-quark production via flavour changing neutral currents at 8 TeV with the ATLAS detector / , A search for single top-quark production via flavour changing neutral current processes from gluon plus up- or charm--quark initial states in proton--proton collisions at the LHC is presented. [...] CERN-PH-EP-2015-172 ; arXiv:1509.00294. - 2015. - 29 p. Previous draft version - Full text - Full text

2015-08-31
16:35
 Search for invisible decays of a Higgs boson using vector-boson fusion in $pp$ collisions at $\sqrt{s}=8$ TeV with the ATLAS detector / , A search for a Higgs boson produced via vector-boson fusion and decaying into invisible particles is presented, using 20.3 fb$^{-1}$ of proton--proton collision data at a centre-of-mass energy of 8 TeV recorded by the ATLAS detector at the LHC. [...] CERN-PH-EP-2015-186 ; arXiv:1508.07869. - 2015. - 26 p. Previous draft version - Full text - Full text

2015-08-27
15:58
 Measurements of fiducial cross-sections for $t\bar{t}$ production with one or two additional $b$-jets in $pp$ collisions at $\sqrt{s}$=8 TeV using the ATLAS detector / , Fiducial cross-sections for $t\bar{t}$ production with one or two additional $b$-jets are reported, using an integrated luminosity of 20.3 fb$^{-1}$ of proton--proton collisions at a centre-of-mass energy of 8 TeV at the Large Hadron Collider, collected with the ATLAS detector. [...] CERN-PH-EP-2015-200 ; arXiv:1508.06868. - 2015. - 38 p. Previous draft version - Full text - Full text

2015-08-26
17:44
 Summary of the ATLAS experiment's sensitivity to supersymmetry after LHC Run 1 - interpreted in the phenomenological MSSM / , A summary of the constraints from the ATLAS experiment on $R$-parity conserving supersymmetry is presented. [...] CERN-PH-EP-2015-214 ; arXiv:1508.06608. - 2015. - 58 p. Previous draft version - Full text - Full text

2015-08-21
17:49
 Search for flavour-changing neutral current top-quark decays to $qZ$ in $pp$ collision data collected with the ATLAS detector at $\sqrt{s}=8$ TeV A search for the flavour-changing neutral-current decay $t\to qZ$ is presented. [...] CERN-PH-EP-2015-183. - 2015. Fulltext - Previous draft version

2015-08-13
06:57
 Search for lepton--flavour--violating $H\to\mu\tau$ decays of the Higgs boson with the ATLAS detector / , A direct search for lepton--flavour--violating (LFV) $H\to\mu\tau$ decays of the recently discovered Higgs boson with the ATLAS detector at the LHC is presented. [...] CERN-PH-EP-2015-184 ; arXiv:1508.03372. - 2015. - 15 p. Previous draft version - Full text - Full text

2015-08-11
01:48
 Constraints on non-Standard Model Higgs boson interactions in an effective field theory using differential cross sections measured in the $H \rightarrow \gamma\gamma$ decay channel at $\sqrt{s} = 8$ TeV with the ATLAS detector / , The strength and tensor structure of the Higgs boson's interactions are investigated within an effective field theory framework, which allows new CP-even and CP-odd interactions that can lead to changes in the kinematic properties of the Higgs boson and associated jet spectra. [...] CERN-PH-EP-2015-182 ; arXiv:1508.02507. - 2015. - 13 p. Previous draft version - Full text - Full text

2015-08-06
05:14
 Measurement of transverse energy--energy correlations in multi-jet events in $pp$ collisions at $\sqrt{s} = 7$ TeV using the ATLAS detector and determination of the strong coupling constant $\alpha_{\mathrm{s}}(m_Z)$ / , High transverse momentum jets produced in pp collisions at a centre of mass energy of 7 TeV are used to measure the transverse energy--energy correlation function and its associated azimuthal asymmetry. [...] arXiv:1508.01579 ; CERN-PH-EP-2015-177. - 2015. - 19 p. Previous draft version - Preprint - Full text

2015-08-04
19:28
 Measurement of the centrality dependence of the charged-particle pseudorapidity distribution in proton--lead collisions at $\sqrt{s_{_\text{NN}}} = 5.02$ TeV with the ATLAS detector / ATLAS Collaboration The centrality dependence of the mean charged-particle multiplicity as a function of pseudorapidity is measured in approximately 1 $\mu$b$^{-1}$ of proton--lead collisions at a nucleon--nucleon centre-of-mass energy of $\sqrt{s_{_\text{NN}}} = 5.02$ TeV using the ATLAS detector at the Large Hadron Collider. [...] arXiv:1508.00848 ; CERN-PH-EP-2015-160. - 2015. - 28 p. Previous draft version - Preprint - Full text


text:
# Display uncertainty on spatialy distributed proportions (visualisation)

This question is related to Distribution of estimator of multiple (spatially related) proportions. We consider here the /visualisation/ issue.

Consider a spatial random process $$Z(s)$$, where $$s$$ denotes the spatial location.

Our objective is to delineate the zone $$\mathcal{Z}$$ where the probability that $$Z$$ exceed a given threshold $$\zeta$$ is above a specified probability $$r$$:

$$\mathcal{Z} = \{s | \mathrm{Pr} (Z(s) > \zeta) > r\} .$$

Displaying the estimate of the zone is straighforward using two colours (“exceed” and “not exceed”), or a drawing a fontier.

• or confidence bounds (for each considered location $$s$$.
How would you display the exceedance zone such that a non mathematician can use the map to take decision regarding where to sparingly perform an expensive action wherever the threshold is exceeded. Put another way, assume $$r$$ is low (1-5%) and we want to ward of the error of non acting despite threshold exceedance.
Following my answer to the other question, you can use the local value of $$\hat{P}(Z>\xi)$$ and display it with a colormap, plus a contour of the zone where it exceeds $$r$$.


text:
Parallel Reinforcement Learning: A Framework and Case Study
 IEEE/CAA Journal of Automatica Sinica  2018, Vol. 5 Issue(4): 827-835 PDF
Parallel Reinforcement Learning: A Framework and Case Study
Teng Liu1, Bin Tian2,3, Yunfeng Ai4, Li Li5, Dongpu Cao6, Fei-Yue Wang1
1. State Key Laboratory of Management and Control for Complex System, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China;
2. State Key Lab. of Management and Control for Complex System, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China;
3. Cloud Computing Center, Chinese Academy of Sciences, Dongguan 523808, China;
4. School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100190, China;
5. Department of Automation, Tsinghua University, Beijing 100190, China;
6. Driver Cognition and Automated Driving Laboratory, University of Waterloo, Waterloo N2L 3G1, Canada
Abstract: In this paper, a new machine learning framework is developed for complex system control, called parallel reinforcement learning. To overcome data deficiency of current data-driven algorithms, a parallel system is built to improve complex learning system by self-guidance. Based on the Markov chain (MC) theory, we combine the transfer learning, predictive learning, deep learning and reinforcement learning to tackle the data and action processes and to express the knowledge. Parallel reinforcement learning framework is formulated and several case studies for real-world problems are finally introduced.
Key words: Deep learning     machine learning     parallel reinforcement learning     parallel system     predictive learning     transfer learning
Ⅰ. INTRODUCTION

Machine learning especially deep reinforcement learning (DRL) experiences an ultrafast development in recent years [1], [2]. No matter in traditional visual detection [3], dexterous manipulation in robotics [4], energy efficiency improvement [5], object localization [6], novel Atari game [7], [8], Leduc poker [9], Doom game [10] and text-based games [11], these data-driven learning approaches show great potential in improving performance and accuracy. However, there are still several issues to impede researchers applying DRL to handle the real complex system problems.

One of the issues is lack of generalization capability to new goals [3]. DRL agents need to collect new data and learn new model parameters for a new target. It is computationally expensive to retrain the learning model. Hence, we need to utilize the limited data well to accommodate the environments via learning.

Another issue is data inefficiency [8]. Acquiring large-scale action and interaction data of real complex systems is arduous. To explore control policy by themselves is very difficult for the learning systems. Thus, it is necessary to create a large number of observations for action and knowledge from the historical available data.

Finally, the issue is data dependency and distribution. In practical systems, data samples dependency is often uncertain and probability distribution is usually variant. So, it is hard for DRL agents to consider the state, action and knowledge of a learning system in an integrated way.

In order to address these difficulties, we develop a new parallel reinforcement learning framework for complex system control in this paper. We construct an artificial system analogy to the real system via modelling to constitute a parallel system. Based on the Markov chain (MC) theory, transfer learning, predictive learning, deep learning and reinforcement learning are exhibited to tackle data and action processes and to express knowledge. Furthermore, several application cases of parallel reinforcement learning are introduced to illustrate its usability. It is noticed that the proposed technique in this paper can be regarded as the specification of the parallel learning in [12].

Fei-Yue Wang first initialized the parallel system theory in 2004 [13], [14]. In [13] and [14], ACP method was proposed to deal the complex system problem. ACP approach represents artificial societies (A) for modelling, computational experiments (C) for analysis, and parallel execution (P) for control. An artificial system is usually built by modelling, to explore the data and knowledge as the real system does. Through executing independently and complementally in these two systems, the learning model can be more efficient and less data-hungry. ACP approach has been applied in several fields to discuss different problems in complex systems [15]-[17].

Transfer learning focuses on storing knowledge gained while solving one problem and applying it to a different but related problem. Taking driving cycles of vehicle as an example, we introduce mean traction force (MTF) components to achieve equivalent transformation of them. By transferring limited data via MTF, the generalization capability problem can be relieved.

Predictive learning tries to use prior knowledge to build a model of environment by trying out different actions in various circumstances. Taking power demand for example, we introduce fuzzy encoding predictor to forecast the future power demand in different time steps. Based on the MC, historical available data can be used to solve the data inefficiency.

Deep learning is defined via learning data representations, including multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer. Reinforcement learning is concerned with how agents ought to take actions in an environment so as to maximize some notion of cumulative reward. The main contribution of this paper is combining parallel system with transfer learning, predictive learning, deep learning and reinforcement learning to formulate the parallel reinforcement learning framework to dispose the data dependency and distribution problems in real-world complex systems.

The rest of this paper is organized as follows. Section Ⅱ introduces the parallel reinforcement learning framework and relevant components, then several case studies for real-world complex system problems are described in Section Ⅲ. Finally, we conclude the paper in Section Ⅳ.

Ⅱ. FRAMEWORK AND COMPONENTS A. The Framework and the Parallel System

The purpose of parallel reinforcement learning is building a closed loop of data and knowledge in the parallel system to determine the next operation in each system, as shown in Fig. 1. The data represents the inputs and parameters in artificial and real systems. The knowledge means the records from state space to action space, which we name in the real system as experience and that in the artificial system as policy. The experience can be used to rectify the artificial model and updated policy is utilized to guide the real actor along with feedback from environment.

Cyber physical systems have attracted increasingly more concerns for their potentials to fuse computational processes with the physical world in the past two decades. Furthermore, cyber-physical-social systems (CPSS) augment the cyber physical system capacity by integrating the human and social characteristics to achieve more effective design and operation [18]. The ACP-driven parallel system framework is depicted in Fig. 2. The integration of the real and artificial system as a whole is called parallel system.

In this framework, the physically-defined real system interacts with the software-defined artificial system via three coupled modules within the CPSS. The three modules are control and management, experiment and evaluation, learning and training. The first module belongs to decision maker in these two systems, the second one represents the evaluator and the final one indicates the learning controller.

ACP = Artificial societies + Computational experiments + Parallel execution. Artificial system is often constructed by descriptive learning based on the observation on the real system due to the development in information and communication technologies. It can help learning controller store more computing results and make more flexible decisions. Thus, the artificial system is parallel to the real system and runs asynchronously to stabilize the learning process and extend the learning capability.

In the computational experiment stage, the specifications of transfer learning, predictive learning and deep learning are illustrated by the MC theory, as we will discuss them later. For the parallel system, combining these learning processes with reinforcement learning, the parallel reinforcement learning is formulated to derive the experience and policy and to clarify the interaction of them. For a general parallel intelligent system, such knowledge can be applied in different tasks because the learning controller can handle several tasks via rational reasoning [19].

Finally, parallel execution between the artificial and real systems is expected to enable an optimal operation of these systems [20]. Although the artificial system is drawn by the prior data of real system, it will be rectified and improved by the further observation. The consecutive updated knowledge in the artificial system is also used to instruct the real system operation in an efficient way. Owing to the communication of data and knowledge by parallel execution, these two systems can improve by self-guidance.

B. Transfer Learning

In this paper, we choose driving cycles as an example to introduce transfer learning, which can be easily popularized for other data in the MC domain. A general driving cycle transformation methodology based on the mean tractive force (MTF) components is introduced in this section. This transformation can convert the existent driving cycles database into an equivalent one with a real MTF value to relieve the data lacking problem.

MTF is defined as the tractive energy divided by the distance traveled for a whole driving cycle, which is integrated over the entire time interval [0, $T$ ] as follows

 $$$\label{eq1} \bar {F}=\frac{1}{x_T }\int_0^T {F(t)} v(t)dt$$$ (1)

where $x_{T}$ is the total distance traveled in a certain driving cycle and calculated as $\smallint v(t)\textit{dt}$ , $v$ is the vehicle speed with respect to a certain driving cycle. $F$ is the longitudinal force to propel the vehicle and computed as

 $$$\label{eq2} \left\{ {\begin{array}{l} F=F_a +F_r +F_m \\ F_a =\frac{1}{2}\rho _a C_d Av^2, F_r =M_v g\cdot f, F_m =M_v a \\ \end{array}} \right.$$$ (2)

where $F_{a}$ is aerodynamic drag, $F_{r}$ is rolling resistance and $F_{m}$ is inertial force. $\rho_{a}$ is the air density, $C_{d}$ is the aerodynamic coefficient, and $A$ is the frontal area. $M_{v}$ is the curb weight, $g$ is the gravitational acceleration, $f$ is the rolling friction coefficient and $a$ is the acceleration.

The vehicle operating modes are divided as traction, coasting, braking and idling according to the force imposed on the vehicle powertrain [21]. Hence, the time interval is depicted as

 $$$\label{eq3}\small \left\{\! \!{\begin{array}{l} T=T_{tr} \cup T_{co} \cup T_{br} \cup T_{id} \\ T_{tr} =\left\{ {\left. t \right|F(t)>0, v(t)\ne 0} \right\}, T_{co} =\left\{ {\left. t \right|F(t)=0, v(t)\ne 0} \right\} \\ T_{br} =\left\{ {\left. t \right|F(t)<0, v(t)\ne 0} \right\}, T_{id} =\left\{ {\left. t \right|v(t)=0} \right\} \\ \end{array}} \right.$$$ (3)

where $T_{tr}$ and $T_{co}$ are the traction-mode and coasting-mode regions, respectively. $T_{br}$ represents the vehicle brakes and $T_{id }$ is the idling set.

From (3), it is obvious that the powertrain only provides positive power to wheels in the traction regions. MTF in (1) is specialized as follows:

 $$$\label{eq4} \bar {F}=\frac{1}{x_L }\int\limits_{t\in T_{tr} } {F(t)} v(t)dt=\bar {F}_a +\bar {F}_r +\bar {F}_m.$$$ (4)

Then, MTF components ( $\alpha$ , $\beta$ , $\gamma$ ) are the statistic characteristics measures for a driving cycle that are defined as [22]

 $$$\label{eq5} \left\{ {\begin{array}{l} \alpha =\dfrac{\bar {F}_a }{\dfrac{1}{2}\rho _a C_d A}=\dfrac{1}{x_L }\int\limits_{t\in T_{tr} } {v^3(t)} dt \\[2mm] \beta =\dfrac{\bar {F}_f }{mg\cdot f}=\dfrac{1}{x_L } \int\limits_{t\in T_{tr} } {v(t)} dt \\[2mm] \gamma =\dfrac{\bar {F}_m }{m}=\dfrac{1}{x_L } \int\limits_{t\in T_{tr} } {a\cdot v(t)} dt. \\ \end{array}} \right.$$$ (5)

Note that MTF components are related to the speed and acceleration for a specific driving cycle. These measures are employed as the constraints for driving cycle transformation.

Definition decides MTF is unique for a specific driving cycle, thus inequality and equality constraints are employed to determine the transferred driving cycle. A cost function can be defined by the designer to choose an optimal equivalent one from a set of feasible solutions. This transformation is formulated as a non-linear program (NLP) as

 $$$\label{eq6} \begin{array}{l} ~~~~~~~~~~~~~~~~~\mathop {\min }\limits_{\tilde {v}} f(\tilde {v}) \\ {\rm s.t.}~~~~~g_i (\tilde {v}, T_{tr} , {\alpha }', {\beta }'or{\gamma }'))=0, i=1, 2, 3 \\ ~~~~~~~~~~~~~h_1 (\tilde {v}, T_{tr} , v_{\rm coast} )<0 \\ ~~~~~~~~~~~~~h_2 (\tilde {v}, T_{co} \cup T_{br} , v_{\rm coast} )\ge 0 \\ \end{array}$$$ (6)

where $\tilde {v}$ is the transferred driving cycle, ( $\alpha'$ , $\beta'$ , $\gamma'$ ) are the target MTF components, $v_{\rm coast}$ is the vehicle coasting speed. $g_{i}$ and $h_{j}$ are the constraints. Through this process, the transferred driving cycle related to the real conditions can be decided, and afterwards can be used for other operations, such as control and management [21], [22], see Fig. 3 for an illustration.

The purpose of transfer learning is converting historical available data into equivalent one to expand the database. The transferred data is strongly associated with the real environments. Thus, it can be used for generating adaptive control and operations in complex systems, so as to solve the generalization capability and data hungry problems.

C. Predictive Learning

Taking power demand of vehicle for example, we introduce predictive learning to forecast the future power demand based on the observed data and processes in parallel system. A better understanding of the real system can then be described and applied to update the artificial system from these new experiences. A power demand prediction technology based on fuzzy encoding predictor (FEP) is illustrated in this section. This approach can also be used to draw more future knowledge from experiences for other parameters in the complex systems.

Power demand is modelled as a finite-state MC [23] and depicted as $P_{dem}={\{}p_{j }\vert j=1, {\ldots}, M{\}}\subset X$ , where $X\subset R$ is bounded. Transition probability of power demand is calculated by maximum likelihood estimator as

 $$$\label{eq7} \left\{ {\begin{array}{l} \pi _{ij} =P(\left. {p^+=p_j } \right|p=p_i )=\dfrac{N_{ij} }{N_i }{\kern 1pt} \\ N_i =\sum\limits_{j=1}^M {N_{ij} } \\ \end{array}} \right.$$$ (7)

where $\pi _{ij}$ is the transition probability from $p_{i}$ to $p_{j}$ . $p$ and $p^{+}$ are the present and next one-step ahead power demands, respectively. Furthermore, $N_{ij }$ indicates the transition count number from $p_{i}$ to $p_{j}$ , and $N_{i }$ is the total transition count number initiated from $p_{i}$ .

All elements $\pi_{ij}$ constitute the transition probability matrix $\Pi$ . For fuzzy encoding technique, $X$ is divided into a finite set of fuzzy subsets $\Phi_{j}, j=1, {\ldots}, M$ , where $\Phi_{j}$ is a pair ( $X$ , $\mu_{j}(\cdot))$ and $\mu_{j}(\cdot )$ is called Lebesgue measurable membership function and defined as

 $$$\label{eq8} \mu _j :X\to [0, {\kern 1pt}{\kern 1pt}1]{\kern 1pt}{\kern 1pt}{\kern 1pt}{\rm s.t.}{\kern 1pt}{\kern 1pt}\forall p\in X, {\kern 1pt}{\kern 1pt}\exists j, 1\le j\le M, \mu _j (p)>0$$$ (8)

where $\mu_{j}(p)$ reflects the membership degree of $p\in X$ in $\mu_{j}$ . It is noticed that a continuous state $p\in X$ in the fuzzy encoding may be associated with several states $p_{j}$ of the underlying finite-state MC model [24].

Two transformations are involved in FEP. The first transformation allocates an $M$ -dimensional possibility (not probability) vector for each $p\in X$ as

 $$$\label{eq9} \tilde {O}^T(p)=\mu ^T(p)\, = [\mu _1 , \mu _2 , \ldots , \mu _M (p)].$$$ (9)

This transformation is named fuzzification and maps power demand in the space $X$ to the vector in $M$ -dimensional possibility vector space $\tilde {X}$ . Note that it is not necessary for the sum of the elements in possibility vector $\tilde {O}(p)$ to equal 1.

The second transformation is called proportional possibility-to-probability transformation, in which the possibility vector $\tilde {O}(p)$ is converted into a probability vector $O(p)$ by normalization [23], [24]:

 $$$\label{eq10} O(p)=\frac{\tilde {O}(p)}{\sum\limits_{j=1}^M {\tilde {O}_j (p)}}$$$ (10)

where this transformation maps $\tilde {X}$ to an $M$ -dimensional probability vector space, $\bar {X}$ . The element $\pi_{ij }$ in the transition probability matrix (TPM) $\Pi$ is interpreted as a transition probability between $\Phi_{i}$ and $\Phi _{j}$ . To decode vectors in $\bar {X}$ back to $X$ , the probability distribution $O^{+}(p)$ is utilized to aggregate the membership function $\mu (p)$ to encode the probability vector of the next state in $X$ :

 $$$\label{eq11} w^+(p)=(O^+(p))^T\mu (p)=(O(p))^T\Pi \mu (p).$$$ (11)

The expected value over the possibility vector leads to the next one-step ahead power demand in FEP:

 $$$\label{eq12} \left\{ {\begin{array}{l} p^+=\int\limits_X {w^+(y)yd} y/\int\limits_X {w^+(y)d} y \\ \int\limits_X {w^+(y)yd} y = \sum\limits_{i=1}^M {O_i (p)\sum\limits_{j=1}^M {\pi _{ij} } } \int\limits_X {y\mu _j (y)d} y \\ \int\limits_X {w^+(y)d} y=\sum\limits_{i=1}^M {O_i (p)\sum\limits_{j=1}^M {\pi _{ij} } } \int\limits_X {\mu _j (y)d} y. \\ \end{array}} \right.$$$ (12)

The centroid and volume of the membership function $\mu_{j}(p)$ is expressed as

 $$$\label{eq13} \left\{ {\begin{array}{l} \bar {c}_i =\int\limits_X {y\mu _j (y)d} y \\ V_j =\int\limits_X {\mu _j (y)d} y. \\ \end{array}} \right.$$$ (13)

Thus, (12) is reformulated as

 $$$\label{eq14} \left\{ {p^+=\frac{\sum\limits_{i=1}^M {O_i (p)\sum\limits_{j=1}^M {\pi _{ij} } } V_j \bar {c}_j }{\sum\limits_{i=1}^M {O_i (p)\sum\limits_{j=1}^M {\pi _{ij} } } V_j }} \right..$$$ (14)

where expression (14) is the predicted one-step ahead power demand using FEP. Fig. 4 shows an example of predictive learning used for power demand prediction. By doing this, the future power demand of vehicle in different time steps can be determined, and then these data will be used for improving the management and operations in the parallel system by self-guidance.

 Download: larger image Fig. 4 Predictive learning for future power demand prediction.

The goal of predictive learning is generating reasonable data from the prior existed data and real-time observations in the real world. We aim to minimize the differences between real samples and generated samples by tuning the parameters in the predictive learning methodology. Therefore, these generated data are responsible for deriving various experiences and guiding the complex system by learning process, so as to settle the data inefficiency and distribution problem.

D. Reinforcement Learning

In the reinforcement learning framework, a learning agent interacts with a stochastic environment. We model the interaction as quintuple (S, A, $\Pi$ , R, $\gamma$ ), where $s\in {\textit{S}}$ and $a\in {\textit{A}}$ are state variables and control actions sets, $\Pi$ is the transition probability matrix, $r\in {\textit{R}}$ is the reward function, and $\gamma$ $\in$ (0, 1) denotes a discount factor.

The action value function $Q(s$ , $a)$ is defined as the expected reward starting from $s$ and taking the action $a$ :

 $$$\label{eq15} Q(s, a)={\rm E}\left\{ {\sum\limits_{l=0}^\infty {\gamma ^lr_{t+1+l} \left| {s_t =s, a_t =a} \right.} } \right\}.$$$ (15)

The action value function associated with an optimal policy can be found by the Q-learning algorithm as in [25]

 $$$\label{eq16} Q(s_t , a_t )\leftarrow Q(s_t , a_t )+\eta (r+\gamma \mathop {\max }\limits_a Q(s_{t+1} , a)-Q(s_t , a_t )).$$$ (16)

When the state and action space is large, for example the action $a_{t}$ consists of several sub-actions, modelling Q-values $Q(s$ , $a)$ becomes difficult. In this situation, we use both state and action representations as input to a deep neural network to approximate the action value function.

A deep neural network is composed of an input layer, one or more hidden layers and an output layer. As shown in Fig. 5(a), the input vector $g=[g_{1}, g_{2}, {\ldots}, g_{R}]$ is weighted by elements $w_{1},w_{2}, {\ldots},w_{R}$ , and then summed with a bias $b$ to form the net input $n$ as

 Download: larger image Fig. 5 Deep neural network and bidirectional long short-term memory.
 $$$\label{eq17} n=\sum\limits_{i=1}^R {w_i g_i } +b.$$$ (17)

Then, the net input $n$ is affected by an activation function $h$ to generate the neuron output $d$ .

 $$$\label{eq18} d=h(n)$$$ (18)

where activation function usually includes activation function in the hidden layer $h_{1}$ and activation function in the output layer $h_{2}$ .

In this paper, we propose a bidirectional long short-term memory [26] based deep reinforcement network (BiLSTM- DRN) to approximate the action value function in reinforcement learning, see Fig. 5 (b) for an illustration. This structure consists of a pair of deep neural networks, one for state variable $s_{t}$ embedding and the other for control sub-actions $c^{i}_{t}$ embeddings. As the bidirectional LSTM has a larger capacity due to its nonlinear structure, we expect it will capture more details on how the embeddings in each sub-action are combined into an action embedding. Finally, a pairwise interaction function (e.g., inner product) is used to compute new $Q(s_{t}$ , $a_{t})$ via combining the state and sub-actions neuron output as

 $$$\label{eq19} Q(s_t , a_t )=\sum\limits_{i=1}^K Q (s_t , c_t^i )$$$ (19)

where $K$ is the number of the sub-actions, and $Q(s_{t}$ , $c^{i}_{t})$ represents the expected accumulated future rewards by including this sub-action.

Combining the ideas of parallel system, transfer learning, predictive learning and reinforcement learning, we can formulate a closed loop of data and knowledge, named parallel reinforcement learning, as described in Fig. 1. Several case studies for real-world complex system problems are introduced and discussed in the next section.

Ⅲ. CASE STUDIES OF PARALLEL REINFORCEMENT LEARNING A. Existing Case Studies in Parallel Reinforcement Learning Framework

Parallel reinforcement learning serves as a reasonable and suitable framework to analyse the real world complex system. It consists of a self-boosting process in the parallel system, a self-adaptive process by transfer learning, a self-guided process by predictive learning and big data screening and generating process by BiLSTM-DRN. Learning process becomes more efficient and continuous in the parallel reinforcement learning framework.

Several complex systems have been researched and analysed in the perspective of parallel reinforcement learning, such as transportation systems [27], [28], and vision systems [29]. A traffic flow prediction system was designed in [27], which considered the spatial and temporal correlations inherently. First, an artificial system named stacked autoencoder model was built to learn generic traffic flow features. Second, the synthetic data were trained by a layer-wise greedy method in the deep learning architecture. Finally, predictive learning was used to achieve traffic flow prediction and self-guidance for the parallel system. A survey on the development of the data-driven intelligent transportation system (D-DITS) was introduced in [28]. The functionality of D-DITS's key components and some deployment issues associated with its future research were addressed in [28].

Also, a parallel reinforcement learning framework has also been applied to address the problems in visual perception and understanding [29]. To draw an artificial vision system based on the observations from real scenes, the synthetic data can be used for feature analysis, object analysis and scene analysis. This novel research methodology, named parallel vision, was proposed for perception and understanding of complex scenes.

Furthermore, autonomous learning system for vehicle energy efficiency improvement in [30] can also be put into parallel reinforcement learning framework. First, a plug-in hybrid electric vehicle was imitated to construct the parallel system. Then, historical driving record for the real vehicle was collected to learn autonomously the optimal fuel use via a deep neural network and reinforcement learning. Finally, this trained policy can guide the real vehicle operations and improve control performance. A better understanding of the real vehicle can then be obtained and used to adjust the artificial system from these new experiences.

B. New Applications Using Parallel Reinforcement Learning Methods

Recently, we designed a driving cycles transformation based adaptive energy management system for a hybrid electric vehicle (HEV). There exist two major difficulties in the energy management problem of HEV. First, most of energy management strategies or predefined rules cannot adapt to changing driving conditions. Second, model-based approaches used in energy management require accurate vehicle models, which bring a considerable model parameter calibration cost. Hence, we apply parallel reinforcement learning framework into the energy management problem of HEV, as depicted in Fig. 6. More precisely, the core idea of this methodology is bi-level.

 Download: larger image Fig. 6 Parallel reinforcement learning for energy management of HEV.

The up-level characterizes how to transform driving cycles using transfer learning by considering the induced matrix norm (IMN). Specially, TPM of power demand are computed and IMN is employed as a critical criterion to identify the differences of TPMs and to determine the alteration of control strategy. The lower-level determines how to set the corresponding control strategies with the transferred driving cycle by using model-free reinforcement learning algorithm. In other words, we simulate the HEV as an artificial system to sample the possible energy management solutions, use transfer learning to make the computed strategies adaptive to real world driving conditions, and use reinforcement learning to generate the corresponding controls. Tests demonstrate that the proposed strategy exceeds the conventional reinforcement learning approach in both calculation speed and control performance.

Furthermore, we construct an energy efficiency improvement system in parallel reinforcement learning framework for a hybrid tracked vehicle (HTV). Specifically, we combine the simulated artificial vehicle with real vehicle to constitute the parallel system, use predictive learning to realize power demand prediction for further self-guidance and use reinforcement learning for control policy calculation. This approach also includes two layers, see Fig. 7 for a visualization of such idea. The first layer discusses how to accurately forecast the future power demand using FEP based on the MC theory. Kullback-Leibler (KL) divergence rate is employed to decide the differences of TPMs and updating of control strategy. The second layer computes the relevant control policy based on the predicted power demand and reinforcement learning technique. Finally, comparison shows that the proposed control policy is superior to the primary reinforcement learning approach in energy efficiency improvement and computational speed.

 Download: larger image Fig. 7 Parallel reinforcement learning for energy efficiency of HTV.

In the future, we plan to apply BiLSTM- DRN to process and train the large real vehicle data for optimal energy management strategy computation. The objective is to realize real-time control using the parallel reinforcement learning method in our self-made tracked vehicle. More importantly, we will apply parallel reinforcement learning framework into multi-missions of automated vehicles [30], such as decision making, trajectory planning and so on. To address the existing disadvantages of traditional data-driven methods, we expect that parallel reinforcement learning can promote the development of machine learning.

Ⅳ. CONCLUSION

The general framework and case studies of parallel reinforcement learning for complex systems are introduced in this paper. The purpose is to build a closed loop of data and knowledge in the parallel system to guide the real system operation or improve the artificial system precision. Particularly, ACP approach is used to construct the parallel system that contains an artificial system and a real system. Transfer learning is utilized to achieve driving cycle transformation by mean of tractive force components. Predictive learning is applied to forecast the future power demand via fuzzy encoding predictor. To train data in the large action and state space, we introduce BiLSTM-DRN to approximate the action value function in reinforcement learning.

Data-driven models are usually viewed as a component irrelevant to the data in learning process, which results in the large-scale exploration and observation-insufficiency problems. Furthermore, data in these models tend to be inadequate, and the general principle to organize these models remains absent. By combining parallel system, transfer learning, predictive learning, deep learning and reinforcement learning, we believe that parallel reinforcement learning can effectively address these problems and promote the development of machine learning.

REFERENCES


text:
## Precalculus (6th Edition)

$\color{blue}{7q^3-9q^2-8q+9}$
Add by combining like terms to obtain: $=(3q^3+4q^3) -9q^2-8q+(6+3) \\=(3+4)q^3-9q^2-8q+9 \\=\color{blue}{7q^3-9q^2-8q+9}$


text:
# Zariski's main theorem in the form of Grothendieck, universal properties

Zariski's main theorem in the form of Grothendieck [EGA 4-3, Thm. 8.12.6] asserts that any quasi-finite morphism $a : V^{\circ} → W$ factors as the composition of an open immersion and a finite morphism, say $$V^{\circ} \xrightarrow{\alpha\text{, open immersion}} V \xrightarrow{\beta\text{, finite}} W.$$

I believe that if $V^{\circ}$ is normal, this factorisation satisfies a number of universal properties. Below, I list three properties which I believe hold true at least for varieties defined over $\mathbb C$. All this must be known to experts, but I was not able not find anything in the literature. I would like to ask if anyone could point me to a reference for these matters.

(1) The intermediate variety $V$ can be chosen to be normal. With this choice, the variety $V$ and the factorisation is unique up to unique isomorphism.

(2) If $G$ is any group which acts algebraically on $V°$ and $W$ and if $a$ is equivariant with respect to these actions, then $G$ also acts on $V$ and the morphisms $α$ and $β$ are equivariant with respect to these actions.

(3) If $W° := \mathrm{Image}(a)$ is open in $W$ and $a : V° \to W°$ is Galois with group $G$, then $β: V → W$ is Galois with group $G$.

-
I don't think that (3) holds. This would imply that every ${\bf Z}/n$-torsor on $W^\circ$ can be extended to a ${\bf Z}/n$-torsor over $W$. There is an obstruction for this to be possible, which lies in a $H^2$ group. – Damian Rössler Apr 29 '13 at 14:00
I guess that "Galois with group $G$" should not be interpreted as being necessarily étale. Anyway, it seems to me that (2) and (3) follows immediately from (1), personally I would not worry with a reference. Of course, I am not very good with references. – Angelo Apr 29 '13 at 14:22
Laumon and Moret-Bailly, Chapitre 16, particularly Corollaire 16.6.2. Although I agree with Angelo in spirit, it is certainly helpful to readers of an article to have a secondary source that they can refer to as necessary. – Jason Starr Apr 29 '13 at 17:04
Thanks for the comments so far. Angelo is right: I did not intent "Galois" to imply "étale", but did not say so clearly. I think of "Galois" as "quotient map for quotient by a finite group". – Stefan Kebekus Apr 29 '13 at 17:59


text:
I have had a little bit of time off for holiday shut down. I’m starting to teach myself CSS animations. This is a pretty cool feature, and I can’t wait to see more of this feature.

The @keyframes are defined as follows. Of course, that might change post-publication.

$time-start: 0%;$time-square-draw-start: 5%;
$time-square-draw-end: 15%;$time-circle-draw-start: 15%;
$time-circle-draw-end: 25%;$time-circle-shift-start: 25%;
$time-circle-shift-end: 35%;$time-rotate-start: 35%;
$time-rotate-end: 45%;$time-heart-draw-start: 45%;
$time-heart-draw-end: 55%;$time-trace-erase-start: 55%;
$time-trace-erase-end: 65%;$time-heart-fill-start: 65%;
$time-heart-fill-end: 75%;$time-fade-out-start: 95%;
\$time-end: 100%;

See the Pen SVG Square Heart Animation by Jarrett M (@jarrettmeyer) on CodePen.


text:
## Cryptology ePrint Archive: Report 2019/1255

Zero-Knowledge Proofs for Set Membership: Efficient, Succinct, Modular

Daniel Benarroch and Matteo Campanelli and Dario Fiore and Kobi Gurkan and Dimitris Kolonelos

Abstract: We consider the problem of proving in zero-knowledge that an element of a public set satisfies a given property without disclosing the element, i.e., for some $u$, $u \in S$ and $P(u)$ holds''. This problem arises in many applications (anonymous cryptocurrencies, credentials or whitelists) where, for privacy or anonymity reasons, it is crucial to hide certain data while ensuring properties of such data. We design new modular and efficient constructions for this problem through new commit-and-prove zero-knowledge systems for set membership, i.e. schemes proving $u \in S$ for a value $u$ that is in a public commitment $c_u$. We also extend our results to support non-membership proofs, i.e. proving $u \notin S$. Being commit-and-prove, our solutions can act as plug-and-play modules in statements of the form $u \in S$ and $P(u)$ holds'' by combining our set (non-)membership systems with any other commit-and-prove scheme for $P(u)$. Also, they work with Pedersen commitments over prime order groups which makes them compatible with popular systems such as Bulletproofs or Groth16. We implemented our schemes as a software library, and tested experimentally their performance. Compared to previous work that achieves similar properties---the clever techniques combining zkSNARKs and Merkle Trees in Zcash---our solutions offer more flexibility, shorter public parameters and $4.7 \times$--$42\times$ faster proving time for a set of size $2^{64}$.

Category / Keywords: cryptographic protocols / public-key cryptography, zero knowledge, applications

Date: received 27 Oct 2019, last revised 6 May 2020

Contact author: matteo campanelli at gmail com

Available format(s): PDF | BibTeX Citation

Short URL: ia.cr/2019/1255

[ Cryptology ePrint archive ]


text:
# A sample of blood or tears has a pH of 7.4. What is the pOH of this solution?

Mar 27, 2018

$p O H = 6.6$

#### Explanation:

$p O H + p H = 14$

If $p H = 7.4$ then $p O H = 14 - p H$

$p O H = 14 - 7.4$

$p O H = 6.6$


text:
• MANJEET SINGH

Articles written in Pramana – Journal of Physics

• Inelastic surface vibrations versus energy-dependent nucleus–nucleus potential in sub-barrier fusion dynamics of $^{6}_{3}$ Li + $^{144} -{62}$Sm system

Limitations of the static Woods–Saxon potential and the applicability of the energy dependent Woods–Saxon potential (EDWSP) model within the framework of one-dimensional Wong formula to explore the sub-barrier fusion data are highlighted. The inelastic surface excitations of the fusing nuclei are found to be dominating in the enhancement of sub-barrier fusion excitation function data and the effects of such dominant vibrational states are exploited through the coupled channel calculations obtained by using the code CCFULL. It is worth mentioning here that the influence of multiphonon vibrational states of the reactants can be simulated by introducing the energy dependence in the nucleus–nucleus potential.

• Analytical evaluation of cesium emission lines using laser-induced breakdown spectroscopy

A laser-induced breakdown spectroscopy-based method has been successfully developed to quantify cesium (Cs) in solution using spectroscopically pure graphite planchets as a sample support. As Cs is a line-poor system, only five usable Cs atomic emission lines could be found and characterised by employing high-resolution system. The calibration curves of these emission lines were constructed under optimised experimental conditions. The analytical properties of these calibration curves were evaluated based on the usable dynamic range, $R^{2}$ of fitting, root mean square error cross-validation and limit of detection (LOD). The dynamic ranges of these five lines were found to be in correlation with the energy level involved in the transition. An LOD of 4 ppm was obtained using Cs(I) 852.11-nm line, which corresponds to 0.16 $\mu$g of Cs on the planchet. Based on the cross-validation approach, the best accuracy and precision ($\sim$6%) were obtained for 852.11 nm in >3000 ppm solutions, and the same is $\sim$8% for 672.33 nm and 697.33 nm in high concentrated solution of Cs.

• Enhanced Raman gain coefficients (under steady-state and transient regimes) of semiconductor magnetoplasmas

Assuming the origination of stimulated Raman scattering (SRS) in Raman susceptibility, we obtain expressions for Raman gain coefficients (under steady-state and transient regimes) of semiconductor magnetoplasmas under various geometrical configurations. The threshold value of excitation intensity and most favourable value of pulse duration (above which transient Raman gain vanishes) are estimated. For numerical calculations, we consider n-InSb crystal at 77K temperature as a Raman-active medium exposed to a frequencydoubled pulsed CO$_2$ laser. The variation of Raman gain coefficients on doping concentration, magnetostatic field and its inclination, scattering angle and pump pulse duration have been explored in detail with an aim to determinesuitable values of these controllable parameters to enhance Raman gain coefficients at lower threshold intensities and to establish the suitability of semiconductor magnetoplasmas as hosts for compression of scattered pulses and fabrication of efficient Raman amplifiers and oscillators based on Raman nonlinearities.

• Hot carrier effects on Brillouin susceptibilities of semiconductor magnetoplasmas

An analytical investigation is made of hot carrier effects on real and imaginary parts of Brillouin susceptibility (Re, Im (χB)) of semiconductor magnetoplasmas. Coupled mode approach is used to obtain expressions for Re, Im (χB) Numerical calculations are made for the n-InSb crystal–CO$_2$ laser system. Efforts are made to obtain enhanced values of Re, Im(χB) and change of their sign by an appropriate selection of external magnetic field (B$_0$) and doping concentration (n$_0$). The hot carrier effects of intense laser radiation modify the momentum transfer collision frequency of carriers and consequently, the nonlinearity of the medium, which in turn (i) further enhances Re, Im(χB), (ii) shifts the enhanced Re, Im (χB) towards smaller values of B$_0$ and (iii) widens the range of B$_0$at which change of sign of Re, Im(χB)occurs. The change of sign of enhanced Re, Im(χB)of semiconductor magnetoplasmas validates the possibility of the chosen Brillouin medium as a potential candidate material for the fabrication of stimulated Brillouin scattering-dependent widely tunable and efficient optoelectronic devices such as optical switches and frequency converters.

• # Pramana – Journal of Physics

Volume 97, 2023
All articles
Continuous Article Publishing mode

• # Editorial Note on Continuous Article Publication

Posted on July 25, 2019


text:
# Math

Below is a partial list of the math research I’ve done.

### Length-$$k$$ Steenrod Operations

In Summer 2013 I worked on determining the structure of the subalgebras $$L(k)$$ of the Steenrod Algebra as modules over the subalgebras $$\mathcal{A}(r)$$, mentored by Mark Behrens. The hope is to show that in a certain sense $$L(k)$$, when considered as a module over $$\mathcal{A}(r)$$, is built up out of copies of the “quotient” algebras $$\mathcal{A}(r)//\mathcal{A}(r-k)$$. I’m still working on writing up the results, but until then, you can see the poster I presented at the 2014 Joint Mathematics Meetings.

### Transposition Trees

In Summer 2012 at the University of Minnesota Duluth REU I worked on algorithms to find the diameters of groups generated by transposition trees. The idea is to consider the Cayley graph $$\Gamma$$ of some permutation group as generated by some minimal set of transpositions, and find the diameter. In general finding the diameter of a Cayley graph is a very hard problem, but in this case we can get some good bounds; my research was on improving those bounds and the speed of the algorithms to compute them. The paper was published in Discrete Applied Mathematics.

### Elements of Random Matrices

At RSI I worked on a project in random matrices with Gregory Minton. The idea was to compute some properties of the distributions of the entries of random matrices in $$U(n)$$. The paper is here and you can see my talk here.

### The Collatz Conjecture

In Summer 2009 Keenan Monks and I worked on understanding some properties of the Collatz map. The idea is that if you take $$\mathbb{Q}$$ or the 2-adics and iterate the Collatz map, you get some directed graph, which has interesting symmetries; we were trying to understand those symmetries. The paper was published in Discrete Mathematics.