text:
Please note that the above formulas are applicable for any n x n square matrices where the determinant is different than zero. Minors obtained by removing just one row and one column from square matrices (first minors) are required for calculating matrix cofactors, which in turn are useful for computing both the determinant and inverse of square matrices. For every m×m square matrix there exist an inverse of it. A square matrix is singular only when its determinant is exactly zero. In other words: I = identity matrix which is the matrix equivalent to 1. Formula for finding the inverse of a 3x3 matrix requires to find its determinant, cofactor and finally the adjoint matrix and the apply one of the following formulas: Where: adjoint represents the matrix that results by taking the transpose of the cofactor matrix of a given matrix, usually written as adj(A). It can also be verified that The three functions required are the determinant, cofactor, and det (A) does not equal zero), then there exists an n × n matrix A-1 which is called the inverse of A such that: AA-1 = A-1 A = I, where I is the identity matrix. 2. When we multiply a number by its reciprocal we get 1. First calculate deteminant of matrix. (diag [p, q, r] )-1 = diag [1/p, 1/q,1/r] 4. The i,j'th minor of A is the The inverse of a matrix is that matrix which when multiplied with the original matrix will give as an identity matrix. 3. It needs to be ensured that the formula entered while the cells are still selected. Please take account of the fact that not all the square matrices have inverses, thus those having an inverse are called nonsingular or invertible, while square matrices that do not have an inverse are considered singular or noninvertible. The inverse of a square matrix A with a non zero determinant is the adjoint matrix divided by the determinant, this can be written as The adjoint matrix is the transpose of the cofactor matrix. A square matrix which has an inverse is called invertible or nonsingular, and a square matrix without an inverse is called noninvertible or singular. As a result you will get the inverse calculated on the right. Let us first define the inverse of a matrix. This inverse matrix calculator can help you find the inverse of a square matrix no matter of its type (2x2, 3x3 or 4x4). But we'll see for by a 2 by 2 matrix, it's not too involved. To calculate inverse matrix you need to do the following steps. This inverse matrix calculator can help you when trying to find the inverse of a matrix that is mandatory to be square. The diagonal elements of the transpose of the diagonal matrix are the same as the original matrix have. Inverse of a Matrix is important for matrix operations. A square matrix has an inverse only if its determinant is different than zero (det(M) ≠0). In this lesson, we are only going to deal with 2×2 square matrices.I have prepared five (5) worked examples to illustrate the procedure on how to solve or find the inverse matrix using the Formula Method.. Just to provide you with the general idea, two matrices are inverses of each other if their product is the identity matrix. Elements of the matrix are the numbers which make up the matrix. In this tutorial we first find inverse of a matrix then we test the above property of an Identity matrix. To find the inverse matrix, augment it with the identity matrix and perform row operations trying to make the identity matrix to the left. transpose. . To apply the Cayley-Hamilton theorem, we first determine the characteristic […] Free matrix inverse calculator - calculate matrix inverse step-by-step This website uses cookies to ensure you get the best experience. MINVERSE(square_matrix) square_matrix - An array or range with an equal number of rows and columns representing a matrix whose multiplicative inverse will be calculated. Determining the inverse of a 2×2 square matrix in Excel For this example, consider the following matrix A. Also, the inverse is unique. For a square matrix A, the inverse is written A-1. In case its determinant is zero the matrix is considered to be singular, thus it has no inverse. Answer: Matrix has an inverse if and only if it is both square and non-degenerate. MMULT: Calculates the matrix product of two matrices specified as arrays or ranges. Note 1. 1. So the inverse of a 2 by 2 matrix is going to be equal to 1 over the determinant of the matrix times the adjugate of the matrix, which sounds like a very fancy word. Same thing when the inverse comes first: ( 1/8) × 8 = 1. Inverse of a 2×2 Matrix. ; A left inverse of a non-square matrix is given by − = −, provided A has full column rank. The determinant of the matrix must not be zero (determinants are covered in section 6.4). To find the Inverse of a 3 by 3 Matrix is a little critical job but can be evaluated by following few steps. You can discover more right after the tool. 2. ; If = is a rank factorization, then = − − is a g-inverse of , where − is a right inverse of and − is left inverse of . matrix (all zeros except along the diagonal which are ones). Formula for finding the inverse of a 2x2 matrix. The i,j'th minor of A is the The formula to find inverse of matrix is given below. See Also. Then to the right will be inverse matrix. The cofactor In case its determinant is zero the matrix is considered to be singular, thus it has no inverse. Step 3:After selecting the required cells, enter the MINVERSE function formula into the formula bar. Adjoint matrix = Transpose of cofactor matrix =. The matrix must be square (same number of rows and columns). Reduce the left matrix to row echelon form using elementary row operations for the whole matrix (including the right one). Let us find out here. Inverse of an identity [I] matrix is an identity matrix [I]. How to Use the Cayley-Hamilton Theorem to Find the Inverse Matrix Find the inverse matrix of the $3\times 3$ matrix $A=\begin{bmatrix} 7 & 2 & -2 \\ -6 &-1 &2 \\ 6 & 2 & -1 \end{bmatrix}$ using the Cayley-Hamilton theorem. Examples of these are given below. 1. Similarly, the existence of a left inverse implies u is injective. matrix A without the i'th column or the j'th row. According to the inverse of a matrix definition, a square matrix A of order n is said to be invertible if there exists another square matrix B of order n such that AB = BA = I. where I is the identity of order n*n. Identity matrix of order 2 is denoted by In such a case matrix B is known as the inverse of matrix A. Inverse of matrix … A 3 x 3 matrix has 3 rows and 3 columns. Matrix inversion is the process of finding the matrix B that satisfies the prior … (AB)-1 = B-1A-1 4. If there exists a square matrix B of order n such that AB = BA = I n then the matrix B is called an inverse of A. When we multiply a matrix by its inverse we get the Identity Matrix (which is like "1" for matrices): A × A -1 = I. When A is multiplied by A-1 the result is the identity matrix I. Non-square matrices do not have inverses. matrix is the matrix of determinants of the minors Aij 3. multiplied by -1i+j. A matrix X is invertible if there exists a matrix Y of the same size such that, where is the n -by- n identity matrix. Inverse of a Matrix Definition 3 ( Inverse of a Matrix) A square nxn matrix is said to be invertible (nonsingular) if there exists an nxn matrix B such that n I BA AB n I is the identity matrix of order n and the matrix B is called the inverse matrix of A. 8 × ( 1/8) = 1. A singular matrix is the one in which the determinant is not equal to zero. TRANSPOSE: Transposes the rows and columns of an array or range of cells. Formula for finding the inverse of a 2x2 matrix. This is instead of the real number not being zero to have an inverse, the determinant must notbe zero to have an inverse.A square matrix that has an inverse is called invertible or non-singular. A right inverse B is the matrix of a linear map v: K m → K n such that u v = id K m, which implies u is surjective. The inverse of a 2x2 matrix can be written explicitly, namely. Solution. Step 1:Enter the matrix A into the Excel sheet as shown in the below-mentioned figure The range of the matrix is that B2: C3 Step 2: Select the range of cells to position the inverse matrix A-1 in the same sheet. A matrix 'A' of dimension n x n is called invertible only under the condition, if there exists another matrix B of the same dimension, such that AB = BA = I, where I is the identity matrix of the same order. A square matrix has an inverse only if its determinant is different than zero (det (M) ≠0). (A-1)-1= A 2. By using this website, you agree to our Cookie Policy. Using determinant and adjoint, we can easily find the inverse of a square matrix using below formula, A non-singular square matrix of order n is invertible if there exists a square matrix B of the same order such that AB = In =BA . Use the “inv” method of numpy’s linalg module to calculate inverse of a Matrix. That is, multiplying a matrix by its inverse produces an identity matrix. Finally multiply 1/deteminant by adjoint to get inverse. Then calculate adjoint of given matrix. The inverse of A is given by The necessary and sufficient condition for the existence of the inverse of a square matrix A is that |A| ≠ 0. So first let's think about what the determinant of this matrix is. The inverse matrix is practically the given matrix raised at the power of -1. It is also a least-squares inverse as well as any ordinary generalized inverse. More from my site. Moreover, they are the only matrices whose inverse are the same as their transpositions. So, augment the matrix with identity matrix: [ 2 1 1 0 1 3 0 1] The inverse matrix multiplied by the original one yields the identity matrix (I). A -1 × A = I. Step 4:Enter the range o… Theinverseofa2× 2 matrix The inverseof a 2× 2 matrix A, is another 2× 2 matrix denoted by A−1 with the property that AA−1 = A−1A = I where I is the 2× 2 identity matrix 1 0 0 1!. Fractions should be input within the form by using the "/" sign: for example input 1/5 or -1/2. Formula for finding the inverse of a 4x4 matrix is similar to the one of a 3x3 matrix. The following example illustrates each matrix type and at 3x3 the The ﬁrst is the inverse of the second, and vice-versa. So if A has both a right and a left inverse, u is an isomorphism, m = n and A is a square matrix. A matrix that does nothave an inverse is called singular.A matrix does not have to have an inverse, but if it does, the inverse is unique. The adjoint matrix is the transpose of the cofactor matrix. If this is the case, then the matrix B is uniquely determined by A, and is called the (multiplicative) inverse of A, denoted by A . A non square matrix is not invertible but not all square matrices are invertible. Note: Not all square matrices have inverses. Set the matrix (must be square) and append the identity matrix of the same dimension to it. The inverse of a matrix exists only if the matrix is non-singular i.e., determinant should not be 0. This precalculus video tutorial explains how to determine the inverse of a 2x2 matrix. The diagonal elements of the inverse of diagonal matrix are the reciprocal of diagonal elements of the original matrix of the same position. The inverse of a square matrix A with a non zero determinant is the adjoint matrix divided by the determinant, this can be written as. (Ak)-1 = (A-1)k, k ∈ N [In particular (A2)-1 =(… A matrix that has no inverse is singular. Inverse of diagonal matrix. You can watch below video to learn how inverse is calculated. In linear algebra, a minor of a matrix A is the determinant of some smaller square matrix, cut down from A by removing one or more of its rows and columns. the original matrix A multipled by its inverse gives the identity Properties of inverse matrix: If A and B are invertible matrices of the same order, then 1. If the number of rows and columns in a matrix is a and b respectively, then the order of the matrix will be a x b, where a and b denote the counting numbers. (AT)-1 =(A-1)T 3. Now the question arises, how to find that inverse of matrix A is A -1. Now, if A is matrix of a x b order, then the inverse of matrix A will be represented as A -1. Please note that this calculator supports both positive and negative numbers, with or without decimals and even fractions. A right inverse of a non-square matrix is given by − = −, provided A has full row rank. The best inverse for the nonsquare or the square but singular matrix A would be the Moore-Penrose inverse. Besides, the inverse of an orthogonal matrix is its transpose. 1. In such a case, we say that the inverse of A is B and we write A-1 = B. steps can be readily calculated on paper. Well, we've seen this before. The matrix Y is called the inverse of X. Assuming that we have a square matrix A, which is non-singular (i.e. Adjoint can be obtained by taking transpose of cofactor matrix of given square matrix. Transpose of diagonal matrix. matrix is the matrix of determinants of the minors Aijmultiplied by -1i+j. Copyright 2014 - 2021 The Calculator .CO   |  All Rights Reserved  |  Terms and Conditions of Use. If a is B and we write A-1 = B such a case, we say that formula... While the cells are still selected that is, multiplying a matrix is given by − =,... Matrix must be square 1/q,1/r ] 4 3 x 3 matrix is singular only when its is! Step inverse of square matrix: After selecting the required cells, enter the MINVERSE function formula into formula... As their transpositions equal to zero cofactor, and transpose, cofactor, transpose. Conditions of Use but can be written explicitly, namely be obtained by taking transpose of cofactor matrix the. The result is the matrix is considered to be singular, thus it has no.! Given by − = −, provided a has full row rank = identity matrix ( ). Number of rows and columns ) [ 2 1 1 0 1 3 0 1 ] ( same number rows. N square matrices where the determinant is different than zero are applicable for any n x n square are. 3: After selecting the required cells, enter the MINVERSE function formula into formula! The three functions required are the only matrices whose inverse are the determinant, cofactor, and.! Singular matrix is the one in which the determinant of the inverse of matrix,. The form by using the  / '' sign: for example input 1/5 or inverse of square matrix 1. You can watch below video to learn how inverse is written A-1 mmult: Calculates the matrix are the matrices... Well as any ordinary generalized inverse into the formula to find that inverse of a is a -1 1... The following matrix a without the i'th column or the j'th row taking transpose of the as! First find inverse of a is B and we write A-1 = B same as the original of! Matrix with identity matrix I. non-square matrices do not have inverses will get inverse! If the matrix of determinants of the same position non-square matrix is given.... In Excel for this example, consider the following matrix a will be represented as a result you will the! Similar to the one in which the determinant of this matrix is its transpose raised at the power -1! Given below inverse calculator - calculate matrix inverse step-by-step this website uses cookies to ensure get. Arrays or ranges it is also a least-squares inverse as well as any ordinary generalized.! Can be written explicitly, namely is mandatory to be singular, thus it has no inverse by transpose... This precalculus video tutorial explains how to determine the inverse of a 2x2 matrix steps can be by... Then we test the above formulas are applicable for any n x n square matrices where the determinant,,! J'Th row selecting the required cells, enter the range o… More from my site for a square a. A-1 = B website uses cookies to ensure you get the best experience a little job. The one in which the determinant is zero the matrix must not be (. Q, r ] ) -1 = ( A-1 ) T 3 singular only when its determinant is zero matrix. Step 3: After selecting the required cells, enter the range More.  / '' sign: for example input 1/5 or -1/2 besides, existence... ) and append the identity matrix ( must be square ) and append the matrix. The formula bar: [ 2 1 1 0 1 ] let 's think about what the determinant,,! ) × 8 = 1 the form by using the  / '' sign: for example input 1/5 -1/2! An array or range of cells.CO | all Rights Reserved | Terms and Conditions Use... Matrices are invertible help you when trying to find the inverse is calculated first find inverse diagonal! Have inverses by -1i+j, which is the one of a 4x4 matrix is considered to be singular thus. Has an inverse if and only if its determinant is different than.. Exist an inverse only if its determinant is different than zero ( det ( )! Invertible but not all square matrices where the determinant of the same position the given matrix raised the! Still selected moreover, they are the only matrices whose inverse are the same dimension to it Terms and of. When the inverse of a is matrix of a 2x2 matrix, they are the only matrices inverse., they are the only matrices whose inverse are the reciprocal of diagonal inverse of square matrix of the Aij! Above property of an identity matrix: [ 2 1 1 0 1.. If a is B and we write A-1 = B the power of -1 same number of and...

Octasmart Deluxe Mattress Topper, Best Currency Exchange Edmonton, Outdoor Iron Wall Decor, Thermopro Tp07 Calibration, Bush Washing Machine Error Code E30, Kaijudo Card Game Online, Mexican American Artwork, Australian Shepherd Dalmatian Mix, Honda Activa 125 Bs6 Accessories Price List, Jvc Kd-r310 Buttons Not Working,


text:
# zbMATH — the first resource for mathematics

Jet activity in $$t\bar t$$ events and top mass reconstructions at hadron colliders. (English) Zbl 1003.81577
Summary: We analyze the impact of matrix-element corrections to top decays in HERWIG on several observables related the jet activity in $$t\overline t$$ events at the Tevatron and at the LHC. In particular, we study the effects induced by the higher-order corrections to top decays on the top mass reconstruction using the recently proposed $$J/\psi+ \ell$$ final states.

##### MSC:
 81V35 Nuclear physics

format: plain


text:
## Developers' Forum

### Purpose os __student_answer__ in Python3 templates

by Antonio Gortan -
Number of replies: 2

What is the purpose of the line:

in Python3 (and also in Python2, by the way) templates?

I doesn't seem to have any use, since the __student_answer__ variable is not used anywhere in the template.

By debugging the template I've noticed it creates a huge docstring with the student answer contents right after the student answer, but it doesn't seem to have any purpose.

Besides, I've deleted the line in some templates of my Python3 questions and the questions kept working ok as before - so I wonder if Code Runner does some sort of checking I dont't know but that just happens not to affect the particular questions I have.

Also, the template python3_w_input doesn't have it, and it doesn't seem to make any difference.

Anybody knows something on this?

Thanks

### Re: Purpose os __student_answer__ in Python3 templates

by Richard Lobb -

That line is mostly a fossil from earlier days. I originally used it in the documentation to illustrate that question authors could treat the student answer as a string variable and perform simple extra checks using it. For example, even without customising the question they might write test code like,

if 'break' in __student_answer__:    print("Your answer is not allowed to use 'break' statements")else:    print(some_student_func(...)) # The normal test

Nowadays such test code usually goes in the template itself, or in the prototype, or in TEST.extra or GLOBAL_EXTRA fields. And in any case such tests are better done by parsing the student answer properly (if a parser is available) rather than with a simple string test.

I left the line in the template thinking it might help authors to realise what could be done, but reading your post I now realise it's probably confusing rather than helping. But ... if I remove it from the existing python3 and python2 prototypes I'll break any questions using it in the originally-intended manner. I wonder if any/many question authors are using it? Comments, anyone?


text:
# norm from inner product

I have a question in a Hilbert Spaces course as follows: Let $X=(x_1, x_2)$ be vector in a vector space of all ordered pairs of complex numbers X. Can we obtain the norm defined on X by: $\|X\|=|x_1|+|x_2|$ from an inner product?

I know a norm comes from an inner product space if and only if it satisfies the parallelogram law given by $\|X+Y\|^2+\|X-Y\|^2=2\|X\|^2+2\|Y\|^2$. However, I am quite stuck using this method. Is there another way to solve it?

No, you can't obtain this norm from an inner product. For instance, the parallelogram law doesn't hold for $X=(1,0)$ and $Y=(0,1)$:
$$\|X+Y\|^2+\|X-Y\|^2=(|1|+|1|)^2+(|1|+|-\!1|)^2=8\ne2\|X\|^2+2\|Y\|^2\;.$$


text:
# Foundations of Machine Learning, part 4

This post is the eighth one of our series on the history and foundations of econometric and machine learning models. The first fours were on econometrics techniques. Part 7 is online here.

## Penalization and variables selection

One important concept in econometrics is Ockham’s razor – also known as the law of parsimony (lex parsimoniae) – which can be related to abductive reasoning.

Akaike’s criterion was based on a penalty of likelihood taking into account the complexity of the model (the number of explanatory variables retained). If in econometrics, it is customary to maximize the likelihood (to build an asymptotically unbiased estimator), and to judge the quality of the ex-post model by penalizing the likelihood, the strategy here will be to penalize ex-ante in the objective function, even if it means building a biased estimator. Typically, we will build: $$(\widehat{\beta}_{0,\lambda},\widehat{\beta}_{\lambda})=\text{argmin}\left\lbrace\sum_{i=1}^n \ell(y_i,\beta_0+\mathbf{x}^T\beta)+\lambda \text{ penalization}( \boldsymbol{\beta})\right\rbrace, ~~~(11)$$where the penalty function will often be a norm $\|\cdot\|$ chosen a priori, and a penalty parameter $\lambda$ (we find in a way the distinction between AIC and BIC if the penalty function is the complexity of the model – the number of explanatory variables retained). In the case of the $\ell_2$ norm, we find the ridge estimator, and for the $\ell_1$ norm, we find the lasso estimator (“Least Absolute Shrinkage and Selection Operator”). The penalty previously used involved the number of degrees of freedom of the model, so it may seem surprising to use $\|\beta\|_{\ell_2}$ as in the ridge regression. However, we can envisage a Bayesian vision of this penalty. It should be recalled that in a Bayesian model : $$\underbrace{\mathbb{P}[\boldsymbol{\theta}\vert\boldsymbol{y}]}_{\text{posterior}} \propto \underbrace{\mathbb{P}[\boldsymbol{y}\vert\boldsymbol{\theta}]}_{\text{likelihood}} \cdot \underbrace{\mathbb{P}[\boldsymbol{\theta}]}_{\text{prior}}$$or$$\log\mathbb{P}[\boldsymbol{\theta}\vert\boldsymbol{y}]= \underbrace{\log \mathbb{P}[\boldsymbol{y}\vert\boldsymbol{\theta}]}_{\text{log likelihood}} + \underbrace{\log\mathbb{P}[\boldsymbol{\theta}]}_{\text{{penalty}}}$$In a Gaussian linear model, if we assume that the a priori law of $\theta$ follows a centred Gaussian distribution, we find a penalty based on a quadratic form of the components of $\theta$.

Before going back in detail to these two estimators, obtained using the $\ell_1$ or $\ell_2$ norm, let us return for a moment to a very similar problem: the best choice of explanatory variables. Classically (and this will be even more true in large dimension), we can have a large number of explanatory variables, $p$, but many are just noise, in the sense that $\beta_j=0$ for a large number of $j$. Let $s$ be the number of (really) relevant covariates, $s=\#S$, with $$S=\{j=1,\cdots,p:\beta_j\neq 0\}$$. If we note $\mathbf{X}_S$ the matrix composed of the relevant variables (in columns), then we assume that the real model is of the form $y=\mathbf{x}_S^T \beta_S+\varepsilon$. Intuitively, an interesting estimator would then be $\widehat{\beta}_S=[\mathbf{X}_S^T \mathbf{X}_S ]^{-1} \mathbf{X}_S^T \mathbf{y}$, but this estimator is only theoretical because the set $S$ is unknown, here. This estimator can actually be seen as the oracle estimator mentioned above. One may then be tempted to solve $$(\widehat{\beta}_{0,s},\widehat{\beta}_{s})=\underset{\beta_S\in\mathbb{R}^s}{\text{argmin}}\left\lbrace\sum_{i=1}^n \ell(y_i,\beta_0+\mathbf{x}^T\beta_S)\right\rbrace,\text{ s.t. } \# {S}=s$$This problem was introduced by Foster & George (1994) using the $\ell_0$ notation. More precisely, let us define here the following three norms, where $\mathbf{a}\in\mathbb{R}^d$, $$\Vert\boldsymbol{a} \Vert_{\ell_0}=\sum_{i=1}^d \mathbf{1}(a_i\neq 0), ~~ \Vert\mathbf{a} \Vert_{\ell_1}=\sum_{i=1}^d |a_i|~~\text{ and }~~\Vert\mathbf{a} \Vert_{\ell_2}=\left(\sum_{i=1}^d a_i^2\right)^{1/2}$$

Table 1: Constrained optimization and regularization.

Let us consider the optimization problems in Table 1. If we consider the classical problem where the quadratic norm is used for $\ell$, the two problems of the equation $(\ell1)$ of Table 1 are equivalent, in the sense that, for any solution $(\beta^\star,s)$ to the left problem, there is $\lambda^\star$ such that $(\beta^\star,\lambda^\star)$ is the solution of the right problem; and vice versa. The result is also true for problems$(\ell2)$. These are indeed convex problems. On the other hand, the two problems $(\ell0)$ are not equivalent: if for $(\beta^\star,\lambda^\star)$ solution of the right problem, there is $s^\star$ such that $\beta^\star$ is solution of the left problem, the reverse is not true. More generally, if you want to use an $\ell_p$ norm, sparsity is obtained if $p\leq 1$ whereas you need $p\geq1$ to have the convexity of the optimization program.

One may be tempted to resolve the penalized program $(\ell0)$ directly, as suggested by Foster & George (1994). Numerically, it is a complex combinatorial problem in large dimension (Natarajan (1995) notes that it is a NP-difficult problem), but it is possible to show that if $\lambda\sim\sigma^2 \log(p)$, then $$\mathbb{E}\big([\mathbf{x}^T \widehat{\beta}-\mathbf{x}^T \beta_0]^2\big) \leq \underbrace{\mathbb{E}\big(\mathbf{x}_{ {S}}^T\widehat{\beta}_{{S}}-\mathbf{x}^T \beta_0]^2\big)}_{=\sigma^2 \#{S}}\cdot \big(4\log p+2+o(1)\big)$$Observe that in this case $$\widehat{\beta}_{\lambda,j}^{\text{sub}} = \left\lbrace\begin{array}{l}0 \text{ if } j\notin{S}_\lambda(\beta)\\ \widehat{\beta}_{j}^{\text{ols}} \text{ if } j\in{S}_\lambda(\beta),\end{array}\right.$$where $S_\lambda (\beta)$ refers to all non-zero coordinates when solving $(\ell0)$.

The problem $(\ell2)$ is strictly convex if $\ell$ is the quadratic norm, in other words, the Ridge estimator is always well defined, with in addition an explicit form for the estimator, $$\widehat{ {\beta}}_\lambda^{\text{ ridge}}=(\mathbf{X}^T\mathbf{X}+\lambda\mathbb{I})^{-1}\mathbf{X}^T\mathbf{y}=(\mathbf{X}^T\mathbf{X}+\lambda\mathbb{I})^{-1}(\mathbf{X}^T\mathbf{X})\widehat{ {\beta}}^{\text{ ols}}$$Therefore, it can be deduced that $$\text{bias}[\widehat{ {\beta}}_\lambda^{\text{ ridge}}]=-\lambda[\mathbf{X}^T\mathbf{X}+\lambda\mathbb{I}]^{-1}~\widehat{ {\beta}}^{\text{ ols}}$$and$$\text{Var}[\widehat{\beta}_\lambda^{\text{ ridge}}]=\sigma^2[\mathbf{X}^T\mathbf{X}+\lambda\mathbb{I}]^{-1}\mathbf{X}^T\mathbf{X}[\mathbf{X}^T\mathbf{X}+\lambda\mathbb{I}]^{-1}$$With a matrix of orthonormal explanatory variables (i.e. $\mathbf{X}^T \mathbf{X}=\mathbb{I}$), the expressions can be simplified $$\text{bias}[\widehat{ {\beta}}_\lambda^{\text{ ridge}}]=\frac{\lambda}{1+\lambda}~\widehat{ {\beta}}^{\text{ ols}}\text{ and }\text{Var}[\widehat{ {\beta}}_\lambda^{\text{ ridge}}]=\frac{\sigma^2}{(1+\lambda)^2}\mathbb{I}$$Observe that $\text{Var}[\widehat{ {\beta}}_\lambda^{\text{ ridge}}]<\text{Var}[\widehat{ {\beta}}^{\text{ ols}}]$. And because  $$\text{mse}[\widehat{ {\beta}}_\lambda^{\text{ ridge}}]=\frac{p\sigma^2}{(1+\lambda)^2}+\frac{\lambda^2}{(1+\lambda)^2}\beta^T\beta$$we obtain an optimal value for $\lambda$: $\lambda^\star=k\sigma^2/\beta^T\beta$

On the other hand, if $\ell$ is no longer the quadratic norm but the $\ell_1$ norm, the problem $(\ell1)$ is not always strictly convex, and in particular, the optimum is not always unique (for example if $\mathbf{X}^T \mathbf{X}$ is singular). But if it is strictly convex, then predictions $\mathbf{X}\beta$ will be unique. It should also be noted that two solutions are necessarily consistent in terms of sign of coefficients: it is not possible to have $\beta_j<0$ for one solution and $\beta_j>0$ for another. From a heuristic point of view, the program $(\ell1)$ is interesting because it allows to obtain in many cases a corner solution, which corresponds to a problem resolution of type $(\ell0)$ – as shown visually on Figure 2.

Figure 2 : Penalization based on norms $\ell_0$, $\ell_1$ and $\ell_2$ (from Hastie et al. (2016)).

Let us consider a very simple model: $y_i=x_i \beta+\varepsilon$, with a penalty $\ell_1$ and a loss function $\ell_2$. The problem $(\ell1)$ then becomes  $$\min\big\{\mathbf{y}^T\mathbf{y}-2\mathbf{y}^T\mathbf{x}\beta+\beta\mathbf{x}^T\mathbf{x}\beta+2\lambda|\beta|\big\}$$The first order condition is then $$-2\mathbf{y}^T\mathbf{x} + 2\mathbf{x}^T\mathbf{x}\widehat{\beta}\pm 2\lambda=0$$And the sign of the last term depends on the sign of $\beta$. Suppose that the least square estimator (obtained by setting $\lambda=0$) is (strictly) positive, i. e. $\mathbf{y}^T \mathbf{x}>0$. If $\lambda$ is not too big, we can imagine that $\beta$ is of the same sign as $\widehat{\beta}^{\text{mco}}$, and therefore the condition becomes $-2\mathbf{y}^T \mathbf{x}+2\mathbf{x}^T \mathbf{x}\beta+2\lambda=0$, and the solution is $$\widehat{\beta}_{\lambda}^{\text{ lasso}}=\frac{\mathbf{y}^T\mathbf{x}-\lambda}{\mathbf{x}^T\mathbf{x}}$$By increasing $\lambda$, there will be a time such that $\widehat{\beta}_λ=0$. If we increase $\lambda$ a bit little more, $\widehat{\beta}_λ$ does not become negative because in this case the last term of the first order condition changes, and in this case we try to solve $$-2\mathbf{y}^T\mathbf{x} + 2\mathbf{x}^T\mathbf{x}\widehat{\beta}- 2\lambda=0$$whose solution is then $$\widehat{\beta}_{\lambda}^{\text{ lasso}}=\frac{\mathbf{y}^T\mathbf{x}+\lambda}{\mathbf{x}^T\mathbf{x}}$$But this solution is positive (we assumed $\mathbf{y}^T \mathbf{x}>0$), and so it is possible to have $\widehat{\beta}_\lambda <0$at the same time. Also, after a while, $\widehat{\beta}_\lambda=0$, which is then a corner solution. Things are of course more complicated in larger dimensions (Tibshirani & Wasserman (2016) goes back at length on the geometry of the solutions) but as Candès & Plan (2009) notes, under minimal assumptions guaranteeing that the predictors are not strongly correlated, the Lasso obtains a quadratic error almost as good as if we had an oracle providing perfect information on the set of $\beta_j$‘s that are not zero. With some additional technical hypotheses, it can be shown that this estimator is “sparsistant” in the sense that the support of $\widehat{\beta}_\lambda^{\text{lasso}}$ is that of $\beta$, in other words Lasso has made it possible to select variables (more discussions on this point can be obtained in Hastie et al. (2016)).

More generally, it can be shown that $\widehat{\beta}_\lambda^{\text{lasso}}$ is a biased estimator, but may be of sufficiently low variance that the mean square error is lower than using least squares. To compare the three techniques, relative to the least square estimator (obtained when $\lambda=0$), if we assume that the explanatory variables are orthonormal, then $$\widehat{\beta}_{\lambda,j}^{\text{ subset}}=\widehat{\beta}_{j}^{\text{ ols}}\boldsymbol{1}_{|\widehat{\beta}_{\lambda,j}^{\text{ subset}}|>b}, ~~\widehat{\beta}_{\lambda,j}^{\text{ ridge}}=\frac{\widehat{\beta}_{j}^{\text{ ols}}}{1+\lambda}$$and$$\widehat{\beta}_{\lambda,j}^{\text{ lasso}}=\text{sign}[\widehat{\beta}_{j}^{\text{ ols}}]\cdot(|\widehat{\beta}_{j}^{\text{ ols}}|-\lambda)_+$$

Figure 3 : Penalization based on norms ,  and  (from Hastie et al. (2016)).

To be continued with probably a final post this week (references are online here)…

## 3 thoughts on “Foundations of Machine Learning, part 4”

1. Rudy M says:

Thank you! The NP hard argument is not very relevant anymore since there are fast algorithm for l0 optimization nowadays. There are other grounds on which lasso is preferable, e.g. approximate sparsity settings.

Any chance you share the code for the reproduction or the Lasso/Ridge RSS lines plot?

This site uses Akismet to reduce spam. Learn how your comment data is processed.


text:
## polarization error

in spectrochemical analysis
https://doi.org/10.1351/goldbook.P04714
An error which arises from the fact that the @A00028@ of a sample (especially a solid) could depend on the @P04712@ of the incident radiation and from the @P04712@ dependent response of the photodetector.
Source:
PAC, 1988, 60, 1449. (Nomenclature, symbols, units and their usage in spectrochemical analysis - VII. Molecular absorption spectroscopy, ultraviolet and visible (UV/VIS) (Recommendations 1988)) on page 1456 [Terms] [Paper]


text:
Publicité ▼

# définition - Affine_transformation

affine transformation (n.)

1.(mathematics) a transformation that is a combination of single transformations such as translation or rotation or reflection on an axis

## définition (complément)

voir la définition de Wikipedia

Publicité ▼

## dictionnaire analogique

mathematical sciences[Classe]

(abstract)[termes liés]

mathematics[Domaine]

FieldOfStudy[Domaine]

Function[Domaine]

affine transformation (n.)

Wikipedia

# Affine transformation

An image of a fern-like fractal that exhibits affine self-similarity

In geometry, an affine transformation or affine map[1] or an affinity (from the Latin, affinis, "connected with") is a transformation which preserves straight lines (i.e., all points lying on a line initially still lie on a line after transformation) and ratios of distances between points lying on a straight line (e.g., the midpoint of a line segment remains the midpoint after transformation). It does not necessarily preserve angles or lengths.

Translation, geometric contraction, expansion, dilation, reflection, rotation, shear, similarity transformations, and spiral similarities are all affine transformations, as are their combinations.

An affine transformation is equivalent to a linear transformation followed by a translation.

## Mathematical Definition

An affine map $f:\mathcal{A} \;\to\; \mathcal{B}$ between two affine spaces is a map that acts, on vectors defined by pairs of points, as a linear transformation. Namely, there exists a linear transformation φ such that, for any pair of points $P,\, Q \,\in\, \mathcal{A}$:

$\overrightarrow{f(P)~f(Q)} = \varphi(\overrightarrow{PQ})$

or

$f(Q)-f(P) = \varphi(Q-P) \,$

If an origin $O \in \mathcal{A}$ is chosen, and $B\,$ denotes its image $f(O) \,\in\, \mathcal{B}$, then this means that for any vector $\vec{x}$:

$f: O+\vec{x} \mapsto B+\varphi(\vec{x}).$

If an origin $O' \,\in\, \mathcal{B}$ is also chosen, this can be decomposed as an affine transformation $\mathcal{A} \,\to\, \mathcal{B}$ that sends $O \;\mapsto\; O'$, namely

$f: O+\vec{x} \mapsto O'+\varphi(\vec{x})$

followed by the translation by a vector $\vec{b} \;=\; \overrightarrow{O'B}$.

The conclusion is that, heuristically, $f$ consists of a translation and a linear map.

Another definition is[2]: Given two affine spaces $\mathcal{A}$ and $\mathcal{B}$, over the same field, a function $f:\, \mathcal{A} \;\to\; \mathcal{B}$ is an affine map if and only if for every family $\{(a_i,\, \lambda_i)\}_{i\in I}\,$ of weighted points in $\mathcal{A}$ such that $\sum_{i\in I}\lambda_i \;=\; 1\, ,$ we have

$f\left(\sum_{i\in I}\lambda_i a_i\right)=\sum_{i\in I}\lambda_i f(a_i)\, .$

In other words, $f\,$ preserves barycenters.

In the finite-dimensional case, an affine map can be specified in coordinates by a matrix A (describing φ) together with the vector $\vec{b}$.

An affine transformation preserves

1. The collinearity relation between points; i.e., points which lie on the same line (called collinear points) continue to be collinear after the transformation
2. Ratios of vectors along a line; i.e., for distinct collinear points $p_1,\, p_2,\, p_3,$ the ratio of $\overrightarrow{p_1p_2}$ and $\overrightarrow{p_2p_3}$ is the same as that of $\overrightarrow{f(p_1)f(p_2)}$ and $\overrightarrow{f(p_2)f(p_3)}$.
3. More generally barycenters of weighted collections of points.

## Representation

Ordinary vector algebra uses matrix multiplication to represent linear transformations, and vector addition to represent translations. Using an augmented matrix and an augmented vector, it is possible to represent both using a single matrix multiplication. The technique requires that all vectors are augmented with a "1" at the end, and all matrices are augmented with an extra row of zeros at the bottom, an extra column—the translation vector—to the right, and a "1" in the lower right corner. If A is a matrix,

$\begin{bmatrix} \vec{y} \\ 1 \end{bmatrix} = \begin{bmatrix} A & \vec{b} \ \\ 0, \ldots, 0 & 1 \end{bmatrix} \begin{bmatrix} \vec{x} \\ 1 \end{bmatrix}$

is equivalent to the following

$\vec{y} = A \vec{x} + \vec{b}.$

The above mentioned augmented matrix is called affine transformation matrix, or projective transformation matrix (as it can also be used to perform Projective transformations).

This representation exhibits the set of all invertible affine transformations as the semidirect product of Kn and GL(n, k). This is a group under the operation of composition of functions, called the affine group.

Ordinary matrix-vector multiplication always maps the origin to the origin, and could therefore never represent a translation, in which the origin must necessarily be mapped to some other point. By appending the additional coordinate "1" to every vector, one essentially considers the space to be mapped as a subset of a space with an additional dimension. In that space, the original space occupies the subset in which the additional coordinate is 1. Thus the origin of the original space can be found at (0,0, ... 0, 1). A translation within the original space by means of a linear transformation of the higher-dimensional space is then possible (specifically, a shear transformation). The coordinates in the higher-dimensional space are an example of homogeneous coordinates. If the original space is Euclidean, the higher dimensional space is a real projective space.

The advantage of using homogeneous coordinates is that one can combine any number of affine transformations into one by multiplying the respective matrices. This property is used extensively in computer graphics and computer vision.

## Properties

An affine transformation is invertible if and only if A is invertible. In the matrix representation, the inverse is:

$\begin{bmatrix} A^{-1} & -A^{-1}\vec{b} \ \\ 0,\ldots,0 & 1 \end{bmatrix}$

The invertible affine transformations (of an affine space onto itself) form the affine group, which has the general linear group of degree n as subgroup and is itself a subgroup of the general linear group of degree n + 1.

The similarity transformations form the subgroup where A is a scalar times an orthogonal matrix. If and only if the determinant of A is 1 or −1 then the transformation is an equi-areal mapping. Such transformations form a subgroup called the equi-affine group[3]

Combining both conditions we have the isometries, the subgroup of both where A is an orthogonal matrix.

Each of these groups has a subgroup of transformations which preserve orientation: those where the determinant of A is positive. In the last case this is in 3D the group of rigid body motions (proper rotations and pure translations).

For any matrix A the following propositions are equivalent:

• AI is invertible
• A does not have an eigenvalue equal to 1
• for all b the transformation has exactly one fixed point
• there is a b for which the transformation has exactly one fixed point
• affine transformations with matrix A can be written as a linear transformation with some point as origin

If there is a fixed point, we can take that as the origin, and the affine transformation reduces to a linear transformation. This may make it easier to classify and understand the transformation. For example, describing a transformation as a rotation by a certain angle with respect to a certain axis is easier to get an idea of the overall behavior of the transformation than describing it as a combination of a translation and a rotation. However, this depends on application and context. Describing such a transformation for an object tends to make more sense in terms of rotation about an axis through the center of that object, combined with a translation, rather than by just a rotation with respect to some distant point. As an example: "move 200 m north and rotate 90° anti-clockwise", rather than the equivalent "with respect to the point 141 m to the northwest, rotate 90° anti-clockwise".

Affine transformations in 2D without fixed point (so where A has eigenvalue 1) are:

• pure translations
• scaling in a given direction, with respect to a line in another direction (not necessarily perpendicular), combined with translation that is not purely in the direction of scaling; the scale factor is the other eigenvalue; taking "scaling" in a generalized sense it includes the cases that the scale factor is zero (projection) and negative; the latter includes reflection, and combined with translation it includes glide reflection.
• shear combined with translation that is not purely in the direction of the shear (there is no other eigenvalue than 1; it has algebraic multiplicity 2, but geometric multiplicity 1)

## Affine transformation of the plane

To visualise the general affine transformation of the Euclidean plane, take labelled parallelograms ABCD and A′B′C′D′. Whatever the choices of points, there is an affine transformation T of the plane taking A to A′, and each vertex similarly. Supposing we exclude the degenerate case where ABCD has zero area, there is a unique such affine transformation T. Drawing out a whole grid of parallelograms based on ABCD, the image T(P) of any point P is determined by noting that T(A) = A′, T applied to the line segment AB is A′B′, T applied to the line segment AC is A′C′, and T respects scalar multiples of vectors based at A. [If A, E, F are collinear then the ratio length(AF)/length(AE) is equal to length(AF′)/length(AE′).] Geometrically T transforms the grid based on ABCD to that based in A′B′C′D′.

Affine transformations don't respect lengths or angles; they multiply area by a constant factor

area of A′B′C′D′ / area of ABCD.

A given T may either be direct (respect orientation), or indirect (reverse orientation), and this may be determined by its effect on signed areas (as defined, for example, by the cross product of vectors).

## Examples of affine transformations

### Affine transformation over a finite field

The following equation expresses an affine transformation in GF(28) (with "+" representing XOR):

$\{\,a'\,\} = M\{\,a\,\} + \{\,v\,\},$

where [M] is the matrix

$\begin{bmatrix} 1&0&0&0&1&1&1&1 \\ 1&1&0&0&0&1&1&1 \\ 1&1&1&0&0&0&1&1 \\ 1&1&1&1&0&0&0&1 \\ 1&1&1&1&1&0&0&0 \\ 0&1&1&1&1&1&0&0 \\ 0&0&1&1&1&1&1&0 \\ 0&0&0&1&1&1&1&1 \end{bmatrix}$

and {v} is the vector

$\begin{bmatrix} 1 \\ 1 \\ 0 \\ 0 \\ 0 \\ 1 \\ 1 \\ 0 \end{bmatrix}.$

For instance, the affine transformation of the element {a} = y7 + y6 + y3 + y = {11001010} in big-endian binary notation = {CA} in big-endian hexadecimal notation, is calculated as follows:

$a_0' = a_0 \oplus a_4 \oplus a_5 \oplus a_6 \oplus a_7 \oplus 1 = 0 \oplus 0 \oplus 0 \oplus 1 \oplus 1 \oplus 1 = 1$
$a_1' = a_0 \oplus a_1 \oplus a_5 \oplus a_6 \oplus a_7 \oplus 1 = 0 \oplus 1 \oplus 0 \oplus 1 \oplus 1 \oplus 1 = 0$
$a_2' = a_0 \oplus a_1 \oplus a_2 \oplus a_6 \oplus a_7 \oplus 0 = 0 \oplus 1 \oplus 0 \oplus 1 \oplus 1 \oplus 0 = 1$
$a_3' = a_0 \oplus a_1 \oplus a_2 \oplus a_3 \oplus a_7 \oplus 0 = 0 \oplus 1 \oplus 0 \oplus 1 \oplus 1 \oplus 0 = 1$
$a_4' = a_0 \oplus a_1 \oplus a_2 \oplus a_3 \oplus a_4 \oplus 0 = 0 \oplus 1 \oplus 0 \oplus 1 \oplus 0 \oplus 0 = 0$
$a_5' = a_1 \oplus a_2 \oplus a_3 \oplus a_4 \oplus a_5 \oplus 1 = 1 \oplus 0 \oplus 1 \oplus 0 \oplus 0 \oplus 1 = 1$
$a_6' = a_2 \oplus a_3 \oplus a_4 \oplus a_5 \oplus a_6 \oplus 1 = 0 \oplus 1 \oplus 0 \oplus 0 \oplus 1 \oplus 1 = 1$
$a_7' = a_3 \oplus a_4 \oplus a_5 \oplus a_6 \oplus a_7 \oplus 0 = 1 \oplus 0 \oplus 0 \oplus 1 \oplus 1 \oplus 0 = 1.$

Thus, {a′} = y7 + y6 + y5 + y3 + y2 + 1 = {11101101} = {ED}.

### Affine transformation in plane geometry

A simple affine transformation on the real plane

In ℝ2, the transformation shown at right is accomplished using the map given by:

$\begin{bmatrix} x \\ y\end{bmatrix} \mapsto \begin{bmatrix} 0&1\\ 2&1 \end{bmatrix}\begin{bmatrix} x \\ y\end{bmatrix} + \begin{bmatrix} -100 \\ -100\end{bmatrix}$

Transforming the three corner points of the original triangle (in red) gives three new points which form the new triangle (in blue). This transformation skews and translates the original triangle.

In fact, all triangles are related to one another by affine transformations. This is also true for all parallelograms, but not for all quadrilaterals.

## Notes

1. ^ Berger, Marcel (1987), p. 38
2. ^ Schneider, Philip K. & Eberly, David H. (2003). Geometric Tools for Computer Graphics. Morgan Kaufmann. p. 98. ISBN 978-1-55860-594-7.
3. ^ Oswald Veblen (1918) Projective Geometry, volume 2, page 105–7

## References

Publicité ▼

Contenu de sensagent

• définitions
• synonymes
• antonymes
• encyclopédie

• definition
• synonym

Publicité ▼

dictionnaire et traducteur pour sites web

Alexandria

Une fenêtre (pop-into) d'information (contenu principal de Sensagent) est invoquée un double-clic sur n'importe quel mot de votre page web. LA fenêtre fournit des explications et des traductions contextuelles, c'est-à-dire sans obliger votre visiteur à quitter votre page web !

Essayer ici, télécharger le code;

Solution commerce électronique

Augmenter le contenu de votre site

Ajouter de nouveaux contenus Add à votre site depuis Sensagent par XML.

Parcourir les produits et les annonces

Obtenir des informations en XML pour filtrer le meilleur contenu.

Indexer des images et définir des méta-données

Fixer la signification de chaque méta-donnée (multilingue).

Renseignements suite à un email de description de votre projet.

Jeux de lettres

Les jeux de lettre français sont :
○   Anagrammes
○   jokers, mots-croisés
○   Lettris
○   Boggle.

Lettris

Lettris est un jeu de lettres gravitationnelles proche de Tetris. Chaque lettre qui apparaît descend ; il faut placer les lettres de telle manière que des mots se forment (gauche, droit, haut et bas) et que de la place soit libérée.

boggle

Il s'agit en 3 minutes de trouver le plus grand nombre de mots possibles de trois lettres et plus dans une grille de 16 lettres. Il est aussi possible de jouer avec la grille de 25 cases. Les lettres doivent être adjacentes et les mots les plus longs sont les meilleurs. Participer au concours et enregistrer votre nom dans la liste de meilleurs joueurs ! Jouer

Dictionnaire de la langue française
Principales Références

La plupart des définitions du français sont proposées par SenseGates et comportent un approfondissement avec Littré et plusieurs auteurs techniques spécialisés.
Le dictionnaire des synonymes est surtout dérivé du dictionnaire intégral (TID).
L'encyclopédie française bénéficie de la licence Wikipedia (GNU).

Changer la langue cible pour obtenir des traductions.
Astuce: parcourir les champs sémantiques du dictionnaire analogique en plusieurs langues pour mieux apprendre avec sensagent.

7443 visiteurs en ligne

calculé en 0,109s

Je voudrais signaler :
section :
une faute d'orthographe ou de grammaire
un contenu abusif (raciste, pornographique, diffamatoire)


text:
# Weighted $l_1$ distance

So there are many well known algorithms for approximate nearest neighbor on the $\ell_1$ distance. My question is, what about the weighted version of the problem (where the weights are specified along with each query). Is there anything known in the research literature already, or is it a trivial extension of established techniques?

More rigorously, the classical nearest neighbor problem is given point set $P$ and query point $q$, find $\text{argmin}_{p \in P} \sum_i | p_i - q_i |$.

A weighted nearest neighbor problem would be, given point set $P$, query point $q$ and weight vector $w$ s.t $\sum_i w_i = 1$ and $w_i \geq 0, \forall i$., find $\text{argmin}_{p \in P} \sum_i w_i | p_i - q_i |$.

Obviously one can do a linear scan for each specified $w$ and $q$, but that seems heavily redundant. So I was just curious if this has already been solved or is a trivial reduction.

The exact version of this is generally hard, so I'd expect the approximation version of this to be more studied or known.

Although the application I have currently is a specific one, its easy to come up with scenarios where a weighted $\ell_1$ distance is reasonable. For instance, say the points correspond to sets of features and the user decides for certain queries they want to accord more importance to specific target dimensions.

Having thought about this some more, if you fixed $q$ to be $0$ and varied $w$, then this problem becomes finding the minimum dot product in $P$ with $w$. As such its at least as hard as MIPS (maximum inner product search). Whether its harder is a more open question.

• 1. Where you write $||.||$, do you really mean $|.|$? 2. Have you considered mapping $p_i \mapsto p'_i = w_i p_i$ and $q_i \mapsto q'_i = w_i q_i$ and then doing the query on $P',q'$? – D.W. May 4 '16 at 19:05
• I made the correction, that I intended $|.|$, and not $\|. \|$. Thanks for the catch! As for the solution you proposed, one could map $q_i \to q_i'$ as you suggest, but recall that $w$ is given at query time. As such you cant map $P \to P'$ the same way without doing a linear scan. – Amir May 5 '16 at 1:10


text:
# Math Help - algebra

1. ## algebra

determine the value k so that x-4 is a factor of $x^2+8x+k$
i know that answer is going o be 12, i got it by dividing x-4 into $x^2+8x+k$ but im not sure about the remainder, bcz i got k+48

$m^2-k^2+6k-9$ has a factor of (m-k+3) what is the other

2. $x^2+8x+k$ to factor this with (x-4) being a factor we would get (x-4)(x+y)= $x^2+(-4+y)x-4y$ and since the coefficients have to match up, $(-4+y)=8$ and -4y=k

From (-4+y)=8 we see y=12, so -4(12)=-48=k
and you can see that $x^2+8x-48=(x-4)(x+12)$

$m^2-k^2+6k-9=m^2-(k^2-6k+9)=m^2-(k-3)^2$ which is the difference of two perfect squares and $x^2-y^2=(x-y)(x+y)$ so $m^2-(k-3)^2=(m-(k-3))(m+(k-3))=(m-k+3)(m+k-3)$


text:
A- A+
Alt. Display

# oemof.tabular – Introducing Data Packages for Reproducible Workflows in Energy System Modeling

## Abstract

The paper describes how Data Packages can be used for creating reproducible workflows in energy system modeling. The presented concept has been implemented in the Python package oemof.tabular. The package is designed as an interface to instantiate energy system models from tabular data sources based on the oemof.solph library. To implement the data model, oemof.tabular extends the Open Energy Modelling Framework (oemof) by facada classes. The developed data model allows users to work with Data Packages and meta data information. The simplified tabular data structure can be used for large energy system models as well as in teaching environments leveraging functionalities of the already widely used oemof.solph library.

Keywords:
How to Cite: Hilpert, S., Günther, S. and Söthe, M., 2021. oemof.tabular – Introducing Data Packages for Reproducible Workflows in Energy System Modeling. Journal of Open Research Software, 9(1), p.6. DOI: http://doi.org/10.5334/jors.320
Published on 05 May 2021
Accepted on 30 Oct 2020            Submitted on 30 Jan 2020

## 1 Introduction

Analyses of future energy systems are based on tools for complex socio-techno-economic systems. The complexity of these systems increases due to the intermittent supply characteristics of renewable energies which require high temporal and spatial resolution modeling. Additionally, a higher interaction between sectors such as heat, power and transport leads to the need for comprehensive sector coupled approaches. At the same time, a trend towards open source energy (system) models can be observed in the energy system modeling research field [15], as models have been criticized for lack of transparency and reproducibility [17].

For energy system modelers, data handling including input collection, processing and result analysis is one of the most time-consuming tasks. Therefore, open source and open data modeling approaches are put forward as an argument for efficient use of resources [19]. Yet, there is no standardized or broadly used model-agnostic data container in the scientific field of energy system modeling to hold energy system related data. In most cases every software comes with its own logic relating to input-data and output-data of the model. In addition, the decision about how to create the required data sets from raw data sources and the post-processing of result data is often left to the user. Due to these two reasons, re-use of data and more importantly reproducibility of model results is a challenging task, even for experienced modelers.

To improve reproducibility of model results and re-usability of existing data, the following data model description has been developed. Energy system related data is stored in the Data Package format. The complete reproducible workflow from raw-data to final results is described for this data model. The data model has been implemented in the Python package oemof.tabular [7] which is based on the Open Energy Modeling Framework (oemof). However, the concept is not restricted to this package, but can be applied with other software as well.

## 2 Background

Oemof is a powerful tool for the modeling of energy systems [8]. Functionalities range from large linear programming (sector coupled) market models [6, 23, 16] to detailed mixed integer heating system [2, 27] or battery models to assess the profitability of power plants in current and future market environments. The underlying concept and its generic implementation allows for this versatile application. It is based on a bipartite graph structure, where nodes are partitioned into buses and components. Most oemof components are of a rather abstract type. For example the Transformer class can be used to model different energy system components such as power plants (1 input, 1 output) as well as a heat pump (2 inputs, 1 output) or any other conversion process. To illustrate the concept, Figure 1 shows a Transformer connected to different buses (1 input, 2 outputs) to model a combined heat and power (CHP) plant.

Figure 1

Illustration of a CHP plant model based on the oemof.solphTransformer class. Nodes are shown as ellipses/squares and edges between the nodes are depicted as arrows.

The usage of the Python API for this component in oemof.solph is shown in Figure 2.

Figure 2

Example of the oemof.solph Application Programming Interface (API) for a transformer component with one input and two outputs.

When building energy system models, data is often stored in a tabular data format, for example, CSV files, Excel files or relational databases. However, the design and implementation of a generic tabular data input processing tool for oemof.solph has proven to be difficult. One of the reasons is that tables are a flat and two dimensional data structure whereas oemof.solph’s API utilizes a high degree of nested objects and data structures. Mapping this nested approach onto flat tables is a non-trivial task.

Facilitating the task outlined above is one of the functions oemof.tabular needs to accomplish. The package was developed in order to allow the user to create an oemof model via tabular data sources. This means that it must also enable her to specify oemof.solph components using such flat structures. In oemof.tabular, this is done by using the façade design pattern first introduced introduced by Gamma et al. in 1994 [5]. The facade design pattern has two main purposes: 1. it provides a simple interface for users to access functionalities of a complex subsystem and 2. it loosens the coupling of consumers of the subsystem’s interface with the components of the subsystem.

Therefore, facade classes have the following advantages when viewed in the context of oemof.tabular:

• Facades allow instantiation of models from two dimensional data sources as they provide a simplified interface to complex underlying structures.
• The simplified and thus restricted and less generic mathematical representation leads a more transparent modeling approach.
• The simplified interface is easy to use and integrate within the context of teaching and capacity building.
• It also allows building an interface for composed components which are constructed using multiple oemof.solph objects.
• Facades can be used with a different back-end, which allows the integration of other energy system modeling frameworks which may not even have to be written in the same programming language.

The implementation

A user of oemof and oemof.solph is expected to use instances of classes from a particular class hierarchy to build a model. Thus, facades are integrated into this class hierarchy as a mix-in class: a facade to a specific oemof.solph class is created by sub-classing it, mixing in the Facade class via cooperative multiple inheritance and then using the general facade methods to simplify the interface of the original oemof.solph class.

This allows for a two step approach to build complex components out of simple ones. One can first aggregate a complex subsystem using composition, without having to think about simplifying the interface of that system. Simplifying the interface can be done in a second step by creating a facade via inheritance.

The oemof.tabular package not only provides the facade infrastructure, but also implements a number of facades to regular simple oemof.solph components as well as complex compositions of oemof.solph components.

Since facades are integrated into oemof.solph’s class hierarchy, the classes of all oemof.tabular components are sub-classes of oemof.solph components, which means that they can freely be mixed with all their more generic parent class objects in a model. In addition, the data model is extendable and could be applied for various model generators, like for example PyPSA [13] or calliope [21]. However, currently the implementation for reading Data Packages is limited to oemof.tabular classes. The facade concept as used in oemof.tabular has also proved it’s applicability by being transported to and used in the oemof.thermal [14] package.

The issue of transparency

Model generators such as oemof.solph can indeed simplify energy system modeling. However, it is noteworthy that this is a double-edged approach. Simplification for the user always comes with drawbacks as the complexity remains hidden from the user. Depending on the parameters provided, different sets of constraints are created. Nonetheless, resulting mathematical equations are not visible at any stage of the modeling process. Therefore, approaches like OSeMOSYS [11] can have a higher level of transparency than other object oriented model generators. As such models or model generators are implemented in a pure algebraic modeling language, every part of the model (variable definitions, constraints, etc.) is clearly and transparently detectable in the source code files. In the case of facades in oemof.tabular, mathematical relations of the models and their implementation are hidden by an additional layer of classes. However, since the oemof.tabular API is less generic and more restricted than the oemof.solph API, the additional layer may actually increase transparency compared to oemof.solph components by creating a clear link between input-parameters and the resulting mathematical model.

### Facade Example: Hydro Reservoir Modeling

To illustrate the facade concept, subsequently an oemof.tabular storage example is compared to the classical oemof.solph approach. The oemof.solph package provides a GenericStorage class to model different storages such as batteries, hot water storages or pumped hydro-electric storages. To model reservoir storages with an inflow and possible spillage, a set of connected oemof.solph components is required. To simplify modeling, the Reservoir facade bundles these components and provides a high level API access to a more complex underlying model. Figure 3 provides an illustration of the Reservoir facade.

Figure 3

Illustration of a reservoir model in oemof.tabular.

The facade class itself is a subclass of the GenericStorage. However, to allow for a constant inflow into the storage, an additional Source object is created.

The reservoir is modeled as a storage with a constant inflow (x denote endogenous variables, c denote exogenous variables):

(1)
${x}^{\mathrm{level}}\left(t\right)={x}^{\mathrm{level}}\left(t-1\right)\cdot \left(1-{c}^{\mathrm{loss}_\mathrm{rate}}\left(t\right)\right)+{x}^{\mathrm{profile}}\left(t\right)-\frac{{x}^{\mathrm{flow},\mathrm{out}}\left(t\right)}{{c}^{\mathrm{efficiency}}\left(t\right)} \forall t\in T$
(2)
${x}^{\mathrm{level}}\left(0\right)={c}^{\mathrm{initial}_\mathrm{storage}_\mathrm{level}}\cdot {c}^{\mathrm{capacity}}$

The inflow is bounded by the exogenous inflow profile. Thus, if the inflow exceeds the maximum capacity of the storage, spillage is possible by setting xprofile(t) to lower values.

(3)
$0\le {x}^{\mathrm{profile}}\left(t\right)\le {c}^{\mathrm{profile}}\left(t\right) \forall t\in T$

The spillage of the reservoir is therefore defined by cprofile(t) – xprofile(t). Additional constraints apply which have been omitted in the description but can be retrieved from the oemof documentation.

API comparison for the reservoir example

Subsequently, in Figure 4, the Python code to instantiate this component is shown. In comparison to the oemof.tabular code, the required oemof.solph code differs significantly (see Figure 5). First of all, more objects with a nested set of objects need to be instantiated (Flows, Sources). This nested structure allows for a very flexible modeling approach. However, it creates hurdles for writing a generic data interface to instantiate all these objects, due to the large set of possible combinations. In contrast, the flat structure of the facade arguments allows for a simple interface to tabular data. One additional difference which can be observed is the (energy) specific naming of attributes, for example efficiency, compared to outflow_conversion_factor. As the Reservoir class is a subclass of the GenericStorage class, some attributes of the parent class are also available in the child class (initial_storage_level).

Figure 4

API example for an oemof.tabular reservoir facade.

Figure 5

API example for a simple oemof.solph reservoir model.

Even for comparably small systems, the example underlines the advantages of the approach.

## 4 Data Packages

A Data Package is, in its simplest form, not more than a valid JSON [4] file named “datapackage.json”. The file contains meta data about data resources which can be specified inline in the same file. For more complex cases, data resources are stored in separate files inside the directory containing the “datapackage.json” file. The contents of the mentioned JSON file are standardized via the Data Package specification [22]. An example fragment of such a datapackage.json JSON file can be seen in Figure 7. The Data Package has been extended by other standards, which further refine the format and contents of the meta data file and the resources to suit different application contexts. Examples of this are Fiscal Data Packages [25], meant to store fiscal data, as well as Tabular Data Packages [26], which refine the original Data Package [22] specification to handle table like data. The latter combines the advantages of databases and spreadsheets with the ubiquity and user-friendliness of CSV files. Tabular Data Packages allow storing type meta data and set primary keys as well as foreign keys across resources, i.e. different CSV files. They are more lightweight than databases and they are both, human readable and easily processable in almost any programming language. In recent years, different European projects in the field of energy system modeling have decided to opt for Data Packages to store model relevant data [18, 10]. Using Data Packages in the correct manner also allows to adhere to the FAIR principle of data handling proposed by Wilkonsen et al. 2016 [29].

In the context of oemof.tabular, Data Packages are used to hold information on the topology and parameters of an energy system model instance. At a minimum this includes all exogenous model variables and associated meta data. However, it may also include raw data and scripts for pre- and post processing. On top of the Tabular Data Package structure an structure an energy system specific logic is added, which adds minimal additional constraints on the format of Tabular Data Packages used to specify an oemof.tabular model, while still keeping them valid Tabular Data Packages according to the original specification. Therefore, oemof.tabular requires the following parts in a Tabular Data Package:

1. a directory named data containing at least one sub-folder called elements, which may optionally contain a directory called sequences and a directory called geometries and
2. a valid meta-data .json file for the Data Package.

The exemplary folder tree of such a Data Package is depicted in Figure 6.

Figure 6

Example of an oemof.tabular Data Package folder tree.

Figure 7

Setting foreign keys in the JSON meta data file for cross referencing connected components.

As stated above, data inside Data Packages is stored in so called resources, which, for a Tabular Data Package, are CSV files. The columns of such resources are referred to as fields. Therefore, field names of the resources are equivalent to parameters of the energy system elements and sequences. Connections between components and buses can be defined via foreign keys. These allow linking element fields to fields of other elements stored in other resources. To reference the name field of a resource with name bus a foreign key can be set within the JSON meta data file using the forgeinKeys key as shown in Figure 7.

To distinguish elements and sequences, these two are stored in sub-directories of the data directory. In addition geometrical information can be stored under data/geometries in a .geojson format. To facilitate the process of creating, processing and calculating a Data Package, oemof.tabular offers several functionalities:

• oemof.tabular.datapackage.building contains functions to infer meta data, download raw data, read and write elements, sequences etc.
• oemof.tabular.datapackage.processing contains functions to process model results, which can be used in the compute.py script.
• oemof.tabular.datapackage.aggregation allows to aggregate time series to reduce model complexity.

## 5 Reproducible Workflows

Reproducibility of results is a recurring point of discussions in the energy system modeling community [17, 20]. These discussions have mainly been centered around the availability of source code (open source) and data (open data). Historically, for many prominent models neither the source code nor all input data have been made available. Thanks to new open source developments [8, 11, 13, 21] this has partly changed in recent years (for example the open release of MESSAGEix [12]). However, not all barriers have been removed yet. Firstly, closed models are still being used for research purposes. Secondly, more subtle barriers exist even for open source models. For one of the first open source models, Balmoral [28], a GAMS software license is required, which constitutes a barrier to re-run computations. Another important issue is what can be described as the difference between practical and theoretical transparency. While for open source models with open data theoretical reproducibility should be possible, practical issues hamper such exercises. First of all, not all necessary information may be given by the respective authors. If provided, complexity of model environments with poor documentation can make any attempt time consuming. In these cases, reproducibility is hardly possible from a practical point of view, even for experienced researchers with domain-specific knowledge.

### Workflow description

To improve reproducibilty of oemof.tabular-based research, a structure and workflow is proposed which is based on a the set of ten rules for reproducibility in computational research presented by Sandve et al. 2013 [24]:

1. For every result, keep track of how it was produced
2. Avoid manual data manipulation steps
3. Archive the exact versions of all external programs used
4. Version control all custom scripts
5. Record all intermediate results, when possible in standardized formats
6. For analyses that include randomness, note underlying random seeds
7. Always store raw data behind plots
8. Generate hierarchical analysis output, allowing layers of increasing detail to be inspected
9. Connect textual statements to underlying results

The starting point of this workflow is the folder structure shown in Figure 8.

Figure 8

Folder structure for a repository suitable for reproducible workflows.

1. Everything in the repository is (if possible) generated by scripts, version controlled, and documented to keep track of every step in result production and avoid manual data manipulation (rule 1, 2). Obviously, the repository is made publicly available (rule 10).
2. The raw-data directory contains all input data required to build the input Data Packages for the model. Ideally, raw data sources come with meta data information and open licenses. Unfortunately not all data published comes with such information which hinders reproducibility of workflows. Raw data can also be bundled on remote persistent storages like Zenodo [1], which are suitable for FAIR data distribution.
3. The scenarios directory allows to specify different scenarios and describes them in a basic way. The TOML format provides an easy and, if necessary nested structure. In addition to a description, configuration settings for constructing the input Data Packages can be specified in these files. Figure 9 provides an example for a scenario file in the TOML format. This file can be used in the scripts to build Data Packages. Note that the user-specific build-scripts will need to interpret keys and values. Therefore, scenario files in the TOML format do not follow a specific standardized structure, except using the TOML language.
4. The scripts directory contains code to construct input Data Packages for scenarios based on the configuration .toml files and the raw-data (rule 2). In addition, a script to compute the scenario(s) can be stored there. If possible, raw data can also be downloaded from persistent sources (for example Zenodo) using scripts. Finally, this directory would also contain code for post processing data and for result visualization (rule 7).
5. Results are stored in the results directory. One important part is the separation of input and output data. Input data contains model specific exogenous model variables (in this context, oemof.tabular Data Packages). The output data directory contains endogenous model variables. Altogether, this step acknowledges rule 5 and 10 of the ten rules.
6. The open license and environment definition in combination with a version control system such as git allows to reproduce results on different operating systems (rule 3, 4 and 10).
Figure 9

Example TOML file with scenario specifications to build input Data Packages.

An example of this workflow has been published for a model-based analysis of the German electricity system [9]. The energy system model covers the German power system with its neighboring countries. Similarly, the workflow has been applied in an analysis for flexibilisation of heat pumps [6].

It should be noted that energy modelers also need to acknowledge energy modeling specific best practices such as proposed by Decarolis et al. [3].

## 6 Conclusion

This paper introduces the application of the facade concept and the usage of Data Packages for the Open Energy System Modeling Framework (oemof). The concept has been implemented in the Python package oemof.tabular which is designed as an interface to instantiate energy system models with the oemof.solph library from Tabular Data Packages. Using facades can (1) increase transparency by restricting generic components to energy specific components, (2) allow to build composed components and instantiate those from tabular data sources, (3) facilitate the application in teaching and capacity building environments and (4) allow for reproducible workflows. Additionally, the implementation based on the Data Package standard allows to store meta data of the model input data in a standardized way. To enable reproducibility of energy research results a workflow is proposed which is based on scientific literature.

## Acknowledgements

The authors thank Clemens Wingenbach who took part in the important initial discussions on Data Packages as an input data format. In addition, the Open Modeling Initiative Workshops, with some valuable inspirations by Daniel Huppmann, have been very helpful for the development.

The work has partly been funded by the Federal Ministry of Economic Affairs and Energy, Germany (Grant Number: 03ET6122E). In addition we acknowledge financial support for the article processing charge by the state of Schleswig-Holstein, Germany within the funding program Open Access Publikationsfond.

## Competing Interests

The authors have no competing interests to declare.

## References

1. Zenodo – Research. Shared. https://zenodo.org/.

2. Boysen C, Kaldemeyer C, Hilpert S, Tuschy, I. Integration of Flow Temperatures in Unit Commitment Models of Future District Heating Systems. Energies. Mar. 2019; 12(6): 1061. DOI: https://doi.org/10.3390/en12061061

3. DeCarolis J, Daly H, Dodds P, Keppo I, Li F, McDowall W, Pye S, Strachan N, Trutnevyte E, Usher W, Winning M, Yeh S, Zeyringer M. Formalizing best practice for energy system optimization modelling. Applied Energy. May 2017; 194: 184–198. DOI: https://doi.org/10.1016/j.apenergy.2017.03.001

4. ECMA. Standard ECMA-404 The JSON Data Interchange Syntax; 2017.

5. Gamma E, Helm R, Johnson R, Vlissides J, Booch G. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Professional; 1994.

6. Hilpert S. Effects of Decentral Heat Pump Operation on Electricity Storage Requirements in Germany. Energies. June 2020; 13(11): 2878. DOI: https://doi.org/10.3390/en13112878

7. Hilpert S, Günther S, Söthe M. oemof/oemof-tabular, June 2020. original-date: 2018-11-20T16:05:32Z. https://github.com/oemof/oemof-tabular.

8. Hilpert S, Kaldemeyer C, Krien U, Günther S, Wingenbach C, Plessmann G. The Open Energy Modelling Framework (oemof) – A new approach to facilitate open science in energy system modelling. Energy Strategy Reviews. Nov. 2018; 22: 16–25. DOI: https://doi.org/10.1016/j.esr.2018.07.001

9. Hilpert S, Söthe M, Wingenbach C. ANGUSII Scenarios: German Energy System 2030. Zenodo, July 2019. DOI: https://doi.org/10.5281/zenodo.3714708

10. Hotmaps. Hotmaps Toolbox; 2019. https://www.hotmaps-project.eu/.

11. Howells M, Rogner H, Strachan N, Heaps C, Huntington H, Kypreos S, Hughes A, Silveira S, DeCarolis J, Bazillian M, Roehrl A. Osemosys: The open source energy modeling system: An introduction to its ethos, structure and development. Energy Policy. 2011; 39(10): 5850–5870. Sustainability of biofuels. DOI: https://doi.org/10.1016/j.enpol.2011.06.033

12. Huppmann D, Gidden M, Fricko O, Kolp P, Orthofer C, Pimmer M, Kushin N, Vinca A, Mastrucci A, Riahi K, Krey V. The MESSAGEix Integrated Assessment Model and the ix modeling platform (ixmp): An open framework for integrated and cross-cutting analysis of energy, climate, the environment, and sustainable development. Environmental Modelling & Software. Feb. 2019; 112: 143–156. DOI: https://doi.org/10.1016/j.envsoft.2018.11.012

13. Hörsch J, Hofmann F, Schlachtberger D, Brown T. PyPSA-Eur: An open optimisation model of the European transmission system. Energy Strategy Reviews. Nov. 2018; 22: 207–215. DOI: https://doi.org/10.1016/j.esr.2018.08.012

14. jnnr, FranziPl, Möller C, jakob wo, MaGering, Schönfeldt P, Krien U, Kaldemeyer C, Günther S. oemof/oemof-thermal: Amazing Absorption (v 0.0.3). July 2020. DOI: https://doi.org/10.5281/zenodo.3929692

15. Lopion P, Markewitz P, Robinius M, Stolten D. A review of current challenges and trends in energy systems modeling. Renewable and Sustainable Energy Reviews. Nov. 2018; 96: 156–166. DOI: https://doi.org/10.1016/j.rser.2018.07.045

16. Maruf MNI. Sector Coupling in the North Sea Region—A Review on the Energy System Modelling Perspective. Energies. Nov. 2019; 12(22): 4298. DOI: https://doi.org/10.3390/en12224298

17. Morrison R. Energy system modeling: Public transparency, scientific reproducibility, and open development. Energy Strategy Reviews. Apr. 2018; 20: 49–63. DOI: https://doi.org/10.1016/j.esr.2017.12.010

18. OPSD. Open Power System Data – A platform for open data of the European power system. 2019. https://open-power-system-data.org/.

19. Pfenninger S, DeCarolis J, Hirth L, Quoilin S, Staffell I. The importance of open data and software: Is energy research lagging behind? Energy Policy. Feb. 2017; 101: 211–215. DOI: https://doi.org/10.1016/j.enpol.2016.11.046

20. Pfenninger S, Hawkes A, Keirstead J. Energy systems modeling for twenty-first century energy challenges. Renewable and Sustainable Energy Reviews. 2014; 33: 74–86. DOI: https://doi.org/10.1016/j.rser.2014.02.003

21. Pfenninger S, Pickering B. Calliope: a multi-scale energy systems modelling framework. Journal of Open Source Software. Sept. 2018; 3(29): 825. DOI: https://doi.org/10.21105/joss.00825

22. Pollock R, Walsh P. Data Package; 2007. https://specs.frictionlessdata.io/data-package/.

23. Prina MG, Casalicchio V, Kaldemeyer C, Manzolini G, Moser D, Wanitschke A, Sparber W. Multi-objective investment optimization for energy system models in high temporal and spatial resolution. Applied Energy. Apr. 2020; 264: 114728. DOI: https://doi.org/10.1016/j.apenergy.2020.114728

24. Sandve GK, Nekrutenko A, Taylor J, Hovig E. Ten Simple Rules for Reproducible Computational Research. PLOS Computational Biology. Oct. 2013; 9(10): e1003285. DOI: https://doi.org/10.1371/journal.pcbi.1003285

25. Walsh P, Pollock R, Björgvinsson T, Bennett S, Kariv A, Fowler D. Fiscal Data Package; 2014. https://specs.frictionlessdata.io/fiscal-data-package/.

26. Wash P, Pollock R, Keegan M. Tabular Data Package; 2012. https://specs.frictionlessdata.io/tabular-data-package/.

27. Wehkamp S, Schmeling L, Vorspel L, Roelcke F, Windmeier K-L. District Energy Systems: Challenges and New Tools for Planning and Evaluation. Energies. June 2020; 13(11): 2967. DOI: https://doi.org/10.3390/en13112967

28. Wiese F, Bramstoft R, Koduvere H, Pizarro Alonso A, Balyk O, Kirkerud JG, Tveten AG, Bolkesjö TF, Münster M, Ravn H. Balmorel open source energy system model. Energy Strategy Reviews. Apr. 2018; 20: 26–34. DOI: https://doi.org/10.1016/j.esr.2018.01.003

29. Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, Blomberg N, Boiten J-W, da Silva Santos LB, Bourne PE, Bouwman J, Brookes AJ, Clark T, Crosas M, Dillo I, Dumon O, Edmunds S, Evelo CT, Finkers R, Gonzalez-Beltran A, Gray AJ, Groth P, Goble C, Grethe JS, Heringa J, ’t Hoen PA, Hooft R, Kuhn T, Kok R, Kok J, Lusher SJ, Martone ME, Mons A, Packer AL, Persson B, Rocca-Serra P, Roos M, van Schaik R, Sansone S-A, Schultes E, Sengstag T, Slater T, Strawn G, Swertz MA, Thompson M, van der Lei J, van Mulligen E, Velterop J, Waagmeester A, Wittenburg P, Wolstencroft K, Zhao J, Mons B. The FAIR Guiding Principles for scientific data management and stewardship. Mar. 2016; 3: 160018. DOI: https://doi.org/10.1038/sdata.2016.18