text:
Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

# Extracting bulk defect parameters in silicon wafers using machine learning models

## Abstract

The performance of high-efficiency silicon solar cells is limited by the presence of bulk defects. Identification of these defects has the potential to improve cell performance and reliability. The impact of bulk defects on minority carrier lifetime is commonly measured using temperature- and injection-dependent lifetime spectroscopy and the defect parameters, such as its energy level and capture cross-section ratio, are usually extracted by fitting the Shockley-Read-Hall equation. We propose an alternative extraction approach by using machine learning trained on more than a million simulated lifetime curves, achieving coefficient of determinations between the true and predicted values of the defect parameters above 99%. In particular, random forest regressors, show that defect energy levels can be predicted with a high precision of ±0.02 eV, 87% of the time. The traditional approach of fitting to the Shockley-Read-Hall equation usually yields two sets of defect parameters, one in each half bandgap. The machine learning model is trained to predict the half bandgap location of the energy level, and successfully overcome the traditional approach’s limitation. The proposed approach is validated using experimental measurements, where the machine learning predicts defect energy level and capture cross-section ratio within the uncertainty range of the traditional fitting method. The successful application of machine learning in the context of bulk defect parameter extraction paves the way to more complex data-driven physical models which have the potential to overcome the limitation of traditional approaches and can be applied to other materials such as perovskite and thin film.

## Introduction

In the wake of the climate crisis, transitioning to a carbon-free society is crucial in creating a sustainable future. In their 2018 report, the Intergovernmental Panel on Climate Change (IPCC) explores sustainable scenarios with PV systems producing up to a third of the planet’s renewable energy1. Today’s most prominent PV technology is silicon (Si) based solar cells2. Improving the efficiency and reliability of Si PV modules and solar cells, as well as reducing the cost of this technology, is paramount in meeting the IPCC targets3. Research focused toward higher efficiency Si solar cells has shone light on Si bulk defects as a key contributor to their total efficiency loss4. It is therefore primordial to identify and characterize bulk defects in order to eliminate or at least reduce their impact. Techniques such as temperature- and injection-dependent lifetimes spectroscopy (TIDLS)5 enables characterization of a wide range of Si bulk defects, such as copper6, aluminium7,8, chromium9,10, iron11,12 and many more13,14,15,16,17. To identify defects, the Shockley-Read-Hall (SRH) equation18,19 that describes the impact of a defect on the overall minority carrier lifetime is often used to fit the lifetime measurements20. Three defect parameters are defined by the SRH equation to represent a defect: its energy level Et and its electron and hole capture cross-sections, σn and σp, respectively. The state of the art fitting method uses Rein’s defect parameter solution surface (DPSS)20 which determines, for each injection-dependent measurement at a fixed temperature, a set of solutions to the SRH equation20. By continuously varying Et, the best fit for other parameters, or combinations of parameters, can be found, creating a continuous surface in the parameter space which represents the entirety of the mathematical solutions for that fit. By repeating the process for each temperature-dependent measurement, different DPSS curves are determined. These curves usually intersect at two points that indicate the possible combinations of the defect parameters, with one set of parameters in the upper bandgap and one in the lower bandgap7,8,9,10,11,12,13,21. This method, henceforth referred to as the ‘traditional approach’, has been refined over the years, such as linearizing the SRH equation under specific conditions14,22 or applying faster convergence techniques to find the intersection points21. However, the technique and its derivatives assume that the defect parameters are independent of temperature and that the defect possesses only one energy level in the Si’s bandgap7,9. In addition, in practice, it fails to identify which of the two mathematical solutions is the correct one.

To overcome these limitations, this study proposes a machine learning (ML) based approach to extract the defect parameters from lifetime curves. ML-based methods are already used at the PV system level, for example for fault detection23,24 or to identify cracks in modules using luminescence imaging techniques25,26,27,28. ML has also been used in non-Si applications to find relevant material parameters for fabrication of CIGS solar cells29, multi-junction solar cells30, organic solar cells31, or perovskite solar cells32. More recently, Kurchin et al.33 proposed a Bayesian-based model to predict the probability distribution of the defect parameters from temperature dependent current-voltage measurements. The model is designed to fit a single combination of defect parameters and needs to be retrained for different measurement scenario, such as iron or copper in silicon. Furthermore, it seems that the use of current-voltage measurements to extract the defect parameters can be influenced by other solar cells parameters, such as edge recombination and shunt resistance, that can mask the impact of the studied defect. This study’s approach brings the technological prowess of ML to defect parameters extraction in order to propose an alternative way to solve this decades-old challenge. Multiple ML training algorithms are considered, and compared, to predict the defect parameters directly from TIDLS measurements with no knowledge of the SRH equation. The developed method is not limited to silicon and can be used for non-Si PV materials, such as perovskites or CIGS.

## Results

### Defect parameter regression

Five iterations of training for each of the ML training algorithms are performed. Each training uses different random model initialization and random training/validation dataset split to ensure the repeatability of the ML algorithm’s predictive power. For neural networks (NN), the evaluation data is held out until the end of the epoch-based training. In Fig. 1, the results of each training’s coefficient of determination (R2) score are presented for the different ML models for both Et (a) and k (b) predictions. The models have been ordered from the best performers to the worst and the average and standard deviation R2 are given for each model and regression target. In the figure, the statistical box plot for each model is shown in green, while the black dots represent each individual training iteration.

Apart from the support vector machine (SVM) models – expected to perform better when the datasets are small and sparse34—and one instance of NN, all trained ML models have am R2 above 95% on the validation dataset, showing the feasibility of using ML as an inverse function, with no a priori knowledge of the SRH equation. The highest average R2 (>99%) is achieved by the random forest (RF) model for both k and Et prediction. From Fig. 1a, b, it can be concluded that the decision tree methods, such as RF, adaptive boosting, and gradient boosting achieve the highest average R2 score in predicting both Et and k, followed by NN and SVM. NN have a higher variance than the other models, due to its sensitivity to the random initial weights and the random order in which the data is processed during training35. Indeed, the training of NN relies on backpropagation steps that shift the weights of the network proportional to the error of the data points that it is trained to predict in order to minimize that error36. Changing the order in which the NN evaluates the data shifts the networks to different local minima because of these initial larger weight shifts.

Figure 2 shows the predicted parameters versus the true parameters for the best performing RF model using the validation dataset. Both (a) Et and (b) k predictions have R2 scores above 99%. In each graph, the data points are semi-transparent, and the darker area outlines the linearity of the correlation between predicted and true values, as most data points lie on the y = x line. The RF model correctly predicts Et across the entire range 87% of the time at ±0.02 eV precision which is typical error bars for reported Et values (between ± 0.02 eV to ±0.04 eV)8,9,10,15,16,17. It is noticeable that far from the mid-gap (|Et | > 0.1 eV), the correct predication is much higher (91%) for this precision level. Close to the mid-gap (|Et | < 0.1 eV), the prediction of Et at ±0.02 eV correctness drops to 75%. This can be explained by examining the SRH equation [Eq. (2)] when Et is close to 0 eV. The dependency on Et comes from the n1 and p1 terms [Eq. (4)]. When Et approaches 0 eV, both n1 and p1 terms becomes negligible compare to n0 + Δn and p0 + Δn, respectively. Thus, the resulting SRH lifetime has a weak dependence on Et, increasing the difficulty of both human and ML prediction near the mid-gap.

To validate this explanation, a simulation of 200 defects with a fixed k = 100 and Et spanning from −0.1 eV to 0.1 eV is performed. Figure 3 presents the lifetime curves of these defects at 400 K. It can be observed that the defects are indistinguishable from one another on a macro scale. The insert is a zoom-in of the ensemble of lifetime curves at low injection level and it reveals that this ensemble of curves is 50 ns wide. Whereas it is extremely difficult to distinguish those lifetime curves with the naked eye, the RF model still correctly predicted 75% of the defects with ±0.02 eV accuracy.

### Defect bandgap location classification

In the traditional DPSS approach two possible solutions are often identified, one in the lower bandgap and one in the upper bandgap7,8,9,21. In certain cases, the half bandgap location can be determined using additional measurements, such as deep level transient spectroscopy17. To address this limitation, ML models are trained to classify in which half bandgap the defect is located. The models have been ordered from the best performers to the worst one in Fig. 4a, and the average and standard deviation of each model’s accuracy. The statistical box plot for each model is shown in green, while the black dots represent each individual training iteration. NN significantly outperforms the other models, reaching an average accuracy of 82.7%. It is worth noting that all the other algorithms produce models above 75% accuracy proving the possibility of identifying the half bandgap’s location with TIDLS measurements. Despite NN’s good performance, the variation in the accuracy score between trained models shows the same behaviour than in the regression’s case of sensitivity to the first few steps of backpropagation (as shown by the standard deviation of this model). From Fig. 4a, it can also be observed that the RF models are significantly better than its boosting counterparts. Boosting techniques add error scoring methods during the training of the different decision trees of the RF37. The difference in performance between boosting and RF is due to the overfitting of the training data by the boosting models38. The best trained NN has an accuracy of 87.7% on the validation dataset and its confusion matrix is shown in Fig. 4b. The confusion matrix is symmetric, as expected, showing that both half bandgaps are treated equally.

To provide more insight into the 12.4% incorrectly predicted half bandgap locations, the corresponding defects are plotted in Fig. 5, as a k vs Et plot. Each data point, representing a single defect parameter combination fed into the NN model, is coloured based on the NN’s classification probability towards their true half bandgap location. The probabilities are obtained directly from the output layer of the NN, which are known to be well-calibrated39. The grey datapoints corresponds to correctly predicted defect half bandgap location while coloured datapoints to incorrectly predicted defect half bandgap location. The colour scale represents the correct label probability, and for incorrectly labelled datapoints, is necessarily between 0 and 0.5. The darker the datapoints, the more inaccurate the classification by the NN is.

The data points aggregate around two trends. For defects with |Et| < 0.1 eV, the classification accuracy drops to 68.7%, which still provides a decent indication regarding the half bandgap location without the need for additional measurements. As previously discussed for Fig. 2, defects with Et near the mid-gap are harder to differentiate for both human and ML. Outside the mid-gap range (|Et| > 0.1 eV), the NN classification model performs extremely well, with a 94.7% half bandgap location classification accuracy. The second trend is that outside the mid-gap region, the most incorrect classification is for defects with k values around 0.83 ± 0.01, represented by the dotted black line in Fig. 5. This corresponds to the ratio between hole and electron thermal velocity. Indeed, when the value k approaches vp/vn, it equalizes τn0 and τp0, resulting in the following simplified SRH equation:

$$\tau _{{\mathrm{SRH}}} = \tau _{{\mathrm{n}}0} \cdot \left( {1 + \frac{{p_1 + n_1 + {\mathrm{{\Delta}}}n}}{{p_0 + n_0 + {\mathrm{{\Delta}}}n}}} \right)$$
(1)

Since n1 and p1 have opposite Et dependence, the SRH equation in the case k = vp/vn is symmetric in respect to Et. This means that for a fixed k value, satisfying k = vp/vn, or equivalently σnp = vp/vn, the lifetime curves for any given Et are identical to the lifetime curves of the opposite energy level −Et. For this reason, it is difficult to pinpoint the exact half bandgap location of defects when the value of k : = σn/σp is close to vp/vn, as shown in Fig. 5 where the prediction probability tends to 50% for each half. However, the regression targets k and Et are not compromised by that regime, as seen in Fig. 2.

### Verification

A comparison between DPSS and the best-trained ML models is given in Fig. 6. The SRH lifetime curves at various temperatures for a selected defect (Et = 0.335 eV and k = 1.1) are presented in Fig. 6a. From these, five DPSS curves are generated following Rein’s method20 [see Fig. 6b] to identify the defect parameters. Figure 6b also includes the combined predications of Et and k by the best of the previously trained ML regressor models of each category (indicated by coloured symbols). As can be noticed, a great agreement is demonstrated between all the ML models and the DPSS method in terms of the prediction of the defect parameters. The best regressor predict Et within ±0.0003 eV and k within ±0.05 of the simulated values.

As stated previously, the DPSS approach usually provides two solutions for the defect parameter extraction, with one in the upper half bandgap and one in the lower half bandgap. To distinguish between the two solutions, the sharpness of the intersection of the DPSS is plotted in Fig. 6c as a function of Et, where the sharpness ε is defined as the standard deviation of the k-DPSS curves divided by the mean of the curves. The sharpness is only defined when all the DPSS curves exists, thus, the discontinuity in the sharpness curve. Using this approach, the minimum of the sharpness curve points to the most probable solution. In Fig. 6e, the DPSS curves sharpness clearly indicates the correct upper bandgap as the solution (also marked by the orange band). The green band represents the best ML classifier model’s classification of the defect energy level half bandgap location. The ML classifier correctly identify the half bandgap location and, in this case, aligns with the DPSS classification.

This analysis is repeated on the same defect parameters, adding random gaussian noise to the SRH lifetime curves. As shown in Fig. 6d, the noise is scaled up with decreasing excess carrier concentration to reproduce similar behaviour of actual lifetime measurements. The DPSS curves of the noisy data [Fig. 6e] shows that the ML regressor models are robust to the noise and continues to show accurate predictions of the defect parameters, with the best regressor predicting Et within ±0.004 eV and k within ±0.05. However, by adding noise to the SRH lifetime curves, the sharpness of the DPSS curves becomes ambiguous [Fig. 6f] and the DPSS approach now predicts the wrong half bandgap energy location. The ML still predicts the correct half bandgap energy location, showing the ML robustness to noise and presenting a case where the ML approach outperforms the DPSS. The comparison done in Fig. 6 is repeated 1,000 times for the same defect with random noise. The ML approach has an accuracy of 87.7% for half bandgap energy location prediction, while DPSS reaches 80.9% accuracy. When increasing the noise intensity, the accuracy of the DPSS drops to 44.7%, while the ML retains 76.4% accuracy. This is a strong demonstration that ML-based predictions are more reliable compared to DPSS-based predications when dealing with noisy signals.

The comparison between the DPSS and the ML has been repeated for 10,000 random defect parameters and set of SRH lifetime curves with random noise. In terms of classification, the DPSS approach predicts the correct half bandgap location 60% of the time, while the ML is correct 80% of the time over those 10,000 defects. Furthermore, the regression of the correct bandgap has shown an average defect energy level residual of 0.093 eV for the DPSS compared to almost of magnitude lower for the ML (0.011 eV). This is a clear indication that using the ML approach has a significant advantage over the DPSS approach in terms of its robustness to noise. This is quite remarkable considering the fact the ML classifier and regressors have been trained only on a dataset with no noise. We assume that the proven good robustness to potential measurement noise can be even further improved by training on a noisy dataset.

### Experimental application

To be able to apply the ML approach to experimental measurements, an adaptation of the simulation parameters (such as wafer type, Ndop, and temperature) to the desired measurement is necessary. Once the ML model is trained, the experimental measurements can be fed to the model in order to extract the defect parameters. This method is validated below by comparison to the traditional approach using the DPSS analysis.

For validation, an n-type Si wafer with Ndop = 5.1 × 1015 cm−3 from Zhu et al.13. is used. Lifetime measurements are done using TIDLS at eight different temperatures. The DPSS method is then used to extract Et and k and the results are compared to ML predictions in Fig. 7. Each DPSS curve represents the best fit to the SRH model at that temperature. As discussed, the defects parameters are identified by a simultaneous minimization of the fit errors at all temperatures. The two solutions found by the DPSS method are marked by the black diamond symbol. The transparent ellipse represents the error-space defined by the error bars of this method. These two solutions are (Et = −0.30 ± 0.03 eV; k = 1.87 ± 1.18) for the lower half bandgap and (Et = 0.28 ± 0.04 eV; k = 1.01 ± 0.44) for the upper half bandgap solution. The ML predictions are obtained by training an instance of different ML training algorithms compared in this study: NN, RF and its boosting counterparts. Figure 7 shows that the ensemble models—RF, adaptive, and gradient boosting—all predict defect parameters within the DPSS solutions uncertainty range for both Et and k, while NN predicts Et within the DPPS solution uncertainty range but overshoot the value of k. Furthermore, calculating the sharpness of the DPSS curves’ intersection points the half bandgap defect energy location in the lower half bandgap. In this experimental case, the ML approach concur with the DPSS results. The agreement between the ML models and the DPSS method highlights the success of this first-ever application of ML techniques for extraction of defect parameters. Based on this study, the approach can be extended to overcome the limitations of the currently used methods. For example, multi-level defects or multiple single-level defects modelling can be used to generate the training dataset and temperature-dependence of the defect parameters, such as the capture cross-sections, can be introduced into the physical models as well.

## Discussion

In this study, we proposed a ML-based approach to defect parameters extraction. By generating more than a million TIDLS lifetime curves with the SRH equation, ML models were trained and evaluated on simulated data. Overall, ensemble methods, such as RF, adaptive, and gradient boosting, seem to be the most promising algorithms to extract the defect parameters with an average R2 score of 99%, whereas NN excels at predicting the half bandgap location of the defect with an average accuracy of 82.7%, outperforming the DPSS in predicting the half bandgap location of noisy SRH lifetime curves, which showcase an important advantage of the ML approach over the traditional approach. With no knowledge of the relationships predicated by the SRH equation, ML models provide insights regarding the physical limitations, such as when the lifetime curves are indistinguishable from one another, while still performing well in those conditions. Experimental measurements validated the ML approach in this successful defect parameter extraction using ML from lifetime measurements.

As more complex models are trained, additional insights can be gained that can contribute to a better understanding of defects. More complex models can also breach the limitations of traditional approaches as more modelling possibilities could be incorporated into the dataset generation step, such as multiple single-level defects and temperature-dependent capture cross-sections. The proposed approach can be generalized to any number of temperature measurements and wafer conditions and is not limited to Si wafers. The adaptability of this method ensures its uses in very varied scenarios and transfers its potential to other measurements techniques or material measured such as thin-film or perovskites with possibly more complex ML models.

## Methods

### Dataset Simulation

The SRH18,19 equation is used to generate a dataset of simulated lifetime curves for a wide range of defect parameters (Et, σn, σp). The SRH lifetime τSRH is given by:

$$\tau _{{\mathrm{SRH}}} = \frac{{\tau _{{\mathrm{p}}0} \cdot \left( {n_0 + n_1 + {\mathrm{{\Delta}}}n} \right) + \tau _{{\mathrm{n}}0} \cdot \left( {p_0 + p_1 + {\mathrm{{\Delta}}}n} \right)}}{{n_0 + p_0 + {\mathrm{{\Delta}}}n}}$$
(2)

with

$$\tau _{{\mathrm{p}}0}: = \left( {\sigma _{\mathrm{p}} \cdot v_{\mathrm{p}} \cdot N_{\mathrm{t}}} \right)^{ - 1};\tau _{{\mathrm{n}}0}: = \left( {\sigma _{\mathrm{n}} \cdot v_{\mathrm{n}} \cdot N_{\mathrm{t}}} \right)^{ - 1}$$
(3)
$$n_1: = n_{\mathrm{i}} \cdot e^{\frac{{E_{\mathrm{t}}}}{{k_{\mathrm{b}}T}}};p_1: = n_{\mathrm{i}} \cdot e^{\frac{{-E_{\mathrm{t}}}}{{k_{\mathrm{b}}T}}}$$
(4)

In Eq. (2), n0 (p0) is the electron (hole) carrier concentration at thermal equilibrium and Δn is the excess carrier concentration. In Eq. (3), vn (vp) is the electron (hole) thermal velocity calculated using the model of Green et al.40 and Nt is the defect density. In Eq. (4), kb is Boltzmann’s constant, T is the temperature and ni is the intrinsic carrier concentration calculated using the model of Couderc et al.41, taking into account bandgap narrowing with the model of Yan et al.42. Et is defined relative to the intrinsic energy of Si Ei (Et = 0 eV is at mid-gap) and grows positive as Et gets closer to the conduction band. At this stage, it is assumed that the defect parameters (Et, σn, σp) are independent of temperature.

For each combination of defect parameters (Et, σn, σp), 500 lifetime points are simulated as a feature vector of 100 Δn points at five different temperatures. Without loss of generality, p-type wafers are chosen with a bulk doping density Ndop fixed at 1015 cm−3, while Nt is fixed at 1012 cm−3. In total, 300,000 defect parameters combinations are chosen randomly within the reasonable physical range resulting in a million and a half-simulated lifetime curves ranging from 0.1 μs to 10 ms. The ranges of the other simulation parameters are given in Table 1.

### Machine learning training

The simulated dataset is randomly split into a training dataset (90% of the simulated data) used to train the ML model and a validation dataset (10%) used to evaluate the model. In the traditional defect parameter extraction approach, such as DPSS20 and its derivatives14,21,22, two solutions for Et are usually obtained, with one in the upper half of the bandgap (Et,+) and the second in the lower half of the bandgap (Et,-). In this study, for the regression of Et, the training of the ML model is done separately for each half bandgap. The results are then combined and presented together. In order to identify in which half-bandgap the defect is located, the same approach is used to train a classifier model to predict the sign of Et. The classifier outputs the probability of the defect being in the upper or lower half bandgap. The capture cross-section regression’s target is the ratio k:= σnp, which is usually reported in defect parameters’ characterization7,9,14,21,22. For the regression of k, the training of the ML model is done on the whole bandgap. In total, four ML models are trained to inverse the SRH equation: two regression models to predict Et, one regression model to predict k; and one classification model to determine the defect energy location. The ML training flow is summarized in Fig. 8.

For each of the four ML models needed, different ML training algorithms can be used, defined in each grey box of Fig. 8. Random forests (RF)43 are an ensemble of decision trees, where each tree is built from a random subset of the training data. The RF model outputs an average decision by all its decision trees. Enhancements of this method are possible through boosting, notably adaptive boosting37,44 and gradient boosting38, which weights the decision of every tree to improve the predictive power of the overall model. A neural network (NN)36,45 is a layered group of nodes, called neurons, with a set of weights that are updated as it processes the training data. NN are widely used for different applications, such as image recognition46, text translation46 or beating Go world champions47. Another widely used algorithm is support vector machines (SVM)48,49 which consists of mapping the training datapoints to a higher dimensional feature space where the data becomes easily separable. For classification problems, the K-nearest neighbours (KNN)50 algorithm simply assigns the predicted label to be the majority vote of the closest known training data points. Figure 8 also shows that most ML training algorithms are used to train both regression and classification models, however, SVM is dedicated to regressions and KNN to classification.

In order to compare the different trained ML models, two scoring techniques are introduced. For the regression of defect parameters, predictions are made on the validation dataset and correlated to the true defect parameter values. To score the correlation, the coefficient of determination (R2) is used51:

$$R^2: = 1 - \frac{{\mathop {\sum }\nolimits_{i = 1}^N ( {y_{\mathrm{i}}^{{\mathrm{pred}}} - y_{\mathrm{i}}^{{\mathrm{true}}}} )^2}}{{\mathop {\sum }\nolimits_{i = 1}^N \left( {y_{\mathrm{i}}^{{\mathrm{true}}} - y_{{\mathrm{mean}}}^{{\mathrm{true}}}} \right)^2}};$$
(5)

where ‘pred’ (‘true) reference to the predicted (true) values of the validation dataset and yi spans the N data points of the validation dataset. The R2 score has a maximum value of 1 which indicates a perfect correlation and enables comparison between prediction targets. For the trained classification models, a confusion matrix is created, comparing predicted and true labels. The considered classification is binary, and predictions can have one of the following four outcomes: true positive (TP), true negative (TN), false negative (FN) and false positive (FP). The accuracy (‘Acc’) is a defined as52:

$${\mathrm{Acc}}: = \frac{{{\mathrm{TP}} + {\mathrm{TN}}}}{{{\mathrm{TP}} + {\mathrm{TN}} + {\mathrm{FP}} + {\mathrm{FN}}}}$$
(6)

The accuracy measures the ratio between correctly predicted points and the total number of points of the training dataset. The lifetime curves dataset simulation, ML model training and evaluation are implemented using Python and Scikit-learn53. More information about the hyper-parameter used can be found in the Supplementary Methods. The machine used for training has an Intel® Xeon® W-2145 processor, a central processing unit up to 3.70 GHz and a random-access memory of 64 GB. The training time is in the order of minutes for all the algorithms presented in this paper.

## Data availability

The authors declare that all data can be found on GitHub [https://github.com/WhyBeU/DPML]. Any additional data can be accessed upon request.

## Code availability

The authors declare that all custom scripts and python code used for this study can be found on GitHub [https://github.com/WhyBeU/DPML].

## References

1. 1.

IPCC. Global warming of 1.5 °C. An IPCC special report on the impacts of global warming of 1.5 °C above pre-industrial levels and related global greenhouse gas emission pathways, in the context of strengthening the global response to the threat of climate change, sustainable development, and efforts to eradicate poverty. In Masson-Delmotte, V. et al. (eds) (2018, In Press). https://archive.ipcc.ch/report/sr15/pdf/sr15_citation.pdf.

2. 2.

Green, M. A. Commercial progress and challenges for photovoltaics. Nat. Energy 1, 15015 (2016).

3. 3.

Needleman, D. B. et al. Economically sustainable scaling of photovoltaics to meet climate targets. Energy Environ. Sci. 9, 2122–2129 (2016).

4. 4.

Schmidt, J. et al. Impurity-related limitations of next-generation industrial silicon solar cells. In IEEE Journal of Photovoltaics vol. 3, pp. 114–118 (2013). https://doi.org/10.1109/JPHOTOV.2012.2210030.

5. 5.

Coletti, G. Sensitivity of state-of-the-art and high efficiency crystalline silicon solar cells to metal impurities. Prog. Photovolt. 21, 1163–1170 (2013).

6. 6.

Inglese, A., Lindroos, J., Vahlman, H. & Savin, H. Recombination activity of light-activated copper defects in p-type silicon studied by injection- and temperature-dependent lifetime spectroscopy. J. Appl. 120, 125703 (2016).

7. 7.

Sun, C., Rougieux, F. E., Degoulange, J., Einhaus, R. & Macdonald, D. Reassessment of the recombination properties of aluminium–oxygen complexes in n- and p-type Czochralski-grown silicon. Phys. Status Solidi B 253, 2079–2084 (2016).

8. 8.

Schmidt, J. Temperature- and injection-dependent lifetime spectroscopy for the characterization of defect centers in semiconductors. Appl. Phys. Lett. 82, 2178–2180 (2003).

9. 9.

Sun, C., Rougieux, F. E. & Macdonald, D. Reassessment of the recombination parameters of chromium in n- and p-type crystalline silicon and chromium-boron pairs in p-type crystalline silicon. J. Appl. 115, 214907 (2014).

10. 10.

Schmidt, J., Krain, R., Bothe, K., Pensl, G. & Beljakowa, S. Recombination activity of interstitial chromium and chromium-boron pairs in silicon. J. Appl. 102, 123701 (2007).

11. 11.

Birkholz, J. E., Bothe, K., Macdonald, D. & Schmidt, J. Electronic properties of iron-boron pairs in crystalline silicon by temperature- and injection-level-dependent lifetime measurements. J. Appl. 97, 103708 (2005).

12. 12.

Paudyal, B. B., McIntosh, K. R. & Macdonald, D. H. Temperature dependent electron and hole capture cross sections of iron-contaminated boron-doped silicon. In 2009 34th IEEE Photovoltaic Specialists Conference (PVSC), Philadelphia, PA, pp. 001588–001593 (2009). https://doi.org/10.1109/PVSC.2009.5411380.

13. 13.

Zhu, Y. et al. New insights into the thermally activated defects in n-type float-zone silicon. AIP Conf. Proc. 2147, 140014 (2019).

14. 14.

Murphy, J. D., Bothe, K., Krain, R., Voronkov, V. V. & Falster, R. J. Parameterisation of injection-dependent lifetime measurements in semiconductors in terms of Shockley-Read-Hall statistics: an application to oxide precipitates in silicon. J. Appl. 111, 113709 (2012).

15. 15.

Vargas, C. et al. Recombination parameters of lifetime-limiting carrier-induced defects in multicrystalline silicon for solar cells. Appl. Phys. Lett. 110, 092106 (2017).

16. 16.

Diez, S., Rein, S., Roth, T. & Glunz, S. W. Cobalt related defect levels in silicon analyzed by temperature- and injection-dependent lifetime spectroscopy. J. Appl. 101, 033710 (2007).

17. 17.

Rosenits, P., Roth, T., Glunz, S. W. & Beljakowa, S. Determining the defect parameters of the deep aluminum-related defect center in silicon. Appl. Phys. Lett. 91, 122109 (2007).

18. 18.

Shockley, W. & Read, W. T. Statistics of the recombinations of holes and electrons. Phys. Rev. 87, 835–842 (1952).

19. 19.

Hall, R. N. Electron-hole recombination in germanium. Phys. Rev. 87, 387–387 (1952).

20. 20.

Rein, S. Lifetime Spectroscopy: A Method of Defect Characterization in Silicon for Photovoltaic Applications (Springer Science & Business Media, 2005).

21. 21.

Zhu, Y., Gia, Q. T. L., Juhl, M. K., Coletti, G. & Hameiri, Z. Application of the Newton–Raphson method to lifetime spectroscopy for extraction of defect parameters. IEEE J. Photovolt. 7, 1092–1097 (2017).

22. 22.

Morishige, A. E. et al. Lifetime spectroscopy investigation of light-Induced degradation in p-type multicrystalline silicon PERC. IEEE J. Photovolt. 6, 1466–1472 (2016).

23. 23.

Peshek, T. J. et al. Insights into metastability of photovoltaic materials at the mesoscale through massive I–V analytics. J. Vac. Sci. Technol. B 34, 050801 (2016).

24. 24.

Riley, D. & Johnson, J. Photovoltaic prognostics and heath management using learning algorithms. In Proc. 38th IEEE Photovoltaic Specialists Conference 001535–001539 (2012).

25. 25.

Fada, J. S. et al. Electroluminescent image processing and cell degradation type classification via computer vision and statistical learning methodologies. In Proc. 44th IEEE Photovoltaic Specialist Conference 3456–3461 (2017).

26. 26.

Haunschild, J. et al. Rating and sorting of mc-Si as-cut wafers in solar cell production using PL imaging. Sol. Energy Mater. Sol. C. 106, 71–75 (2012).

27. 27.

Anwar, S. A. & Abdullah, M. Z. Micro-crack detection of multicrystalline solar cells featuring shape analysis and support vector machines. In Proc. IEEE International Conference on Control System, Computing and Engineering 143–148 (2012).

28. 28.

Ni, B., Zou, P., Li, Q. & Chen, Y. Intelligent defect detection method of photovoltaic modules based on deep learning. Adv. Intel. Syst. Res. 161, 7 (2018).

29. 29.

Ulaczyk, J., Morawiec, K., Zabierowski, P., Drobiazg, T. & Barreau, N. Finding relevant parameters for the thin-film photovoltaic cells production process with the application of data mining methods. Mol. Inform. 36, 1600161 (2017).

30. 30.

Brandt, R. E. et al. Rapid photovoltaic device characterization through bayesian parameter estimation. Joule 1, 843–856 (2017).

31. 31.

Sahu, H., Rao, W., Troisi, A. & Ma, H. Toward predicting efficiency of organic solar cells via machine learning and improved descriptors. Adv. Energy Mater. 8, 1801032 (2018).

32. 32.

Lu, S. et al. Accelerated discovery of stable lead-free hybrid organic-inorganic perovskites via machine learning. Nat. Commun. 9, 3405 (2018).

33. 33.

Kurchin, R. C. et al. Semiconductor parameter extraction via current-voltage characterization and Bayesian inference methods. In Proc. 7th IEEE World Conference on Photovoltaic Energy Conversion 3271–3275 (IEEE, 2018).

34. 34.

Fernández-Delgado, M., Cernadas, E., Barro, S. & Amorim, D. Do we need hundreds of classifiers to solve real world classification problems? J. Mach. Learn. Res. 15, 3133–3181 (2014).

35. 35.

Nielsen, M. A. Neural Networks and Deep Learning (Determination Press, 2015).

36. 36.

Hinton, G. E. Connectionist learning procedures. Artif. Intell. 40, 185–234 (1989).

37. 37.

Drucker, H. Improving regressors using boosting techniques. In Proc. International Conference on Machine Learning (1997).

38. 38.

Friedman, J. H. Stochastic gradient boosting. Comput. Stat. Data. 38, 367–378 (2002).

39. 39.

Niculescu-Mizil, A. & Caruana, R. Predicting good probabilities with supervised learning. In Proc. 22nd International Conference on Machine Learning 625–632 (2005).

40. 40.

Green, M. A. Intrinsic concentration, effective densities of states, and effective mass in silicon. J. Appl. 67, 2944–2954 (1990).

41. 41.

Couderc, R., Amara, M. & Lemiti, M. Reassessment of the intrinsic carrier density temperature dependence in crystalline silicon. J. Appl. 115, 093705 (2014).

42. 42.

Yan, D. & Cuevas, A. Empirical determination of the energy band gap narrowing in highly doped n+ silicon. J. Appl. 114, 044508 (2013).

43. 43.

Breiman, L. Random forests. Mach. Learn. 45, 5–32 (2001).

44. 44.

Freund, Y. & Schapire, R. E. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997).

45. 45.

Kingma, D. P. & Ba, J. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs] (2014).

46. 46.

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nat 521, 436–444 (2015).

47. 47.

Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nat. 529, 484–489 (2016).

48. 48.

Platt, J. C. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in Large Margin Classifiers 61–74 (MIT Press, 1999).

49. 49.

Chang, C.-C. & Lin, C.-J. LIBSVM: A library for support vector machines. ACM T. Intel. Syst. Tec. 2, 1–27 (2011).

50. 50.

Altman, N. S. An introduction to Kernel and nearest-neighbor nonparametric regression. Am. Stat. 46, 175–185 (1992).

51. 51.

Glantz, S. A. & Slinker, B. K. Primer of Applied Regression and Analysis of Variance. (McGraw-Hill: New York, NY, 1990).

52. 52.

Fawcett, T. An introduction to ROC analysis. Pattern Recognit. Lett. 27, 861–874 (2006).

53. 53.

Pedregosa, F. et al. Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011).

## Acknowledgements

This work was supported by the Australian Government through the Australian Renewable Energy Agency [ARENA; Projects 2017/RND001 and 2017/RND017]. Yan Zhu acknowledges the support of the Australian Government through the Australian Centre for Advanced Photovoltaics (ACAP, project RG200768-G). The views expressed herein are not necessarily the views of the Australian Government, and the Australian Government does not accept responsibility for any information or advice contained herein.

## Author information

Authors

### Contributions

Y.B. wrote the simulation and machine learning codes and the initial version of the manuscript. Z.H. supervised the work and contributed to the development of the methodology. Q.L. and J.D. contributed to the development of the methodology. Y.Z. contributed to the analysis of the experimental validation.

### Corresponding author

Correspondence to Yoann Buratti.

## Ethics declarations

### Competing interests

The authors declare no competing interests.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

Reprints and Permissions

Buratti, Y., Le Gia, Q.T., Dick, J. et al. Extracting bulk defect parameters in silicon wafers using machine learning models. npj Comput Mater 6, 142 (2020). https://doi.org/10.1038/s41524-020-00410-7


text:
# Rolle theorem

If a real-valued function $f$ is continuous on some closed interval $[a,b]$, has at each interior point of this interval a finite derivative or an infinite derivative of definite sign and at the end points of this interval takes equal values, then in the open interval $(a,b)$ there exists at least one point at which the derivative of $f$ vanishes.

The geometric sense of Rolle's theorem is that on the graph of a function $f$ satisfying the requirements of the theorem there exists a point $(\xi,f(\xi))$, $a<\xi<b$, such that the tangent to the graph at this point is parallel to the $x$-axis.

The mechanical interpretation of Rolle's theorem is that for any material point moving continuously along a straight line and which has returned after a certain period of time to the initial point there exists an instant at which the instantaneous velocity has been zero.

This theorem was first obtained by M. Rolle [1] for algebraic polynomials.

#### References

 [1] M. Rolle, "Traité d'algèbre" , Paris (1690) [2] S.M. Nikol'skii, "A course of mathematical analysis" , 1 , MIR (1977) (Translated from Russian)

format: plain


text:
## Wednesday, May 30, 2012

### Understanding the impacts of a risk occurring

Recently, we have had several clients inquire about using Inkling to understand not only the likelihood of a risk occurring but the potential impact of that risk.

Prediction markets handle this nicely because of the flexibility you have in asking the question and also the fact they output a quantitative value to your question.

For example, we can ask:

"If risk X occurs, what will happen?"
• Impact A
• Impact B
• Impact C
In this instance, the question type in Inkling would allow for multiple possible right answers so each impact can be judged individually. If the risk does not occur (the impacts cannot be assessed,) each impact would be cashed out at 0.

We can also directly correlate the occurrence of a risk (or any other event) with impacts. To do so, we simply multiply the chances of the impact and the risk to understand the likelihood the impact will occur in association with that risk.

For example, we can ask: "Will risk X occur?" and let's say the current price is \$75, representing a 75% chance the risk will occur. In a separate question, we can ask "Will impact X occur if risk X occurs?" and let's say the current price is \$25, representing a 25% chance the impact will occur.

Multiplying (.25 * .75) the two, we get ~.19 or a 19% chance Impact X will occur.

Assessing the impacts of risks is important for decision makers to understand as it will directly influence what they may or may not want to do to mitigate a risk. For example if the likelihood of a risk is high but only non-harmful impacts have a high probability of occurring, it may not be necessary to mitigate the risk. But if the reverse is true, mitigating that risk may take on a higher priority.


text:
# [−][src]Struct bulletproofs::generators::BulletproofGens

pub struct BulletproofGens {
pub gens_capacity: usize,
pub party_capacity: usize,
G_vec: Vec<Vec<RistrettoPoint>>,
H_vec: Vec<Vec<RistrettoPoint>>,
}

The BulletproofGens struct contains all the generators needed for aggregating up to m range proofs of up to n bits each.

# Extensible Generator Generation

Instead of constructing a single vector of size m*n, as described in the Bulletproofs paper, we construct each party's generators separately.

To construct an arbitrary-length chain of generators, we apply SHAKE256 to a domain separator label, and feed each 64 bytes of XOF output into the ristretto255 hash-to-group function. Each of the m parties' generators are constructed using a different domain separation label, and proving and verification uses the first n elements of the arbitrary-length chain.

This means that the aggregation size (number of parties) is orthogonal to the rangeproof size (number of bits), and allows using the same BulletproofGens object for different proving parameters.

This construction is also forward-compatible with constraint system proofs, which use a much larger slice of the generator chain, and even forward-compatible to multiparty aggregation of constraint system proofs, since the generators are namespaced by their party index.

## Fields

gens_capacity: usize

The maximum number of usable generators for each party.

party_capacity: usize

Number of values or parties

G_vec: Vec<Vec<RistrettoPoint>>

Precomputed $$\mathbf G$$ generators for each party.

H_vec: Vec<Vec<RistrettoPoint>>

Precomputed $$\mathbf H$$ generators for each party.

## Methods

### impl BulletproofGens[src]

#### pub fn new(gens_capacity: usize, party_capacity: usize) -> Self[src]

Create a new BulletproofGens object.

# Inputs

• gens_capacity is the number of generators to precompute for each party. For rangeproofs, it is sufficient to pass 64, the maximum bitsize of the rangeproofs. For circuit proofs, the capacity must be greater than the number of multipliers, rounded up to the next power of two.

• party_capacity is the maximum number of parties that can produce an aggregated proof.

#### pub fn share(&self, j: usize) -> BulletproofGensShare[src]

Returns j-th share of generators, with an appropriate slice of vectors G and H for the j-th range proof.

#### pub(crate) fn G(    &self,     n: usize,     m: usize) -> impl Iterator<Item = &RistrettoPoint>[src]

Return an iterator over the aggregation of the parties' G generators with given size n.

#### pub(crate) fn H(    &self,     n: usize,     m: usize) -> impl Iterator<Item = &RistrettoPoint>[src]

Return an iterator over the aggregation of the parties' H generators with given size n.

## Trait Implementations

### impl Clone for BulletproofGens[src]

#### fn clone_from(&mut self, source: &Self)1.0.0[src]

Performs copy-assignment from source. Read more

## Blanket Implementations

### impl<T, U> TryFrom for T where    U: Into<T>, [src]

#### type Error = !

🔬 This is a nightly-only experimental API. (try_from)

The type returned in the event of a conversion error.

### impl<T, U> TryInto for T where    U: TryFrom<T>, [src]

#### type Error = <U as TryFrom<T>>::Error

🔬 This is a nightly-only experimental API. (try_from)

The type returned in the event of a conversion error.

### impl<T> Same for T

#### type Output = T

Should always be Self


text:
# Find the length of $w$ using a numerical method

I need to find the value of $w$ using a numerical method for finding function's root (like Newton's method).

By the Pythagorean theorem we know that $BF = \sqrt {100 -w^2}$ and $AD = \sqrt {144 - w^2}$.

I'm not so sure how to utilize $OE$ to create a function so that it's root is the answer for $w$.

Thanks!

Put the origin at $D$, $A=(0,a)$, $B=(w,b)$. Then the diagonal lines have the equations $y=a·(1-\frac xw)$ and $y=b·\frac xw$. They intersect at $x=w·\frac{a}{a+b}$ at height $y=\frac{ab}{a+b}$.

The three unknowns $a,b,w$ are thus connected to the 3 given lengths via $$a^2+w^2=144\\ b^2+w^2=100\\ ab=5(a+b)$$ leading to $$\sqrt{(144-w^2)(100-w^2)}=5(\sqrt{144-w^2}+\sqrt{100-w^2}).$$ This can now be solved using your favorite univariate solver, like bisection, secant method, Newton method and others to find the root close to $w=4.29732800472$.

From triangle $ADF$ and triangle $ODF$: $$\frac{AD}{DF}=\frac{OE}{EF}$$

$$\frac{AD}{w}=\frac{5}{EF}$$

$$EF=\frac{5w}{AD}$$

From triangle $BFD$ and $OED$:

$$\frac{BF}{DF}=\frac{OE}{DE}$$

$$\frac{BF}{w}=\frac{5}{DE}$$

$$DE=\frac{5w}{BF}$$

Since $DE+EF=w$

$$\frac{5}{AD}+\frac{5}{BF}=1$$

Hence,

$$\frac{5}{\sqrt{144-w^2}}+\frac{5}{\sqrt{100-w^2}}=1$$

Starting from @Siong Thye Goh's answer, $$\frac{5}{\sqrt{144-w^2}}+\frac{5}{\sqrt{100-w^2}}=1$$ let $$x=w^2$$ and consider that you look for the zero of function $$f(x)=\frac{5}{\sqrt{144-x}}+\frac{5}{\sqrt{100-x}}-1$$ If you plot it, it looks very linear and this is very good for a root-finding algorithm.

Being very lazy, using $$x_0=0$$, the first iteration of Newton, Halley and Householder methods will be $$\frac{7200}{341} \qquad \frac{89280}{4823}\qquad \frac{173628000}{9399763}$$ Converted to decimals, th first estimate of $$w$$ is $$4.59504$$, $$4.30248$$ and $$4.29785$$ while the exact solution is $$4.29733$$.


text:
# Org Latex Fragment Succeeded by a Dash Is Not Showing

I'm using the org-mode to write some mathematical notes, so I'm using latex fragments like $\frac{a}{b}$ a lot. However, if I succeed the fragment with a dash, like $\frac{a}{b}$-fraction, it will not render (preview) the fragment.

Questions: (1) How can I override this behavior and make it show? (2) Is this the intended behavior or strictly a bug?

Use $$...$$ instead of $...$ and it should work just fine.
Org-mode recognizes Latex fragments by the $ symbol at the start and end of the fragment. If you add a trailing -, then the fragment doesn't end with a $ anymore, so Org won't recognize it as a fragment. You can work around this by placing a space between the closing $ and the -: $\frac{a}{b}$-fraction  • But how can one then supress the space during export? Note that this is possible when dealing with emphasis: /wh/-pronoun. – Timm Nov 30 '17 at 21:35 • The code for font-locking, responsible for bold and emphasis, is different than the code for identifying code fragments, so they will not necessarily behave the same. Another work around: try $\frac{a}{b} - \text{fraction}$. Do you need to have the '-fraction' outside the fragment for some reason? Nov 30 '17 at 21:45 • Does the following work? $\frac{a}{b}\$ \hspace{-\em} (or -\thinspace)? Dec 7 '17 at 20:18


text:
# §16.8(i) Classification of Singularities

An ordinary point of the differential equation

 16.8.1 $\frac{{d}^{n}w}{{dz}^{n}}+f_{n-1}(z)\frac{{d}^{n-1}w}{{dz}^{n-1}}+f_{n-2}(z)% \frac{{d}^{n-2}w}{{dz}^{n-2}}+\dots+f_{1}(z)\frac{dw}{dz}+f_{0}(z)w=0$

is a value $z_{0}$ of $z$ at which all the coefficients $f_{j}(z)$, $j=0,1,\dots,n-1$, are analytic. If $z_{0}$ is not an ordinary point but $(z-z_{0})^{n-j}f_{j}(z)$, $j=0,1,\dots,n-1$, are analytic at $z=z_{0}$, then $z_{0}$ is a regular singularity. All other singularities are irregular. Compare §2.7(i) in the case $n=2$. Similar definitions apply in the case $z_{0}=\infty$: we transform $\infty$ into the origin by replacing $z$ in (16.8.1) by $1/z$; again compare §2.7(i).

For further information see Hille (1976, pp. 360–370).

# §16.8(ii) The Generalized Hypergeometric Differential Equation

With the notation

 16.8.2 $\displaystyle D$ $\displaystyle=\frac{d}{dz},$ $\displaystyle\vartheta$ $\displaystyle=z\frac{d}{dz},$ Symbols: $\frac{df}{dx}$: derivative of $f$ with respect to $x$, $z$: complex variable, $D$: differential operator and $\vartheta$: differential operator Permalink: http://dlmf.nist.gov/16.8.E2 Encodings: TeX, TeX, pMML, pMML, png, png

the function $w=\mathop{{{}_{p}F_{q}}\/}\nolimits\!\left(\mathbf{a};\mathbf{b};z\right)$ satisfies the differential equation

 16.8.3 $\left(\vartheta(\vartheta+b_{1}-1)\cdots(\vartheta+b_{q}-1)-z(\vartheta+a_{1})% \cdots(\vartheta+a_{p})\right)w=0.$

Equivalently,

 16.8.4 $z^{q}D^{q+1}w+\sum_{j=1}^{q}z^{j-1}(\alpha_{j}z+\beta_{j})D^{j}w+\alpha_{0}w=0,$ $p\leq q$,

or

 16.8.5 $z^{q}(1-z)D^{q+1}w+\sum_{j=1}^{q}z^{j-1}(\alpha_{j}z+\beta_{j})D^{j}w+\alpha_{% 0}w=0,$ $p=q+1$,

where $\alpha_{j}$ and $\beta_{j}$ are constants. Equation (16.8.4) has a regular singularity at $z=0$, and an irregular singularity at $z=\infty$, whereas (16.8.5) has regular singularities at $z=0$, $1$, and $\infty$. In each case there are no other singularities. Equation (16.8.3) is of order $\max(p,q+1)$. In Letessier et al. (1994) examples are discussed in which the generalized hypergeometric function satisfies a differential equation that is of order 1 or even 2 less than might be expected.

When no $b_{j}$ is an integer, and no two $b_{j}$ differ by an integer, a fundamental set of solutions of (16.8.3) is given by

 16.8.6 $\displaystyle w_{0}(z)$ $\displaystyle=\mathop{{{}_{p}F_{q}}\/}\nolimits\!\left({a_{1},\dots,a_{p}\atop b% _{1},\dots,b_{q}};z\right),$ ${w_{j}(z)=z^{1-b_{j}}\mathop{{{}_{p}F_{q}}\/}\nolimits\!\left({1+a_{1}-b_{j},% \dots,1+a_{p}-b_{j}\atop 2-b_{j},1+b_{1}-b_{j},\ldots*\dots,1+b_{q}-b_{j}};z% \right),}$ $j=1,\dots,q$,

where $*$ indicates that the entry $1+b_{j}-b_{j}$ is omitted. For other values of the $b_{j}$, series solutions in powers of $z$ (possibly involving also $\mathop{\ln\/}\nolimits z$) can be constructed via a limiting process; compare §2.7(i) in the case of second-order differential equations. For details see Smith (1939a, b), and Nørlund (1955).

When $p=q+1$, and no two $a_{j}$ differ by an integer, another fundamental set of solutions of (16.8.3) is given by

 16.8.7 $\widetilde{w}_{j}(z)=(-z)^{-a_{j}}\mathop{{{}_{q+1}F_{q}}\/}\nolimits\!\left({% a_{j},1-b_{1}+a_{j},\dots,1-b_{q}+a_{j}\atop 1-a_{1}+a_{j},\ldots*\dots,1-a_{q% +1}+a_{j}};\frac{1}{z}\right),$ $j=1,\dots,q+1$,

where $*$ indicates that the entry $1-a_{j}+a_{j}$ is omitted. We have the connection formula

 16.8.8 $\mathop{{{}_{q+1}F_{q}}\/}\nolimits\!\left({a_{1},\dots,a_{q+1}\atop b_{1},% \dots,b_{q}};z\right)=\sum_{j=1}^{q+1}\left({\textstyle\ifrac{\prod\limits_{% \substack{k=1\\ k\neq j}}^{q+1}\frac{\mathop{\Gamma\/}\nolimits\!\left(a_{k}-a_{j}\right)}{% \mathop{\Gamma\/}\nolimits\!\left(a_{k}\right)}}{\prod\limits_{k=1}^{q}\frac{% \mathop{\Gamma\/}\nolimits\!\left(b_{k}-a_{j}\right)}{\mathop{\Gamma\/}% \nolimits\!\left(b_{k}\right)}}}\right)\widetilde{w}_{j}(z),$ $|\mathop{\mathrm{ph}\/}\nolimits\!\left(-z\right)|\leq\pi$.

More generally if $z_{0}$ ($\in\Complex$) is an arbitrary constant, $|z-z_{0}|>\max{(|z_{0}|,|z_{0}-1|)}$, and $|\mathop{\mathrm{ph}\/}\nolimits\!\left(z_{0}-z\right)|<\pi$, then

 16.8.9 $\left({\textstyle\ifrac{\prod\limits_{k=1}^{q+1}\mathop{\Gamma\/}\nolimits\!% \left(a_{k}\right)}{\prod\limits_{k=1}^{q}\mathop{\Gamma\/}\nolimits\!\left(b_% {k}\right)}}\right)\mathop{{{}_{q+1}F_{q}}\/}\nolimits\!\left({a_{1},\dots,a_{% q+1}\atop b_{1},\dots,b_{q}};z\right)=\sum_{j=1}^{q+1}\left(z_{0}-z\right)^{-a% _{j}}\sum_{n=0}^{\infty}\frac{\mathop{\Gamma\/}\nolimits\!\left(a_{j}+n\right)% }{n!}\*\left({\textstyle\ifrac{\prod\limits_{\substack{k=1\\ k\neq j}}^{q+1}\mathop{\Gamma\/}\nolimits\!\left(a_{k}-a_{j}-n\right)}{\prod% \limits_{k=1}^{q}\mathop{\Gamma\/}\nolimits\!\left(b_{k}-a_{j}-n\right)}}% \right)\*\mathop{{{}_{q+1}F_{q}}\/}\nolimits\!\left({a_{1}-a_{j}-n,\dots,a_{q+% 1}-a_{j}-n\atop b_{1}-a_{j}-n,\dots,b_{q}-a_{j}-n};z_{0}\right)\left(z-z_{0}% \right)^{-n}.$

(Note that the generalized hypergeometric functions on the right-hand side are polynomials in $z_{0}$.)

When $p=q+1$ and some of the $a_{j}$ differ by an integer a limiting process can again be applied. For details see Nørlund (1955). In this reference it is also explained that in general when $q>1$ no simple representations in terms of generalized hypergeometric functions are available for the fundamental solutions near $z=1$. Analytical continuation formulas for $\mathop{{{}_{q+1}F_{q}}\/}\nolimits\!\left(\mathbf{a};\mathbf{b};z\right)$ near $z=1$ are given in Bühring (1987b) for the case $q=2$, and in Bühring (1992) for the general case.

# §16.8(iii) Confluence of Singularities

If $p\leq q$, then

 16.8.10 $\lim_{|\alpha|\to\infty}\mathop{{{}_{p+1}F_{q}}\/}\nolimits\!\left({a_{1},% \dots,a_{p},\alpha\atop b_{1},\dots,b_{q}};\frac{z}{\alpha}\right)=\mathop{{{}% _{p}F_{q}}\/}\nolimits\!\left({a_{1},\dots,a_{p}\atop b_{1},\dots,b_{q}};z% \right).$

Thus in the case $p=q$ the regular singularities of the function on the left-hand side at $\alpha$ and $\infty$ coalesce into an irregular singularity at $\infty$.

Next, if $p\leq q+1$ and $|\mathop{\mathrm{ph}\/}\nolimits\beta|\leq\pi-\delta$ ($<\pi$), then

 16.8.11 $\lim_{|\beta|\to\infty}\mathop{{{}_{p}F_{q+1}}\/}\nolimits\!\left({a_{1},\dots% ,a_{p}\atop b_{1},\dots,b_{q},\beta};\beta z\right)=\mathop{{{}_{p}F_{q}}\/}% \nolimits\!\left({a_{1},\dots,a_{p}\atop b_{1},\dots,b_{q}};z\right),$

provided that in the case $p=q+1$ we have $|z|<1$ when $|\mathop{\mathrm{ph}\/}\nolimits\beta|\leq\frac{1}{2}\pi$, and $|z|<|\mathop{\sin\/}\nolimits\!\left(\mathop{\mathrm{ph}\/}\nolimits\beta% \right)|$ when $\frac{1}{2}\pi\leq|\mathop{\mathrm{ph}\/}\nolimits\beta|\leq\pi-\delta$ ($<\pi$).


text:
Impact Factor: 3.9

Dec.  2022

Article Contents

# Application of a Neural Network to Store and Compute the Optical Properties of Non-Spherical Particles

• Radiative transfer simulations and remote sensing studies fundamentally require accurate and efficient computation of the optical properties of non-spherical particles. This paper proposes a deep learning (DL) scheme in conjunction with an optical property database to achieve this goal. Deep neural network (DNN) architectures were obtained from a dataset of the optical properties of super-spheroids with extensive shape parameters, size parameters, and refractive indices. The dataset was computed through the invariant imbedding T-matrix method. Four separate DNN architectures were created to compute the extinction efficiency factor, single-scattering albedo, asymmetry factor, and phase matrix. The criterion for designing these neural networks was the achievement of the highest prediction accuracy with minimal DNN parameters. The numerical results demonstrate that the determination coefficients are greater than 0.999 between the prediction values from the neural networks and the truth values from the database, which indicates that the DNN can reproduce the optical properties in the dataset with high accuracy. In addition, the DNN model can robustly predict the optical properties of particles with high accuracy for shape parameters or refractive indices that are unavailable in the database. Importantly, the ratio of the database size (~127 GB) to that of the DNN parameters (~20 MB) is approximately 6810, implying that the DNN model can be treated as a highly compressed database that can be used as an alternative to the original database for real-time computing of the optical properties of non-spherical particles in radiative transfer and atmospheric models.
摘要: 辐射传输模拟和遥感反演需要准确和快速地计算非球形粒子的光学特性。传统上一般采用查找表方法来解决电磁散射计算效率低的问题。但随着粒子参数增加，查找表数据体量变大，不便于模式使用。本文提出了一种深度学习方法用于存储和计算非球形粒子光学特性。我们将基于不变嵌入T-矩阵方法计算的超椭球粒子光学特性作为训练数据库，选取长宽比、圆滑度、粒径和复折射指数作为训练参数，设计了四种最优的神经网络架构分别计算或预测消光效率因子、单次散射消光比、不对称因子以及相矩阵元素。结果表明：神经网络预测值与数据库真值之间的决定系数大于0.999，可以准确再现数据库的光学特性信息。另外，神经网络模型还能够可靠预测出未知参数（尺寸和折射指数）的粒子光学特性值。通过将大型数据库近乎无损压缩为四个神经网络后，可将小巧的网络模型代替原始查找表接入辐射传输算法中，从而实现非球形粒子光学特性的高效计算。
• Figure 1.  Twenty-five representative super-spheroid shapes including oblate ($a/c > 1.0$), prolate ($a/c < 1.0$), concave ($n > 2.0$), and convex particles ($n < 2.0$).

Figure 3.  Deep Neural Network (DNN) architectures for ${Q_{{\text{ext}}}}$, SSA, g, and ${P_{ij}}$. The number of layers is 5, 5, 6, and 7 for ${Q_{{\text{ext}}}}$, SSA, g, and ${P_{ij}}$, respectively. The number of neuron nodes in the layer is included in the parentheses. For example, $\left( {{Q_{{\text{ext}}}},{\text{ }}128} \right)$ indicates 128 neuron nodes in a layer for the prediction of ${Q_{{\text{ext}}}}$.

Figure 2.  Scatter plot of RMSE (orange dots) and the coefficient of determination ${R^2}$ (green dots) from several established DNN models for (a) ${Q_{{\text{ext}}}}$, (b) SSA, (c) g, and (d) ${P_{11}}$. In each subfigure, the left y-axis represents the RMSE, and the right y-axis represents ${R^2}$. As the network parameters increase, the DNN models have an upper limit of the prediction accuracy for each optical property.

Figure 4.  Scatter plot of (a) ${Q_{{\text{ext}}}}$, (b) SSA, (c) g, (d) ${P_{11}}$, and (e) – (i) ${P_{ij}}/{P_{11}}$ by DNN predictions against the true values.

Figure 5.  Absolute errors of ${Q_{{\text{ext}}}}$ predicted by the DNNs and the true values from the database.

Figure 6.  Absolute errors of SSA predicted by the DNNs and the true values from the database.

Figure 7.  Absolute errors of g predicted by the DNNs and the true values from the database.

Figure 8.  Comparison of ${Q_{{\text{ext}}}}$, SSA, and g predicted by the DNNs against the true values in the database. The upper row (a–c) in the figure shows the results for parameters in light-colored areas, and the lower row (d–f) shows the results for parameters in dark-colored areas (Figs. 5–7). The mean absolute errors (MAEs) are also included in each subplot.

Figure 9.  Comparison of (a) ${P_{11}}$, (b) ${P_{12}}/{P_{11}}$, (c) ${P_{22}}/{P_{11}}$, (d) ${P_{33}}/{P_{11}}$, (e) ${P_{43}}/{P_{11}}$, and (f) ${P_{44}}/{P_{11}}$ predicted by the DNNs against the true values. The size parameter xsize is unavailable in the database.

Figure 10.  Comparison of (a) ${P_{11}}$, (b) ${P_{12}}/{P_{11}}$, (c) ${P_{22}}/{P_{11}}$, (d) ${P_{33}}/{P_{11}}$, (e) ${P_{43}}/{P_{11}}$, and (f) ${P_{44}}/{P_{11}}$ predicted from the DNNs against the true values. The refractive index is unavailable in the database.

Figure 11.  Comparison of (a) ${P_{11}}$, (b) ${P_{12}}/{P_{11}}$, (c) ${P_{22}}/{P_{11}}$, (d) ${P_{33}}/{P_{11}}$, (e) ${P_{43}}/{P_{11}}$, and (f) ${P_{44}}/{P_{11}}$ predicted from the DNNs against the true values. Both the size parameter xsize and refractive index are unavailable in the database.

Export:

## Manuscript History

Manuscript revised: 17 November 2021
Manuscript accepted: 23 November 2021
###### 通讯作者: 陈斌, bchen63@163.com
• 1.

沈阳化工大学材料科学与工程学院 沈阳 110142

## Application of a Neural Network to Store and Compute the Optical Properties of Non-Spherical Particles

###### Corresponding author: Wei HAN, hanwei@cma.gov.cn;
• 1. Key Laboratory of Geoscience Big Data and Deep Resource of Zhejiang Province, School of Earth Sciences, Zhejiang University, Hangzhou 310027, China
• 2. Center for Earth System Modeling and Prediction, China Meteorological Administration, Beijing 100081, China
• 3. Numerical Weather Prediction Center, China Meteorological Administration, Beijing 100081, China
• 4. Key Laboratory of Atmospheric Chemistry of China Meteorological Administration, Institute of Atmospheric Composition, Chinese Academy of Meteorological Sciences, Beijing 100081, China

Abstract: Radiative transfer simulations and remote sensing studies fundamentally require accurate and efficient computation of the optical properties of non-spherical particles. This paper proposes a deep learning (DL) scheme in conjunction with an optical property database to achieve this goal. Deep neural network (DNN) architectures were obtained from a dataset of the optical properties of super-spheroids with extensive shape parameters, size parameters, and refractive indices. The dataset was computed through the invariant imbedding T-matrix method. Four separate DNN architectures were created to compute the extinction efficiency factor, single-scattering albedo, asymmetry factor, and phase matrix. The criterion for designing these neural networks was the achievement of the highest prediction accuracy with minimal DNN parameters. The numerical results demonstrate that the determination coefficients are greater than 0.999 between the prediction values from the neural networks and the truth values from the database, which indicates that the DNN can reproduce the optical properties in the dataset with high accuracy. In addition, the DNN model can robustly predict the optical properties of particles with high accuracy for shape parameters or refractive indices that are unavailable in the database. Importantly, the ratio of the database size (~127 GB) to that of the DNN parameters (~20 MB) is approximately 6810, implying that the DNN model can be treated as a highly compressed database that can be used as an alternative to the original database for real-time computing of the optical properties of non-spherical particles in radiative transfer and atmospheric models.

Reference

/


text:
# Card probability problem

Possible Duplicate: ¢

I located the adhering to trouble in Rosen is Discrete Mathematics and also Its Applications 6th ed. :

There are 3 cards in a box. Both sides of one card are black, both sides of one card are red, and also the 3rd card has one black side and also one red side. We select a card randomly and also observe just one side.

If the side is black, what is the probability that the opposite side is additionally black?

Should not the response to the inquiry be $1/3$, as amongst $3$ cards just $1$ of them has both sides black. Yet guide claims solution is $2/3$, do I miss out on anything?

2
2022-07-25 17:47:14
Source Share


text:
# Generate Uniform Random Variates with Constant Norm [duplicate]

How can one generate $$k$$ uniform random variates centered at zero, $$X_1, X_2, ..., X_k$$, given a constant Euclidean norm, $$c =\sqrt{X_1^2+X_2^2+...X_k^2}$$?

• – kjetil b halvorsen Sep 14 '20 at 22:48
• There is no solution --- if they obey the constraint then they are not independent. Perhaps also related to this question. – Ben Sep 15 '20 at 1:46
• @QuestionAsker did you mean that the X's are marginally uniform, or uniform on the hypersurface? – Glen_b Sep 15 '20 at 22:37
• @Glen_b Uniform on the hypersurface. I was under the impression that if they were marginally uniform, they would be be uniform on the surface, but I see now this is not the case. – QuestionAsker Sep 16 '20 at 1:54
• For a sphere, i.e. $k=3$, uniform on the surface does imply marginally uniform (though not independent) - think Archimedes On the Sphere and Cylinder. This is not true in other dimensions. You can then easily construct $X_1,X_2,X_3$ from two independent uniform random variables. – Henry Sep 16 '20 at 7:02

Generate $$k$$ independent standard normal random variates $$Y_1, Y_2, \ldots, Y_k$$
and then scale to give the desired $$c$$ with $$X_i= \frac{c}{\sqrt{Y_1^2+Y_2^2+\cdots+Y_k^2}}Y_i$$
• @Glen_b Whether it's a duplicate depends on how we interpret the "uniform" of the question: does it mean marginally uniform or uniform on the sphere's surface? The question, as currently phrased, strongly implies the former meaning is intended, whereas the duplicate addresses the latter meaning only. The answer is in the negative as one can see by drawing a picture of the case $k=2.$ – whuber Sep 15 '20 at 15:10


text:
In my understanding, "the absolute correlation" is not such a common expression. Can anyone kindly explain the significance of the built-in symbol AbsoluteCorrelation, especially compared with Pearson's and Spearman's correlation coefficients?

I'd say it's not only "not such a common expression" but at best a very misleading expression. What one obtains is not a correlation measure in the sense that the values are only between 0 and 1.

As an example:

d = MultinormalDistribution[{3, 2, 1}, {{2, -2/3, -1/3}, {-2/3, 3, -1/3}, {-1/3, -1/3, 7}}];
AbsoluteCorrelation[d] // MatrixForm

So one is not getting correlations. I would steer clear of it for any statistical applications at least until the documentation gets better (or even never).

d = MultinormalDistribution[{0, 0, 0}, {{1, -2/3, -1/3}, {-2/3, 1, -1/3}, {-1/3, -1/3, 1}}];


text:
1. Home
2. Docs
3. WeStatiX Documentation EN
4. Verification
5. Beam subjected to uniform temperature rise

# Beam subjected to uniform temperature rise

In this page we consider a one-dimensional beam fixed at both ends subjected to an uniform temperature rise T.

If you want, you can build the FE model with the data shown in the following table.

On the other hand, you can find it through our tutorials, so you just have to start the calculation.

First you have to determine the analytical solution: in order to do it, consider the axial direction. [1] The strain on the beam due to uniform temperature change is:

$$\epsilon_T=\alpha \cdot T$$

The stress/strain law is linear, therefore the nodal forces must be

$$F=A \cdot \sigma=A \cdot E \cdot \epsilon = A \cdot E \cdot \alpha \cdot T$$

Therefore you can compare the analytical solution with WeStatiX’s results, as shown in the following table.

In the pictures you can see the diagram of the normal force.

WeStatiX catches the solution perfectly.

[1] DARYL L. LOGAN, A First Course in the Finite Element Method, 4th edition, Thomson


text:
Filters cases based on their processing_time.

This filter can be used by using an interval or by using a percentage. The percentage will always start with the shortest cases first and stop including cases when the specified percentile is reached. On the other hand, an absolute interval can be defined instead to filter cases which have a processing time in this interval. The time units in which this interval is defined can be supplied with the units argument.

## Usage

filter_processing_time(
log,
interval = NULL,
percentage = NULL,
reverse = FALSE,
units = c("secs", "mins", "hours", "days", "weeks"),
eventlog = deprecated()
)

# S3 method for log
filter_processing_time(
log,
interval = NULL,
percentage = NULL,
reverse = FALSE,
units = c("secs", "mins", "hours", "days", "weeks"),
eventlog = deprecated()
)

# S3 method for grouped_log
filter_processing_time(
log,
interval = NULL,
percentage = NULL,
reverse = FALSE,
units = c("secs", "mins", "hours", "days", "weeks"),
eventlog = deprecated()
)

## Arguments

log

log: Object of class log or derivatives (grouped_log, eventlog, activitylog, etc.).

interval, percentage

Provide either interval or percentage.
interval (numeric vector of length 2): A duration interval. Half open interval can be created using NA.
percentage (numeric): A percentage to be used for relative filtering.

reverse

logical (default FALSE): Indicating whether the selection should be reversed.

units

character (default "secs"): The time unit in which the processing times should be reported. Should be one of the following values: "secs" (default), "mins", "hours", "days", "weeks". See also the units argument of difftime().

eventlog

; please use log instead.

## Value

When given an object of type log, it will return a filtered log. When given an object of type grouped_log, the filter will be applied in a stratified way (i.e. each separately for each group). The returned log will be grouped on the same variables as the original log.

## Methods (by class)

• filter_processing_time(log): Filters cases for a log.

• filter_processing_time(grouped_log): Filters cases for a grouped_log.

## References

Swennen, M. (2018). Using Event Log Knowledge to Support Operational Exellence Techniques (Doctoral dissertation). Hasselt University.

processing_time(),difftime()
Other filters: filter_activity_frequency(), filter_activity_instance(), filter_activity_presence(), filter_activity(), filter_case_condition(), filter_case(), filter_endpoints_condition(), filter_endpoints(), filter_flow_time(), filter_idle_time(), filter_infrequent_flows(), filter_lifecycle_presence(), filter_lifecycle(), filter_precedence_condition(), filter_precedence_resource(), filter_precedence(), filter_resource_frequency(), filter_resource(), filter_throughput_time(), filter_time_period(), filter_trace_frequency(), filter_trace_length(), filter_trace(), filter_trim_lifecycle(), filter_trim()


text:
# Homogeneous Coordinates

A Concrete Explanation

Computer graphics programs usually work with homogeneous coordinates, whose main purpose is to allow for some operations like translation to be linear and thus vectorizable. Vectorized operations can be heavily parallelized on the processing unit which results in a tremendous performance boost. Video games make heavy use of this technique.

Homogeneous coordinates are often introduced as a trick. I think this is a misconception and it paves the way for a lot of misunderstandings and errors when working with computer graphics.

In the following article I will detail a more natural approach to the concept.

## Notations

In the following we will consider canonical Euclidean planes and spaces. $$x$$, $$y$$ and $$z$$ are the canonical Euclidean dimensions or the vector components depending on the context.

The initial space is called the projective space.

All the transformations are supposed to be endomorphisms.

## Usual transformations

Linear transformations in a Euclidean space can be written as a matrix $$M$$ so that the transformed point $$P'$$ of $$P$$ is:

$P' = M \cdot P$

In 2D, the matrix is of size $$(2,2)$$. A few examples follow:

• 2D scaling $$(x,y) \to (sx, sy)$$:
\begin{pmatrix} s & 0 \\ 0 & s \\ \end{pmatrix}
• Rotation of angle $$\theta$$:

In 2D:

\begin{pmatrix} \cos(\theta) & -\sin(\theta) \\ \sin(\theta) & \cos(\theta) \\ \end{pmatrix}

In 3D, around the $$z$$-axis:

\begin{pmatrix} \cos(\theta) & -\sin(\theta) & 0 \\ \sin(\theta) & \cos(\theta) & 0 \\ 0 & 0 & 1 \\ \end{pmatrix}

Other linear transformations include shearing and mirroring.

## Non-linear operations

In the following section we will consider the projective space to be a 2D plane. This is only to make the explanation more visual, since this can be easily generalized to any dimension.

Some operations are not linear in the projective plane. But what if we consider the projective plane as being a plane in a 3D space? Then we can apply any 3D transformation on the points of this plane.

We say that we promote the plane to the next dimension.

The fundamental principle here is that the higher dimension allows for more complex transformations, e.g. some specific shearing operations in 3D can be seen as a translation on the plane.

After the desired transformations have been applied, we project the resulting points back to the plane.

There are some parameters we need to specify:

• How do we promote the points? I.e. where does the projective plane lie in 3D?
• How do we project the transformation result back to the plane?

In order to answer this we need to analyze how some non-linear operations behave.

### Translation operation

#### Promotion

Translations are not linear in the 2D-Euclidean space. Now if we promote the input points to a 3D space, they will all lie in a plane. It is obviously simpler to keep the $$x$$ and $$y$$ vectors identical. Then the only parameter for defining the plane is $$z$$.

Let’s have a look at the following shearing of some point $$P=(i,j,k)$$ by the scalars $$a$$ and $$b$$:

$\begin{pmatrix} 1 & 0 & a \\ 0 & 1 & b \\ 0 & 0 & 1 \\ \end{pmatrix} \cdot (i, j, k) = (i+ka, j+kb, k)$

If $$k=1$$, then the transformed point is $$(i+a, j+b, 1)$$, which is a translation on the plane $$z=1$$.

Considering the projective plane at $$z=1$$ makes it easy to write a translation matrix.

Conclusion: the points from a projective space of any dimension can be promoted to the next dimension by adding a coordinate with the scalar value 1.

#### Projection

If the transformed points lie in the plane, the projection back to the projective space is trivial: we simply cut off the last dimension.

However, the transformed points might end up somewhere outside the plane. In that case we need some way to map the 3D points to the 2D points. There are several ways of doing this.

• One would be to simply cut off the last dimension.
• Another solution would be to divide all dimensions by the last one when it is non-zero. The last dimension would then always be 1, which matches the position of the plane $$z=1$$ as we defined it previously.

First let’s consider some point $$P=(i,j,k)$$ and let’s see what happens if we apply the same transformation as before:

$\begin{pmatrix} 1 & 0 & a \\ 0 & 1 & b \\ 0 & 0 & 1 \\ \end{pmatrix} \cdot (i,j,k) = (i+ka, j+ka, k) = P'$

• If we project by cutting off the last coordinate, the projection of $$P$$ is $$(i, j)$$, the projection of $$P'$$ is $$(i+ka, j+kb)$$. The transformation is generally not the translation by $$(a, b)$$.
• Using the division by the last coordinate, the projection of $$P$$ is $$(i/k, j/k)$$, the projection of $$P'$$ is $$(i/k+a, j/k+b)$$. This time the transformation is the translation by $$(a, b)$$.

Conclusion: the projection back to the projective space is done by dividing all dimensions by the last one when non-zero. This allows for translations in the projective space.

### Perspective operation

We have seen one example that covers it all. But what if the above analysis was good enough for translations only? Let us have a look at a very common operation in computer graphics: perspective correction.

In this section we consider a 3D projective space. The higher dimension is 4D and the 4th coordinates we be written $$w$$.

When the camera and the screen are centered on the $$z$$-axis, perspective projection results from simple triangular relations, i.e. by dividing $$x$$ and $$y$$ by $$\alpha+\beta\cdot z$$, where $$\alpha$$ and $$\beta$$ are scalars depending on the distance from the camera and the screen to the origin.

In the previous scheme, $$i' = i\frac{z_s-z_c}{k-z_c}$$ and $$j' = j\frac{z_s-z_c}{k-z_c}$$.

In the simplest case the camera is at the origin and the screen is at $$z=1$$, as above. We get the simple transformation where $$\alpha=1$$ and $$\beta=1$$, that is, in our example, $$i' = i/k$$ and $$j' = j/k$$.

But there is no way a matrix multiplication can induce a division by one of the dimension. After all, matrices are a notation for linear transformations.

There is one way out though: a division by one dimension is performed during the projection back to the projective space. As such, our lead is to “inject” $$z$$ into $$w$$:

$\begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & \beta & \alpha \\ \end{pmatrix} \cdot (i, j, k, 1) = (i, j, k, \alpha+\beta k)$

If we project the result back to the projective space, we get the 3D point $$(\frac{i}{\alpha+\beta k}, \frac{j}{\alpha+\beta k}, \frac{k}{\alpha+\beta k})$$. If we project the result on the plane $$z=z_s$$, we obtain the perspective projection of $$(i, j, k)$$.

Conclusion: one more time, the division by the last coordinate is a projection technique that allows for operations in the next dimension to be properly mapped to the projective space.

## Homogeneous coordinates

The above analysis issued a relation between points in the projective space and points in the next dimension.

Making use of the above reasoning, homogeneous coordinates offer an alternative notation to cartesian coordinates. We insist on the word ’notation’ as there are no new points nor new spaces. It means that $$(i,j,k)$$ and $$(i,j,k,1)_h$$ specify the same point in 3D.

We use the $$_h$$ index to avoid confusion between homogeneous coordinates and cartesian coordinates in a higher dimension.

The mapping between the two coordinate systems is central: the cartesian coordinates are obtained by dividing all dimensions by the homogeneous component if not null, and discarding this last component. This has several consequences:

• Infinity can be represented using finite coordinates, e.g. $$(x, y, 0)_h$$ in 2D.
• The $$0_h$$ vector is ignored.
• The origin is the vector where all components are 0 but the last one is 1, e.g. $$(0,0,1)_h$$ in 2D.
• All points on a line going through $$0_h$$ represent the same point. A non-null scalar multiplication over homogeneous coordinates does not change the point. For instance $$(i, j) = (i, j, 1)_h = (2i, 2j, 2)_h = (\alpha i, \alpha j, \alpha)_h$$ for all scalars $$\alpha$$.

## Examples

### Rendering pipeline

In computer graphics, homogeneous coordinates play an important role as they allow for perspective transformations in 3D.

The 3D rendering pipeline can be summarized as follows (C stands for cartesian, H stands for homogeneous):

Vertex data (3D C) → Object data (3D H) → Eye coordinates (3D H) → Clip coordinates (3D H) → Normalized device coordinates (3D C) → Window coordinates (2D C)

• Vertex data are 3D vectors. The graphic pipeline uses homogeneous coordinates for object data, with the homogeneous component being 1, for the reason detailed above.
• Object data is then translated and rotated as needed via $$(4,4)_h$$-matrices. This yields the homogeneous eye coordinates.
• A $$(4,4)_h$$-perspective matrix is applied on the eye coordinates, which leads to the homogeneous clip coordinates.
• The homogeneous clip coordinates are converted to cartesian coordinates, a.k.a. normalized device coordinates (NDC), by dividing every coordinate by the last one, as usual.
• The NDC are converted to the 2D window coordinates by removing the third dimension and applying some scaling.

### 2D perspective correction

Let us consider the following problem: we want to rectify the perspective of a picture, e.g. we want a tilted building facade picture to appear as if it had been taken orthogonally.

We proceed with control points: we select pixels in the input picture and tell the program where we would like it to be on the transformed picture.

A $$(2,2)$$-matrix will not do the trick as it is not able to handle translations for instance.

Using homogeneous coordinates, we can transform the entire plane to the orthogonal picture we would like. In 3D, this can be seen as tilting the plane to make it match the screen.

This transformation can be embodied within a $$(3,3)_h$$-matrix $$M$$. The answer to our problem is $$M$$.

Let us consider a set of known point pairs $$\langle \text{ control point } (i,j), \text{ target point } (I,J)\rangle$$, so that

$\begin{pmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \\ \end{pmatrix} \cdot (i, j, 1)_h = (\alpha I, \alpha J, \alpha)_h$

We could be tempted to think we only need the first 2 rows of the matrix: this is wrong! The result of the matrix multiplication yields the homogeneous coordinates of a point, which still needs to be projected to the 2D plane to be significant. Therefore we need to compute the homogeneous component of every point.

The previous matrix operation yields the following equations:

$\left\{ \begin{array}{l l} m_{11}i + m_{12}j + m_{13} &= \alpha I \\ m_{21}i + m_{22}j + m_{23} &= \alpha J \\ m_{31}i + m_{32}j + m_{33} &= \alpha \\ \end{array} \right.$

or, cancelling the scale factor:

$\left\{\begin{array}{l l} \frac{m_{11}i + m_{12}j + m_{13}}{m_{31}i + m_{32}j + m_{33}} &= I \\ \frac{m_{21}i + m_{22}j + m_{23}}{m_{31}i + m_{32}j + m_{33}} &= J \\ \end{array} \right.$

The matrix has 9 unknown coefficients, but since it is homogeneous, it is defined up to a scale factor. For instance, we have $$M=M/m_{11}$$ if $$m_{11}≠0$$. As such, only 8 coefficient determines a homogeneous matrix uniquely.

We can solve the system of 8 linearly independent equations constructed as above from 4 well chosen points. The above equation pair yields

$\left\{\begin{array}{l l} m_{11}i + m_{12}j + m_{13} - m_{31}iI - m_{32}jI - m_{33}I &= 0 \\ m_{21}i + m_{22}j + m_{23} - m_{31}iJ - m_{32}jJ - m_{33}J &= 0 \\ \end{array} \right.$

which can be written as a multiplication

$\left\{\begin{array}{l l} (i, j, 1, 0, 0, 0, -iI, -jI, -I) \cdot m^T &= 0 \\ (0, 0, 0, i, j, 1, -iJ, -jJ, -J) \cdot m^T &= 0 \\ \end{array} \right.$

with $$m=(m_{11}, m_{12}, m_{13}, m_{21}, m_{22}, m_{23}, m_{31}, m_{32}, m_{33})$$.

We can build the following matrix from 4 point sets:

$S = \begin{pmatrix} i_1 & j_1 & 1 & 0 & 0 & 0 & -i_1I_1 & -j_1I_1 & -I_1 \\ 0 & 0 & 0 & i_1 & j_1 & 1 & -i_1J_1 & -j_1J_1 & -J_1 \\ ... \\ i_4 & j_4 & 1 & 0 & 0 & 0 & -i_4I_4 & -j_4I_4 & -I_4 \\ 0 & 0 & 0 & i_4 & j_4 & 1 & -i_4J_4 & -j_4J_4 & -J_4 \\ \end{pmatrix}$

If $$det(S) ≠ 0$$, then the result can be found by solving

$S\cdot m = 0$

which is equivalent to finding the eigenvalues of $$S$$. A singular-value decomposition will numerically find $$m$$.

## Final note

The present article does not claim to provide any new results when it comes to homogeneous coordinates. It only aims at explaining the theory more visually, and thus making it easier to understand and remember. It all makes perfect sense, it is not just a trick. Mathematically speaking, the key argument that I introduced above is to consider the projective space as an affine hyperplane in the next dimension. Then everything falls into place:

• the link between cartesian and homogeneous coordinates;
• why this actually works;
• when we should manipulate homogeneous coordinates, when we should project back to the projective space.

The central idea can be seen as a hook in the process: we momentarily promote our data to the next dimension, manipulate it in there, then project the result back to the original dimension. It generally does not make sense to exploit a result in homogeneous coordinates.


text:
Previous | Next --- Slide 30 of 46
Back to Lecture Thumbnails
Tdog

What exactly is d representing here? By offset do you mean the distance from the plane to the origin? Or is it like the distance from the point to the line along the vector (a,b,c).

motoole2

@Tdog The value for d effectively represents the distance of the plane from the origin. That is, for any point (x,y,z), the plane is characterized by the equation a*x + b*y + c*z + d = 0. (a,b,c) represents the normal of the plane and d represents the offset from the origin in this normal direction.