text:
# 35 Debugging

How to identify the source of a problem.

### 35.0.1 Using tests/workflow.R

This script, along with model-specific settings files in the tests folder, provide a working example. From inside the tests folder, R CMD --vanilla -- --settings pecan.<model>.xml < workflow.R should work.

The next step is to add debugonce(<broken.function.name>) before running the test workflow.

This allows you can step through the function and evaluate the different objects as they are created and/or transformed.

### 35.0.2 Useful scripts

The following scripts (in qaqc/vignettes identify, respectively:

1. relationships among functions across packages
2. function inputs and outputs (e.g. that will identify which functions and outputs are used in a workflow).

### 35.0.3 Debugging Shiny Apps

When developing shiny apps you can run the application from rstudio and place breakpoints int he code. To do this you will need to do the following steps first (already done on the VM) before starting rstudio: - echo “options(shiny.port = 6438)” >> ${HOME}/.Rprofile - echo “options(shiny.launch.browser = ‘FALSE’)” >>${HOME}/.Rprofile

Next you will need to create a tunnel for port 6438 to the VM, which will be used to open the shiny app, the following command will creat this tunnel: ssh -l carya -p 6422 -L 6438:localhost:6438 localhost.

Now you can from rstudio run your application using shiny::runApp() and it will show the output from the application in your console. You can now place breakpoints and evaluate the output.

## 35.1 Troubleshooting PEcAn

### 35.1.1 Cookies and pecan web pages

You may need to disable cookies specifically for the pecan webserver in your browser. This shouldn’t be a problem running from the virtual machine, but your installation of php can include a ‘PHPSESSID’ that is quite long, and this can overflow the params field of the workflows table, depending on how long your hostname, model name, site name, etc are.

### 35.1.2Warning: mkdir() [function.mkdir]: No such file or directory

If you are seeing: Warning: mkdir() [function.mkdir]: No such file or directory in /path/to/pecan/web/runpecan.php at line 169 it is because you have used a relative path for \$output_folder in system.php.

### 35.1.3 After creating a new PFT the tag for PFT not passed to config.xml in ED

This is a result of the rather clunky way we currently have adding PFTs to PEcAn. This is happening because you need to edit the ./pecan/models/ed/data/pftmapping.csv file to include your new PFTs.

This is what the file looks like:

PEcAn;ED
ebifarm.acru;11
ebifarm.acsa3;11
...

You just need to edit this file (in a text editor, no Excel) and add your PFT names and associated number to the end of the file. Once you do this, recompile PEcAn and it should then work for you. We currently need to reference this file in order to properly set the PFT number and maintain internal consistency between PEcAn and ED2.

## 35.2 PEcAn Project use to teach Ecological model-data synthesis

### 35.2.1 University classes

#### 35.2.1.1 GE 375 - Environmental Modeling - Spring 2013, 2014 (Mike Dietze, Boston University)

The final “Case Study: Terrestrial Ecosystem Models” is a PEcAn-based hands-on activity. Each class has been 25 students.

GE 585 - Ecological forecasting Fall 2013 (Mike Dietze, Boston University)

### 35.2.2 Summer Courses / Workshops

#### 35.2.2.1 Annual summer course in flux measurement and advanced modeling (Mike Dietze, Ankur Desai) Niwot Ridge, CO

About 1/3 lecture, 2/3 hands-on (the syllabus is actually wrong as it list the other way around). Each class has 24 students.

2013 Syllabus see Tuesday Week 2 Data Assimilation lectures and PEcAn demo and the Class projects and presentations on Thursday and Friday. (Most students use PEcAn for their group projects. 2014 will be the third year that PEcAn has been used for this course.

#### 35.2.2.2 Assimilating Long-Term Data into Ecosystem Models: Paleo-Ecological Observatory Network (PalEON) Project

Here is a link to the course: https://www3.nd.edu/~paleolab/paleonproject/summer-course/

This course uses the same demo as above, including collecting data in the field and assimilating it (part 3)

#### 35.2.2.3 Integrating Evidence on Forest Response to Climate Change: Physiology to Regional Abundance

http://blue.for.msu.edu/macrosystems/workshop

May 13-14, 2013

Session 4: Integrating Forest Data Into Ecosystem Models

#### 35.2.2.4 Ecological Society of America meetings

Workshop: Combining Field Measurements and Ecosystem Models

### 35.2.3 Selected Publications

1. Dietze, M.C., D.S LeBauer, R. Kooper (2013) On improving the communication between models and data. Plant, Cell, & Environment doi:10.1111/pce.12043
2. LeBauer, D.S., D. Wang, K. Richter, C. Davidson, & M.C. Dietze. (2013). Facilitating feedbacks between field measurements and ecosystem models. Ecological Monographs. doi:10.1890/12-0137.1

## 35.3 Data assimilation with DART

In addition to the state assimilation routines found in the assim.sequential module, another approach for state data assimilation in PEcAn is through the DART workflow created by the DARES group in NCAR.

This section gives a straight-forward explanation how to implement DART, focused on the technical aspects of the implementation. If there are any questions, feel free to send @Viskari an email (tt.viskari@gmail.com) or contacting DART support as they are quite awesome in helping people with problems. Also, if there are any suggestions on how to improve the wiki, please let me know.

Running with current folders in PEcAn

Currently the DART folders in PEcAn are that you can simply copy the structure there over a downloaded DART workflow and it should replace/add relevant files and folders. The most important step after that is to check and change the run paths in the following files: Path_name files in the work folders T_ED2IN file, as it indicates where the run results be written. advance_model.csh, as it indicates where to copy files from/to.

Second thing is setting the state vector size. This is explained in more detail below, but essentially this is governed by the variable model_size in model_mod.f90. In addition to this, it should be changed in utils/F2R.f90 and R2F.f90 programs, which are responsible for reading and writing state variable information for the different ensembles. This also will be expanded below. Finally, the new state vector size should be updated for any other executable that runs it.

Third thing needed are the initial condition and observation sequence files. They will always follow the same format and are explained in more detail below.

Finally the ensemble size, which is the easiest to change. In the work subfolder, there is a file named input.nml. Simply changing the ensemble size there will set it for the run itself. Also remember that initial conditions file should have the equal amount of state vectors as there are ensemble members.

The central file for the actual workflow is advance_model.csh. It is a script DART calls to determine how the state vector changes between the two observation times and is essentially the only file one needs to change when changing state models or observations operators. The file itself should be commented to give a good idea of the flow, but beneath is a crude order of events. 1. Create a temporary folder to run the model in and copy/link required files in to it. 2. Read in the state vector values and times from DART. Here it is important to note that the values will be in binary format, which need to be read in by a Fortran program. In my system, there is a program called F2R which reads in the binary values and writes out in ascii form the state vector values as well as which ED2 history files it needs to copy based on the time stamps. 3. Run the observation operator, which writes the state vector state in to the history files and adjusts them if necessary. 4. Run the program. 5. Read the new state vector values from output files. 6. Convert the state vector values to the binary. In my system, this is done by the program R2F.

Initial conditions file

The initial conditions file, commonly named filter_ics although you can set it to something else in input.nml, is relatively simple in structure. It has one sequence repeating over the number of ensemble members. First line contains two times: Seconds and days. Just use one of them in this situation, but it has to match the starting time given in input.nml. After that each line should contain a value from the state vector in the order you want to treat them. R functions filter_ics.R and B_filter_ics.R in the R folder give good examples of how to create these.

Observations files

The file which contains the observations is commonly known as obs_seq.out, although again the name of the file can be changed in input.nml. The structure of the file is relatively straight-forward and the R function ObsSeq.R in the R subfolder has the write structure for this. Instead of writing it out here, I want to focus on a few really important details in this file. Each observations will have a time, a value, an uncertainty, a location and a kind. The first four are self-explanatory, but the kind is really important, but also unfortunately really easy to misunderstand. In this file, the kind does not refer to a unit or a type of observation, but which member of the state vector is this observation of. So if the kind was, for example, 5, it would mean that it was of the fifth member of the state vector. However, if the kind value is positive, the system assumes that there is some sort of an operator change in comparing the observation and state vector value which is specified in a subprogram in model_mod.f90.

So for an direct identity comparison between the observation and the state vector value, the kind needs to be negative number of the state vector component. Thus, again if the observation is of the fifth state vector value, the kind should be set as -5. Thus it is recommendable that the state vector values have already been altered to be comparable with the observations.

As for location, there are many ways to set in DART and the method needs to be chosen when compiling the code by giving the program which of the location mods it is to use. In our examples we used a 1-dimensional location vector with scaled values between 0 and 1. For future it makes sense to switch to a 2 dimensional long- and lat-scale, but for the time being the location does not impact the system a lot. The main impact will be if the covariances will be localized, as that will be decided on their locations.

State variable vector in DART

Creating/adjusting a state variable vector in DART is relatively straight-forward. Below are listed the steps to specify a state variable vector in DART.

I. For each specific model, there should be an own folder within the DART root models folder. In this folder there is a model_mod.f90, which contains the model specific subroutines necessary for a DART run.

At the beginning of this file there should be the following line:

integer, parameter :: model_size = [number]

The number here should be the number of variables in the vector. So for example if there were three state variables, then the line should look like this:

integer, parameter :: model_size = 3

This number should also be changed to match with any of the other executables called during the run as indicated by the list above.

1. In the DART root, there should be a folder named obs_kind, which contains a file called DEFAULT_obs_kind_mod.F90. It is important to note that all changes should be done to this file instead of obs_kind_mod.f90, as during compilation DART creates obs_kind_mod.f90 from DEFAULT_obs_kind_mod.F90. This program file contains all the defined observation types used by DART and numbers them for easier reference later. Different types are classified according to observation instrument or relevant observation phenomenon. Adding a new type only requires finding an unused number and starting a new identifying line with the following:

integer, parameter, public :: & KIND_…

Note that the observation kind should always be easy to understand, so avoid using unnecessary acronyms. For example, when adding an observation type for Leaf Area Index, it would look like below:

integer, parameter, public :: & KIND_LEAF_AREA_INDEX = [number]

1. In the DART root, there should be a folder named obs_def, which contains several files starting with obs_def_. There files contain the different available observation kinds classified either according to observation instrument or observable system. Each file starts with the line

! BEGIN DART PREPROCESS KIND LIST

And end with line

! END DART PREPROCESS KIND LIST

The lines between these two should contain

! The desired observation reference, the observation type, COMMON_CODE.

For example, for observations relating to phenology, I have created a file called obs_def_phen_mod.f90. In this file I define the Leaf Area Index observations in the following way.

! BEGIN DART PREPROCESS KIND LIST ! LAI, TYPE_LEAF_AREA_INDEX, COMMON_CODE ! END DART PREPROCESS KIND LIST

Note that the exclamation marks are necessary for the file.

1. In the model specific folder, in the work subfolder there is a namelist file input.nml. This contains all the run specific information for DART. In it, there is a subtitle &preprocess, under which there is a line

input_files = ‘….’

This input_files sections must be set to refer to the obs_def file created in step III. The input files can contain references to multiple obs_def files if necessary.

As an example, the reference to the obs_def_phen_mod.f90 would look like input_files = ‘../../../obs_def/obs_def_phen_mod.f90’

V. Finally, as an optional step, the different values in state vector can be typed. In model_mod, referred to in step I, there is a subroutine get_state_meta_data. In it, there is an input variable index_in, which refers to the vector component. So for instance for the second component of the vector index_in would be 2. If this is done, the variable kind has to be also included at the beginning of the model_mod.f90 file, at the section which begins

use obs_kind_mod, only ::

The location of the variable can be set, but for a 0-dimensional model we are discussing here, this is not necessary.

Here, though, it is possible to set the variable types by including the following line

if(index_in .eq. [number]) var_type = [One of the variable kinds set in step II]

1. If the length of the state vector is changed, it is important that the script ran with DART produces a vector of that length. Change appropriately if necessary.

After these steps, DART should be able to run with the state vector of interest.


text:
# How do you compute the tangent space vectors with normals given in the mesh?

Suppose you are given a mesh with normals for each vertex. These normals might not correspond to the face normals, as the artist could have tweaked them. In this case, how would you go about computing the tangent and binormal vectors for the vertex?

The best algorithm I can think of, and the one that looks to be used by assimp, is to take all the faces in a given smoothing group and compute the tangent and binormal per face. Once that is done, project them into the plane determined by the normal and average over all the faces in the smoothing group.

I am uncertain as to whether this algorithm works exactly as needed. For example, does the projection operator preserve the angle between the tangent and binormal?

Another issue I have is that the assimp implementation of the algorithm does not seem to use the smoothing groups specified in the mesh. They instead seem to calculate the smoothing groups manually using nearby vertices. What is the advantage of doing this?

Finally, is this the ideal algorithm? Some other things I have found (such as this) seem to suggest that it is not, but they do not go into any more details.

EDIT:

So I checked out the MikkTSpace algorithm that was suggested in the comments, and while it answers my question of what the industry standard method for generating tangent spaces is, I do not fully understand why the algorithm works.

As described in the paper, the algorithm works by collecting triangles into groups and then averaging the tangent spaces for each individual triangle in the group. I ran into two problems understanding this: first and once again, why discard the smoothing groups specified in the mesh? It seems wasteful to discard information that is already given to you about how the mesh is supposed to appear.

Secondly, how do the groups even work? The paper specifies that two triangles are considered to be in a group if the share two vertices, two texture coordinates, and two normals. The first two requirements make sense to me, but the third does not. Suppose you are approximating a sphere by something like an icosahedron. In this case there will be no vertex normals shared by any two vertices, so each individual triangle is going to have its own tangent space. This seems counterintuitive.

• There are many methods to compute tangent spaces, which can add to frustration when trying to get two parts of your pipeline to agree. ;) One popular one though is called MikkTSpace, after Morten S. Mikkelsen. Search that name and you'll find publicly released source code for working with this tangent convention. – DMGregory Aug 7 '17 at 17:27
• The problem is that the author says they must share two normals, ie when two triangles meet in two vertices, both vertices must have the same normal. This means that since no two normals of a sphere are the same, no two triangles will share a tangent space. – Josiah Aug 9 '17 at 18:06
• I don't think that's what they're saying. I think they mean if triangle A B C and triangle a b c share an edge between vertices A/a and B/b, that A & a should have the same position, texture coordinate, and normal (our first shared normal), AND B & b should have the same position, texture coordinate, and normal (our second shared normal), giving us a total of two shared normals on this shared edge. Contrast this to the case where b has a different normal than B, then there's a sharp crease along the edge A/a-B/b, and we cannot guarantee a continuous tangent space across the crease. – DMGregory Aug 9 '17 at 18:11
• Ok, I guess I was misunderstanding what I read, that makes a lot more sense. That answers my main problem with the algorithm, but I would still like to know why the smoothing group information is ignored. If this information is artist generated, then ignoring it seems like it would lead to results that are not quite what the artist intended. – Josiah Aug 9 '17 at 18:15
• Where do you think shared and unshared normals come from? ;) – DMGregory Aug 9 '17 at 18:16

Summarizing the clarifications above as an answer:

There are multiple ways to generate tangent spaces for a mesh, and not all of them agree on the result.

This is a common source of rendering errors in games, where the normal map baking tool generated the texture with respect to one tangent space, and the 3D modelling software or game's mesh importer decided on a different tangent space, leading to mismatches and artifacts, as shown in these examples from the Handplane documentation:

So, we need to pick a standard tangent space to use.

A popular choice is "MikkTSpace" a method for generating tangent spaces that Morten S. Mikkelsen developed as part of his master's thesis. He specifically designed the algorithm to be robust for use by multiple tools in an asset pipeline, so they can independently generate the same tangent basis regardless of quirks like choices of vertex welding or the order of the vertices & faces.

Code for the MikkTSpace algorithm is freely available online, and I'm not an expert in all of its workings, so I won't describe a complete implementation here. Instead I'll address the specific questions about it raised above.

Why do these algorithms disregard smoothing groups?

"Smoothing groups" don't really exist on GPUs or in most game engines - they're a concept used in 3D modelling tools to make manipulating normals more intuitive.

By the time a mesh is pushed down the graphics pipeline, it's just a raw stream of vertices combining position, texture coordinates, normals, etc.

Wherever two entries in this stream coincide at the same position, but have different normals, you'll get a hard crease or lighting seam along any edges they share (or a point discontinuity like the tip of a cone if they don't share any edges)

Wherever you have an edge where all vertices at the start of the edge agree on their normal, and all vertices at the end of the edge agree on their (possibly different) normal, you'll get a smooth join with no discontinuity.

Smoothing groups exist to tell the 3D package where it should force vertices along a shared edge to share a normal, versus where the normal can be independently chosen for each vertex.

By the time the mesh gets to the tangent space baking step, these smoothing groups have typically already been converted to vertex splits, so the tangent space algorithm doesn't need to be aware of a particular tool's smoothing conventions - it can just work with the literal vertex data.

No artist-authored smoothing information is lost this way, it's just already been translated into the lower level form these algorithms understand.

Why does the MikkTSpace algorithm group only faces that share two vertices, two texture coordinates, AND two normals?

"Suppose you are approximating a sphere by something like an icosahedron. In this case there will be no vertex normals shared by any two vertices, so each individual triangle is going to have its own tangent space. This seems counterintuitive."

This looks like a misunderstanding of the algorithm, as though it said:

"Two triangles can be grouped only if the vertices at both ends of the shared edge agree on a single shared vertex normal."

But it actually says:

1. two vertices [(vertex positions)] are shared.
2. vertex normals at the two vertices are shared.
3. texture coordinates at the two vertices are shared.
4. the triangles must have the same sign of det(T). [(ie. Either neither, or both, are mirrored in texture space - not one of each)]

Point 2 means that:

• triangle A and triangle B's vertices at the START of the shared edge must share one vertex normal
• triangle A and triangle B's vertices at the END of the shared edge must share a (possibly distinct) second vertex normal

...for a total of two shared normals along the common edge.

So you can see this is the same condition we described above with regard to smoothing groups. If this condition is not met - the vertices at at least one end of the edge disagree about their normal direction - then we'll have a sharp crease along this edge. Since the normal experiences a sharp discontinuity along this edge, the tangent basis (which includes the normal) will also be discontinuous along this edge.

Why does the algorithm accumulate magnitudes of vectors?

Checking through the code, you'll find the magnitudes of the partial derivatives are not used for the "basic" version of the tangent space.

For the more advanced version, here is Mikkelsen's original comment:

// This function is used to return tangent space results to the application.
// fvTangent and fvBiTangent are unit length vectors and fMagS and fMagT are their
// true magnitudes which can be used for relief mapping effects.

Relief Mapping is an effect where we approximate parallax and self-occlusion of surface structure by ray-marching through a height field texture from our initial surface sample point. We imagine the surface has some depth to it, and that our view ray can continue some distance from where it hit the bounding volume of the polygon geometry, before it actually hits the displaced surface underneath.

To make it work, we need to transform our view vector from eye/world/object space into texture space:

To do that precisely, we need to know more than just how the texture is oriented with regard to the 3D geometry (what we get from tangent & normal directions), we also need to know how it's scaled. If we ignore this, then a ray entering a compressed part of the texture will behave like it's refracted in water, covering less worldspace distance parallel to the surface for each unit of travel perpendicular, distorting the effect. We can use the magnitude information provided by the algorithm to compensate and ensure our texture space ray matches the direction of our view ray in the world.

There are other effects that might benefit from having this type of scale information about the texture mapping (maybe some forms of tesselation using a control texture?) but if you're just using the tangent space for standard normal mapping and only care about directions, then you can safely ignore the magnitude tracking the MikkTSpace algorithm does and use the basic version instead.

• Wow, this answer is great, I had found the comment in the code but was not sure if it was the only reason it stored the magnitudes. Thank you very much. – Josiah Aug 12 '17 at 1:02
• I'll confess I don't know for sure if there are other reasons to use these magnitudes, but from examining the code they didn't seem to be used to determine the tangent vectors themselves... Hopefully if someone finds other important cases for these they'll post another answer. :) – DMGregory Aug 12 '17 at 1:03
• Also, just for anyone coming by and wanting to know where in the paper the algorithm is outlined, the specific steps are given on pages 47-48. The tangent and bitangent/binormal are described in the paper as the first order derivatives. – Josiah Aug 12 '17 at 1:17

format: plain


text:
# CORNELIA PARKER // COLD DARK MATTER: AN EXPLODED VIEW // 1991 | PART 1 // Research by Connie Elizabeth Burlton

OTHER INFLUENTIAL ARTISTS:CORNELIA PARKER // COLD DARK MATTER: AN EXPLODED VIEW // 1991// Cornelia Parker's Cold Dark Matter installation is the view of an exploded shed suspended in space. She has successfully frozen and preserved a second in time. The calamity can violence of the scene is juxtaposed by the tranquillity of the actual installation. It is heavily lit from the centre and as a result dramatic shadows are cast on the surrounding walls so that the explosion  appears to be expanding outwards.
Like Parker, our installation involving broken glass, illustrates the force of destruction by freezing a brief moment in time. As a result, the expected brutality is contrasted by the peace and beauty of the work.CORNELIA PARKER:"This piece came out of a series of works I was doing about cartoon deaths - things like, things falling off cliffs, things being run over by a steam roller, things being blown up, shot full of bullets, like Roadrunner or Tom and Jerry. The garden shed came about because I was trying to find something universal and archetypal and that we all identified with and that was familiar to us. It's not the house but it's this kind of attic-y private place at the bottom of the garden which we put all our left-over stuff in. And so it seemed like a depository rather than the place that you live. We took it out to the Banbury Army School of Ammunition, to their demolition grounds where they do all these experiments with explosives and they were really keen to blow it up. I actually pressed the button that detonated it.The whole point of suspending it is to rob it of its pathos. After it was blown up and all the objects were lying on the floor, all very distressed, they had a pathos and somehow putting it back in the air where they were a little while before, it sort of re-animates them.The title of the piece is called 'Cold Dark Matter: An Exploded View'. It's a two-part title really. The 'cold dark matter' I really like because obviously explosions have all kinds of connotations, dark ones being the most prominent. Also, 'cold dark matter' sounds like a psychological state - a mood or an atmosphere or a depression - I like the sound of that. It's a scientific term: it was coined to describe all the stuff in the universe you can't quantify, all the stuff they know is there but you can't see, which seemed a perfect description. And then 'an exploded view' is the kind of diagram you get in technical manuals to describe how a car works or a bike or a lawnmower, a very pragmatic laying out of stuff. And so that's what I was trying to do, to organise something tat was totally beyond our control and emotional control."


text:
# Staking Pool Operations

This document describes how to perform staking pool operations. Note that you can stake only when you meet the minimal staking requirement.

Minimum staking requirement

The current required minimum for staking is 1M APT tokens.

## Initializing the stake pool

Make sure that this initializing the stake pool step was performed by the owner. See Initialize staking pool in the owner documentation section.

## Joining validator set

Errors?

The validator node cannot sync until the stake pool becomes active. If you see errors, see the Issues and Workarounds.

Follow the below steps to set up the validator node using the operator account and join the validator set.

Mainnet vs Testnet

The below CLI command examples use mainnet. See the --rest-url value for testnet or devnet in Aptos Blockchain Deployments.

### 1. Initialize Aptos CLI

aptos init --profile mainnet-operator \--private-key <operator_account_private_key> \--rest-url https://fullnode.mainnet.aptoslabs.com/v1 \--skip-faucet
tip

### 5. Join the validator set

aptos node join-validator-set \  --pool-address <pool-address> \  --profile mainnet-operator

The validator set is updated at every epoch change. You will see your validator node joining the validator set only in the next epoch. Both validator and validator fullnode will start syncing once your validator is in the validator set.

When is next epoch?

Run the command aptos node get-stake-pool as shown in Checking your stake pool information. You can also follow these steps: How to find out when the next epoch starts.

### 6. Check the validator set

When you join the validator set, your validator node will be in "Pending Active" state until the next epoch occurs. During this time you might see errors like "No connected AptosNet peers". This is normal. Run the below command to look for your validator in the "pending_active" list.

aptos node show-validator-set --profile mainnet-operator | jq -r '.Result.pending_active' | grep <pool_address>

When the next epoch happens, the node will be moved into "active_validators" list. Run the below command to see your validator in the "active_validators" list:

aptos node show-validator-set --profile mainnet-operator | jq -r '.Result.active_validators' | grep <pool_address>

## Checking your stake pool information

How validation works

Before you proceed, see Validation on the Aptos blockchain for a brief overview.

To check the details of your stake pool, run the below CLI command with the get-stake-pool option by providing the --owner-address and --url fields.

The below command is for an example owner address e7be097a90c18f6bdd53efe0e74bf34393cac2f0ae941523ea196a47b6859edb.

tip

For testnet or devnet --url field values, see Aptos Blockchain Deployments.

aptos node get-stake-pool \  --owner-address e7be097a90c18f6bdd53efe0e74bf34393cac2f0ae941523ea196a47b6859edb \  --profile mainnet-operator

Example output:

{  "Result": [    {      "state": "Active",       "pool_address": "25c3482850a188d8aa6edc5751846e1226a27863643f5ebc52be4f7d822264e3",      "operator_address": "3bec5a529b023449dfc86e9a6b5b51bf75cec4a62bf21c15bbbef08a75f7038f",      "voter_address": "3bec5a529b023449dfc86e9a6b5b51bf75cec4a62bf21c15bbbef08a75f7038f",      "pool_type": "StakingContract",      "total_stake": 100525929489123,      "commission_percentage": 10,      "commission_not_yet_unlocked": 15949746439,      "lockup_expiration_utc_time": "2022-10-07T07:12:55Z",      "consensus_public_key": "0xb3a7ac1491b0165f08f136c2b02739846b6610084984d5298c2983c4f8e5553284bffca2e3fe2b99167da82717501732",      "validator_network_addresses": [        "/ip4/35.91.145.164/tcp/6180/noise-ik/0xeddf05470520af91b847f353dd804a04399e1213d130a4260e813527f2c49262/handshake/0"      ],      "fullnode_network_addresses": [],      "epoch_info": {        "epoch": 594,        "epoch_interval_secs": 3600,        "current_epoch_start_time": {          "unix_time": 1665087178789891,          "utc_time": "2022-10-06T20:12:58.789891Z"        },        "next_epoch_start_time": {          "unix_time": 1665090778789891,          "utc_time": "2022-10-06T21:12:58.789891Z"        }      }    }  ]}

### Description of output fields

state

• "Active": Validator is already in the validator set and proposing.
• "Pending_active": Validator will be added to the validator set in the next epoch. Do not try to join the validator set again before the arrival of next epoch, or else you will receive an error.

• Use this "pool_address" (not the operator address) in you validator.yaml file. If you mistakenly used the operator address, you will receive the message: "Validator not in validator set".

commission_percentage

• This can be set only by the stake pool owner. Operator receives the "commission_percentage" of the generated staking rewards. If you request the commission (you can do so by running the command aptos stake request-commission), then at the end of the lockup_expiration_utc_time the commission part of the rewards will go to the operator address while the rest will stay in the stake pool and belong to the owner. Here "the commission part of the rewards" means the value of commission_not_yet_unlocked.

For example, in a scenario with a lock up of one month, you call aptos stake request-commission every month. This will pay out the commission that was accrued during the previous month but only when unlocked at the end of the previous month. Regardless of how often you run aptos stake request-commission during the month, the commission is only paid out upon the completion of lockup_expiration_utc_time.

Compounding

Note that if you do not request commission for multiple months, your commission will accrue more due to compounding of the commission_percentage during these months.

commission_not_yet_unlocked

• The amount of commission (amount of APT) that is not yet unlocked. It will be unlocked at the lockup_expiration_utc_time. This is the total commission amount available to the operator, i.e., the staking rewards only to the operator. This does not include the staking rewards to the owner.

lockup_expiration_utc_time

• The date when the commission will unlock. However, this unlocked commission will not be auto-disbursed. It will only disburse when the command aptos stake request-commission is called again.

epoch_info

• Use the Epoch Converter to convert the unix_time into human readable time.

## Requesting commission

Either an owner or an operator can request commission. You can request commission at the end of a lockup period, i.e., at the end of lockup_expiration_utc_time, by running the aptos stake request-commission command. Make sure to provide the operator and the owner addresses. See an example command below:

aptos stake request-commission \  --operator-address 0x3bec5a529b023449dfc86e9a6b5b51bf75cec4a62bf21c15bbbef08a75f7038f \  --owner-address 0xe7be097a90c18f6bdd53efe0e74bf34393cac2f0ae941523ea196a47b6859edb \  --profile mainnet-operator

## Frequently used staking operations commands

To see your validator performance in the current and past epochs and the rewards earned, run the below command. The output will show the validator's performance in block proposals, and in governance voting and governance proposals. Default values are used in the below command. Type aptos node get-performance --help to see default values used.

aptos node get-performance \   --pool-address <pool address> \  --profile mainnet-operator

Example output:

{  "Result": {    "current_epoch_successful_proposals": 56,    "current_epoch_failed_proposals": 0,    "previous_epoch_rewards": [      "12312716242",      "12272043711",      "12312912674",      "12313011054",      "12313109435",      "12180092056",      "12313305136",      "12313403519",      "12313501903",      "12313600288"    ],    "epoch_info": {      "epoch": 68,      "epoch_interval": 3600000000,      "last_epoch_start_time": {        "unix_time": 1665074662417326,        "utc_time": "2022-10-06T16:44:22.417326Z",        "local_time": "Thu Oct  6 16:44:22 2022"      },      "next_epoch_start_time": {        "unix_time": 1665078262417326,        "utc_time": "2022-10-06T17:44:22.417326Z",        "local_time": "Thu Oct  6 17:44:22 2022"      }    }  }}

#### Description of fields

current_epoch_successful_proposals

• Successful leader-validator proposals during the current epoch. Also see Validation on the Aptos blockchain for the distinction between leader-validator and the voter-validator.

previous_epoch_rewards

• An ordered list of rewards earned (APT amounts) for the previous 10 epochs, starting with the 10 epoch in the past. In the above example, a reward of 12312716242 APT was earned 10 epochs past and a reward of 12313600288 APT was earned in the most recent epoch. If a reward is 0 for any epoch, then:
• Either the validator was not part of the validator set in that epoch (could have been in either inactive or pending_active validator state), or
• The validator missed all the leader proposals.

### Checking the performance for all epochs

To check the performance of all the epochs since the genesis, run the below command. You can filter the results for your pool address with grep, as shown below:

aptos node analyze-validator-performance \  --analyze-mode detailed-epoch-table \  --profile mainnet-operator \  --start-epoch 0 | grep <pool address>


text:
# Electronic structure of NaWO$3$: Role of the impurity potential - Condensed Matter > Materials Science

Electronic structure of NaWO$3$: Role of the impurity potential - Condensed Matter > Materials Science - Download this document for free, or read online. Document in PDF available to download.

Abstract: We have performed {\it ab-initio} electronic structure calculations todetermine the evolution of the electronic structure of WO$3$ with Na doping.Na doping introduces an additional electron when introduced into WO$3$. Theensuing electronic structure of Na doped WO$3$, we find, is very similar tothe electronic structure of an electron introduced into WO$3$, thus clarifyingthe role of the impurity potential due to Na. While the electronic structure ofNaWO$3$ allows a rigid band like description over a certain energy range,modifications introduced in the electronic structure can be related back to theelectron due to Na and not the impurity potential that one generally believesto be responsible.

Author: Priya Mahadevan, Roby Cherian, K.J. Sankaran, D.D. Sarma

Source: https://arxiv.org/


text:
# Developing an Isolated Word Recognition System in MATLAB

By Daryl Ning, MathWorks

Speech-recognition technology is embedded in voice-activated routing systems at customer call centres, voice dialling on mobile phones, and many other everyday applications. A robust speech-recognition system combines accuracy of identification with the ability to filter out noise and adapt to other acoustic conditions, such as the speaker’s speech rate and accent. Designing a robust speech-recognition algorithm is a complex task requiring detailed knowledge of signal processing and statistical modeling.

This article demonstrates a workflow that uses built-in functionality in MATLAB® and related products to develop the algorithm for an isolated digit recognition system. The system is speaker-dependent—that is, it recognizes speech only from one particular speaker’s voice.

### Classifying Speech-Recognition Systems

Most speech-recognition systems are classified as isolated or continuous. Isolated word recognition requires a brief pause between each spoken word, whereas continuous speech recognition does not. Speech-recognition systems can be further classified as speaker-dependent or speaker-independent. A speaker-dependent system only recognizes speech from one particular speaker's voice, whereas a speaker-independent system can recognize speech from anybody.

### The Development Workflow

There are two major stages within isolated word recognition: a training stage and a testing stage. Training involves “teaching” the system by building its dictionary, an acoustic model for each word that the system needs to recognize. In our example, the dictionary comprises the digits ‘zero’ to ‘nine’. In the testing stage we use acoustic models of these digits to recognize isolated words using a classification algorithm.

The development workflow consists of three steps:

• Speech acquisition
• Speech analysis
• User interface development

## Acquiring Speech

For training, speech is acquired from a microphone and brought into the development environment for offline analysis. For testing, speech is continuously streamed into the environment for online processing.

During the training stage, it is necessary to record repeated utterances of each digit in the dictionary. For example, we repeat the word ‘one’ many times with a pause between each utterance.

Using the following MATLAB code with a standard PC sound card, we capture ten seconds of speech from a microphone input at 8000 samples per second:

Fs = 8000; % Sampling Freq (Hz)
Duration = 10; % Duration (sec)
y = wavrecord(Duration*Fs,Fs);

We save the data to disk as ‘mywavefile.wav’:

This approach works well for training data. In the testing stage, however, we need to continuously acquire and buffer speech samples, and at the same time, process the incoming speech frame by frame, or in continuous groups of samples.

We use Data Acquisition Toolbox™ to set up continuous acquisition of the speech signal and simultaneously extract frames of data for processing.

The following MATLAB code uses a Windows sound card to capture data at a sampling rate of 8000 Hz. Data is acquired and processed in frames of 80 samples. The process continues until the “RUNNING” flag is set to zero.

% Define system parameters
framesize = 80;       % Framesize (samples)
Fs = 8000;            % Sampling Frequency (Hz)
RUNNING = 1;          % A flag to continue data capture

% Setup data acquisition from sound card

set(ai, 'SampleRate', Fs);
set(ai, 'SamplesPerTrigger', framesize);
set(ai, 'TriggerRepeat',inf);
set(ai, 'TriggerType', 'immediate');

% Start acquisition
start(ai)

% Keep acquiring data while "RUNNING" ~= 0
while RUNNING
% Acquire new input samples
newdata = getdata(ai,ai.SamplesPerTrigger);
% Do some processing on newdata
...
<DO _ SOMETHING>
...

% Set RUNNING to zero if we are done
if <WE _ ARE _ DONE>
RUNNING = 0;
end

end

% Stop acquisition
stop(ai);

% Disconnect/Cleanup
delete(ai);
clear ai;

## Analyzing the Acquired Speech

We begin by developing a word-detection algorithm that separates each word from ambient noise. We then derive an acoustic model that gives a robust representation of each word at the training stage. Finally, we select an appropriate classification algorithm for the testing stage.

## Developing a Speech-Detection Algorithm

The speech-detection algorithm is developed by processing the prerecorded speech frame by frame within a simple loop. For example, this MATLAB code continuously reads 160 sample frames from the data in ‘speech’.

% Define system parameters
seglength = 160;                    % Length of frames
overlap = seglength/2;              % # of samples to overlap
stepsize = seglength - overlap;     % Frame step size
nframes = length(speech)/stepsize-1;

% Initialize Variables
samp1 = 1; samp2 = seglength; %Initialize frame start and end

for i = 1:nframes
% Get current frame for analysis
frame = speech(samp1:samp2);

% Do some analysis
...
<DO _ SOMETHING>
...

% Step up to next frame of speech
samp1 = samp1 + stepsize;
samp2 = samp2 + stepsize;
end

To detect isolated digits, we use a combination of signal energy and zero-crossing counts for each speech frame. Signal energy works well for detecting voiced signals, while zero-crossing counts work well for detecting unvoiced signals. Calculating these metrics is simple using core MATLAB mathematical and logical operators. To avoid identifying ambient noise as speech, we assume that each isolated word will last at least 25 milliseconds.

## Developing the Acoustic Model

A good acoustic model should be derived from speech characteristics that will enable the system to distinguish between the different words in the dictionary.

We know that different sounds are produced by varying the shape of the human vocal tract and that these different sounds can have different frequencies. To investigate these frequency characteristics we examine the power spectral density (PSD) estimates of various spoken digits. Since the human vocal tract can be modeled as an all-pole filter, we use the Yule-Walker parametric spectral estimation technique from Signal Processing Toolbox™ to calculate these PSDs.

After importing an utterance of a single digit into the variable ‘speech’, we use the following MATLAB code to visualize the PSD estimate:

      order = 12;
nfft = 512;
Fs = 8000;
pyulear(speech,order,nfft,Fs);

Since the Yule-Walker algorithm fits an autoregressive linear prediction filter model to the signal, we must specify an order of this filter. We select an arbitrary value of 12, which is typical in speech applications.

Figures 1a and 1b plot the PSD estimate of three different utterances of the words ‘one’ and ‘two’. We can see that the peaks in the PSD remain consistent for a particular digit but differ between digits. This means that we can derive the acoustic models in our system from spectral features.

Figure 1a. Yule Walker PSD estimate of three different utterances of the word “ONE.” Click on image to see enlarged view.
Figure 1b. Yule Walker PSD estimate of three different utterances of the word “TWO.” Click on image to see enlarged view.

From the linear predictive filter coefficients, we can obtain several feature vectors using Signal Processing Toolbox functions, including reflection coefficients, log area ratio parameters, and line spectral frequencies.

One set of spectral features commonly used in speech applications because of its robustness is Mel Frequency Cepstral Coefficients (MFCCs). MFCCs give a measure of the energy within overlapping frequency bins of a spectrum with a warped (Mel) frequency scale1.

Since speech can be considered to be short-term stationary, MFCC feature vectors are calculated for each frame of detected speech. Using many utterances of a digit and combining all the feature vectors, we can estimate a multidimensional probability density function (PDF) of the vectors for a specific digit. Repeating this process for each digit, we obtain the acoustic model for each digit.

During the testing stage, we extract the MFCC vectors from the test speech and use a probabilistic measure to determine the source digit with maximum likelihood. The challenge then becomes to select an appropriate PDF to represent the MFCC feature vector distributions.

Figure 2a shows the distribution of the first dimension of MFCC feature vectors extracted from the training data for the digit ‘one’.

Figure 2a. Distribution of the first dimension of MFCC feature vectors for the digit ‘one.’ Click on image to see enlarged view.

We could use dfittool in company/newsletters/articles/developing-an-isolated-word-recognition-system-in-matlab.html™ to fit a PDF, but the distribution looks quite arbitrary, and standard distributions do not provide a good fit. One solution is to fit a Gaussian mixture model (GMM), a sum of weighted Gaussians (Figure 2b).

Figure 2b. Overlay of estimated Gaussian components (red) and overall Gaussian mixture model (green) to the distribution in 4(a). Click on image to see enlarged view.

The complete Gaussian mixture density is parameterized by the mixture weights, mean vectors, and covariance matrices from all component densities. For isolated digit recognition, each digit is represented by the parameters of its GMM.

To estimate the parameters of a GMM for a set of MFCC feature vectors extracted from training speech, we use an iterative expectation-maximization (EM) algorithm to obtain a maximum likelihood (ML) estimate. Given some MFCC training data in the variable MFCCtraindata, we use the Statistics and Machine Learning Toolbox gmdistribution function to estimate the GMM parameters. This function is all that is required to perform the iterative EM calculations.

  %Number of Gaussian component densities
M = 8;
model = gmdistribution.fit(MFCCtraindata,M);

## Selecting a Classification Algorithm

After estimating a GMM for each digit, we have a dictionary for use at the testing stage. Given some test speech, we again extract the MFCC feature vectors from each frame of the detected word. The objective is to find the digit model with the maximum a posteriori probability for the set of test feature vectors, which reduces to maximizing a log-likelihood value. Given a digit model gmmmodel and some test feature vectors testdata, the log-likelihood value is easily computed using the posterior function in Statistics and Machine Learning Toolbox:

[P, log _ like] =
posterior(gmmmodel,testdata);

We repeat this calculation using each digit’s model. The test speech is classified as the digit with the GMM that produces the maximum log-likelihood value.

## Building the User Interface

After developing the isolated digit recognition system in an offline environment with prerecorded speech, we migrate the system to operate on streaming speech from a microphone input.We use MATLAB GUIDE tools to create an interface that displays the time domain plot of each detected word as well as the classified digit (Figure 3).

Figure 3. Interface to final application. Click on image to see enlarged view.

### Extending the Application

The algorithm described in this article can be extended to recognize isolated words instead of digits, or to recognize words from several speakers by developing a speaker-independent system.

If the goal is to implement the speech recognition algorithm in hardware, we could use MATLAB and related products to simulate fixed-point effects, automatically generate embedded C code, and verify the generated code.

Published 2010 - 91805v00


text:
# Does a FTS work on the same principle as a michelson (amplitude division) interferometer?

As far as I can tell within an Fourier Transform Spectrometer the spectral information is gained from changing the path length along one arm, this sounds very similar to a michelson interferometer but using two apertures instead of one.

So are there any underlying differences between the two?

-

You are right, fourier transform spectrometer is just a scanning Michelson interferometer. In spectroscopic applications these are just synonims. Spectral information is a fourier transform of intensity dependence on path length, thus the name. Usually a term wavemeter is used, meaning some interferometric device to measure wavelength, including FTS as well as Fabry–Pérot interferometers.

-