text:
# The Making of FAB 9000

INFORMATION: This section is structured like a blog. The most recent notes are at the top and the oldest at the bottom.

### AUGUST 8, 2013. Thursday

In about 12 hours we are presenting again final projects and our web sites, and this will be the cutoff for graduation this year. This is the situation right now (1:49 AM): I have to assemble the arm again which will take me about half an hour. I do not have transmission for the elbow joint, tomorrow we'll see what I can do. Motors still do not have enough torque and I don't know why. I thought they needed more power and I just fried stepper driver number 2 using another power supply. Lucky me, before switching the power supply on, I remembered the spiral development and unplugged all the boards except this one, just in case. And finally, I still have to update two tutorials that I left unfinished. I don't know how much time I am going to sleep today, or if I am going to sleep at all. But there is one thing that I know for sure:

### JULY 24, 2013. Wednesday

Today I re-worked the lasercut parts. When you lasercut a part and you notice that you forgot some shapes you have two and only two options. Option 1: Lasercut the whole part again. Option 2: lasercut only the missing shapes in the part that you already have. It is somewhat tricky and I am preparing a tutorial of it. But it can save you a lot of money in materials without spending too much time. The reason I did that is because I changed the motors, and new mount holes were required.

### JULY 23, 2013. Tuesday

I've been like two weeks trying to solve the error of the weight calculation. The arm actually weights 67 grams, not 48 as I calculated. At the beginning I could not find out why. I checked the formulas and there were some errors due to radians and degrees. I fixed them, but still, calculations said 58 g instead of 67g. I double checked every line of code and everything seemed OK.

This evening I was looking at the code and when I raised a little bit my eyes I saw the word density. My density was set at 0.94 g/cm3, I found this value in the Internet. When I saw it for the first time I thought: lighter than water? But hey, it is The Internet, it knows it all. Just in case, I looked again for acrylic density and I found another value: from 1.15 to 1.20 g/cm3. I put 1.20 in the density parameter and you know what was the calculated weight, don't you? This is the second time The Internet fools me.

### JULY 15, 2013. Monday

Wow, I have just received an email from the European Edition of Maker Faire in Rome!!

Dear Francisco Sanchez Arroyo,

we are happy to inform you that your booth proposal Robotic Arm FAB 9000 for the Maker Faire Rome, based on the Call for Makers was accepted. The Maker Faire will take place in Rome at Palazzo dei Congressi, from 3rd till 6th of October 2013.

blah, blah, blah

Massimiliano Colella
www.makerfairerome.eu

That is amazing, I'm going to meet all the best makers of Europe. Here we go Rome!

### JUNE 19, 2013. Wednesday

Yesterday I was thinking again about integrating physics simulation inside the CAD design file. About a month ago I emailed Matt Keeter asking him the way of calculating the area of a shape in kokopelli. It looks like there is no easy solution but he found a way to do it by counting the white pixels of the shape. The resulting area is not analytical and hence the precision lays in the resolution of the image. So yesterday I thought that I could calculate analytically the area of my shapes so I started doing this. I always start with a piece of paper:

In the above image there is an error. Beta is actually 180 - alpha. Later I created a custom function for calculating the area, volume and mass of the red arm:

My next step is disassemble the arm and check that the arm weights about 48.87 g, I will do it next Tuesday, when I will try also to finish a couple of unfinished assignments.

### JUNE 15, 2013. Saturday

I have successfully implemented linear acceleration motion to the motors in ArmControl.py. When I first used Easydriver with the accelstepper library I thought it was very difficult, and that it was out of my understanding. But it's just a matter of adjusting the delays between steps. The thing I like the most is that I implemented the acceleration without reprogramming the boards, it's all done in the Virtual Control side. I can think of a few advantages of this, like having the arm in a place you cannot reach, or a hostile environment.

def shoulder(self,steps,sp):
engine.say('Rotating shoulder')
engine.say(steps)
engine.say('steps at')
engine.say(sp)
engine.say('steps per second')
engine.runAndWait()
# Moving with linear acceleration
constant_delay=1./sp
ramp_steps=int((max_delay-constant_delay)/accel_delay)
if 2*ramp_steps>steps: ramp_steps=int(steps/2)
flat_steps=steps-(2*ramp_steps)
actual_delay=max_delay # Init delay
if steps >=0: # Moving CW
for k in range (ramp_steps): # Ramping up
ser.write('(')
time.sleep(actual_delay)
actual_delay-=accel_delay
for k in range (flat_steps): # Constant speed
ser.write('(')
time.sleep(constant_delay)
for k in range (ramp_steps): # Ramping down
ser.write('(')
actual_delay+=accel_delay
time.sleep(actual_delay)
else: # Moving CCW
for k in range (ramp_steps): # Ramping up
ser.write(')')
time.sleep(actual_delay)
actual_delay-=accel_delay
for k in range (flat_steps): # Constant speed
ser.write(')')
time.sleep(constant_delay)
for k in range (ramp_steps): # Ramping down
ser.write(')')
actual_delay+=accel_delay
time.sleep(actual_delay)
self.arm_sho+=steps
print 'Current angular position ('+ str(self.arm_rot)+','+ str(self.arm_sho)+','+ str(self.arm_elb)+ ','+str(self.arm_wri)+')'
return

### JUNE 14, 2013. Friday

I have been talking to Guillem lately. He convinced me to go back to the original idea of Virtual Control (the computer program controls the hardware step by step). So I am not going to follow Neil's advice for the first time. I hope he doesn't mind. He told me not to send serial commands for every step but to send "move N steps". He told me that the clock is not reliable in my computer, it is more reliable in the micro-controller. I started reading how to control time in Python, it is done with the time.sleep() method, but it is only reliable down to about 10-15 ms. If you want to lower it to near 1 ms you need a RTOS (Real Time Operating System). I also read that Ubuntu has a kernel that can do RTOS. But I am on a Mac now.

So let's be in the safe area. Assume my lower delay (maximum speed) will be 20 ms. That means that the maximum speed I can reach with the arm will be 1/20/1000=50 steps per second. Since my motors have 200 steps per revolution that means it will take 2 seconds for moving 180 degrees (which is the usual span for an arm) or 4 seconds in half stepping mode. I don't need it to be faster that this.

So that is what I did. I reprogrammed original Neil's code (I am so happy that I returned back to C and leaved the Arduino IDE). The code is constantly listening to the serial port. When it receives a certain character it moves one step clockwise. When it reads another certain character it moves one step counterclockwise. That simple! Here is the loop for one of the boards:

while (1) {

get_char(&serial_pins, serial_pin_in, &chr);
if (chr == 40) {  // This is the '(' Character
step_cw();}
if (chr == 41) {  // This is the ')' Character
step_ccw();}

}

In the computer side, this is the code to move a certain number of steps at a certain speed:

def rotate(self,steps,sp):
engine.say('Rotating Arm')
engine.say(steps)
engine.say('steps at')
engine.say(sp)
engine.say('steps per second')
engine.runAndWait()
# Moving step by step
constant_delay=1./sp
if steps >= 0:
for k in range (steps):
ser.write('(')
time.sleep(constant_delay)
else:
for k in range (steps):
ser.write(')')
time.sleep(constant_delay)
self.arm_rot+=steps
print 'Current angular position ('+ str(self.arm_rot)+','+ str(self.arm_sho)+','+ str(self.arm_elb)+ ','+str(self.arm_wri)+')'
return

I like this solution because it's simple, and simple is better that complex (The Zen of Python). It has some drawbacks though. I will sweat when I try to code the movement of all 4 motors at the same time

### JUNE 4, 2013. Tuesday

While having breakfast today I got an idea. Since the serial communication seems to be interfering with the stepper delays I should create a function for moving n steps so that the micro-controller executes this function before listening again to the serial port. The only drawback is that once initiated a movement, there will be no way to stop the arm. I will try to code this today.

(Evening) So I coded two functions. The first one is move n steps and the second one is move n steps with linear acceleration. The first one has two parameters: number of steps and speed (in steps per second). What I do is compute the delay in ms between each step (1000/speed) and then I enter a loop for moving a step followed by a delay. It should work but it doesn't because the compiler complains that _delay_ms expects a constant integer. So no floats and even no integer variables are accepted. So far I haven't found a way to solve it.

                //
// move number steps at speed steps per second
//
void steps_speed(int number, int speed) {
static uint8_t k;
int  constant_delay = 1000/speed;
if (number >= 0) {
for (k = 0; k < number; ++k){
step_cw();
_delay_ms(constant_delay);}
}
else if (number < 0) {
for (k = 0; k < number; ++k){
step_ccw();

_delay_ms(constant_delay);}
}
}

Even though I did not solve the above problem I started coding the linear acceleration subroutine. This is my strategy. I set the acceleration parameters at the beginning, that is, what would be the delay to start and what is the rate of acceleration. Then I compute how many steps will have the three different parts of the movement: acceleration, the constant speed and the deceleration. And for each one I apply a routine similar to the above routine. It doesn't work because of the same _delay_ms constant integer issue. I am heading to the Iaac. If I find Guillem I'll ask him.

               //
// stepper movement with linear acceleration
//

int max_delay = 500;
int accel_delay = 50;   // Adjust these for
acceleration

void steps_accel(int number, int speed) {
static uint8_t k;
int constant_delay = 1000/speed;
int ramp_steps =(max_delay-constant_delay)/accel_delay;
if (2*ramp_steps > number) {
ramp_steps = number/2;
}
int flat_steps = number - (2*ramp_steps);
int actual_delay = max_delay; // Init delay
if (number >= 0) {  // Moving CW
for (k = 0; k < ramp_steps; ++k){ // ramping up
step_cw();
_delay_ms(actual_delay);
actual_delay = actual_delay - accel_delay;
}
for (k = 0; k < flat_steps; ++k){ // Constant
speed
step_cw();
_delay_ms(constant_delay);
}
for (k = 0; k < ramp_steps; ++k){ // ramping down
step_cw();
actual_delay=actual_delay +
accel_delay;
_delay_ms(actual_delay);
}
}

else { // Moving CCW
for (k = 0; k < ramp_steps;
++k){ // ramping up
step_ccw();
_delay_ms(actual_delay);
actual_delay = actual_delay -
accel_delay;
}
for (k = 0; k < flat_steps;
++k){ // Constant speed
step_ccw();

_delay_ms(constant_delay);
}
for (k = 0; k < ramp_steps;
++k){ // ramping down
step_ccw();

actual_delay=actual_delay + accel_delay;

_delay_ms(actual_delay);
}

}
}

Hola

### MAY 30, 2013. Thursday

Today I slept for 8 hours in a row at last. Yesterday I finally presented FAB 9000 to the world! Some things like the stepper motors, were not working properly but I am happy that I could progress that much. Guillem Camprodon told me that it could be caused because the serial bus is implemented in software and the delays may interfere with the clock cycles. Since Guillem is an expert I will follow his advice, so this is my next milestone. It was important to be forced by a deadline because this is the only way if you want to finish things. No deadline = unfinished work. I will keep working in the project until it works.

After the worldwide presentation we had a cool party over the new Polycom MCU. In our Lab we had beers and chips. It's sort of funny using so expensive equipment for an online party. Fablab Leon were the best dancers! Cesar Garcia was so funny!

### MAY 28, 2013. Tuesday Evening

This is probably going to be the last post in the final project section before the presentation. I will dedicate the remaining hours to improve the code and the mechanics of the arm.

Today I tried a Nema 17 motor that was in a tray. The motor kept stepping forward and backwards no matter what combination or wire coils I used. One of the times I removed the motor connector to change the combination of wires I heard what sounded like a spark and my computer suddenly shutted down and rebooted. This is not a good signal, definitely not. I suddenly remembered that in the instructions of the Easydriver there is a big warning like this:

WARNING! Never plug or unplug the stepper motor while the easydriver is powered, you may damage the chip.

Looks like the same warning applies to Neil's stepper board. Since the board was not working I checked with a multimeter and found that one of the Alegro's A4953 wasn't delivering power to the motor. I didn't have more Allegro chips but I remembered that Ramin, one of my buddys did the Stepper assignment in the output devices week. So I kindly asked him if he would donate it to science and he did. Thanks man, you saved my final project. I owe you a big one. I quickly unsoldered the chip and resoldered it to my board to find that everything was working again.

Later I tried to clean the marks that the laser makes to the acrylic. The laser beam leaves some marks around the cutting edges that seem water, but you cannot clean them. I found a can of acetone so I put on a pair of gloves (please be safe) and used a cloth to clean them. While I was cleaning I noticed that my gloves were dissolving so I thought this was not good at all and stopped cleaning. I used then some alcohol and the marks removed quite well. You have to press a little bit hard sometimes but the alcohol removes the "watermarks". When I was cleaning one of the green fluor arms I pressed too hard and it broke a bit in the side. I fixed it with a nut and a bolt. I spent all the afternoon in the arm assembly. It's 90% done. The rest is for tomorrow right after I bring my kid to the kindergarden I will head to the Iaac.

My code is ready, all the basic movements have been programmed and also angular coordinates. Tonight I will add more and more movements until I fall sleep. We'll see how long I can keep my eyes open. 17 hours remaining for the deadline.

### MAY 28, 2013. Tuesday

It is curious how different you see things at late evening and early morning. Yesterday at 03:00 AM the last thing I thought before falling sleep was "I am not going to make it". Today I woke up, washed my face, and thought: "I am going to make it".

Yesterday I could send and receive serial commands with more than one character using Neil's code if I send them as strings in Python like this: ser.write('50'). The only bug is that it it only reads numbers and symbols (like "." and "!" and "*"). Letters are not recognised. I will use the characters that are working and after the presentation and I will debug the code again for the error. For the RGB y will use Neil's based raw C code. But for the motors, I will use the Arduino IDE code. Although the raw C code works, I could not manage to properly set the delays of the coils and the PWM cycle count. The salvaged motors I am using either block or stall or make random movements and jitters. Today I will try with a Nema 17 standard motor and will have a last look at how the Stepper.h manages this task. The following are the current set of functions that I have defined in the class arm in Python. The commands make some calculations and then send commands over the serial connection to the motors and the LED nodes.

              Welcome to ArmControl.py version 0.5
Created by Francisco Sanchez Arroyo. Fab Academy 2013
Type help() for help
>>> help()
#####################################
# ArmControl.py currently supports: #
#####################################

help() # This help
clearscreen() # Clears the console

Arm movements:
--------------
arm.goto(theta,r,z,time) # Goto theta,r,Z cylindrical
coordinates
arm.rotate(angle,time) # Rotate arm angle
arm.shoulder(angle,time) # Rotate shoulder angle
arm.elbow(angle,time) # Rotate elbow angle
arm.wrist(angle,time) # Rotate wrist angle

Colors:
-------
arm.red() # Change color to red
arm.green() # Change color to green
arm.blue() # Change color to blue
arm.yellow() # Change color to yellow
arm.cyan() # Change color to cyan
arm.magenta() # Change color to magenta
arm.white() # Change color to white
arm.dark() # Turn off the LED

Other:
------
arm.quit() # Try to shut down a 9000 series
arm.whoami() # Find for yourself

exit() # Quit to shell
######################

I will deal with coding in the morning because I can adjust the mechanical issues of the arm even 5 minutes before the presentation. But definitely I cannot code under a pressure of 90 Giga Pascal. My brain will just freeze. I feel so tired now. 28 hours for the deadline.

### MAY 27, 2013. Monday

Yesterday I implemented the Arduino IDE code with one of my motors. Footage will appear below soon. I could stop here and finish the coding of my project now. But there are a couple of things that I don't like. The first one is that the libraries Stepper.h and SoftwareSerial.h are consuming most of the memory space of the ATtiny44 uC. Actually you have less remaining bytes than a twitter message to express your creativity. The second problem is that I don't want to be a hostage of the Arduino IDE implementation. It's working now, but what will happen tomorrow? I think I decoded part of Maurice's code. I thought that void get_char was reading characters. But I think that get_char actually reads an array string. So if you send 'hello' it will read that. So I will design a set of strings to control my arm and will give it a try. I feel I'm so close to the end of the tunnel now. I hope I can solve this today, cut the gears and mounting the arm. That is the goal so that tomorrow I can assemble it all and play with scipy in Python for some stunning movements in the presentation. I still have 52 hours left. Let's see what we can give under some pressure Francisco.

### MAY 26, 2013. Sunday

In the university, my Calculus professor used to say: "Time x Effort remains Constant. So given an initial large time, the effort you put will be small. As remaining time approaches to zero, the effort you put will raise to infinite. Conclusion: If it wasn't because of the last minute, nothing would be done." 75 hours left for the final project presentations.

Yesterday we had a crazy day in the Iaac. There were a lot of people sewing and ironing in the lab because there was a textiles workshop. I can feel how the maker movement is rising, don't you? Suddenly Sara Ezz came and showed me the Barduino she had redesigned. She did a great work because she adapted the Barduino in order to accept standard Arduino shields, which is great. She was sad because she had just soldered the micro-controller in the wrong place for the second time in a row. Her morale was very low because she was doing everything wrong and she was stuck with her final project as well. I know her feeling very well. I have felt the same so many times in my life. I convinced her to leave aside the Barduino and tried to help her to get on track with her final project. I think her final project is the best one this year in Fab Lab Barcelona. Her original idea was to make a clock with a custom hypnotic pattern code so that you know what time is it by knowing the code. I'll keep an eye on her.

Yesterday my goal was to deal with the C code of the boards trying to make the steppers and the RGB respond to serial commands. I managed to mix the code of the stepper and the RGB LED with the bus node code, and after compiling 250 times the code and debugging the errors the program finally uploaded. And it worked!!! Here's footage:

The problem is that Neil's example code was only sending one character over serial and I had no idea how to send more characters in order to achieve a more complex behavior. So I broadcasted help to the class and the class responded to me. Maurice gave the hint to follow: arrays, pointers, atoi's and itoa's. With this I found a guy who managed to program an ATtiny85 with the Arduino IDE using the libraries stepper.h and SoftwareSerial.h. With those libraries the code is very easy so I was happy. Then Chris shared with me his code to send more characters using Neil's code. If I can make it work, I will use Chris and Neil's code because you can control the motor at the coil level and hence you can adapt the code to fit non standard motors like the ones you can salvage from printers and scanners. With the libraries and the Arduino IDE you can only control the motor at the step level and has been coded with standard stepper motors in mind.

At the end of the day I had yet another problem. I didn't find my RGB board and while I was looking for it I heard a crack in my foot. Don't ask me why, but I knew it was my board. All the connectors broke removing a lot of pads and traces. So the first thing today is advanced surgery to fix it since I don't want to mill another one. Well, my train is arriving to Barcelona, and I have and exciting and busy day ahead!

### MAY 25, 2013. Saturday

Yesterday I made big advances. I fixed the bridge board, it was not working because the ATtiny45 was broken. This is the second time that happens me in the Academy so maybe we got a bad batch. Also I could test that all the boards were working.

Later I was dealing with the Python interpreter and the library of commands ArmControl.py. I readed all the pyserial documentation and started creating the commands that will control the arm. I will command the arm using cylindrical coordinates (theta, radius and z) which are the natural coordinates given the geometry of the arm . At night, hyper-jumping in google I found a cool Python module named pyttsx which is a multiplatform text to speech module. Is pretty cool to listen the computer saying the commands aloud. I implemented the module and that gave me the idea to add an easter egg that you will have to find for yourself.

Now I'm in the train, heading again to the Iaac. I wish I could work on this at home but I can't, my older son (4) keeps asking me all the time: "Daddy is that your homework?, Daddy, why do you go to school? What is your teacher's name? Do you have friends Daddy?". I love you son, but daddy must finish this. It is important for us.

### MAY 19, 2013. Sunday

Dealing with forward and inverse kinematics. From now on, you may not understand a single word of what I'm about to say. A nice source of information about robot kinematics is the course Introduction to Robotics, from Stanford University taught by Oussama Khatib. I really recommend you to take lessons four and five.

Forward kinematics is obtaining x,y,z coordinates and orientation of the end effector of the robot arm by knowing the angles of the joints. On the other hand, inverse kinematics is obtaining the angles of the joints needed to place the end effector in given x,y,z coordinates and orientation. Both forward and inverse kinematics are needed.

The most used are inverse kinematics, because if you want the arm to draw a line, you will describe that line using cartesian coordinates. You need to obtain the joint angles so that you can tell the microcontroller to move the stepper motors a certain number of steps in order to reach that angle. Those are also the most difficult set of equations to solve and includes solving the inverse of the Jacobian of the 4x4 forward kinematics matrix. Forward kinematics is used in a closed loop. The encoder will give you the angles of the joints, and you will obtain the coordinates of the end effector using forward kinematics. Comparing these coordinates with the target coordinates will help you to program a Proportional-Integrative-Derivative (PID) algorythim for accelerating/slowing down smoothly.

I told you that you would not understand anything.

### MAY 16, 2013. Thursday

I have made the basic wireframe interface of the Virtual Control Machine in light painting mode. There is a top right button to load the image, which previews in a window.  There are three settings that the user must enter: speed (this is related to camera aperture in bulb mode). Number of columns (see MIT example image) and physical width of the image. There is also a big button for sending the job to the arm. Python, here we go!

### MAY 15, 2013. Wednesday

I am very happy because I managed to fit most of the electronics of this project using only half of a 6x4" single side copper board. Which will reduce the cost of the arm and also make it easier to fabricate.

Yesterday Monica Pedro gave me a a very good advice on programming the PC Virtual Control. She told me to program based on Use Case, that is to think as an user, thinking what the users of the arm will need and how are they going to interact with it, how is it going the interface to look like. And then, when you have it, you start programming to achieve this. That sounds a pretty good idea, and might seem obvious to you, but I didn't even think about it. Thank you Monica! I have just find a webapp to make simple wireframe interfaces which I highly recommend due to its simplicity.

### MAY 14, 2013. Tuesday

I have finished modifying Neil's bipolar stepper driver to become a node of a network. Given that my goal is to manage it from Python in a personal computer, no Arduinos or microcontrollers should be needed. Tomorrow morning I will mill and stuff it and hopefully I will be able to make some smoke tests. I need four of these. Time is running out.

Later in the afternoon I also modified Neil's hello.rgb.45 to become a node of the network. This is my light painting module.

And the unmodified bridge board to interface with the computer:

I will add input devices once I've dealed with all of this mess.

### MAY 13, 2013. Monday

Last weekend we all received an email from Luciano saying that Monday (for today) was the deadline for ordering parts for our final projects. I have a dilemma, which makes me remember The Matrix Reloaded dialogue. The Architect:

"Which brings us at last to the moment of truth, wherein the fundamental flaw is ultimately expressed, and the Anomaly revealed as both beginning and end. There are two doors, Francisco. The door to your right leads to the Easydriver and the salvation of your project. The door to your left leads back to Neil's boards, to the source code, and to the complete failure of your robotic arm. As you adequately put, the problem is choice. But we already know what you are going to do, don't we? Hope. It is the quintessential human delusion, simultaneously the source of your greatest strength, and your greatest weakness."

I came here to learn, not to success.

### MAY 6, 2013. Monday

Today I felt like starting the project and lasercutted some parts of the arm. It is curious how fast you can develop under some pressure. I was bored and even modified the shoulder to become a blue Pac-man ghost. I have been all this time refining the design in kokopelli, but someday you have to take a step forward and start cutting and doing something if you really want to face problems that do not arise in the paper (computer, I mean).

### MAY 2, 2013. Thursday

I am getting more and more confident to kokopelli. I can now represent 3D shapes, and even simulate the robot arm movements with sliders. My next goal is to integrate physical properties like weight and inertia in order to display torques on each joint for any given angle.

Now the problem is other. I m stuck in making big key decisions. Every time I listen to Neil I change my mind about the way I should control the arm. Yesterday he talked about Virtual Machine control. My initial idea was to have a micro-controller "brain" that receives data from sensors and send data to actuators. But there is a tendency in moving the processing power to a personal computer so that all the boards in the machine are semi-idiot sensor boxes just for communication between the computer and the sensors/actuators. This way the system becomes modular and adding or removing modules from the design does not affect the rest of the boards. I wonder if the personal computer could even be a Raspberry PI. I like modularity so I should proceed this way. But I have almost no experience with Python programming. So I am taking a Python course in codeacademy and looking at the examples that Jose Pablo showed me. Thanks buddy for pointing me all these resources. I wonder if I will be able to achieve this.

My modules are clear now: A piezo module for sound and tones. A RGB led module for light painting. A stepper module for motor control. An encoder module for closed loop control. A camera trigger module. And a bridge module for interfacing with the computer. And I think nothing else is needed. Now in the computer side I will program what to do with the arm , say draw a circle (no idea of how it is made) and the arm must go to an initial known position and from there, start the movement with a PID scheme. So I must receive feedback from the encoders to make that calculations and refresh the commands.

### APRIL 19, 2013. Friday

Today I started designing a system to join the acrylic parts together. I started with a clip system like the one below.

I lasercut the piece in 3 mm acrylic using the Epilog 36 EXT. The design ultimately proved to result a miserable failure since acrylic has a very poor elastic performance and the prototype broke at the first trial with very little force. So I abandoned that idea.

Then I designed a tab system. The most important feature of a tab system is making it tight enough so that you don't need any glue. What happens is that if you cut a hole to the exact dimensions it feels too loose. The hole is too big. This is because of the kerf. When you cut the material, part of it disappears. So you have to play with the kerf in order to make it fit tight. This is where kokopelli starts to show its potential. I made a lot of testings in just a few minutes by assigning a variable the value of the kerf.

I tried several values. With values of kerf above 0.2 mm it was not possible to make it fit, if you try to force it too much, the material breaks. A value of 0.1 mm is quite tight, maybe a bit too much. A value of 0.0 mm is too loose, it won't resist any dynamic forces, it even falls to the ground if you flip it. Finally I will adopt a value of 0.05 or maybe 0.07 mm depending upon the force that the part has to resist. For a very strong fit I might use a kerf of 0.09 mm.

Today I also have very good news. For the first time in the academy I was able to laser-cut all those pieces directly from kokopelli with my Macbook using fab modules. You might think: So what? But this is really important because it saves me a lot of time since the workflow has been reduced a lot. I will make a tutorial in order for you to do the same. The are a couple of things that I still have to solve but I found workarounds for the meantime.

Hopefully by Monday I will start laser-cutting iteration 1! Stay tuned!

### APRIL 10, 2013. Wednesday

I have been experimenting with settings for cutting and engraving with the Epilog Laser. Iteration 1 is going to be made with acrylic leftovers from other students. The most unusable the leftover seems, the best it is for me. All the acrylic leftovers I found are 3 mm thick. I had a look at the Fablab BCN Wiki section and I experimented with the recommended settings:

• CUTTING 3 mm acrylic: Power 60%, Speed 15%
• ENGRAVING 3 mm acrylic: Power 10%, Speed 65%

I found the recommended settings perfect to me. Please find below some quick tests I made.

My next step is going to be designing a tab and clipping system. Because this method of construction may be used from people all over the world and they will probably have different leftovers sizes and thickness, it is important to design parametric parts so that by changing a few numbers everybody can obtain a suitable fit. I will do this in kokopelli.

### APRIL 4, 2013. Thursday

It's raining cats and dogs outside, a perfect weather to sit back in the sofa and think about my final project. I am just remembering Joe Justice now, he is the guy from Wikispeed. I remember when he came to the Iaac and explained how he made his car in three months. A process that takes years to the established industry. He uses extreme manufacturing techniques borrowed from software development: Making it short he makes iterations of the design and obtains a working version every 7 days. The second week he improves the design and maybe adds new features and so on. I will try this now.

### Robot Arm Iteration 1

This week I will make a working version of the robot arm. Small in size, made out of acrylic leftovers. Since I don't know the size of the leftovers that I will find, I will design the arm in Kokopelli. That way I will quickly adjust the robot arm to the available acrylic pieces. I will use servo motors without a transmission system. In electronics I will make a servo controller and an Arduino. That's should be enough to have a working version. Will see how it looks like in a few days. Follow this week Kanban in my Scrumy page.

### MARCH 26, 2013. Tuesday

It has been a long time with no updates. I have been very busy both at the Academy and at work. Last Friday we had a local session in Fab Lab Barcelona where we talked again about our final projects. Our instructors do not want us to leave the final project aside and encouraged us to build a working prototype in the following two weeks. This is a good advice. If you don't know how to make it work in the following couple of weeks, probably you won't make it. At the beginning of the Fab Academy I did some research about what past students thought they would make and what they really did for their final projects. I found a significant percentage of people who changed their mind.

When choosing your final project take into account that you are going to be given a set of tools and how to use them. You should be able to make your final project in combination with your background. Do not forget that. Fab Academy won't transform you into a NASA engineer. I would like to make an electric vehicle for my final project if you ask me. I am sure it can be done with the Fab Lab tools but I don't have the background to make it. Well I don't have the background to make the robot arm either, but I like it so much that I am acquiring it on my own.

In our talk I explained how I changed my mind about the Final Project. At the beginning I imagined a high-tech robot arm made out of carbon fiber and stainless steel. I don't see it like this in my mind now. I would like to create the very same project but from other point of view. The main questions that I want to answer are: Is it possible to create something high tech using common low tech things that people trash? Is it possible to convert trashed items into something nice and pleasant to the eye? To achieve this goal I created my own project boundaries, which I have named The Ten Commandments.

THE TEN COMMANDMENTS (DRAFT)

1. I shall push to the limits personal fabrication. The project will serve a market of one person: just me.
2. I shall design and build all my electronics.
3. I shall not buy what I can make.
5. I shall use materials from leftovers. Now new materials shall be used.
6. I shall modify my design in order to reuse electronic components from trashed appliances.
7. I shall carefully analyze my design to create something pleasant to the eye.
8. I shall use as many Fab processes as I can.
9. The project shall be Open Source hardware and software.
10. I shall document the project so that anyone can use, modify and improve it.

### Kokopelli

I've been all the weekend playing with Kokopelli. The more I use it, the more I like it. I have made 2 items. The first one is a gear for a timing belt. Look, it took me all the weekend to do this, but now, I can make hundreds of variants in seconds just by typing 3 numbers: Pitch, number of teeth and shaft diameter. Isn't that incredible?

The second item that I made is the castellated beam. When I say I made it, read Evan Jones made it. Today he was about to leave the lab when he had a last look at my screen and said: Is that Python? I said.. yes, sort of... it is the Kokopelli tool... I'm just trying to figure out how to make the holes smaller as they move to the end of the beam. He opened his laptop and started coding. I don't know what he was doing, but he didn't even finish typing the commands and variables, they auto completed. In two minutes he showed me my beam with he holes getting smaller as they move to the end. Then I asked... umm now you are here, can you evenly divide the distance between the holes? He paused 5 seconds (this should be like thousands of billions of operations in his brain). Then he typed a couple more lines and the holes where evenly distributed. Evan Jones did in two and a half minutes what I could not even figure out in a whole weekend. There are some people that do not think, they code directly, Evan Jones is one of them.

Well, now I have 2 problems: the first problem is that I spent all of this time in Kokopelli and when I had a look at my mates work today they all were rendering real 3D models of their projects in Rhino. And all that I have is a hand drawed sketch, a Kokopelli script for a gear and a castellated beam, and the same castellated beam, done in Rhino. And I still want model and render it in Rhino and I also would like to test the castellated beam with finite element analysis (FEA/FEM) in COMSOL to see the differences between acrylic and carbon fiber. The second problem is that it is Monday 21:19 in my watch and the deadline is tomorrow at 24:00 Barcelona time. Someone is not going to sleep today.

### Rhino tutorial

Arian is a great guy. He is architect and Rhino and Grasshopper expert. I am proud of being in such a multidisciplinar, multicultural and talented class. I have had one lecture of rhino a couple of years ago and I found it very powerful and with a quick learning curve. But I did not have time to practice with it. Now I feel it even more powerful. One of the tools that made an impression on me was PROJECT, I don't think I will use it in this project but since I like truss structures it can help me a lot. Arian also pointed me to a great plugin: KARAMBA for structural design (I don't know who decides those names for rhino plugins). I practised a little bit and I made my robot arms in no time. Rhino is a must!

### After class hangout

When we finished the master class (actually when we were fed up of trims, offsets and rebuilds) we walked downtown to El Borne district because people from the Iaac and others were performing a Live Jam session. We had a great time listening to the music, drinking beers, talking to each other and getting us to know ourselves better. I am happy I went there, I left with the feeling that coming to the Academy has been one of the best things that have happened to me. We'll repeat guys!

### Designing

From now on, I will publish here my progress in the project. Neil told us yesterday in the lecture that the goal of the Archive is not to show the final result, but the process and progress, including all of our miserable failures -actually he did't say that last thing, I just completed the sentence-. I also need to start this project as early as possible. This is not a tweeting cauliflower, you know, that sends help when someone boils it, this is serious, I have a lot of coding, mechanics, electronics, all of them with many uncertainies that I haven't even come across. And I want to finish it, for me it is important.

Yesterday in the train trip to the Iaac I started drawing the arms of the robot, and trying to fit them inside the carbon fiber board. It is important to try not to waste material, not only because carbon fiber is expensive, but also because it is stupid to waste material if you can save it.

As you can see in the image below, my initial design is based on a Castellated Beam. These kind of beams have been used since the early 1900's in order to increase beam depth and strength without increasing material. They cut the beam in a particular shape so that when they welded it again, the beam became taller (increased bending resistance). In my case I cut the holes to decrease even more weight and also because I like Castellated Beams. They are so nice, don't they? The members sizes are initially derived from the equations that determine the maximum arm that could be fitted in the board. It is like some sort of functional and parametrical design actually.

I'm going to try now draw a piece of the arm the the Fab Modules.

### Surfing the web

In a desperate try to find someone who has dealed with inverse kinematics I found this guy:

http://www.circuitsathome.com/mcu/controlling-robotic-arm-with-arduino-and-usb-mouse

He managed to control a Lynxmotion (servo motors) with Arduino and some sort of inverse kinematics. It's not exactly what I want to do but I will print the code and try to decipher it. With servos is easier to move a motor because you just tell the servo go to that angle. In my case, with steppers and a transmission system you have to tell it move n steps forward or backward, and you have to calculate how many steps you need to achieve that angle. It depends upon many factors, like pulley diameters, microstepping, etc. Anyway it is a good start, at least I know Arduino has enough processing power to deal with it. He also managed to manually move the arm with a mouse. I will also need a method to manually move the arm to learn some 'keyframes' for the time-lapse scene. But a mouse is not something I could use in the forest. I prefer something more analog, like just grabbing the arm and moving it to its position.

### JANUARY 29, 2013. Tuesday

I created this video trying to explain in one minute what is Fab Academy.


text:
• Segal-Bargmann Transform and Paley-Wiener Theorems on $M(2)$

• # Fulltext

https://www.ias.ac.in/article/fulltext/pmsc/120/02/0169-0183

• # Keywords

Segal–Bargmann transform; Poisson integrals; Paley–Wiener theorem.

• # Abstract

We study the Segal–Bargmann transform on $M(2)$. The range of this transform is characterized as a weighted Bergman space. In a similar fashion Poisson integrals are investigated. Using a Gutzmer’s type formula we characterize the range as a class of functions extending holomorphically to an appropriate domain in the complexification of $M(2)$. We also prove a Paley–Wiener theorem for the inverse Fourier transform.

• # Author Affiliations

1. Department of Mathematics, Indian Institute of Science, Bangalore 560 012, India

• # Proceedings – Mathematical Sciences

Volume 130, 2020
All articles
Continuous Article Publishing mode

• # Editorial Note on Continuous Article Publication

Posted on July 25, 2019

format: plain


text:
# FAQ overflow

#### QUESTION

I somewhat understand the concept of attacking the king and pieces, but "attacking" squares is a bit of a vague concept for me.

What are weak and strong squares? When do you use them? How do you create them?

Weak squares are squares which cannot be defended by a pawn. [1][2]

... that all of the squares behind the pawns are weak by default.

Counter-intuitively: Strong squares are squares which are defended by a pawns.

These square are very powerful places for opponents Knights or Bishops. No pawn can attack to these pieces. In below position, square d6 is a weak square for black and also strong square for its opponent.

1r1q2k1/1pbr2pp/2pNp3/p1P1Pp2/PP1P4/3R1QP1/6KP/1R6 w - - 0 1

or square f5, h4 and e6 are some weak squares for black and also f3 and d4 for white (however it's too late for black to put his Knight there)

1n3q2/p6p/1p1p1pk1/3PpN1p/4P3/1PQ3P1/P4P2/6K1 w - - 0 1

They're useful to place a Knight or Bishop in near the heart of opponents position. Usually making a weak square in the opponents land needs long term strategy. You have to force the opponent to pass its pawns...


text:
Technical name (radiometric/photometric/ray-optic) of the physical field that is measured with a camera

A camera (that is FPA and objective) is a device that measures [intensity][1] or radiant intensity vs. two angles (horizontal & vertical). So it generates a representation of the physical quantity

$$I(\phi, \theta)$$

How is this "field" actually named in physics? I am looking for a radiometric, photometric, ray-optics or similar term.

A few options that come to my mind:

• Intensity field
• The field is the electromagnetic field $\mathbf E(\mathbf r, t)$. The intensity is just $|\mathbf E|^2$. – John Rennie Jan 26 at 11:16


text:
## Where do the Kähler Identities first appear?

The Kähler identities (sometimes known as the Hodge identities) are an important collection of relationships between operators on the exterior algebra of a Kähler manifold. These relationships generalise to hermitian manifolds, sections of hermitian holomorphic vector bundles, and many other situations.

I know that the notion of a Kähler metric was introduced by Kähler himself in 1933 and that the Kähler identities were first generalised to hermitian manifolds by Demailly in 1985 (although he mentions that the ideas were present in a paper by Griffiths in 1966).

Can anyone fill in the historical gap and tell me where or when the Kähler identities first appeared?

-

As you hinted yourself, they were indeed discovered by W.V.D. Hodge. They appear explicitly in his 1941 book "The Theory and Applications of Harmonic Integrals", which you can find here, see e.g. the lemmas on pages 172 and following. They were discovered by him some times in the 30s, see e.g. this 1935 paper of his.

Hodge's notation is a bit different from the modern one. The first place that I know where they appear in the modern form is A.Weil's book "Introduction a l'étude des variétés kählériennes" (1957), available here, Theoreme 1, p.42. The modern notation was probably introduced by Weil, since Kodaira in 1952 called the operator $\Lambda$ the "Hodge-Weil operator".

-
 May I ask, how did you find this out? – Michael Albanese Dec 17 at 20:01 I remembered that Weil's book, which is arguably the first textbook written on Kahler geometry, so I looked there. He had the reference to Hodge, as part of his theory of "harmonic integrals". – YangMills Dec 18 at 17:56 I've had a look at Hodge's paper, but I can't see where exactly the Kähler identities appear. – Michael Albanese Dec 19 at 3:29 It's true, that paper does not really contain the Kahler identities (the 1941 book does), but for example equation (15) in the paper follows from one of the simpler Kahler identities. There is probably some other paper of Hodge that is a more suitable reference, but MR does not do a good job at indexing them. You can find some references in Hodge's book, page 225, but I haven't looked at them. – YangMills Dec 19 at 6:13

(By the way, just a comment, not an answer: Kähler metrics seem to appear firstly in: J. A. Schouten and D. van Dantzig, Über unitäre Geometrie, Math. Ann. 103 (1930), no. 1, 319–346 )

-
In 'Eugenio Calabi and Kähler Metrics' by Bourguinon, he attributes the introduction of the notion of a Kähler metric to Kähler's 1933 paper, but makes a parenthetical remark listing the paper you mention, as well as a paper by Schouten from 1929, as "earlier attempts". I'm not sure why these two papers are only listed as attempts; personally, I have not had a look at either of them. – Michael Albanese Dec 17 at 9:39


text:
Home > Error Correction > Error Correction Model Economics

# Error Correction Model Economics

## Contents

We show that error-correction models may also arise because one variable forecasts another. pp.662–711. From the econometrician's point of view, this long run relationship (aka cointegration) exists if errors from the regression C t = β Y t + ϵ t {\displaystyle C_{t}=\beta Y_{t}+\epsilon _{t}} C t − 1 = 0.9 Y t − 1 {\displaystyle C_{t-1}=0.9Y_{t-1}} . his comment is here

Please register to: Save publications, articles and searchesGet email alertsGet all the benefits mentioned below! E. JSTOR1913236. In Baltagi, Badi H. https://en.wikipedia.org/wiki/Error_correction_model

## Error Correction Model Stata

New York: John Wiley & Sons. Oxford: Blackwell. In particular, Monte Carlo simulations show that one will get a very high R squared, very high individual t-statistic and a low Durbin–Watson statistic.

Suppose that in the period t Y t {\displaystyle Y_{t}} increases by 10 and then returns to its previous level. Generated Tue, 11 Oct 2016 03:53:53 GMT by s_ac15 (squid/3.5.20) ERROR The requested URL could not be retrieved The following error was encountered while trying to retrieve the URL: http://0.0.0.8/ Connection H.; Hendry, D. Vector Error Correction Model Tutorial doi:10.1002/9780470996249.ch31.

Given two completely unrelated but integrated (non-stationary) time series, the regression analysis of one on the other will tend to produce an apparently statistically significant relationship and thus a researcher might Vector Error Correction Model Please try the request again. Granger, C.W.J.; Newbold, P. (1978). "Spurious regressions in Econometrics". Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.

Shiller NBER Working Paper No. 2568 (Also Reprint No. Vector Error Correction Model Sas Enders, Walter (2010). This can be done by standard unit root testing such as Augmented Dickey–Fuller test. Journal of the Royal Statistical Society. 89 (1): 1–63.

## Vector Error Correction Model

Econometrica. 55 (2): 251–276. ISBN978-0-470-50539-7. Error Correction Model Stata Suppose, consumption C t {\displaystyle C_{t}} and disposable income Y t {\displaystyle Y_{t}} are macroeconomic time series that are related in the long run (see Permanent income hypothesis). Error Correction Model Eviews In this setting a change Δ C t = C t − C t − 1 {\displaystyle \Delta C_{t}=C_{t}-C_{t-1}} in consumption level can be modelled as Δ C t = 0.5

In order to still use the Box–Jenkins approach, one could difference the series and then estimate models such as ARIMA, given that many commonly used time series (e.g. this content Then the predicted residuals ϵ t ^ = y t − β 0 − β 1 x t {\displaystyle {\hat {\epsilon _{t}}}=y_{t}-\beta _{0}-\beta _{1}x_{t}} from this regression are saved and used Shiller. "Interpreting Cointegrated Models ," from the Journal of Economic Dynamics and Control, Vol. 12,pp. 505-522, (1988). shocks of consumer confidence that affect consumption). Error Correction Model Interpretation

in Econometric Analysis for National Economic Planning, ed. Whittaker. Campbell, Robert J. http://celldrifter.com/error-correction/error-correction-model-in-r.php JSTOR2231972.

Even in deterministically detrended random walks walks spurious correlations will eventually emerge. Error Correction Model Impulse Response Function Phillips, Peter C.B. (1985). "Understanding Spurious Regressions in Econometrics" (PDF). Generated Tue, 11 Oct 2016 03:53:53 GMT by s_ac15 (squid/3.5.20)

## Contents 1 History of ECM 2 Estimation 2.1 Engel and Granger 2-Step Approach 2.2 VECM 2.3 An example of ECM 3 Further reading History of ECM Yule (1936) and Granger and

Technically speaking, Phillips (1986) proved that parameter estimates will not converge in probability, the intercept will diverge and the slope will have a non-degenerate distribution as the sample size increases. Even in deterministically detrended random walks walks spurious correlations will eventually emerge. D. (1964). "Wages and Prices in the United Kingdom: A Study in Econometric Methodology", 16, 25–54. Error Correction Model Fixed Effects N.

In practice, econometricians often first estimate the cointegration relationship (equation in levels), and then insert it into the main model (equation in differences). JSTOR2231972. pp.272–355. check over here Take the case of two different series x t {\displaystyle x_{t}} and y t {\displaystyle y_{t}} .

F.; Srba, F.; Yeo, J. Retrieved from "https://en.wikipedia.org/w/index.php?title=Error_correction_model&oldid=738124940" Categories: Error detection and correctionTime series modelsEconometric models Navigation menu Personal tools Not logged inTalkContributionsCreate accountLog in Namespaces Article Talk Variants Views Read Edit View history More Search Our last assumption is that the gap between current and equilibrium consumption decreases each period by 20%. we need weak exogeneity for x t {\displaystyle x_{t}} as determined by Granger causality One can potentially have a small sample bias The cointegration test on α {\displaystyle \alpha } does

http://wiley.force.com/Interface/ContactJournalCustomerServices_V2. Thus detrending doesn't solve the estimation problem. Econometric Modelling with Time Series. Engle, Robert F.; Granger, Clive W.

In order to still use the Box–Jenkins approach, one could difference the series and then estimate models such as ARIMA, given that many commonly used time series (e.g. E. Then the predicted residuals ϵ t ^ = y t − β 0 − β 1 x t {\displaystyle {\hat {\epsilon _{t}}}=y_{t}-\beta _{0}-\beta _{1}x_{t}} from this regression are saved and used Sargan, J.

One can then test for cointegration using a standard t-statistic on α {\displaystyle \alpha } . Ordinary least squares will no longer be consistent and commonly used test-statistics will be non-valid. New Introduction to Multiple Time Series Analysis. ECMs are a theoretically-driven approach useful for estimating both short-term and long-term effects of one time series on another.

Applied Econometric Time Series (Third ed.).


text:
# Want to improve ANN presentation adding the activation functions

Well, there is actually two problems that I want to resolve, but I don't know if I should do two posts. Anyway...

I have this representation of my ANN and want to include the action functions, but I also don't know how to make it looks better

            \documentclass{article}
\usepackage{tikz}
\begin{document}
\pagestyle{empty}
\def\layersep{1.5cm}
\begin{tikzpicture}[
shorten >=1pt,->,
draw=black!50,
node distance=\layersep,
every pin edge/.style={<-,shorten <=1pt},
neuron/.style={circle,fill=black!25,minimum size=17pt,inner sep=0pt},
input neuron/.style={neuron, fill=green!50},
output neuron/.style={neuron, fill=red!50},
hidden neuron/.style={neuron, fill=blue!50},
annot/.style={text width=4em, text centered}
]
% Draw the input layer nodes
\foreach \name / \y in {1,...,3}
% This is the same as writing \foreach \name / \y in {1/1,2/2,3/3,4/4}
\node[input neuron, pin=left:Input \#\y] (I-\name) at (0,-\y) {};
% set number of hidden layers
\newcommand\Nhidden{2}
% Draw the hidden layer nodes
\foreach \N in {1,...,\Nhidden} {
\foreach \y in {1,...,12} {
\path[yshift=4cm]
node[hidden neuron] (H\N-\y) at (\N*\layersep,-\y cm) {};
}
\node[annot,above of=H\N-1, node distance=1cm] (hl\N) {Hidden layer \N};
}
% Draw the output layer node
\node[output neuron,pin={[pin edge={->}]right:Output}, right of=H\Nhidden-6] (O) {};

% Connect every node in the input layer with every node in the
% hidden layer.
\foreach \source in {1,...,3}
\foreach \dest in {1,...,12}
\path (I-\source) edge (H1-\dest);
% connect all hidden stuff
\foreach [remember=\N as \lastN (initially 1)] \N in {2,...,\Nhidden}
\foreach \source in {1,...,12}
\foreach \dest in {1,...,12}
\path (H\lastN-\source) edge (H\N-\dest);
% Connect every node in the hidden layer with the output layer
\foreach \source in {1,...,12}
\path (H\Nhidden-\source) edge (O);
% Annotate the layers
\node[annot,left of=hl1] {Input layer};
\node[annot,right of=hl\Nhidden] {Output layer};
\end{tikzpicture}
% End of code
\end{document}

• you can write text in the {} at the end of the \node lines. You'll have to spend some time tweaking the loops and distances though – Thorbjørn E. K. Christensen Nov 15 '17 at 8:28

You can write text in the curly brackets after the nodes (as below), however this ruins the spacing of the original drawing, and it looks really clouded.

\documentclass{standalone}
\usepackage{tikz}
\begin{document}
\pagestyle{empty}
\def\layersep{3cm}
\def\nodeinlayersep{1.5cm}
\begin{tikzpicture}[
shorten >=1pt,->,
draw=black!50,
node distance=\layersep,
every pin edge/.style={<-,shorten <=1pt},
neuron/.style={circle,fill=black!25,minimum size=17pt,inner sep=0pt},
input neuron/.style={neuron, fill=green!50},
output neuron/.style={neuron, fill=red!50},
hidden neuron/.style={neuron, fill=blue!50},
annot/.style={text width=4em, text centered}
]
% Draw the input layer nodes
\foreach \name / \y in {1,...,3}
% This is the same as writing \foreach \name / \y in {1/1,2/2,3/3,4/4}
\node[input neuron, pin=left:Input \#\y] (I-\name) at (0,-\y) {};
% set number of hidden layers
\newcommand\Nhidden{2}
% Draw the hidden layer nodes
\foreach \N in {1,...,\Nhidden} {
\foreach \y in {1,...,12} {
\path[yshift=6cm]
node[hidden neuron] (H\N-\y) at (\N*\layersep,-\y*\nodeinlayersep ) {$\frac{1}{1+e^{-x}}$};
}
\node[annot,above of=H\N-1, node distance=1cm] (hl\N) {Hidden layer \N};
}
% Draw the output layer node
\node[output neuron,pin={[pin edge={->}]right:Output}, right of=H\Nhidden-6] (O) {};
%
% Connect every node in the input layer with every node in the
% hidden layer.
\foreach \source in {1,...,3}
\foreach \dest in {1,...,12}
\path (I-\source) edge (H1-\dest);
% connect all hidden stuff
\foreach [remember=\N as \lastN (initially 1)] \N in {2,...,\Nhidden}
\foreach \source in {1,...,12}
\foreach \dest in {1,...,12}
\path (H\lastN-\source) edge (H\N-\dest);
% Connect every node in the hidden layer with the output layer
\foreach \source in {1,...,12}
\path (H\Nhidden-\source) edge (O);
% Annotate the layers
\node[annot,left of=hl1] {Input layer};
\node[annot,right of=hl\Nhidden] {Output layer};
\end{tikzpicture}
% End of code
\end{document}

• Is there a way to draw the graph of the function instead of the function? Is it too hard? @ThorbjørnEK – Caio Custódio Nov 16 '17 at 17:43
• Yes, this can quite easily be done by exchanging $\frac{1}{1+e^{-x}}$ With \tikz\draw[scale=0.05,samples=100,domain=-6:6,smooth,variable=\x,red,-] plot ({\x},{4/(1+exp(-\x))}); @CaioCustódio – Thorbjørn E. K. Christensen Nov 16 '17 at 18:18
• @ThorbjørnEK worked with an error: ' Package pgfkeys Error: I do not know the key '/tikz/variable', to which you passed '\x ', and I am going to ignore it. Perhaps you misspelled it. }' – Caio Custódio Nov 16 '17 at 19:33
• I don't get any error, so that's wierd. – Thorbjørn E. K. Christensen Nov 16 '17 at 19:38
• I did it again and worked, maybe a case of bad copy & past. Thank you! So it works with every other kind of function? @ThorbjørnEK – Caio Custódio Nov 16 '17 at 19:41


text: CSM analyses of $t\bar t H, t\bar t Z, t\bar b W^-$ production in gluon-gluon and photon-photon collisions Auteur(s): Renard F. M. (Document sans référence bibliographique) 2017-06-14 Ref HAL: hal-01562846_v1 Ref Arxiv: 1706.04484 Ref. & Cit.: NASA ADS Exporter : BibTex | endNote Résumé: We extend our previous analysis of $t\bar t H, t\bar t Z, t\bar b W$ production in $e^+e^-$ collisions within the CSM (Composite Standard Model) concept of Higgs boson and top quark compositeness to the case of gluon-gluon and photon-photon collisions. We show that remarkable differences may indeed appear between the effects generated by CSM conserving and by CSM violating sets of form factors. Commentaires: 16 pages, 9 figures


text:
# How do you write 0.000000027 in scientific notation?

##### 1 Answer
Mar 6, 2018

We need to move the decimal point 8 places to the right so the exponent for the 10s term will be negative:

$0.000000027 = 2.7 \times {10}^{-} 8$


text:
## Exploding Dots

### 1.1 Base Machines

Lesson materials located below the video overview.

OPENING COMMENT: The entire text to this Exploding Dots course appears in the COMPANION GUIDE to the course. Plus ALL THE SOLUTIONS to the questions asked throughout this course, and some additional material, appear in it too. Check out the “Guides and Solutions” section at the home page.

Here is a story that isn’t true.

When I was a young child I invented a machine.  And this machine was nothing more than a row of boxes that extends as far to the left as I could ever desire.

I called this machine a “ $$1 \leftarrow 2$$ machine” and one would work this machine by placing dots into the rightmost box.

Placing one dot in into the machine doesn’t do much. It stays as one dot.

But in placing two dots into the machine, something exciting happens: Two dots in a box explode– kapow!–and produce one dot one place to the left.

The code for “two” in a $$1 \leftarrow 2$$ machine is “no dots, no dots, … , no dots, one dot, no dots.” Let’s not bother writing the beginning lack of dots and just write this code for two as:

### $$10$$

Placing a third dot into the machine (always the rightmost box) produces:

Putting in a fourth dot is exciting! One explosion is followed by a second explosion!

A fifth dot yields the code:

And hours of fun are to be had placing dots in a  $$1 \leftarrow 2$$ machine and working out their codes.

Comment: One can place dots all at once in the rightmost box of the machine. For example, here’s what happens with six dots placed into the machine right away:

This shows that six has code: $$110$$. (Is this what you expect given that $$5$$ has code $$101$$?)

 Question 1: a) Verify that the code for thirteen is $$1101$$  in a $$1 \leftarrow 2$$ machine.  b) What is the code for fifty?

Recall that alll the solutions to these questions appear in the COMPANION GUIDE to this EXPLODING DOTS course.

 Question 2: Could a number ever have $$100211$$  as its code in a $$1 \leftarrow 2$$ machine?

 Question 3:  (HARD!) Which number has code $$10101$$  in a $$1 \leftarrow 2$$ machine?

### OTHER MACHINES

But then one day, in this untrue story, I had an epiphany! Instead of playing with dots in a $$1 \leftarrow 2$$ machine, I realized I could instead play with dots in a $$1 \leftarrow 3$$ machine.

THE  $$1 \leftarrow 3$$ MACHINE: Whenever there are three dots in any one box they “explode,” that is, disappear, and become one dot in the next box one place to their left.

Here’s what happens to fifteen  in a $$1 \leftarrow 3$$ machine:

We have:

THE  $$1 \leftarrow 3$$ CODE FOR FIFTEEN IS: $$120$$ .

 Question 4: a) Show that the $$1 \leftarrow 3$$ code for four is $$11$$.  b) Show that the $$1 \leftarrow 3$$ code for twenty is $$202$$.

 Question 5: What is the $$1 \leftarrow 3$$ code for thirteen? For twenty-five?

 Question 6: Is it possible for a number to have $$1 \leftarrow 3$$ code $$2031$$? Explain.

 Question 7: HARD CHALLENGE: What number has $$1 \leftarrow 3$$ code $$1022$$?

Let’s keep going …

 Question 8: What do you think the $$1 \leftarrow 4$$ rule is?  What is the $$1 \leftarrow 4$$ code for the number thirteen?

 Question 9: What is the $$1 \leftarrow 5$$  code for the number thirteen?

 Question 10: What is the $$1 \leftarrow 9$$ code for the number thirteen?

 Question 11:  What is the $$1 \leftarrow 5$$  code for the number twelve?

 Question 12: What is the $$1 \leftarrow 9$$ code for the number thirty?

AHA MOMENT!

 Question 13: What is the $$1 \leftarrow 10$$ code for the number thirteen? What is the $$1 \leftarrow 10$$ code for the number thirty-seven? What is the $$1 \leftarrow 10$$ code for the number $$273$$? What is the $$1 \leftarrow 10$$ code for the number $$5846$$ ?

BONUS: Here’s a cute little animation. Does it make sense to you?

## Books

Take your understanding to the next level with easy to understand books by James Tanton.

BROWSE BOOKS

## Guides & Solutions

Dive deeper into key topics through detailed, easy to follow guides and solution sets.

BROWSE GUIDES

## Donations

Consider supporting G'Day Math! with a donation, of any amount.

Your support is so much appreciated and enables the continued creation of great course content. Thanks!


text:
# Find the equation of a tangent line to y=f(x)

## Homework Statement

Find the equation of the tangent line to the curve at the given point.
$$y = \sqrt{2x+1} , (4, 3)$$

## Homework Equations

a = 4
$$M =\lim_{x \to a} \frac{f(x) - f(a)}{x-a}$$

or

$$Mpq = \lim_{h \to 0} \frac{f(a+h) - f(a)}{h}$$

## The Attempt at a Solution

$$M =\lim_{x \to 4} \frac{f(x) - f(4)}{x-a}$$

$$M =\lim_{x \to 4} \frac{\sqrt{2x+1} - \sqrt{2 * 4 + 1}}{x-4}$$

$$M =\lim_{x \to 4} \frac{\sqrt{2x+1} - 3}{x-4}$$

$$M =\lim_{x \to 4} \frac{\sqrt{2x+1} - 3}{x-4} * \frac{\sqrt{2x+1} + 3}{\sqrt{2x+1} + 3}$$

$$M =\lim_{x \to 4} \frac{2x - 3}{x-4(\sqrt{2x+1} + 3)}$$

At this point I get stuck... I'm not sure what I can do from here. did I go wrong some where?

It should be 2x-8 since $$(\sqrt{2x+1}-3) * (\sqrt{2x+1}+3) = 2x+1 -9 = 2x-8$$
From that point you can divide by (x-4) quite easily resulting in a function for which you can directly substitute x=4 since the result is not in an 'indeterminate form' of $$\frac{0}{0}$$ or $$\frac{\infty}{\infty}$$


text:
3.5.3.3.6 IncF

Definition

The function is used to calculate the incomplete F-table function. x is the upper limit of integration. m and n are the degrees of freedom.

Syntax

double incf(double x, int m, int n)

Parameters

x

the upper limit of integration

m

the degrees of freedom of the numerator variance

n

the degrees of freedom of the denominator variance

Return

Returns the value of the incomplete F-table function at x with parameters a and b.

Example

aa = IncF(0.5,1,2);
aa = ; //0.55278640450326


text:
# .

In abstract algebra, the endomorphism ring of an abelian group X, denoted by End(X), is the set of all homomorphisms of X into itself.[1][2] The addition operation is defined by pointwise addition of functions and the multiplication operation is defined by function composition.

The functions involved are restricted to what is defined as a homomorphism in the context, which depends upon the category of the object under consideration. The endomorphism ring consequently encodes several internal properties of the object. As the resulting object is often an algebra over some ring R, this may also be called the endomorphism algebra.

Description

Let (A, +) be an abelian group and we consider the group homomorphisms from A into A. Then addition of two such homomorphisms may be defined pointwise to produce another group homomorphism. Explicitly, given two such homomorphisms f and g, the sum of f and g is the homomorphism (f + g)(x) := f(x) + g(x). Under this operation End(A) is an abelian group. With the additional operation of composition of homomorphisms, End(A) is a ring with multiplicative identity. This composition is explicitly (fg)(x) := f(g(x)). The multiplicative identity is the identity homomorphism on A.

If the set A does not form an abelian group, then the above construction is not necessarily additive, as then the sum of two homomorphisms need not be a homomorphism.[3] This set of endomorphisms is a canonical example of a near-ring which is not a ring.

Properties

Endomorphism rings always have additive and multiplicative identities, respectively the zero map and identity map.
Endomorphism rings are associative, but typically non-commutative.
If a module is simple, then its endomorphism ring is a division ring (this is sometimes called Schur's lemma).[4]
A module is indecomposable if and only if its endomorphism ring does not contain any non-trivial idempotent elements.[5] If the module is an injective module, then indecomposability is equivalent to the endomorphism ring being a local ring.[6]
For a semisimple module, the endomorphism ring is a von Neumann regular ring.
The endomorphism ring of a nonzero right uniserial module has either one or two maximal right ideals. If the module is Artinian, Noetherian, projective or injective, then the endomorphism ring has a unique maximal ideal, so that it is a local ring.
The endomorphism ring of an Artinian uniform module is a local ring.[7]
The endomorphism ring of a module with finite composition length is a semiprimary ring.
The endomorphism ring of a continuous module or discrete module is a clean ring.[8]
If an R module is finitely generated and projective (that is, a progenerator), then the endomorphism ring of the module and R share all Morita invariant properties. A fundamental result of Morita theory is that all rings equivalent to R arise as endomorphism rings of progenerators.

Examples

In the category of R modules the endomorphism ring of an R-module M will only use the R module homomorphisms, which are typically a proper subset of the abelian group homomorphisms.[9] When M is a finitely generated projective module, the endomorphism ring is central to Morita equivalence of module categories.
$$End(\mathbb{Z}_2\times \mathbb{Z}_2, +)\cong M_2(\mathbb{Z}_2)$$. The endomorphism ring of the additive abelian group $$(\mathbb{Z}_2\times \mathbb{Z}_2, +)$$ is isomorphic to the 2\times 2 matrix ring over $$\mathbb{Z}_2$$. (see Dummit-Foote, Abstract Algebra 3rd edition, example (5), pp. 338 and example (5), pp. 346)
If K is a field and we consider the K-vector space Kn, then the endomorphism ring of Kn consists of all K-linear maps from Kn to Kn: it is a K-algebra. After a basis for the vector space is chosen, this ring is naturally identified with the ring of n-by-n matrices with entries in K.[10] More generally, the endomorphism algebra of the free module M = Rn is naturally n-by-n matrices with entries in the ring R.
As a particular example of the last point, for any ring R with unity, End(RR) = R, where the elements of R act on R by left multiplication.
In general, endomorphism rings can be defined for the objects of any preadditive category.

Notes

Fraleigh (1976, p. 211)
Passman (1991, pp. 4–5)
Dummit (Foote, p. 347)
Jacobson 2009, p. 118.
Jacobson 2009, p. 111, Prop. 3.1.
Wisbauer 1991, p.163.
Wisbauer 1991, p. 263.
Camillo et al. Zhou.
Abelian groups may also be viewed as modules over the ring of integers.

Drozd & Kirichenko 1994, pp. 23–31.

References

Camillo, V. P.; Khurana, D.; Lam, T. Y.; Nicholson, W. K.; Zhou, Y. (2006), "Continuous modules are clean", J. Algebra 304 (1): 94–111, doi:10.1016/j.jalgebra.2006.06.032, ISSN 0021-8693, MR 2255822
Drozd, Yu. A.; Kirichenko, V.V. (1994), Finite Dimensional Algebras, Berlin: Springer-Verlag, ISBN 3-540-53380-X
Dummit, David; Foote, Richard, Algebra

Fraleigh, John B. (1976), A First Course In Abstract Algebra (2nd ed.), Reading: Addison-Wesley, ISBN 0-201-01984-1

Hazewinkel, Michiel, ed. (2001), "Endomorphism ring", Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4

Jacobson, Nathan (2009), Basic algebra 2 (2nd ed.), Dover, ISBN 978-0-486-47187-7

Passman, Donald S. (1991), A Course in Ring Theory, Pacific Grove: Wadsworth & Brooks/Cole, ISBN 0-534-13776-8

Wisbauer, Robert (1991), Foundations of module and ring theory, Algebra, Logic and Applications 3 (Revised and translated from the 1988 German ed.), Philadelphia, PA: Gordon and Breach Science Publishers, pp. xii+606, ISBN 2-88124-805-5, MR 1144522 A handbook for study and research


text:
Articles under category:
Approximation
 Vol 10, Article 2 (pp 27-53) A Regularity Lemma and Low-Weight Approximators for Low-Degree Polynomial Threshold Functions
 Vol 9, Article 27 (pp 845-862) [Boolean Spec Issue] Satisfying Degree-$d$ Equations over $GF[2]^n$
 Vol 9, Article 20 (pp 653-663) Approximating the AND-OR Tree
 Vol 9, Article 18 (pp 593-615) [Boolean Spec Issue] Making Polynomials Robust to Noise


text:
# Principal Component Analysis (PCA) – Better Explained

Principal Components Analysis (PCA) is an algorithm to transform the columns of a dataset into a new set of features called Principal Components. By doing this, a large chunk of the information across the full dataset is effectively compressed in fewer feature columns. This enables dimensionality reduction and ability to visualize the separation of classes or clusters if any. In this tutorial, I will first implement PCA with scikit-learn, then, I will discuss the step-by-step implementation with code and the complete concept behind the PCA algorithm in an easy to understand manner.

Principal Components Analysis (PCA) – Better Explained. Photo by RockyClub.

## Content

1. Introduction: What is PCA?

Part 1: Implementing PCA using scikit learn

1. Building PCA with Scikit-learn
2. Weights of Principal Components
3. Percentage of Variance Explained with each PC
4. Plot the clustering tendency
5. How to get the original features back

Part 2: Understanding Concepts behind PCA

1. Objective function of PCA

How to understand the rotation of coordinate axes

1. What is Eigen Value and Eigen Vector

Part 3: Steps to Compute Principal Components from Scratch

1. Import Data
2. Step 1: Standardize each column
3. Step 2 Compute Covariance Matrix
4. Step 3: Compute Eigen values and Eigen Vectors
5. Step 4: Derive Principal Component Features by taking dot product of eigen vector and standardized columns
6. Conclusion

## 1. Introduction: What is PCA?

PCA is a fundamentally a simple dimensionality reduction technique that transforms the columns of a dataset into a new set features called Principal Components (PCs).

The information contained in a column is the amount of variance it contains. The primary objective of Principal Components is to represent the information in the dataset with minimum columns possible.

But, How to compute the PCs using a package like scikit-learn and how to actually compute it from scratch (without using any packages)? and importantly how to understand PCA and what is the intuition behind it?

I will try to answer all of these questions in this post using the of MNIST dataset. Structure of the Post: Part 1: Implementing PCA using scikit-Learn package Part 2: Understanding Concepts behind PCA Part 3: PCA from Scratch without scikit-learn package. Let’s first understand the data at hand.

## Part 1: Implementing PCA using scikit learn

##### Dataset Description and Practical Uses of PCA

I’ll use the MNIST dataset, where each row represents a square image of a handwritten digit (0-9). The values in each cell ranges between 0 and 255 corresponding to the gray-scale color. Each image is of 28×28=784 pixels, so the flattened version of this will have 784 column entries. why use PCA? Practically PCA is used for two reasons:

1. Dimensionality Reduction: The information distributed across a large number of columns is transformed into principal components (PC) such that the first few PCs can explain a sizeable chunk of the total information (variance). These PCs can be used as explanatory variables in Machine Learning models.
2. Visualize Classes: Visualising the separation of classes (or clusters) is hard for data with more than 3 dimensions (features). With the first two PCs itself, it’s usually possible to see a clear separation.

Is PCA a feature selection technique? It is not a feature selection technique. Rather, it is a feature combination technique. Because each PC is a weighted additive combination of all the columns in the original dataset. More on this when you implement it in the next section. However, the PCs are formed in such a way that the first Principal Component (PC1) explains more variance in original data compared to PC2. Likewise, PC2 explains more than PC3, and so on.

## 2. Building PCA with Scikit-learn

Using scikit-learn package, the implementation of PCA is quite straight forward. The module named sklearn.decomposition provides the PCA object which can simply fit and transform the data into Principal components.

# Load Packages
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
pd.set_option('max_rows', 500)
pd.set_option('max_columns', 500)
np.set_printoptions(suppress=True)
%matplotlib inline

##### Import Data

Let’s import the mnist dataset. For ease of learning, I am importing a smaller version containing records for digits 0, 1 and 2 only.

# Import Data

# Prepare X and Y
Y = df.loc[:, '0']
X = df.drop(['0'], axis=1)

print(df.shape)

(3147, 785) This dataset has 784 columns as explanatory variables and one Y variable names '0' which tells what digit the row represents. We won’t use the Y when creating the principal components.

#### Get Free Complete Python Course

Facing the same situation like everyone else?

Build your data science career with a globally recognised, industry-approved qualification. Get the mindset, the confidence and the skills that make Data Scientist so valuable.

#### Get Free Complete Python Course

Build your data science career with a globally recognised, industry-approved qualification. Get the mindset, the confidence and the skills that make Data Scientist so valuable.

Because I don’t want the PCA algorithm to know which class (digit) a particular row belongs to.

Rather, I create the PCs using only the X. Later you will see, we draw a scatter plot using the first two PCs and color the points based in the actual Y. Typically, if the X’s were informative enough, you should see clear clusters of points belonging to the same category.

#### Building the Principal Components

Let’s first create the Principal components of this dataset.

First, I initialize the PCA() class and call the fit_transform() on X to simultaneously compute the weights of the Principal components and then transform X to produce the new set of Principal components of X. This I am storing in the df_pca object, which is converted to a pandas DataFrame.

# PCA
pca = PCA()
df_pca = pca.fit_transform(X=X)

# Store as dataframe and print
df_pca = pd.DataFrame(df_pca)
print(df_pca.shape)  #> (3147, 784)

The first column is the first PC and so on. This dataframe (df_pca) has the same dimensions as the original data X.

## 3. Weights of Principal Components

The pca has been built.

The pca.components_ object contains the weights (also called as ‘loadings’) of each Principal Component. It is using these weights that the final principal components are formed.

But what exactly are these weights? how are they related to the Principal components we just formed and how it is calculated?

Well, in part 2 of this post, you will learn that these weights are nothing but the eigenvectors of X. More details on this when I show how to implement PCA from scratch without using sklearn’s built-in PCA module.

The key thing to understand is that, each principal component is the dot product of its weights (in pca.components_) and the mean centered data(X). What I mean by ‘mean-centered’ is, each column of the ‘X’ is subtracted from its own mean so that the mean of each column becomes zero. Let’s actually compute this, so its very clear.

# Principal Components Weights (Eigenvectors)

Each row actually contains the weights of Principal Components, for example, Row 1 contains the 784 weights of PC1.

No need to pay attention to the values at this point, I know, the picture is not that clear anyway.

Step 2: Compute the mean centered data.

X_mean = X - X.mean()

In above dataframe, I’ve subtracted the mean of each column from each cell of respective column itself. So the mean of each column now is zero.

Step 3: If you go by the formula, take a dot product of of the weights in the first row of pca.components_ and the first row of the mean centered X to get the value -134.27.

This equals to the value in position (0,0) of df_pca. Do refer back to the pic in section 2 to confirm this.

# Compute PC1 for row 1.
#> -134.272

Likewise, all the cells of the principal components matrix (df_pca) is computed this way internally.

## 4. Percentage of Variance Explained with each PC

The PCs are usually arranged in the descending order of the variance(information) explained. To see how much of the total information is contributed by each PC, look at the explained_variance_ratio_ attribute.

print(pca.explained_variance_ratio_.round(2)[:10])
#> [0.22 0.1  0.06 0.06 0.04 0.04 0.03 0.03 0.02 0.02]

How to read this? PC1 contributed 22%, PC2 contributed 10% and so on. The further you go, the lesser is the contribution to the total variance. Plotting a cumulative sum gives a bigger picture.

variance_exp_cumsum = pca.explained_variance_ratio_.cumsum().round(2)
fig, axes = plt.subplots(1,1,figsize=(16,7), dpi=100)
plt.plot(variance_exp_cumsum, color='firebrick')
plt.title('Screeplot of Variance Explained %', fontsize=22)
plt.xlabel('# of PCs', fontsize=16)
plt.show()

## 5. Plot the clustering tendency

If you draw a scatterplot against the first two PCs, the clustering of data points of 0, 1 and 2 is clearly visible. Such graphs are good to show your team/client. Let me define the encircle function to enable encircling the points within the cluster.

from scipy.spatial import ConvexHull

def encircle(x,y, ax=None, **kw):
if not ax: ax=plt.gca()
p = np.c_[x,y]
hull = ConvexHull(p)
poly = plt.Polygon(p[hull.vertices,:], **kw)

Let’s plot the first two principal components along the X and Y axis.

# Scatterplot against PC1 and PC2
fig, ax = plt.subplots(1,1, figsize=(16,12))

# Row masks for each category
rows_0 = Y==0;
rows_1 = Y==1;
rows_2 = Y==2;

# Plot
ax.scatter(df_pca.loc[rows_0.tolist(), 1], df_pca.loc[rows_0.tolist(), 2], c='blue', edgecolor='k', s=120, label='Zero')
ax.scatter(df_pca.loc[rows_1.tolist(), 1], df_pca.loc[rows_1.tolist(), 2], c='red', edgecolor='k', s=120, label='One')
ax.scatter(df_pca.loc[rows_2.tolist(), 1], df_pca.loc[rows_2.tolist(), 2], c='green', edgecolor='k', s=120, label='Two')

# Encircle the boundaries
encircle(df_pca.loc[rows_0.tolist(), 1], df_pca.loc[rows_0.tolist(), 2], ec="blue", fc="none", linewidth=2.5)
encircle(df_pca.loc[rows_1.tolist(), 1], df_pca.loc[rows_1.tolist(), 2], ec="firebrick", fc="none", linewidth=2.5)
encircle(df_pca.loc[rows_2.tolist(), 1], df_pca.loc[rows_2.tolist(), 2], ec="green", fc="none", linewidth=2.5)

encircle(df_pca.loc[rows_1.tolist(), 1], df_pca.loc[rows_1.tolist(), 2], ec="k", fc="firebrick", alpha=0.05)
encircle(df_pca.loc[rows_0.tolist(), 1], df_pca.loc[rows_0.tolist(), 2], ec="k", fc="blue", alpha=0.05)
encircle(df_pca.loc[rows_2.tolist(), 1], df_pca.loc[rows_2.tolist(), 2], ec="k", fc="green", alpha=0.05)

# Labels
ax.set_title("MNIST Data for 1, 2 & 3: Scatterplot of First Two PCA directions", fontsize=22)
ax.set_xlabel("1st Principal Component", fontsize=22)
ax.set_ylabel("2nd Principal Component", fontsize=22)
ax.legend(loc='best', title='Transaction Type', fontsize=16)
plt.show();

In the picture, though there is a certain degree of overlap, the points belonging to same category are distinctly clustered and region bound. This proves that the data captured in the first two PCs is informative enough to discriminate the categories from each other.

In other words, we now have evidence that the data is not completely random, but rather can be used to discriminate or explain the Y (the number a given row belongs to).

Refer to the 50 Masterplots with Python for more visualization ideas.

## 6. How to get the original features back

The fitted pca object has the inverse_transform() method that gives back the original data when you input principal components features.

df_orig = pca.inverse_transform(df_pca)

The above code outputs the original input dataframe.

## Part 2: Understanding Concepts behind PCA

To simplify things, let’s imagine a dataset with only two columns. Using these two columns, I want to find a new column that better represents the ‘data’ contributed by these two columns.

This new column can be thought of as a line that passes through these points. Such a line can be represented as a linear combination of the two columns and explains the maximum variation present in these two columns.

In what direction do you think the line should stop so that it covers the maximum variation of the data points? Such a line should be in a direction that minimizes the perpendicular distance of each point from the line. It may look something like this: But how to determine this line?

I am only interested in determining the direction(u1) of this line. Because, by knowing the direction u1, I can compute the projection of any point on this line. This line u1 is of length 1 unit and is called a unit vector. This unit vector eventually becomes the weights of the principal components, also called as loadings which we accessed using the pca.components_ earlier.

Remember the PCA weights you calculated in Part 1 under ‘Weights of Principal Components’? It is same as the ”u1′ I am talking about here.

The PCA weights (Ui) are actually unit vectors of length 1. Because, it is meant to represent only the direction.

# PCA weights has length = 1
#> 0.9999999999999993

## 7. Objective function of PCA

So, how to find this unit vector (u1)?

Remember, we wanted to minimize the distances of the points from PC1’s direction?

u1 is the unit vector that direction.

To determine u1, we use Pythagoras theorem to arrive at the objective function as shown in pic.

In the pic below, u1 is the unit vector of the direction of PC1 and Xi is the coordinates of the blue dot in the 2d space. Remember, Xi is nothing but the row corresponding the datapoint in the original dataset.

The lengths of the lines can be computed using the Pythagoras theorem as shown in the pic below.

### Objective function:

The objective is to determine u1 so that the mean perpendicular distance from the line for all points is minimized. Here is the objective function: It can be proved that the above equation reaches a minimum when value of u1 equals the Eigen Vector of the covariance matrix of X. This Eigen Vector is same as the PCA weights that we got earlier inside pca.components_ object.

We’ll see what Eigen Vectors are shortly.

Alright. But there can be a second PC to this data. The next best direction to explain the remaining variance is perpendicular to the first PC. Actually, there can be as many Eigen Vectors as there are columns in the dataset. And they are orthogonal to each other. Thanks to this excellent discussion on stackexchange that provided these dynamic graphs.

## How to understand the rotation of coordinate axes

Now you know the direction of the unit vector. To compute the Principal components, we rotate the original XY axis of to match the direction of the unit vector.

The Principal components are nothing but the new coordinates of points with respect to the new axes.

## 8. What is Eigen Value and Eigen Vector?

If you were like me, Eigenvalues and Eigenvectors are concepts you would have encountered in your matrix algebra class but paid little attention to. Well, Eigen Values and Eigen Vectors are at the core of PCA.

What is Eigen vector?

v is an eigenvector of matrix A if A(v) is a scalar multiple of v. The actual computation of Eigenvector and Eigen value is quite straight forward using the eig() method in numpy.linalg module.

Refer to this guide if you want to learn more about the math behind computing Eigen Vectors.

## 9. Import Data

# Import
url = 'https://raw.githubusercontent.com/selva86/datasets/master/mnist_012.csv'

# Prepare X and Y
Y = df.loc[:, '0']
X = df.drop(['0'], axis=1)

## Step 1: Standardize each column

Subtract each column by its own mean. As a result, the mean of each column becomes zero.

X_standard = X - X.mean()
X_standard

## Step 2 Compute Covariance Matrix

But what is covariance and covariance matrix?

Covariance measures how two variables are related to each other, that is, if two variables are moving in the same direction with respect to each other or not.

When covariance is positive, it means, if one variable increases, the other increases as well. The opposite true when covariance is negative. The covariance matrix calculates the covariance of all possible combinations of columns.

As a result, it becomes a square matrix with the same number of rows and columns. But, How to actually compute the covariance matrix in Python?

Using pandas dataframe, covariance matrix is computed by calling the df.cov() method.

df_cov = X_standard.cov()
print(df.shape)

## Step 3: Compute Eigen values and Eigen Vectors

Eigen values and Eigen vectors represent the amount of variance explained and how the columns are related to each other.

The length of Eigenvectors is one.

from numpy.linalg import eig
eigvalues, eigvectors = eig(df_cov)
print(eigvalues[:10])
print(eigvectors.shape)
#> [764074.70509694+0.j 333409.88084323+0.j 206248.30545139+0.j
#>  190073.11430869+0.j 149373.67054257+0.j 140893.79592355+0.j
#>  103478.87643359+0.j  87549.48702637+0.j  82347.51266608+0.j
#>  66776.33611636+0.j]

#> (784, 784)

The j in the above output implies the resulting eigenvectors are represented as complex numbers.

## Step 4: Derive Principal Component Features by taking dot product of eigen vector and standardized columns

X_pca = np.dot(X_standard, eigvectors)
df_pca_calc = pd.DataFrame(X_pca)

The result is the Principal componenents, which, is the same as the PC’s computed using the scikit-learn package.

## Conclusion

Congratulations if you’ve completed this, because, we’ve pretty much discussed all the core components you need to understand in order to crack any question related to PCA.

PCA can be a powerful tool for visualizing clusters in multi-dimensional data.

Plus, it is also while building machine learning models as it can be used as an explanatory variable as well.

You saw the implementation in scikit-learn, the concept behind it and how to code it out algorithmically as well.

Course Preview


text:
## Scott is trying to make a profit on the stock market by investing in some stock that his friends recommended. He has $1000 invested and was Question Scott is trying to make a profit on the stock market by investing in some stock that his friends recommended. He has$1000 invested and was hoping to gain 10 points. Instead, his stocks are currently declining 5 points every hour. What is the rate of change or slope of his stocks?

in progress 0
6 months 2021-08-24T23:37:49+00:00 1 Answers 2 views 0


text:
# Measuring Shifts in Attitudes Towards COVID-19 Measures in Belgium Using Multilingual BERT

This model can be used to determine if a tweet expresses support or not for a curfew. The model was trained on manually labeled tweets from Belgium in Dutch, French and English.

We categorized several months worth of these Tweets by topic (government COVID measure) and opinion expressed. Below is a timeline of the relative number of Tweets on the curfew topic (middle) and the fraction of those Tweets that find the curfew too strict, too loose, or a suitable measure (bottom), with the number of daily cases in Belgium to give context on the pandemic situation (top).

Models used in this paper are on HuggingFace:

Mask token: [MASK]