text:
# [Aptitude-devel] r3083 - branches/aptitude-0.3/aptitude/src/generic/problemresolver

Daniel Burrows dburrows@costa.debian.org
Mon, 25 Apr 2005 03:18:24 +0000

Author: dburrows
Date: Mon Apr 25 03:18:23 2005
New Revision: 3083

Modified:
branches/aptitude-0.3/aptitude/src/generic/problemresolver/model.tex
Log:
Correct an error in the reduction from apt

Modified: branches/aptitude-0.3/aptitude/src/generic/problemresolver/model.tex
==============================================================================
--- branches/aptitude-0.3/aptitude/src/generic/problemresolver/model.tex	(original)
+++ branches/aptitude-0.3/aptitude/src/generic/problemresolver/model.tex	Mon Apr 25 03:18:23 2005
@@ -269,14 +269,16 @@
$\D$ contains $v -> S$ for every such dependency.
\item For each conflict declared by a version $v$ on the package $p$,
accumulate a set $S$ containing all the \emph{non}-matching versions
-  of $p$ and insert $v -> S$ into $\D$.  Furthermore, if the conflict
-  is not versioned, then for each package $p'$ and version $v'$ of
-  $p'$ such that $v'$ provides $p$, let $S=\{v'' \st \pkgof{v''}=p' \land - \text{$v''$does not provide$p$}\}$ and insert $v -> S$ into $\D$.
+  of $p$, including the uninstalled'' version, and insert $v -> S$
+  into $\D$.  Furthermore, if the conflict is not versioned, then for
+  each package $p'$ and version $v'$ of $p'$ such that $v'$ provides
+  $p$, let $S=\{v'' \st \pkgof{v''}=p' \land \text{$v''$does not + provide$p$}\}$ and insert $v -> S$ into $\D$.

For instance, if $v$ conflicts with \pkg{A}, of which versions 3.2
-  and 3.3 are available, and versions 2 and 3 of \pkg{B} provide
-  \pkg{A}, then $S=\{\pkg{A}:3.2,\pkg{A}:3.3,\pkg{B}:2,\pkg{B}:3\}$.
+  and 3.3 are available, versions 2 and 3 of \pkg{B} provide \pkg{A},
+  and no other versions of \pkg{B} are available, then
+  $S=\{\pkg{A}:3.2,\pkg{A}:3.3,\pkg{A}:\pkg{UNINST},\pkg{B}:2,\pkg{B}:\pkg{UNINST}\}$.

\begin{note}
In reality, extra care must be taken to screen out self-conflicts


text:
# Turning bookmarks into Markdown reference links

For some time now, I’ve been writing almost all of my blog posts in Drafts on an iPad. I usually have Drafts and Safari set up in Split View, and I copy the URL of the current Safari page whenever I need to add a link to a post. I’ve always preferred Markdown’s reference format for links, and this action inserts reference links in Drafts the same way an AppleScript I wrote a long time ago does it in BBEdit.1

With a URL on the clipboard, I select the text that’s going to be the link and run the action. It surrounds the selected text with brackets, puts a numbered reference after it,

after reading [this John Voorhees review][3] in MacStories.

and then adds the reference and URL at the bottom of the draft:

[3]: https://www.macstories.net/reviews/highlights-for-iphone-and-ipad-an-excellent-companion-for-researchers/

The reference number is incremented each time.

Sometimes, though, I have a bunch of websites open in tabs in Safari, and because I know I’ll be linking to each of them, I’d like to add all the references to the bottom of the draft in a single step. I can do this via AppleScript on the Mac, but Shortcuts currently isn’t smart enough to do this sort of thing. So I decided to use the Mac to get around iPadOS’s limitations.

First, on the iPad, I bookmark all the tabs to a “Blogging” sublist I’ve set up in my Favorites (which is the Bookmarks bar on the Mac). This can be done in a single step by long-pressing the bookmark button and choosing the command from the popup menu.

Because iCloud syncs my bookmarks, all of these links are now saved in the ~/Library/Safari/Bookmarks.plist file on my Mac. If I have a script on my Mac that can extract them from that file, I can run it from my iPad via the Shortcuts command.

Such a script isn’t that hard to write in Python, as the standard library has a plistlib module that knows how to read and write plist files.

Here’s the mdbookmarks script that pulls out all the URLs in the “Blogging” sublist and prints them in Markdown reference format:

python:
1:  #!/usr/bin/env python
2:
3:  import plistlib
4:  import os
5:  from docopt import docopt
6:
7:  # Handle the command line options
8:  usage = '''Usage:
9:    mdbookmarks [options]
10:
11:  Options:
12:    -n NNN  Starting number for reference links [default: 1]
13:    -h      Show this help message
14:
15:  Return a set of lines with Markdown reference links to all the
16:  bookmarks in the "Blogging" section of Safari's Bookmarks bar.'''
17:
18:  args = docopt(usage)
19:  n = int(args['-n'])
20:
21:  # Load the Safari Bookmarks plist
22:  fn = os.environ['HOME'] + '/Library/Safari/Bookmarks.plist'
23:  with open(fn, 'rb') as f:
25:
26:  # Work through the plist and assemble all the URLs in the
27:  # "Blogging" section of the Bookmarks bar into a list
28:  blist = []
29:  for a in bm['Children']:
30:    if a['Title'] == 'BookmarksBar':
31:      for b in a['Children']:
32:        try:
33:          if b['Title'] == 'Blogging':
34:            for c in b['Children']:
35:              blist.append(f"[{n}]: {c['URLString']}")
36:              n += 1
37:        except KeyError:
38:          pass
39:
40:  # Print the list, one per line
41:  print('\n'.join(blist), end='')

As usual, I use the nonstandard docopt module to handle the command-line switches.

The trickiest part of this script was figuring out where the “Blogging” bookmarks are kept within the labyrinth of Bookmarks.plist. Lines 28–38 were built up through trial and error. I dug through the layers of the plist, testing each layer using type and keys to eventually find my way to the URLs.

Although this script is written for Python 3.7, it shouldn’t be too hard to get it to run on the Python 2.7 that comes preinstalled on the Mac. I think you can just change the f-string in Line 35 to a format call.

I ran mdbookmarks on my Mac to test it, but it’s meant to be run via a shortcut on my iPad. The shortcut that does it is called “Blogging Bookmarks,” and it has only two steps:

StepActionComment
1 Get the starting number from the user.
2 Log into my Mac and run the mdbookmarks script. Use the starting number from the previous step.

You’ll note that this shortcut doesn’t do anything with the output of mdbookmarks. That’s because it’s meant to be run from a simple Drafts action that can be downloaded from here. There are only two steps in the action:2

1. Run the “Blogging Bookmarks” shortcut. This makes the output of mdbookmarks script available in the [[shortcut_result]] template.
2. Insert [[shortcut_result]] at the current cursor point. The intention is to run this action with the cursor at the bottom of the draft.

With all three pieces in place, adding a bunch of references to blog post is fairly easy. I just add bookmarks to the “Blogging” sublist as I do my research, then run the “Blogging Bookmarks” action in Drafts to insert all the references when I start writing the post.

I can make this system sound ridiculously complex: It’s a Drafts action that runs a shortcut that connects to my Mac over SSH and runs a Python script that reads a plist file and extracts information that then works its way back along the chain to insert text with that information into Drafts. But each link in the chain is straightforward. Only mdbookmarks took more than a couple of minutes to write, and that was because the Bookmarks.plist file is a nested mess.

The biggest limitation of this system is that I have to clean out the “Blogging” sublist whenever I’m done with a post. Otherwise, links from older posts will be inserted into newer posts.

I’m pleased I was able to build this system, but I wish I didn’t have to. I’d be much happier if Apple made all of this obsolete by adding features to Shortcuts that, among other things, allow us to get all of Safari’s tab links directly.

1. Going even farther back, I had a similar system set up in TextMate.

2. Unfortunately, Drafts actions aren’t especially screenshotable. You have to go at least three levels deep to see all of an action’s properties, and I’m just not interested in trying to figure out a good way to present all that scattered information. It’s better to just download the action and see it within Drafts itself.

format: plain


text:
more useful options PerlMonks

Deleting windows environment variables

 on Aug 19, 2004 at 09:28 UTC ( #384224=perlquestion: print w/ replies, xml ) Need Help??
qadwjoh has asked for the wisdom of the Perl Monks concerning the following question:

Hi

I'm writing a CGI script to run on Windows (2000 Server to be precise) and I was thinking of deleting environment variables when the script starts, for extra security.

Does anyone have any suggestions on what I should delete?

thanks,
A

Comment on Deleting windows environment variables
Re: Deleting windows environment variables
by TStanley (Canon) on Aug 19, 2004 at 09:54 UTC
Check out Ovid's CGI course for some very good advice.

TStanley
--------
The only thing necessary for the triumph of evil is for good men to do nothing -- Edmund Burke
Is there Windows specific advice in this guide? I couldn't seem to find any.

thanks,
A

Firstly you have to know what environment variables are presented to a CGI program on windows that differ from those on other platforms - you can determine this by running something like:

```#!perl -w

use CGI qw(:standard);

foreach \$var (keys %ENV )
{
print "\$var : \$ENV{\$var}<br />\n";
}
print end_html;
On IIS 5.0 with w2k I get:
```USERPROFILE : C:\Documents and Settings\Default User
SCRIPT_NAME : /test.pl
PATH_INFO : /test.pl
OS2LIBPATH : C:\WINNT\system32\os2\dll;
REQUEST_METHOD : GET
HTTP_ACCEPT : image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
COMMONPROGRAMFILES : C:\Program Files\Common Files
SERVER_SOFTWARE : Microsoft-IIS/5.0
PROGRAMFILES : C:\Program Files
OS : Windows_NT
PATHEXT : .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH
HTTP_USER_AGENT : Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .
+NET CLR 1.1.4322)
HTTP_ACCEPT_LANGUAGE : en-gb
NUMBER_OF_PROCESSORS : 1
PATH : C:\WINNT\system32;C:\WINNT;C:\WINNT\System32\Wbem;C:\Program Fi
+les\Microsoft SQL Server\80\Tools\BINN
GATEWAY_INTERFACE : CGI/1.1
INCLUDE : C:\Program Files\Microsoft Visual Studio .NET 2003\SDK\v1.1\
+include\
HTTPS : off
REMOTE_HOST : 127.0.0.1
PROCESSOR_ARCHITECTURE : x86
PATH_TRANSLATED : c:\inetpub\wwwroot\test.pl
SERVER_NAME : localhost
TEMP : C:\WINNT\TEMP
HTTP_ACCEPT_ENCODING : gzip, deflate
SYSTEMDRIVE : C:
HTTP_CONNECTION : Keep-Alive
PROCESSOR_REVISION : 0308
VS71COMNTOOLS : C:\Program Files\Microsoft Visual Studio .NET 2003\Com
+mon7\Tools\
SYSTEMROOT : C:\WINNT
CONTENT_LENGTH : 0
INSTANCE_ID : 1
COMSPEC : C:\WINNT\system32\cmd.exe
WINDIR : C:\WINNT
SERVER_PORT_SECURE : 0
PROCESSOR_LEVEL : 15
SERVER_PORT : 80
SERVER_PROTOCOL : HTTP/1.1
PROCESSOR_IDENTIFIER : x86 Family 15 Model 3 Stepping 8, GenuineIntel
LIB : C:\Program Files\Microsoft Visual Studio .NET 2003\SDK\v1.1\Lib\
ALLUSERSPROFILE : C:\Documents and Settings\All Users
COMPUTERNAME : VM-DEV-US
HTTP_HOST : localhost
TMP : C:\WINNT\TEMP
Now obviously some of these are going to be needed for the proper operation of the CGI, and some are just set by windows for the convenience of the application. Of course most of the superfluous ones here will have no affect whatsoever on the operation of a Perl program and just so long as you aren't letting them leak out into the outside world then they shouldn't be a problem. The standard sanitising of PATH should generally be enough. The ones that are windows specific and can be got rid of in their entirety are:
```USERPROFILE
OS2LIBPATH
COMMONPROGRAMFILES
PROGRAMFILES
OS
PATHEXT
NUMBER_OF_PROCESSORS
INCLUDE
PROCESSOR_ARCHITECTURE
TEMP
SYSTEMDRIVE
PROCESSOR_REVISION
VS71COMNTOOLS
SYSTEMROOT
INSTANCE_ID
COMSPEC
WINDIR
PROCESSOR_LEVEL
PROCESSOR_IDENTIFIER
LIB
ALLUSERSPROFILE
COMPUTERNAME
TMP
Of course those on your machine will almost certainly be different.

And before anyone chimes in, no I don't believe it is a security risk to be showing the contents of these variables as a) they are not giving away any secrets and, b) the machine is on a private network behind a firewall.

/J\

Re: Deleting windows environment variables
by Chady (Priest) on Aug 19, 2004 at 11:52 UTC

Instead of looking for what to delete, why not keep only what you want ?

```BEGIN {
my %e = %ENV;
%ENV = (
PATH_INFO => \$e{PATH_INFO},
QUERY_STRING => \$e{QUERY_STRING},
etc...
);
}

He who asks will be a fool for five minutes, but he who doesn't ask will remain a fool for life.

Re: Deleting windows environment variables
by davido (Archbishop) on Aug 19, 2004 at 15:55 UTC

%ENV may also be localized:

```{
local %ENV;        # Old %ENV saved away in case you need it
# outside this scope.
%ENV = ( ...... ); # Create your own custom %ENV.
# Most of your script here...
}
# Here, %ENV will be restored to the original, in case you need
# it for something else.

Dave

Create A New User
Node Status?
node history
Node Type: perlquestion [id://384224]
Approved by TStanley
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (2)
As of 2014-03-09 21:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?


text:
Definitions
Nearby Words

# Rocket engine

A rocket engine is a jet engine that uses only propellant mass for forming its high speed propulsive jet. Rocket engines are reaction engines and obtain thrust in accordance with Newton's third law. Since they need no external material to form their jet, rocket engines can be used for spacecraft propulsion as well as terrestrial uses, such as missiles. Most rocket engines are internal combustion engines, although non combusting forms also exist.

A rocket motor is a synonymous term that usually refers to solid rocket engines.

## Principle of operation

Most rocket engines produce thrust by the expulsion of a high-temperature, high-speed gaseous exhaust. This is typically created by high pressure (10-200 bar) combustion of solid or liquid propellants, consisting of fuel and oxidiser components, within a combustion chamber.

### Introducing propellant into a combustion chamber

Liquid-fueled rockets typically pump separate fuel and oxidiser components into the combustion chamber, where they mix and burn. Solid rocket propellants are prepared as a mixture of fuel and oxidizing components and the propellant storage chamber becomes the combustion chamber. Hybrid rocket engines use a combination of solid and liquid or gaseous propellants. Alternatively, a chemically inert reaction mass can be heated using a high-energy power source via a heat exchanger, and then no combustion chamber is used.

Rocket propellant is mass that is stored, usually in some form of propellant tank, prior to being used as the propulsive mass that is ejected from a rocket engine in the form of a fluid jet to produce thrust.

Chemical rocket propellants are most commonly used, which undergo exothermic chemical reactions which produce hot gas which is used by a rocket for propulsive purposes.

### Combustion chamber

For chemical rockets the combustion chamber is typically just a cylinder. The dimensions of the cylinder is such that the propellant is able to combust thoroughly; different propellants require different combustion chamber sizes for this to occur. This leads to a number called $L*$:
$L* = frac \left\{V_c\right\} \left\{A_t\right\}$
where:
$V_c$ is the Volume of the chamber
$A_t$ is the area of the throat
L* is typically in the range of 25-60 inches (0.6 - 1.5 m)

The combination of temperatures and pressures typically reached in a combustion chamber are usually extreme by any standards. Unlike air-breathing jet engines no atmospheric nitrogen is present to dilute and cool the combustion and the temperature can reach true stochiometric. The high pressures mean that the rate of conduction of heat through the walls is very high indeed.

### Rocket nozzles

The large bell or cone shaped expansion nozzle gives a rocket engine its characteristic shape.

In rockets the hot gas produced in the combustion chamber is permitted to escape from the combustion chamber through an opening (the "throat"), within a high expansion-ratio 'de Laval nozzle'.

Provided sufficient pressure is provided to the nozzle (about 2.5-3x above ambient pressure) the nozzle chokes and a supersonic jet is formed, dramatically accelerating the gas, converting most of the thermal energy into kinetic energy. Exhaust speeds as high as ten times the speed of sound at sea level are not uncommon.

A portion of the rocket engine's thrust comes from the unbalanced pressures inside the combustion chamber but the majority comes from the pressures against the inside of the nozzle (see diagram). As the gas expands (adiabatically) the pressure against the nozzle's walls forces the rocket engine in one direction while accelerating the gas in the other.

### Propellant efficiency

For a rocket engine to be propellant efficient, it is important that the maximum pressures possible be created by a specific amount of propellant acting on the chamber and nozzle. This can be achieved by all of:

• heating the propellant to as high a temperature as possible (using a high energy fuel, containing hydrogen and carbon and sometimes metals such as aluminium, or even using nuclear energy)
• using a low specific density gas (as hydrogen rich as possible)
• using propellants which are, or decompose to, simple molecules with few degrees of freedom to maximise translational velocity

Since all of these things minimise the mass of the propellant used, and since pressure is proportional to the amount of propellant present to be accelerated as it pushes on the engine, and since from Newton's third law the pressure that acts on the engine also reciprocally acts on the propellant, it turns out that the speed that the propellant leaves the chamber is unaffected by the chamber pressure (although the thrust is proportional). However, speed is significantly affected by all three of the above factors and the exhaust speed is an excellent measure of the engine propellant efficiency.

For aerodynamic reasons the flow goes sonic ("chokes") at the narrowest part of the nozzle, the 'throat'. Since the speed of sound in gases increases with the square root of temperature, the use of hot exhaust gas greatly improves performance. By comparison, at room temperature the speed of sound in air is about 340 m/s while the speed of sound in the hot gas of a rocket engine can be over 1700 m/s; much of this performance is due to the higher temperature, but additionally rocket propellants are chosen to be of low molecular mass, and this also gives a higher velocity compared to air.

Expansion in the rocket nozzle then further multiplies the speed, typically between 1.5 and 2 times, giving a highly collimated hypersonic exhaust jet. The speed increase of a rocket nozzle is mostly determined by its area expansion ratio—the ratio of the area of the throat to the area at the exit, but detailed properties of the gas are also important. Larger ratio nozzles are more massive but are able to extract more heat from the combustion gases, increasing the exhaust velocity.

Nozzle efficiency is affected by operation in the atmosphere because atmospheric pressure changes with altitude; but due to the supersonic speeds of the gas exiting from a rocket engine, the pressure of the jet may be either below or above ambient, and equilibrium between the two is not reached at all altitudes (See Diagram).

### Back pressure and optimal expansion

For optimal performance the pressure of the gas at the end of the nozzle should just equal the ambient pressure: if the exhaust's pressure is lower than the ambient pressure, then the vehicle will be slowed by the difference in pressure between the top of the engine and the exit; on the other hand, if the exhaust's pressure is higher, then exhaust pressure that could have been converted into thrust is not converted, and energy is wasted.

To maintain this ideal of equality between the exhaust's exit pressure and the ambient pressure, the diameter of the nozzle would need to increase with altitude, giving the pressure a longer nozzle to act on (and reducing the exit pressure and temperature). This increase is difficult to arrange. A compromise nozzle is generally used and some reduction in performance occurs. To improve on this, various exotic nozzle designs such as the plug nozzle, stepped nozzles, the expanding nozzle and the aerospike have been proposed, each providing some way to adapt to changing ambient air pressure and each allowing the gas to expand further against the nozzle, giving extra thrust at higher altitudes.

When exhausting into a sufficiently low ambient pressure (vacuum) several issues arise. One is the shear weight of the nozzle- beyond a certain point, for a particular vehicle, the extra weight of the nozzle outweighs any performance gained. Secondly, as the exhaust gases adiabatically expand within the nozzle they cool, and eventually some of the chemicals can freeze, producing 'snow' within the jet. This causes instabilities in the jet and must be avoided.

## Overall rocket engine performance

Rocket technology can combine very high thrust (meganewtons), very high exhaust speeds (around 10 times the speed of sound at sea level) and very high thrust/weight ratios (>100) simultaneously as well as being able to operate outside the atmosphere.

Rockets can be further optimised to even more extreme performance along one or more of these axes at the expense of the others.

### Specific impulse

The most important metric for the efficiency of a rocket engine is impulse per unit of propellant, this is called specific impulse (usually written $I_\left\{sp\right\}$). This is either measured as a speed ($V_\left\{e\right\}$ in metres/second or ft/s) or as a time (seconds). An engine that gives a large specific impulse is normally highly desirable.

### Net thrust

Below is an approximate equation for calculating the net thrust of a rocket engine:

$F_n = dot\left\{m\right\};V_\left\{e\right\} = dot\left\{m\right\};V_\left\{e-act\right\} + A_\left\{e\right\}\left(P_\left\{e\right\} - P_\left\{amb\right\}\right)$

where:

$dot\left\{m\right\} = ,$exhaust gas mass flow

$V_\left\{e\right\} =,$effective exhaust velocity

$V_\left\{e-act\right\} =,$actual jet velocity at nozzle exit plane

$A_\left\{e\right\} =,$flow area at nozzle exit plane

$P_\left\{e\right\} =,$static pressure at nozzle exit plane

$P_\left\{amb\right\} =,$ambient (or atmospheric) pressure

Since, unlike a jet engine, a conventional rocket motor lacks an air intake, there is no 'ram drag' to deduct from the gross thrust. Consequently the net thrust of a rocket motor is equal to the gross thrust (apart from static back pressure).

The $dot\left\{m\right\};V_\left\{e-act\right\},$ term represents the momentum thrust, which remains constant at a given throttle setting, whereas the $A_\left\{e\right\}\left(P_\left\{e\right\} - P_\left\{amb\right\}\right),$ term represents the pressure thrust term. At full throttle, the net thrust of a rocket motor improves slightly with increasing altitude, because as atmospheric pressure decreases with altitude, the pressure thrust term increases.

Maximum thrust for a rocket engine is achieved by maximizing the momentum contribution of the equation without incurring penalties from over expanding the exhaust. This occurs when $P_\left\{e\right\} = P_\left\{amb\right\}$. Since ambient pressure changes with altitude, most rocket engines spend very little time operating at peak efficiency.

### Vacuum Isp

Due to the specific impulse varying with pressure, a quantity that is easy to compare and calculate with is useful. Because rockets choke at the throat, and because the supersonic exhaust prevents external pressure influences travelling upstream, it turns out that the pressure at the exit is ideally exactly proportional to the propellant flow $dot\left\{m\right\}$, provided the mixture ratios and combustion efficiencies are maintained. It is thus quite usual to rearrange the above equation slightly:

$Fvac = C_f dot\left\{m\right\} c^*$

and so define the vacuum Isp to be:

$V_\left\{evac\right\} = C_f c^*$

Where:

$c^* =,$ the speed of sound constant at the throat
$C_f =,$ the thrust coefficient constant of the nozzle (typically between 0.8 and 1.9)

And hence:

$F_n = dot\left\{m\right\} V_\left\{evac\right\} - A_\left\{e\right\} P_\left\{amb\right\}$

### Throttling

Rockets can be throttled by controlling the propellant rate $dot\left\{m\right\}$ (usually measured in kg/s or lb/s).

In principle rockets can be throttled down to an exit pressure of about one-third of ambient pressure (due to flow separation in nozzles) and up to a maximum limit determined only be the mechanical strength of the engine.

In practice, the degree to which rockets can be throttled varies greatly, but most rockets may be throttled by a factor of 2 without great difficulty; the typical limitation is combustion stability, as for example, injectors need a minimum pressure to avoid triggering damaging oscillations (chugging or combustion instabilities); but injectors can often be optimised and tested for wider ranges. Additionally, it is important that the exit pressure not be too far below ambient to avoid flow separation in the nozzle.

### Energy efficiency

Rocket engine nozzles are surprisingly efficient heat engines for generating a high speed jet, as a consequence of the high combustion temperature and high compression ratio in accordance with the carnot cycle. For a vehicle employing a rocket engine the energetic efficiency is very good if the vehicle speed approaches or somewhat exceeds the exhaust velocity (relative to launch); but at low speeds the energy efficiency goes to 0% at zero speed (as with all jet propulsion.) See Rocket energy efficiency for more details.

## Cooling

For efficiency reason, and because they physically can, rockets run with reaction mass's combustion temperatures that can reach ~3500 K (~5800 °F).

The temperatures employed are very often far higher than the melting point of the nozzle and combustion chamber materials, two exceptions are graphite and tungsten (~1200 K for copper). Indeed many construction materials can make perfectly acceptable propellants in their own right. It is important that these materials be prevented from combusting, melting or vapourising to the point of failure. Materials technology could potentially place an upper limit on the exhaust temperature of chemical rockets.

Alternatively, rockets may use more common construction materials such as aluminum, steel, nickel or copper alloys and employ cooling systems that prevent the construction material itself becoming too hot. Regenerative cooling, where the propellant is passed through tubes around the combustion chamber or nozzle, and other techniques, such as curtain cooling or film cooling, are employed to give longer nozzle and chamber life. These techniques ensure that a gaseous thermal boundary layer touching the material is kept below the temperature which would cause the material to catastrophically fail.

In rockets, the heat fluxes that can pass through the wall are among the highest in engineering, fluxes are generally in the range of 1-200 MW/m^2. The strongest heat fluxes are found at the throat, which often sees twice that found in the associated chamber and nozzle. This is due to the combination of high speeds (which gives a very thin boundary layer), and the high temperatures seen there.

Most other jet engines have gas turbines in the hot exhaust. Due to their larger surface area, they are harder to cool and hence need to run their combustion processes at much lower temperatures, losing efficiency.

In rockets the coolant methods include:

1. uncooled (used for short runs mainly during testing)
2. ablative walls (walls are lined with a material that is continuously vapourised and carried away).
3. radiative cooling (the chamber becomes almost white hot and radiates the heat away)
4. dump cooling (a propellant, usually hydrogen, is passed around the chamber and dumped)
5. regenerative cooling (uses the propellant to cool the chamber via a cooling jacket before being injected)
6. curtain cooling (propellant injection is arranged so the temperature of the gases is cooler at the walls)
7. film cooling (surfaces are wetted with liquid propellant, which cools as it evaporates)

In all cases the cooling effect that prevents the wall from being destroyed is caused by a thin layer of insulating fluid (a boundary layer) that is in contact with the walls that is far cooler than the combustion temperature. Provided this boundary layer is intact the wall will not be damaged.

Disruption of the boundary layer may occur during cooling failures or combustion instabilities, and wall failure typically occurs soon after.

With regenerative cooling a second boundary layer is found in the coolant channels around the chamber. This boundary layer thickness needs to be as small as possible, since the boundary layer acts as an insulator between the wall and the coolant. This may be achieved by making the coolant velocity in the channels as high as possible.

## Mechanical issues

Rocket combustion chambers are normally operated at fairly high pressure, typically 10-200 bar (1 to 20 MPa), higher pressures often give better performance (by permitting a more efficient nozzle to be fitted). This causes the outermost part of the chamber to be under very large hoop stresses.

Worse, due to the high temperatures created in rocket engines the materials used tend to have a significantly lowered working tensile strength.

## Acoustic issues

In addition, the extreme vibration and acoustic environment inside a rocket motor commonly results in peak stresses well above mean values, especially in the presence of organ pipe-like resonances and gas turbulence.

### Combustion instabilities

Three different types of combustion instabilities occurChugging

This is a low frequency oscillation at a few Hertz in chamber pressure usually caused by pressure variations in feed lines due to variations in acceleration of the vehicle. This can cause cyclic variation in thrust, and the effects can vary from merely annoying to actually damaging the payload or vehicle. Chugging can be minimised by using gas filled damping tubes on feed lines of high density propellant's.Buzzing

This can be caused due to insufficient pressure drop across the injectors. It generally is mostly annoying, rather than being damaging. However, in extreme cases combustion can end up being forced backwards through the injectors- this can cause explosions with monopropellants.Screeching

This the most immediately damaging, and the hardest to control. It is due to acoustics within the combustion chamber that often couples to the chemical combustion processes that are the primary drivers of the energy release, and can lead to unstable resonant "screeching" that commonly leads to catastrophic failure due to thinning of the insulating thermal boundary layer. Such effects are very difficult to predict analytically during the design process, and have usually been addressed by expensive, time consuming and extensive testing, combined with trial and error remedial correction measures.

Screeching is often dealt with by detailed changes to injectors, or changes in the propellant chemistry, or vaporizing the propellant before injection, or use of Helmholtz dampers within the combustion chambers to change the resonant modes of the chamber.

Testing for the possibility of screeching is sometimes done by exploding small explosive charges in the combustion chamber to determine the engine's impulse response and then evaluating the time response of the chamber pressure- a fast recovery indicates a stable system.

### Exhaust noise

For all but the very smallest sizes, rocket exhaust compared to other engines is generally very noisy. As the hypersonic exhaust mixes with the ambient air, shock waves are formed. The sound intensity from these shock waves depends on the size of the rocket, and on large rockets could potentially kill at close range.

The Saturn V launch was detectable on seismometers a considerable distance from the launch site. The sound intensity from the shock waves generated depends on the size of the rocket and on the exhaust velocity. Such shock waves seem to account for the characteristic crackling and popping sounds produced by large rocket engines when heard live. These noise peaks typically overload microphones and audio electronics, and so are generally weakened or entirely absent in recorded or broadcast audio reproductions. For large rockets the acoustic effects could actually kill. The Space Shuttle generates over 200 dB(A) of noise around its base.

Generally speaking noise is most intense when a rocket is close to the ground, since the noise from the engines radiates up away from the plume, as well as reflecting off the ground. Also, when the vehicle is moving slowly, little of the chemical energy input to the engine can go into increasing the kinetic energy of the rocket (since useful power P transmitted to the vehicle is $P = F*V$ for thrust F and speed V). Then the largest portion of the energy is dissipated in the exhaust's interaction with the ambient air, producing noise. This noise can be reduced somewhat by flame trenches with roofs, by water injection around the plume and by deflecting the plume at an angle.

## Testing

Rocket engines are usually statically tested at a test facility before being put into production. For high altitude engines, either a shorter nozzle must be used, or the rocket must be tested in a large vacuum chamber.

## Safety

Rockets have a reputation for unreliability and danger; especially catastrophic failures. Contrary to this reputation, carefully designed rockets can be made arbitrarily reliable. In military use, rockets are not unreliable. However, one of the main non-military uses of rockets is for orbital launch. In this application, the premium is on minimum weight, and it is difficult to achieve high reliability and low weight simultaneously. In addition, if the number of flights launched is low, there is a very high chance of a design, operations or manufacturing error causing destruction of the vehicle. Essentially all launch vehicles are test vehicles by normal aerospace standards (as of 2006).

The X-15 rocket plane achieved a 0.5% failure rate, with a single catastrophic failure during ground test, and the SSME has managed to avoid catastrophic failures in over 350 engine-flights.

## Chemistry

Rocket propellants require a high specific energy (energy per unit mass), because ideally all the reaction energy appears as kinetic energy of the exhaust gases, and exhaust velocity is the single most important performance parameter of an engine, on which vehicle performance depends.

Aside from inevitable losses and imperfections in the engine, incomplete combustion, etc., after specific reaction energy, the main theoretical limit reducing the exhaust velocity obtained is that, according to the laws of thermodynamics, a fraction of the chemical energy may go into rotation of the exhaust molecules, where it is unavailable for producing thrust. Monatomic gases like helium have only three degrees of freedom, corresponding to the three dimensions of space, {x,y,z}, and only such spherically symmetric molecules escape this kind of loss. A diatomic molecule like H2 can rotate about either of the two axes perpendicular to the one joining the two atoms, and as the equipartition law of statistical mechanics demands that the available thermal energy be divided equally among the degrees of freedom, for such a gas in thermal equilibrium 3/5 of the energy can go into unidirectional motion, and 2/5 into rotation. A triatomic molecule like water has six degrees of freedom, so the energy is divided equally among rotational and translational degrees of freedom. For most chemical reactions the latter situation is the case. This issue is traditionally described in terms of the ratio, gamma, of the specific heat of the gas at constant volume to that at constant pressure. The rotational energy loss is largely recovered in practice if the expansion nozzle is large enough to allow the gases to expand and cool sufficiently, the function of the nozzle being to convert the random thermal motions of the molecules in the combustion chamber into the unidirectional translation that produces thrust. As long as the exhaust gas remains in equilibrium as it expands, the initial rotational energy will be largely returned to translation in the nozzle.

Although the specific reaction energy per unit mass of reactants is key, low mean molecular weight in the reaction products is also important in practice in determining exhaust velocity. This is because the high gas temperatures in rocket engines pose serious problems for the engineering of survivable motors. Because temperature is proportional to the mean energy per molecule, a given amount of energy distributed among more molecules of lower mass permits a higher exhaust velocity at a given temperature. This means low atomic mass elements are favored. Liquid hydrogen (LH2) and oxygen (LOX, or LO2), are the most effective propellants in terms of exhaust velocity that have been widely used to date, though a few exotic combinations involving boron or liquid ozone are potentially somewhat better in theory if various practical problems could be solved.

It is important to note in computing the specific reaction energy, that the entire mass of the propellants, including both fuel and oxidizer, must be included. The fact that air-breathing engines are typically able to obtain oxygen "for free" without having to carry it along, accounts for one factor of why air-breathing engines are very much more propellant-mass efficient, and one reason that rocket engines are far less suitable for most ordinary terrestrial applications. Fuels for automobile or turbojet engines, utilize atmospheric oxygen and so have a much better effective energy output per unit mass of fuel that must be carried.

Computer programs that predict the performance of propellants in rocket engines are available.

## Ignition

With liquid and hybrid rockets, immediate ignition of the propellant(s) as they first enter the combustion chamber is essential.

With liquid propellants (but not gaseous), failure to ignite within milliseconds usually causes too much liquid propellant to be within the chamber, and if/when ignition occurs the amount of hot gas created will often exceed the maximum design pressure of the chamber. The pressure vessel will often fail catastrophically. This is sometimes called a Hard start.

Ignition can be achieved by a number of different methods; a pyrotechnic charge can be used, a plasma torch can be used, or electric spark plugs may be employed. Some fuel/oxidizer combinations ignite on contact (hypergolic), and non-hypergolic fuels can be "chemically ignited" by priming the fuel lines with hypergolic propellants (popular in Russian engines).

Gaseous propellants generally will not cause hard starts, with rockets the total injector area is less than the throat thus the chamber pressure tends to ambient prior to ignition and high pressures cannot form even if the entire chamber is full of flammable gas at ignition.

Solid propellants are usually ignited with one-shot pyrotechnic devices.

Once ignited, rocket chambers are self sustaining and igniters are not needed. Indeed chambers often spontaneously reignite if they are restarted after being shut down for a few seconds. However, when cooled, many rockets cannot be restarted without at least minor maintenance, such as replacement of the pyrotechnic igniter.

## Plume physics

Rocket plume varies depending on the rocket engine, design altitude, altitude, thrust and other factors.

Carbon rich exhausts from kerosene fuels are often orangey colour due to the emission lines. Peroxide oxidiser based rockets and hydrogen rocket plumes contain largely steam are nearly invisible to the naked eye but shine brightly in the ultraviolet and infrared. Plumes from solid rockets often are highly visible as the propellant contains metals such as elemental aluminium which burns with a white flame.

Some exhausts, notably alcohol fuelled rockets can show visible Shock diamonds.

The shape of the plume varies from the design altitude, at high altitude all rockets are grossly underexpanded, and a quite small percentage of exhaust gases actually end up expanding forwards.

## Types of rocket engines

### Physically powered

water rocket Partially filled pressurised carbonated drinks container with tail and nose weighting Very simple to build Altitude typically limited to a few hundred feet or so (world record is 623 meters/2044 feet)
cold gas thruster A non combusting form, used for attitude jets Non contaminating exhaust Extremely low performance
hot water rocket Hot water is stored in a tank at high temperature/pressure and turns to steam in exhaust Simple, fairly safe Low performance due to heavy tank

### Chemically powered

Solid rocket Ignitable, self sustaining solid fuel/oxidiser mixture ("grain") with central hole and nozzle Simple, often no moving parts, reasonably good mass fraction, reasonable Isp. A thrust schedule can be designed into the grain. Once lit, extinguishing it is difficult although often possible, cannot be throttled in real time; handling issues from ignitable mixture, lower performance than liquid rockets, if grain cracks it can block nozzle with disastrous results, cracks burn and widen during burn. Refuelling grain harder than simply filling tanks, Lower specific Impulse than Liquid Rockets.
Hybrid rocket Separate oxidiser/fuel, typically oxidiser is liquid and kept in a tank, the other solid with central hole Quite simple, solid fuel is essentially inert without oxidiser, safer; cracks do not escalate, throttleable and easy to switch off. Some oxidisers are monopropellants, can explode in own right; mechanical failure of solid propellant can block nozzle, central hole widens over burn and negatively affects mixture ratio.
Monopropellant rocket Propellant such as Hydrazine, Hydrogen Peroxide or Nitrous Oxide, flows over catalyst and exothermically decomposes and hot gases are emitted through nozzle Simple in concept, throttleable, low temperatures in combustion chamber catalysts can be easily contaminated, monopropellants can detonate if contaminated or provoked, Isp is perhaps 1/3 of best liquids
Bipropellant rocket Two fluid (typically liquid) propellants are introduced through injectors into combustion chamber and burnt Up to ~99% efficient combustion with excellent mixture control, throttleable, can be used with turbopumps which permits incredibly lightweight tanks, can be safe with extreme care Pumps needed for high performance are expensive to design, huge thermal fluxes across combustion chamber wall can impact reuse, failure modes include major explosions, a lot of plumbing is needed.
Dual mode propulsion rocket Rocket takes off as a bipropellant rocket, then turns to using just one propellant as a monopropellant Simplicity and ease of control Lower performance than bipropellants
Tripropellant rocket Three different propellants (usually hydrogen, hydrocarbon and liquid oxygen) are introduced into a combustion chamber in variable mixture ratios, or multiple engines are used with fixed propellant mixture ratios and throttled or shut down Reduces take-off weight, since hydrogen is lighter; combines good thrust to weight with high average Isp, improves payload for launching from Earth by a sizeable percentage Similar issues to bipropellant, but with more plumbing, more R&D
Air-augmented rocket Essentially a ramjet where intake air is compressed and burnt with the exhaust from a rocket Mach 0 to Mach 4.5+ (can also run exoatmospheric), good efficiency at Mach 2 to 4 Similar efficiency to rockets at low speed or exoatmospheric, inlet difficulties, a relatively undeveloped and unexplored type, cooling difficulties, very noisy, thrust/weight ratio is similar to ramjets.
Turborocket A combined cycle turbojet/rocket where an additional oxidizer such as oxygen is added to the airstream to increase maximum altitude Very close to existing designs, operates in very high altitude, wide range of altitude and airspeed Atmospheric airspeed limited to same range as turbojet engine, carrying oxidizer like LOX can be dangerous. Much heavier than simple rockets.
Precooled jet engine / LACE (combined cycle with rocket) Intake air is chilled to very low temperatures at inlet before passing through a ramjet or turbojet engine. Can be combined with a rocket engine for orbital insertion. Easily tested on ground. High thrust/weight ratios are possible (~14) together with good fuel efficiency over a wide range of airspeeds, mach 0-5.5+; this combination of efficiencies may permit launching to orbit, single stage, or very rapid intercontinental travel. Exists only at the lab prototyping stage. Examples include RB545, SABRE, ATREX

### Electrically powered

Resistojet rocket (electric heating) A monopropellant is electrically heated by a filament for extra performance Higher Isp than monopropellant alone, about 40% higher. Uses a lot of power and hence gives typically low thrust
Arcjet rocket (chemical burning aided by electrical discharge) Similar to resistojet in concept but with inert propellant, except an arc is used which allows higher temperatures 1600 seconds Isp Very low thrust and high power, performance is similar to Ion drive.
Pulsed plasma thruster (electric arc heating; emits plasma) Plasma is used to erode a solid propellant High Isp , can be pulsed on and off for attitude control Low energetic efficiency
Variable specific impulse magnetoplasma rocket Microwave heated plasma with magnetic throat/nozzle Variable Isp from 1000 seconds to 10,000 seconds similar thrust/weight ratio with ion drives (worse), thermal issues, as with ion drives very high power requirements for significant thrust, really needs advanced nuclear reactors, never flown, requires low temperatures for superconductors to work

### Solar powered

The Solar thermal rocket would make use of solar power to directly heat reaction mass, and therefore does not require an electrical generator as most other forms of solar-powered propulsion do. A solar thermal rocket only has to carry the means of capturing solar energy, such as concentrators and mirrors. The heated propellant is fed through a conventional rocket nozzle to produce thrust. The engine thrust is directly related to the surface area of the solar collector and to the local intensity of the solar radiation.

Solar thermal rocket Propellant is heated by solar collector Simple design. Using hydrogen propellant, 900 seconds of Isp is comparable to Nuclear Thermal rocket, without the problems and complexity of controling a fission reaction. Using higher mollecular weight propellants, for example water water, lowers performance. Only useful once in space, as thrust is fairly low, but hydrogen is not easily stored in space, otherwise moderate/low Isp if higher molecular mass propellants are used

### Beam powered

light beam powered rocket Propellant is heated by light beam (often laser) aimed at vehicle from a distance, either directly or indirectly via heat exchanger simple in principle, in principle very high exhaust speeds can be achieved ~1 MW of power per kg of payload is needed to achieve orbit, relatively high accelerations, lasers are blocked by clouds, fog, reflected laser light may be dangerous, pretty much needs hydrogen monopropellant for good performance which needs heavy tankage, some designs are limited to ~600 seconds due to reemission of light since propellant/heat exchanger gets white hot
microwave beam powered rocket Propellant is heated by microwave beam aimed at vehicle from a distance microwaves avoid reemission of energy, so ~900 seconds exhaust speeds might be achieveable ~1 MW of power per kg of payload is needed to achieve orbit, relatively high accelerations, microwaves are absorbed to a degree by rain, reflected microwaves may be dangerous, pretty much needs hydrogen monopropellant for good performance which needs heavy tankage, transmitter diameter is measured in kilometres to achieve a fine enough beam to hit a vehicle at up to 100 km.

### Nuclear powered

Nuclear propulsion includes a wide variety of propulsion methods that use some form of nuclear reaction as their primary power source. Various types of nuclear propulsion have been proposed, and some of them tested, for spacecraft applications:

Radioisotope rocket/"Poodle thruster" (radioactive decay energy) Heat from radioactive decay is used to heat hydrogen about 700-800 seconds, almost no moving parts low thrust/weight ratio.
Nuclear thermal rocket (nuclear fission energy) propellant (typ. hydrogen) is passed through a nuclear reactor to heat to high temperature Isp can be high, perhaps 900 seconds or more, above unity thrust/weight ratio with some designs Maximum temperature is limited by materials technology, some radioactive particles can be present in exhaust in some designs, nuclear reactor shielding is heavy, unlikely to be permitted from surface of the Earth, thrust/weight ratio is not high.
Gas core reactor rocket (nuclear fission energy) Nuclear reaction using a gaseous state fission reactor in intimate contact with propellant Very hot propellant, not limited by keeping reactor solid, Isp between 1500 and 3000 seconds but with very high thrust difficulties in heating propellant without losing fissionables in exhaust, exhaust inherently highly radioactive, massive thermal issues particularly for nozzle/throat region.
Fission-fragment rocket (nuclear fission energy) Fission products are directly exhausted to give thrust Theoretical only at this point.
Fission sail (nuclear fission energy) A sail material is coated with fissionable material on one side No moving parts, works in deep space Theoretical only at this point.
Nuclear salt-water rocket (nuclear fission energy) Nuclear salts are held in solution, caused to react at nozzle Very high Isp, very high thrust Thermal issues in nozzle, propellant could be unstable, highly radioactive exhaust. Theoretical only at this point.
Nuclear pulse propulsion (exploding fission/fusion bombs) Shaped nuclear bombs are detonated behind vehicle and blast is caught by a 'pusher plate' Very high Isp, very high thrust/weight ratio, no show stoppers are known for this technology Never been tested, pusher plate may throw off fragments due to shock, minimum size for nuclear bombs is still pretty big, expensive at small scales, nuclear treaty issues, fallout when used below Earth's magnetosphere.
Antimatter catalyzed nuclear pulse propulsion (fission and/or fusion energy) Nuclear pulse propulsion with antimatter assist for smaller bombs Smaller sized vehicle might be possible Containment of antimatter, production of antimatter in macroscopic quantities isn't currently feasible. Theoretical only at this point.
Fusion rocket (nuclear fusion energy) Fusion is used to heat propellant Very high exhaust velocity Largely beyond current state of the art.
Antimatter rocket (annihilation energy) Antimatter annihilation heats propellant Extremely energetic, very high theoretical exhaust velocity Problems with antimatter production and handling; energy losses in neutrinos, gamma rays, muons; thermal issues. Theoretical only at this point

## History of rocket engines

According to the writings of the Roman Aulus Gellius, in c. 400 BC, a Greek Pythagorean named Archytas, propelled a wooden bird along wires using steam. However, it would not appear to have been powerful enough to take off under its own thrust.

The aeolipile invented in the first century (known as Hero's engine) essentially consists of a hot water rocket on a bearing. It was created almost two millennia before the industrial revolution. Apparently Hero's steam engine was taken to be little more than a toy, the principles behind it were not well understood, and its full potential not realized for a millennium.

The availability of black powder to propel projectiles was a precursor to the development of the first solid rocket. Ninth Century Chinese Taoist alchemists discovered black powder in a search for the Elixir of life; this accidental discovery led to fire arrows which were the first rocket engines to leave the ground.

Rocket engines were also brought in use by Tippu Sultan, The king of Mysore. These rockets could be of various sizes, but usually consisted of a tube of soft hammered iron about 8" long and 1½ - 3" diameter, closed at one end and strapped to a shaft of bamboo about 4ft. long. The iron tube acted as a combustion chamber and contained well packed black powder propellant. A rocket carrying about one pound of powder could travel almost 1,000 yards. These 'rockets', fitted with swords used to travel long distance, several meters above in air before coming down with swords edges facing the enemy. These rockets were used against British empire very effectively.

Slow development of this technology continued up to the later 20th Century, when the writings of Konstantin Tsiolkovsky first talked about liquid fuelled rocket engines.

These independently became a reality thanks to Robert Goddard. Goddard also used a De Laval nozzle for the first time on a rocket, doubling the thrust and multiplying up the efficiency by several times.

Liquid hydrogen engines were first successfully developed in America, the RL-10 engine first flew in 1962. Hydrogen engines were used as part of the Project Apollo; the liquid hydrogen fuel giving a rather lower stage mass and thus reducing the overall size and cost of the vehicle.

The Space Shuttle's SSME is the highest specific impulse rocket engine to fly.


text:
Video Tutorial – Rolling 2 Dice: An Intuitive Explanation of The Central Limit Theorem

According to the central limit theorem, if

• $n$ random variables, $X_1, ..., X_n$, are independent and identically distributed,
• $n$ is sufficiently large,

then the distribution of their sample mean, $\bar{X_n}$, is approximately normal, and this approximation is better as $n$ increases.

One of the most remarkable aspects of the central limit theorem (CLT) is its validity for any parent distribution of $X_1, ..., X_n$.  In my new Youtube channel, you will find a video tutorial that provides an intuitive explanation of why this is true by considering a thought experiment of rolling 2 dice.  This video focuses on the intuition rather than the mathematics of the CLT.  In a later video, I will discuss the technical details of the CLT and how it applies to this example.

You can also watch the video below the fold!

Side-by-Side Box Plots with Patterns From Data Sets Stacked by reshape2 and melt() in R

Introduction

A while ago, one of my co-workers asked me to group box plots by plotting them side-by-side within each group, and he wanted to use patterns rather than colours to distinguish between the box plots within a group; the publication that will display his plots prints in black-and-white only.  I gladly investigated how to do this in R, and I want to share my method and an example of what the final result looks like with you.

In generating a fictitious data set for this example, I will also demonstrate how to use the melt() function from the “reshape2” package in R to stack a data set while keeping categorical labels for the individual observations.  For now, here is a sneak peek at what we will produce at the end; the stripes are the harder pattern to produce.

Read the rest of this post to learn how to generate side-by-side box plots with patterns like the ones above!

Machine Learning Lesson of the Day – Linear Gaussian Basis Function Models

I recently introduced the use of linear basis function models for supervised learning problems that involve non-linear relationships between the predictors and the target.  A common type of basis function for such models is the Gaussian basis function.  This type of model uses the kernel of the normal (or Gaussian) probability density function (PDF) as the basis function.

$\phi_j(x) = exp[-(x - \mu_j)^2 \div 2\sigma^2]$

The $\sigma$ in this basis function determines the spacing between the different basis functions that combine to form the model.

Notice that this is just the normal PDF without the scaling factor of $1/\sqrt{2\pi \sigma^2}$; the scaling factor ensures that the normal PDF integrates to 1 over its support set.  In a linear basis function model, the regression coefficients are the weights for the basis functions, and these weights will scale the normal PDF to fit the data that are local to $\mu_j$.  Thus, there is no need to include that scaling factor of $1/\sqrt{2\pi \sigma^2}$, because the scaling is already being handled by the regression coefficients.

The Gaussian basis function model is useful because

• it can model many non-linear relationships between the predictor and the target surprisingly well,
• each basis function is non-zero over a very small interval and is zero everywhere else.  These local basis functions result in a very sparse design matrix (i.e. one with mostly zeros) that make the computation much faster.

Applied Statistics Lesson of the Day – Fractional Factorial Design and the Sparsity-of-Effects Principle

Consider again an experiment that seeks to determine the causal relationships between $G$ factors and the response, where $G > 1$.  Ideally, the sample size is large enough for a full factorial design to be used.  However, if the sample size is small and the number of possible treatments is large, then a fractional factorial design can be used instead.  Such a design assigns the experimental units to a select fraction of the treatments; these treatments are chosen carefully to investigate the most significant causal relationships, while leaving aside the insignificant ones.

When, then, are the significant causal relationships?  According to the sparsity-of-effects principle, it is unlikely that complex, higher-order effects exist, and that the most important effects are the lower-order effects.  Thus, assign the experimental units so that main (1st-order) effects and the 2nd-order interaction effects can be investigated.  This may neglect the discovery of a few significant higher-order effects, but that is the compromise that a fractional factorial design makes when the sample size available is low and the number of possible treatments is high.

Machine Learning Lesson of the Day – Overfitting and Underfitting

Overfitting occurs when a statistical model or machine learning algorithm captures the noise of the data.  Intuitively, overfitting occurs when the model or the algorithm fits the data too well.  Specifically, overfitting occurs if the model or algorithm shows low bias but high variance.  Overfitting is often a result of an excessively complicated model, and it can be prevented by fitting multiple models and using validation or cross-validation to compare their predictive accuracies on test data.

Underfitting occurs when a statistical model or machine learning algorithm cannot capture the underlying trend of the data.  Intuitively, underfitting occurs when the model or the algorithm does not fit the data well enough.  Specifically, underfitting occurs if the model or algorithm shows low variance but high bias.  Underfitting is often a result of an excessively simple model.

Both overfitting and underfitting lead to poor predictions on new data sets.

In my experience with statistics and machine learning, I don’t encounter underfitting very often.  Data sets that are used for predictive modelling nowadays often come with too many predictors, not too few.  Nonetheless, when building any model in machine learning for predictive modelling, use validation or cross-validation to assess predictive accuracy – whether you are trying to avoid overfitting or underfitting.

Mathematical and Applied Statistics Lesson of the Day – The Central Limit Theorem Applies to the Sample Mean

Having taught and tutored introductory statistics numerous times, I often hear students misinterpret the Central Limit Theorem by saying that, as the sample size gets bigger, the distribution of the data approaches a normal distribution.  This is not true.  If your data come from a non-normal distribution, their distribution stays the same regardless of the sample size.

Remember: The Central Limit Theorem says that, if $X_1, X_2, ..., X_n$ is an independent and identically distributed sample of random variables, then the distribution of their sample mean is approximately normal, and this approximation gets better as the sample size gets bigger.

Video Tutorial – The Hazard Function is the Probability Density Function Divided by the Survival Function

In an earlier video, I introduced the definition of the hazard function and broke it down into its mathematical components.  Recall that the definition of the hazard function for events defined on a continuous time scale is

$h(t) = \lim_{\Delta t \rightarrow 0} [P(t < X \leq t + \Delta t \ | \ X > t) \ \div \ \Delta t]$.

Did you know that the hazard function can be expressed as the probability density function (PDF) divided by the survival function?

$h(t) = f(t) \div S(t)$

In my new Youtube video, I prove how this relationship can be obtained from the definition of the hazard function!  I am very excited to post this second video in my new Youtube channel.  You can also view the video below the fold!

Applied Statistics Lesson of the Day – The Independent 2-Sample t-Test with Unequal Variances (Welch’s t-Test)

A common problem in statistics is determining whether or not the means of 2 populations are equal.  The independent 2-sample t-test is a popular parametric method to answer this question.  (In an earlier Statistics Lesson of the Day, I discussed how data collected from a completely randomized design with 1 binary factor can be analyzed by an independent 2-sample t-test.  I also discussed its possible use in the discovery of argon.)  I have learned 2 versions of the independent 2-sample t-test, and they differ on the variances of the 2 samples.  The 2 possibilities are

• equal variances
• unequal variances

Most statistics textbooks that I have read elaborate at length about the independent 2-sample t-test with equal variances (also called Student’s t-test).  However, the assumption of equal variances needs to be checked using the chi-squared test before proceeding with the Student’s t-test, yet this check does not seem to be universally done in practice.  Furthermore, conducting one test based on the results of another results possible inflation of Type 1 error (Ruxton, 2006).

Some books give due attention to the independent 2-sample t-test with unequal variances (also called Welch’s t-test), but some barely mention its value, and others do not even mention it at all.  I find this to be puzzling, because the assumption of equal variances is often violated in practice, and Welch’s t-test provides an easy solution to this problem.  There is a seemingly intimidating but straightforward calculation to approximate the number of degrees of freedom for Welch’s t-test, and this calculation is automatically incorporated in most software, including R and SAS.  Finally, Welch’s t-test removes the need to check for equal variances, and it is almost as powerful as Student’s t-test when the variances are equal (Ruxton, 2006).

For all of these reasons, I recommend Welch’s t-test when using the parametric approach to comparing the means of 2 populations.

Reference

Graeme D. Ruxton.  “The unequal variance t-test is an underused alternative to Student’s t-test and the Mann–Whitney U test“.  Behavioral Ecology (July/August 2006) 17 (4): 688-690 first published online May 17, 2006

Applied Statistics Lesson of the Day – Additive Models vs. Interaction Models in 2-Factor Experimental Designs

In a recent “Machine Learning Lesson of the Day“, I discussed the difference between a supervised learning model in machine learning and a regression model in statistics.  In that lesson, I mentioned that a statistical regression model usually consists of a systematic component and a random component.  Today’s lesson strictly concerns the systematic component.

An additive model is a statistical regression model in which the systematic component is the arithmetic sum of the individual effects of the predictors.  Consider the simple case of an experiment with 2 factors.  If $Y$ is the response and $X_1$ and $X_2$ are the 2 predictors, then an additive linear model for the relationship between the response and the predictors is

$Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \varepsilon$

In other words, the effect of $X_1$ on $Y$ does not depend on the value of $X_2$, and the effect of $X_2$ on $Y$ does not depend on the value of $X_1$.

In contrast, an interaction model is a statistical regression model in which the systematic component is not the arithmetic sum of the individual effects of the predictors.  In other words, the effect of $X_1$ on $Y$ depends on the value of $X_2$, or the effect of $X_2$ on $Y$ depends on the value of $X_1$.  Thus, such a regression model would have 3 effects on the response:

1. $X_1$
2. $X_2$
3. the interaction effect of $X_1$ and $X_2$

For example, a full factorial design with 2 factors uses the 2-factor ANOVA model and assumes a linear relationship between the response and the above 3 effects.

$Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_1 X_2 + \varepsilon$

Note that additive models and interaction models are not confined to experimental design; I have merely used experimental design to provide examples for these 2 types of models.

Applied Statistics Lesson of the Day – The Full Factorial Design

An experimenter may seek to determine the causal relationships between $G$ factors and the response, where $G > 1$.  On first instinct, you may be tempted to conduct $G$ separate experiments, each using the completely randomized design with 1 factor.  Often, however, it is possible to conduct 1 experiment with $G$ factors at the same time.  This is better than the first approach because

• it is faster
• it uses less resources to answer the same questions
• the interactions between the $G$ factors can be examined

Such an experiment requires the full factorial design.  After controlling for confounding variables and choosing the appropriate range and number of levels of the factor, the different treatments are applied to the different groups, and data on the resulting responses are collected.

The simplest full factorial experiment consists of 2 factors, each with 2 levels.  Such an experiment would result in $2 \times 2 = 4$ treatments, each being a combination of 1 level from the first factor and 1 level from the second factor.  Since this is a full factorial design, experimental units are independently assigned to all treatments.  The 2-factor ANOVA model is commonly used to analyze data from such designs.

In later lessons, I will discuss interactions and 2-factor ANOVA in more detail.

Machine Learning Lesson of the Day – Overfitting

Any model in statistics or machine learning aims to capture the underlying trend or systematic component in a data set.  That underlying trend cannot be precisely captured because of the random variation in the data around that trend.  A model must have enough complexity to capture that trend, but not too much complexity to capture the random variation.  An overly complex model will describe the noise in the data in addition to capturing the underlying trend, and this phenomenon is known as overfitting.

Let’s illustrate overfitting with linear regression as an example.

• A linear regression model with sufficient complexity has just the right number of predictors to capture the underlying trend in the target.  If some new but irrelevant predictors are added to the model, then they “have nothing to do” – all the variation underlying the trend in the target has been captured already.  Since they are now “stuck” in this model, they “start looking” for variation to capture or explain, but the only variation left over is the random noise.  Thus, the new model with these added irrelevant predictors describes the trend and the noise.  It predicts the targets in the training set extremely well, but very poorly for targets in any new, fresh data set – the model captures the noise that is unique to the training set.

(This above explanation used a parametric model for illustration, but overfitting can also occur for non-parametric models.)

To generalize, a model that overfits its training set has low bias but high variance – it predicts the targets in the training set very accurately, but any slight changes to the predictors would result in vastly different predictions for the targets.

Overfitting differs from multicollinearity, which I will explain in later post.  Overfitting has irrelevant predictors, whereas multicollinearity has redundant predictors.

Applied Statistics Lesson of the Day – Blocking and the Randomized Complete Blocked Design (RCBD)

A completely randomized design works well for a homogeneous population - one that does not have major differences between any sub-populations.  However, what if a population is heterogeneous?

Consider an example that commonly occurs in medical studies.  An experiment seeks to determine the effectiveness of a drug on curing a disease, and 100 patients are recruited for this double-blinded study – 50 are men, and 50 are women.  An abundance of biological knowledge tells us that men and women have significantly physiologies, and this is a heterogeneous population with respect to gender.  If a completely randomized design is used for this study, gender could be a confounding variable; this is especially true if the experimental group has a much higher proportion of one gender, and the control group has a much higher proportion of the other gender.  (For instance, purely due to the randomness, 45 males may be assigned to the experimental group, and 45 females may be assigned to the control group.)  If a statistically significant difference in the patients’ survival from the disease is observed between such a pair of experimental and control groups, this effect could be attributed to the drug or to gender, and that would ruin the goal of determining the cause-and-effect relationship between the drug and survival from the disease.

To overcome this heterogeneity and control for the effect of gender, a randomized blocked design could be used.  Blocking is the division of the experimental units into homogeneous sub-populations before assigning treatments to them.  A randomized blocked design for our above example would divide the males and females into 2 separate sub-populations, and then each of these 2 groups is split into the experimental and control group.  Thus, the experiment actually has 4 groups:

1. 25 men take the drug (experimental)
2. 25 men take a placebo (control)
3. 25 women take the drug (experimental)
4. 25 women take a placebo (control)

Essentially, the population is divided into blocks of homogeneous sub-populations, and a completely randomized design is applied to each block.  This minimizes the effect of gender on the response and increases the precision of the estimate of the effect of the drug.

Statistics and Chemistry Lesson of the Day – Illustrating Basic Concepts in Experimental Design with the Synthesis of Ammonia

To summarize what we have learned about experimental design in the past few Applied Statistics Lessons of the Day, let’s use an example from physical chemistry to illustrate these basic principles.

Ammonia (NH3) is widely used as a fertilizer in industry.  It is commonly synthesized by the Haber process, which involves a reaction between hydrogen gas and nitrogen gas.

N2 + 3 H2 → 2 NH3   (ΔH = −92.4 kJ·mol−1)

Recall that ΔH is the change in enthalpy.  Under constant pressure (which is the case for most chemical reactions), ΔH is the heat absorbed or released by the system.

Applied Statistics Lesson of the Day – The Completely Randomized Design with 1 Factor

The simplest experimental design is the completely randomized design with 1 factor.  In this design, each experimental unit is randomly assigned to each factor level.  This design is most useful for a homogeneous population (one that does not have major differences between any sub-populations).  It is appealing because of its simplicity and flexibility – it can be used for a factor with any number of levels, and different treatments can have different sample sizes.  After controlling for confounding variables and choosing the appropriate range and number of levels of the factor, the different treatments are applied to the different groups, and data on the resulting responses are collected.  The means of the response variable in the different groups are compared; if there are significant differences, then there is evidence to suggest that the factor and the response have a causal relationship.  The single-factor analysis of variance (ANOVA) model is most commonly used to analyze the data in such an experiment, but it does assume that the data in each group have a normal distribution, and that all groups have equal variance.  The Kruskal-Wallis test is a non-parametric alternative to ANOVA in analyzing data from single-factor completely randomized experiments.

If the factor has 2 levels, you may think that an independent 2-sample t-test with equal variance can also be used to analyze the data.  This is true, but the square of the t-test statistic in this case is just the F-test statistic in a single-factor ANOVA with 2 groups.  Thus, the results of these 2 tests are the same.  ANOVA generalizes the independent 2-sample t-test with equal variance to more than 2 groups.

Some textbooks state that “random assignment” means random assignment of experimental units to treatments, whereas other textbooks state that it means random assignment of treatments to experimental units.  I don’t think that there is any difference between these 2 definitions, but I welcome your thoughts in the comments.

Applied Statistics Lesson of the Day – Positive Control in Experimental Design

In my recent lesson on controlling for confounders in experimental design, the control group was described as one that received a neutral or standard treatment, and the standard treatment may simply be nothing.  This is a negative control group.  Not all experiments require a negative control group; some experiments instead have positive control group.

A positive control group is a group of experimental units that receive a treatment that is known to cause an effect on the response.  Such a causal relationship would have been previously established, and its inclusion in the experiment allows a new treatment to be compared to this existing treatment.  Again, both the positive control group and the experimental group experience the same experimental procedures and conditions except for the treatment.  The existing treatment with the known effect on the response is applied to the positive control group, and the new treatment with the unknown effect on the response is applied to the experimental group.  If the new treatment has a causal relationship with the response, both the positive control group and the experimental group should have the same responses.  (This assumes, of course, that the response can only be changed in 1 direction.  If the response can increase or decrease in value (or, more generally, change in more than 1 way), then it is possible for the positive control group and the experimental group to have the different responses.

In short, in an experiment with a positive control group, an existing treatment is known to “work”, and the new treatment is being tested to see if it can “work” just as well or even better.  Experiments to test for the effectiveness of a new medical therapies or a disease detector often have positive controls; there are existing therapies or detectors that work well, and the new therapy or detector is being evaluated for its effectiveness.

Experiments with positive controls are useful for ensuring that the experimental procedures and conditions proceed as planned.  If the positive control does not show the expected response, then something is wrong with the experimental procedures or conditions, and any “good” result from the new treatment should be considered with skepticism.

Machine Learning Lesson of the Day – Supervised Learning: Classification and Regression

Supervised learning has 2 categories:

• In classification, the target variable is categorical.
• In regression, the target variable is continuous.

Thus, regression in statistics is different from regression in supervised learning.

In statistics,

• regression is used to model relationships between predictors and targets, and the targets could be continuous or categorical.
• a regression model usually includes 2 components to describe such relationships:
• a systematic component
• a random component.  The random component of this relationship is mathematically described by some probability distribution.
• most regression models in statistics also have assumptions about the between the predictors and/or between the observations.
• many statistical models also aim to provide interpretable relationships between the predictors and targets.
• For example, in simple linear regression, the slope parameter, $\beta_1$, predicts the change in the target, $Y$, for every unit increase in the predictor, $X$.

In supervised learning,

• target variables in regression must be continuous
• categorical target variables are modelled in classification
• regression has less or even no emphasis on using probability to describe the random variation between the predictor and the target
• Random forests are powerful tools for both classification and regression, but they do not use probability to describe the relationship between the predictors and the target.
• regression has less or even no emphasis on providing interpretable relationships between the predictors and targets.
• Neural networks are powerful tools for both classification and regression, but they do not provide interpretable relationships between the predictors and the target.

***The last 2 points are applicable to classification, too.

In general, supervised learning puts much more emphasis on accurate prediction than statistics.

Since regression in supervised learning includes only continuous targets, this results in some confusing terminology between the 2 fields.  For example, logistic regression is a commonly used technique in both statistics and supervised learning.  However, despite its name, it is a classification technique in supervised learning, because the response variable in logistic regression is categorical.

Applied Statistics Lesson of the Day – Basic Terminology in Experimental Design #1

Experiment: A procedure to determine the causal relationship between 2 variables – an explanatory variable and a response variable.  The value of the explanatory variable is changed, and the value of the response variable is observed for each value of the explantory variable.

• An experiment can have 2 or more explanatory variables and 2 or more response variables.
• In my experience, I find that most experiments have 1 response variable, but many experiments have 2 or more explanatory variables.  The interactions between the multiple explanatory variables are often of interest.
• All other variables are held constant in this process to avoid confounding.

Explanatory Variable or Factor: The variable whose values are set by the experimenter.  This variable is the cause in the hypothesis.  (*Many people call this the independent variable.  I discourage this usage, because “independent” means something very different in statistics.)

Response Variable: The variable whose values are observed by the experimenter as the explanatory variable’s value is changed.  This variable is the effect in the hypothesis.  (*Many people call this the dependent variable.  Further to my previous point about “independent variables”, dependence means something very different in statistics, and I discourage using this usage.)

Factor Level: Each possible value of the factor (explanatory variable).  A factor must have at least 2 levels.

Treatment: Each possible combination of factor levels.

• If the experiment has only 1 explanatory variable, then each treatment is simply each factor level.
• If the experiment has 2 explanatory variables, X and Y, then each treatment is a combination of 1 factor level from X and 1 factor level from Y.  Such combining of factor levels generalizes to experiments with more than 2 explanatory variables.

Experimental Unit: The object on which a treatment is applied.  This can be anything – person, group of people, animal, plant, chemical, guitar, baseball, etc.

Trapezoidal Integration – Conceptual Foundations and a Statistical Application in R

Introduction

Today, I will begin a series of posts on numerical integration, which has a wide range of applications in many fields, including statistics.  I will introduce trapezoidal integration by discussing its conceptual foundations, write my own R function to implement trapezoidal integration, and use it to check that the Beta(2, 5) probability density function actually integrates to 1 over its support set.  Fully commented and readily usable R code will be provided at the end.

Given a probability density function (PDF) and its support set as vectors in an array programming language like R, how do you integrate the PDF over its support set to ensure that it equals to 1?  Read the rest of this post to view my own R function to implement trapezoidal integration and learn how to use it to numerically approximate integrals.

Detecting Unfair Dice in Casinos with Bayes’ Theorem

Introduction

I saw an interesting problem that requires Bayes’ Theorem and some simple R programming while reading a bioinformatics textbook.  I will discuss the math behind solving this problem in detail, and I will illustrate some very useful plotting functions to generate a plot from R that visualizes the solution effectively.

The Problem

The following question is a slightly modified version of Exercise #1.2 on Page 8 in “Biological Sequence Analysis” by Durbin, Eddy, Krogh and Mitchison.

An occasionally dishonest casino uses 2 types of dice.  Of its dice, 97% are fair but 3% are unfair, and a “five” comes up 35% of the time for these unfair dice.  If you pick a die randomly and roll it, how many “fives”  in a row would you need to see before it was most likely that you had picked an unfair die?”

Read more to learn how to create the following plot and how it invokes Bayes’ Theorem to solve the above problem!

Exploratory Data Analysis: Quantile-Quantile Plots for New York’s Ozone Pollution Data

Introduction

Continuing my recent series on exploratory data analysis, today’s post focuses on quantile-quantile (Q-Q) plots, which are very useful plots for assessing how closely a data set fits a particular distribution.  I will discuss how Q-Q plots are constructed and use Q-Q plots to assess the distribution of the “Ozone” data from the built-in “airquality” data set in R.

Previous posts in this series on EDA include

Learn how to create a quantile-quantile plot like this one with R code in the rest of this blog!