text:
# Lesson 6: Sample Size and Power - Part a

Lesson 6: Sample Size and Power - Part a

## Overview

The underlying theme of sample size calculation in all clinical trials is precision. ; Validity and unbiasedness do not necessarily relate to sample size.

Usually, sample size is calculated with respect to two circumstances. The first involves precision for an estimator, e.g., requiring a 95% confidence interval for the population mean to be within ± $$\delta$$ units. The second involves statistical power for hypothesis testing, e.g., requiring 0.80 or 0.90 statistical power $$\left(1- \beta\right)$$ for a hypothesis test when the significance level ($$\alpha$$) is 0.05 and the effect size (the clinically meaningful effect) is $$\Delta$$ units.

The formulae for many sample size calculations will involve percentiles from the standard normal distribution. The graph below illustrates the 2.5th percentile and the 97.5th percentile.

Fig. 1 Standard normal distribution centered on zero.

For a two-sided hypothesis test with significance level α and statistical power $$1 - \beta$$, the percentiles of interest are $$z_{(1-\alpha/2)}$$ and $$z_{(1 - \beta)}$$.

For a one-sided hypothesis test, $$z_{(1 - \alpha)}$$ is used instead. Usual choices of $$\alpha$$ are 0.05 and 0.01, and usual choices of $$\beta$$ are 0.20 and 0.10, so the percentiles of interest usually are:

$$z_{0.995} = 2.58, z_{0.99} = 2.33, z_{0.975} = 1.96, z_{0.95} = 1.65, z_{0.90} = 1.28, z_{0.80} = 0.84$$.

In SAS, the PROBIT function is available to generate percentiles from the standard normal distribution function, e.g., Z = PROBIT(0.99) yields a value of 2.33 for Z. So, if you ever need to generate z-values you can get SAS to do this for you.

It is important to realize that sample size calculations are approximations. The assumptions that are made for the sample size calculation, e.g., the standard deviation of an outcome variable or the proportion of patients who succeed with placebo, may not hold exactly.

Also, we may base the sample size calculation on a t statistic for a hypothesis test, which assumes an exact normal distribution of the outcome variable when it only may be approximately normal.

In addition, there will be loss-to-follow-up, so not all of the subjects who initiate the study will provide complete data. .Some will deviate from the protocol, including not taking the assigned treatment or adding on a treatment. Sample size calculations and recruitment of subjects should reflect these anticipated realities.

## Objectives

Upon completion of this lesson, you should be able to:

• Identify studies for which sample size is an important issue.
• Estimate the sample size required for a confidence interval for p for given $$\delta$$ and $$\alpha$$, using normal approximation and Fisher's exact methods.
• Estimate the sample size required for a confidence interval for $$\mu$$ for given $$\delta$$ and $$\alpha$$, using normal approximation when the sample size is relatively large.
• Estimate the sample size required for a test of $$H_0 \colon \mu_{1} = \mu_{2}$$ to have $$\left(1-\beta\right)\%$$ power for given $$\delta$$ and $$\alpha$$, using normal approximation, with equal or unequal allocation.
• Estimate the sample size required for a test of $$H_0 \colon p_{1} = p_{2}$$ for given $$\delta$$ and $$\alpha$$ and $$\beta$$, using normal approximation and Fisher's exact methods.
• Use a SAS program to estimate the number of events required for a logrank comparison of two hazard functions to have $$\left(1-\beta\right)\%$$ power with given $$\alpha$$
• Use Poisson probability methods to determine the cohort size required to have a certain probability of detecting a rare event that occurs at a rate = ξ.
• Adjust sample size requirements to account for multiple comparisons and the anticipated withdrawal rate.

## Reference:

Friedman, Furberg, DeMets, Reboussin and Granger. (2015) Sample size. In: FFDRG. Fundamentals of Clinical Trials. 5th ed. Switzerland: Springer.

Piantadosi Steven. (2005) Sample size and power. In: Piantadosi Steven. Clinical Trials: A Methodologic Perspective. 2nd ed. Hoboken, NJ: John Wiley and Sons, Inc.

Wittes, Janet. (2002) "Sample Size Calculations for Randomized Controlled Trials." Epidemiologic Reviews. Vol. 24. No 1. pp. 39-53.

# 6a.1 - Treatment Mechanism and Dose Finding Studies

6a.1 - Treatment Mechanism and Dose Finding Studies

For many treatment mechanism (TM) studies, sample size is not an important issue because usually only a few subjects are enrolled to investigate treatment mechanisms. Here you are taking a lot of measurements on a few subjects in order to find out what might be going on with your treatment.

As presented last week, dose-finding (DF) and dose-ranging studies typically involve a design scheme, such as a modified Fibonacci design or continual reassessment. An example of phase I cytotoxic drug trials is as follows. A set of doses is determined a priori, such as 100 mg, 200 mg, 300 mg, 500 mg, 800 mg, etc. Subjects are recruited into the DF study in groups of three. The first group receives the lowest dose of 100 mg. If none of the subjects experience the effect (toxicity, side effect, etc.), then the next group of three subjects is escalated to the next dose of 200 mg. If one of the three subjects at 100 mg experiences the effect, however, then the next group of three subjects will receive the same dose of 100 mg. Whenever six subjects at the same dose reveal at least two subjects that experience the effect, then the study is terminated and the chosen dose for a safety and efficacy study is the previous dose level.

With such mechanisms in place to determine initial dosage levels, selection of. the study sample size is not a major consideration. In fact, the final sample size is dependent on patient outcomes.

# 6a.2 - Safety and Efficacy Studies

6a.2 - Safety and Efficacy Studies

The U.S. FDA mandates that efficacy is proven prior to the approval of a drug. Efficacy means that the tested dose of the drug is effective at ameliorating the treated condition. Phase II trials evaluate the potential for efficacy; Phase III trials confirm efficacy. These trials can also be referred to as safety and activity studies.

A typical goal of a safety and efficacy (SE) study is to estimate certain clinical endpoints with a specified amount of precision. Confidence intervals are useful for reflecting the amount of precision, and the width of a confidence interval is a function of sample size.

The simplest example occurs when the outcome response is binary (success or failure). Let p denote the true (but unknown) proportion of successes in the population that will be estimated from a sample.

The sample size is denoted as n and the number of observed successes is r. Thus, the point estimate of p is:

$$\hat{p}=\frac{r}{n}$$

If the sample size is large enough, then the $$100(1 - \alpha)\%$$ confidence interval can be approximated as:

$$\hat{p}\pm z_{1-\alpha/2}\sqrt{\hat{p}(1-\hat{p})/n}$$

Prior to the conduct of the study, however, the point estimate is undetermined so that an educated guess is necessary for the purposes of a sample size calculation.

If it is desirable for the confidence interval to have limits of $$\hat{p} \pm \delta$$ .

for a $$100(1 - \alpha)\%$$ confidence interval, and the researcher has a reasonable guess as to the value of p, reworking through the sample size equation, the target sample size is:

$$n=z_{1-\alpha/2}^{2}p(1-p)/\delta^2$$

If a researcher guesses that $$p ≈ 0.4$$ and wants a 95% confidence interval to have limits of $$\delta = 0.10$$, then the required sample size is

$$n = \dfrac{(1.96)^{2}(0.4)(0.6)}{(0.10)^2} = 92$$

Notice that $$p(1 - p)$$ is maximized when $$p = 0.5$$. Therefore, because p has to be guessed, it is more conservative to use p = 0.5 in the sample size calculation. In the above example this yields $$n = \dfrac{(1.96)^2(0.5)(0.5)}{(0.10)^2} = 96$$, a slightly larger sample size.

Notice that the sample size is a quadratic function of precision. If $$\delta = 0.05$$ is desired instead of 0.10 in the above example, then

$$n = \dfrac{(1.96)^2(0.5)(0.5)}{(0.05)^2} = 384$$

If you want the confidence interval to be tighter remember that splitting the width of the confidence interval in half will involve quadrupling the number of subjects in the sample size!

The normal approximation for calculating the $$100(1 - \alpha)\%$$ confidence for p works well if

$$n \hat{p}\left(1-\hat{p}\right) \ge 5$$

Otherwise, exact binomial methods should be used.

In the exact binomial method, the lower $$100(\alpha/2)\%$$ confidence limit for $$p$$ is determined as the value $$p_L$$ that satisfies

$$\alpha/2=\sum_{k=r}^{n}C(n,k)\ast (p_L)^k(1-p_L)^{n-k}$$

The upper $$100(1 - \alpha/2)\%$$ confidence limit for $$p$$ is determined as the value $$p_U$$ that satisfies

$$\alpha/2=\sum_{k=0}^{r}C(n,k)\ast (p_U)^k(1-p_U)^{n-k}$$

SAS PROC FREQ provides the exact and asymptotic $$100(1 - \alpha)\%$$ confidence intervals for a binomial proportion, p.

## SAS® Example

### Using PROC FREQ in SAS for determining an exact confidence interval for a binomial proportion

6.1_binomial_proportion.sas (from Piantadosi, 2005) This is a program that illustrates the use of PROC FREQ in SAS for determining an exact confidence interval for a binomial proportion.

***********************************************************************
* This is a program that illustrates the use of PROC FREQ in SAS for  *
* determining an exact confidence interval for a binomial proportion. *
***********************************************************************;

proc format;
value succfmt 1='yes' 2='no';
run;

data Example_1;
input success count;
format success succfmt.;
cards;
1 03
2 16
;
run;

proc freq data=Example_1;
tables success/binomial alpha=0.05;
weight count/zeros;
title "Exact and Asymptotic 95% Confidence Intervals for a Binomial Proportion";
run;

In the example above, $$n = 19$$ is the sample size and $$r = 3$$ successes are observed in a binomial trial. The point estimate of p is

$$\hat{p} = 0.16$$

Note, however, that

$$n\hat{p}(1-\hat{p}) = 19(0.16)(0.84)=2.55 <5$$

The 95% confidence interval for p, based on the exact method, is [0.03, 0.40]. The 95% confidence interval for p, based on the normal approximation, is [-0.01, 0.32], which is modified to [0.00, 0.32] because p represents the probability of success that is supposed to be restricted to lie within the [0, 1] interval. Even with the correction to the lower endpoint, the confidence interval based on the normal approximation does not appear to be very accurate in this example.

Modify the SAS program above to reflect 11 successes out of 75 trials. Run the program. Do the results round to (0.08, 0.25) for the 95% exact confidence limits?

If an investigator estimates $$p = 0.15$$ and wants a 95% exact confidence interval with $$\delta = 0.1$$, what sample size is needed? One way to solve this is to use SAS PROC FREQ in a "guess and check" manner. In this case, $$n = 73$$ with 11 successes will result in a 95% exact confidence interval of (0.07, 0.25). It may impossible to exactly achieve the desired $$\delta$$, but an estimate of the required sample size can be provided.

Using the exact confidence interval for a binomial proportion is the better option if you are not sure you are working in a standard normally distributed population.

# 6a.3 - Example: Discarding Ineffective Treatment

6a.3 - Example: Discarding Ineffective Treatment

An approach for discarding an ineffective treatment in an SE study, based on the exact binomial method, is as follows. Suppose that the lowest success rate acceptable to an investigator for the treatment is 0.20. Suppose that the investigator decides to administer the treatment consecutively to a series of patients. When can the investigator terminate the SE trial if he continues to find no treatment successes?

## SAS® Example

### Determine when the exact confidence interval for p no longer contains a certain value

SAS Example: Modifications to the exact confidence interval program used earlier can be made to determine when the exact confidence interval for p no longer contains a certain value.

***********************************************************************
* This is a program that illustrates the use of PROC FREQ in SAS for  *
* determining an exact confidence interval for a binomial proportion. *
***********************************************************************;

proc format;
value succfmt 1='yes' 2='no';
run;

data Example_1;
input success count;
format success succfmt.;
cards;
1 03
2 16
;
run;

proc freq data=Example_1;
tables success/binomial alpha=0.05;
weight count/zeros;
title "Exact and Asymptotic 95% Confidence Intervals for a Binomial Proportion";
run;

SAS PROF FREQ (trial-and-error) indicates that the exact one-sided 95% upper confidence limit for p, when 0 out of 14 successes are observed, is 0.19. Thus, if the treatment fails in each of the first 14 patients, then the study is terminated.

Try it!
What is the upper 95% one-sided confidence limit for p when you have seen no successes in 5 trials?

Did you get 45% with the exact limits?

Notice also how clearly wrong the asymptotic limit is in this situation.

Exact and Asymptotic 95% Confidence Intervals for Sensitivity
The FREQ Procedure
Positive Frequency Percent Cumulative Frequency Cumulative Percent
yes 0 0.00 0 0.00
no 5 100.00 5 100.00

Binomial Proportion
for success = yes

Proportion 0.0000
ASE 0.0000
90% Lower Conf Limit 0.0000
90% Upper Conf Limit 0.0000
Exact Conf Limits
90% Lower Conf Limit 0.0000
90% Upper Conf Limit 0.4507

Test of H0: Proportion = 0.5

ASE Under H0 0.2236
Z -2.2361
One-Sided Pr < Z 0.0127
Two-Sided Pr > |Z| 0.0253

Sample Size = 5

Try it!
Here is another one to try... How many straight failures would it take to rule out a 30% success rate?

Exact and Asymptotic 95% Confidence Intervals for Sensitivity
The FREQ Procedure
Positive Frequency Percent Cumulative Frequency Cumulative Percent
yes 0 0.00 0 0.00
no 9 100.00 9 100.00

Binomial Proportion
for success = yes

Proportion 0.0000
ASE 0.0000
90% Lower Conf Limit 0.0000
90% Upper Conf Limit 0.0000
Exact Conf Limits
90% Lower Conf Limit 0.0000
90% Upper Conf Limit 0.2831

Test of H0: Proportion = 0.5

ASE Under H0 0.1667
Z -3.0000
One-Sided Pr < Z 0.0013
Two-Sided Pr > |Z| 0.0027

Sample Size = 9

# 6a.4 - Confidence Intervals for Means

6a.4 - Confidence Intervals for Means

For a clinical endpoint that can be approximated by a normal distribution in an SE study, the $$100(1 - \alpha)\%$$ confidence interval for the population mean, $$\mu$$, is

$$\bar{Y} \pm \left [ t_{n-1, 1-\alpha/2}s/\sqrt{n} \right ]$$

where

$$\bar{Y}=\sum_{i=1}^{n}Y_i/n$$ is the sample mean,

$$t_{n-1, 1-\alpha/2}$$ is the appropriate percentile from the $$t_{n-1}$$ distribution, and

$$s^2= \sum_{i=1}^{n}(Y_{i} - \bar{Y})^2 / (n-1)$$ is the sample variance and estimates $$σ^{2}$$.

If σ is known, then a z-percentile can replace the t-percentile in the $$100(1 - \alpha)\%$$ confidence interval for the population mean, $$\mu$$, that is,

$$\bar{Y} \pm \left( z_{1- \alpha /2}\sigma / \sqrt{n} \right)$$

If n is relatively large, say n ≥ 60, then $$z_{1 - \alpha/2} ≈ t_{n - 1,1 - \alpha/2}$$.

If it is desired for the $$100(1 - \alpha)\%$$ confidence interval to be

$$\bar{Y} \pm \delta$$

then

$$n= z_{1-\alpha/2}^{2}\sigma^2/\delta^2$$

For example, the necessary sample size for estimating the mean reduction in diastolic blood pressure, where $$σ = 5$$ mm Hg and $$δ = 1$$ mm Hg, is $$n = \dfrac{1.96^{2} \times 5^{2}}{1^2} = 96$$.

# 6a.5 - Comparative Treatment Efficacy Studies

6a.5 - Comparative Treatment Efficacy Studies

Suppose that a comparative treatment efficacy (CTE) trial consists of comparing two independent treatment groups with respect to the means of the primary clinical endpoint. Let $$\mu_1$$ and $$\mu_2$$ denote the unknown population means of the two groups, and let $$\sigma$$ denote the known standard deviation common to both groups. Also, let $$n_1$$ and $$n_2$$ denote the sample sizes of the two groups.

The treatment difference in means is $$\Delta = \mu_1 -\mu_2$$ and the null hypothesis is $$H_0\colon \Delta = 0$$. The test statistic is

$$Z = \left( \bar{Y}_1 - \bar{Y}_2 \right) / \sigma \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}$$

which follows a standard normal distribution when the null hypothesis is true. If the alternative hypothesis is two-sided, i.e., $$H_1 \colon \Delta \ne 0$$, then the null hypothesis is rejected for large values of |Z|.

Under a particular alternative where there might be some difference $$\Delta, \Delta = \mu_1 - \mu_2$$,

$$Z = \left( \bar{Y}_1 - \bar{Y}_2 - \Delta \right)/ \sigma \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}$$

Suppose we let $$AR = \dfrac{n_1}{n_2}$$ denote the allocation ratio $$\left(AR\right)$$, (in most cases we will assign $$AR = 1$$ to get equal sample sizes). If we wish to a have large enough sample size to detect an effect size Δ with a two-sided, α-significance level test with $$100 \left(1 - \beta \right)\%$$ statistical power, then

$$n_2 = \left( \frac{AR+1}{AR}\right) \left( z_{1-\alpha/2}+z_{1-\beta} \right)^2\sigma^2/\Delta^2$$

and $$n_1 = AR \times n_2$$.

Note this formula matches the sample size formula in our FFDRG text on p. 180, assuming equal allocation to the two treatment groups and multiplying the result here by 2 to get 2N, which FFDRG uses to denote the total sample size.

If the alternative hypothesis is one-sided, then $$Z_{1 - α}$$ replaces $$Z_{1 - \frac{\alpha}{2}}$$ in either formula.

Notice that the sample size expression contains $$\left(\dfrac{\sigma}{\Delta}\right)^2$$, the square of the effect size expressed in standard deviation units. Thus, sample size is a quadratic function of the effect size and precision. As the variance gets larger, it has a quadratic effect on the sample size. For example, reducing the effect size by one-half quadruples the required sample size.

Although this sample size formula assumes that the standard deviation is known so that a z test can be applied, it works relatively well when the standard deviation must be estimated and a t-test applied. A preliminary guess of σ must be available, however, either from a small pilot study or a report in the literature. For smaller sample sizes $$\left(n_1 ≤ 30, n_2 ≤ 30 \right)$$ percentiles from a t distribution can be substituted, although this results in both sides of the formula involving $$n_2$$ so that it must be solved iteratively:

$$n_2 = \left( \dfrac{AR+1}{AR}\right) \left( t_{n_1+n_2-2,1-\alpha/2}+t_{n_1+n_2-2,1-\beta} \right)^2\sigma^2/\Delta^2$$

# 6a.6 - Example: Comparative Treatment Efficacy Studies

6a.6 - Example: Comparative Treatment Efficacy Studies

## SAS® Example

### Using PROC POWER to calculate sample size when comparing two normal means

An investigator wants to determine the sample size for comparing two asthma therapies with respect to the forced expiratory volume in one second $$\left(FEV_1\right)$$. A two-sided, 0.05-significance level test with 90% statistical power is desired. The effect size is $$\Delta = 0.25 \text{ L}$$ and the standard deviation reported in the literature for a similar population is $$\sigma = 0.75 \text{ L}$$. The investigator plans to have equal allocation to the two treatment groups (AR = 1).

The first step is to identify the primary response variable. In this example, $$FEV_1$$ is a continuous response variable. Assuming that $$FEV_1$$ has an approximate normal distribution, the number of patients required for the second treatment group based on the z formula is

$$n_2 = \dfrac{(2)(1.96 + 1.28)^2(0.75)^2}{(0.25)^2 } = 189$$

Thus, the total sample size required is $$n_1 + n_2 = 189 + 189 = 378$$. SAS Example: This is a program that illustrates the use of PROC POWER to calculate sample size when comparing two normal means.

***********************************************************************
* This is a program that illustrates the use of PROC POWER to         *
* calculate sample size when comparing two normal means.              *
***********************************************************************;

proc power;
twosamplemeans dist=normal groupweights=(1 1) alpha=0.05 power=0.9 stddev=0.75
meandiff=0.25 test=diff sides=2 ntotal=.;
plot min=0.1 max=0.9;
title "Sample Size Calculation for Comparing Two Normal Means (1:1 Allocation)";
run;

proc power;
twosamplemeans dist=normal groupweights=(2 1) alpha=0.05 power=0.9 stddev=0.75
meandiff=0.25 test=diff sides=2 ntotal=.;
plot min=0.1 max=0.9;
title "Sample Size Calculation for Comparing Two Normal Means (2:1 Allocation)";
run;

SAS PROC POWER, based on the t formula, yields $$n_1 + n_2 = 191 + 191 = 382$$.

If the investigator had wanted an allocation ratio of $$AR = 2$$ (twice as many subjects in the first group), then $$n_2 = \dfrac{(1.5)(1.96 + 1.28)^2(0.75)^2}{(0.25)^2} = 142$$ and $$n_1 = 2 \times 142 = 284$$.

The total sample size required is $$n_1 + n_2 = 142 + 284 = 426$$.

SAS PROC POWER, based on the t formula, yields $$n_1 + n_2 = 143 + 286 = 429$$.

Notice that the 2:1 allocation, when compared to the 1:1 allocation, requires an overall larger sample size (429 versus 382).

Now it is your turn to give it a try!

## Try it!

Here is another one to try... How many subjects are needed to have 80% power in testing equivalence of two means when subjects were allocated 2:1, using a $$\alpha = 0.05$$ two sided test? The standard deviation is 10 and the hypothesized difference in means is 5.

144 subjects

Here is the output that you should have obtained ...

Fixed Scenario Elements

The POWER Procedure
Two-sample t Test for Mean Difference

Fixed Scenario Elements
Distribution Normal
Method Exact
Number of Sides 2
Alpha 0.05
Mean Difference 5
Standard Deviation 10
Group 1 Weight 2
Group 2 Weight 1
Nominal Power 0.8
Null Difference 0
Computed N Total
Computed N Total
Actual Power N Total
0.802 144

# 6a.7 - Example: Comparative Treatment Efficacy Studies

6a.7 - Example: Comparative Treatment Efficacy Studies

What if the primary response variable is binary?

When the outcome in a CTE trial is a binary response and the objective is to compare the two groups with respect to the proportion of success, the results can be expressed in a 2 × 2 table as

 Group # 1 Group # 2 Success $$r_1$$ $$r_2$$ Failure $$n_1 - r_1$$ $$n_2 - r_2$$

There are a variety of methods for performing the statistical test of the null hypothesis $$H_0\colon p_1 = p_2$$, such as a z-test using a normal approximation, a $$χ^2$$ test (basically, a square of the z-test), a $$χ^2$$ test with continuity correction, and Fisher's exact test.

The normal and $$χ^2$$ approximations for comparing two proportions are relatively accurate when these conditions are met:

$$\dfrac{n_1(r_1+r_2)}{(n_1+n_2}\ge 5, \dfrac{n_2(r_1+r_2)}{(n_1+n_2}\ge 5, \dfrac{n_1(n_1+n_2-r_1-r_2)}{(n_1+n_2}\ge 5, \dfrac{n_2(n_1+n_2-r_1-r_2)}{(n_1+n_2}\ge 5$$

Basically when the expected number in each cell is greater than 5, the normal or Chi Square approximation is useful.

Otherwise, Fisher's exact test is recommended. All of these tests are available in SAS PROC FREQ of SAS and will be discussed later in the course.

A sample size formula for comparing the proportions $$p_1$$ and $$p_2$$ using the normal approximation is given below:

$$n_2=\left( \dfrac{AR+1}{AR}\right)\left( z_{1-\alpha/2}+z_{1-\beta}\right)^2\bar{p}(1-\bar{p})/(p_1-p_2)^2$$

where $$p_1 - p_2$$ represents the effect size and

$$\bar{p}= (AR \cdot p_1+p_2) / (AR+1)$$

is the weighted average of the proportions.

NOTE! this formula is the same as p. 173 in our text FFDRG if you assume the allocation ratio is 1:1 and double the sample size here to get total sample size 2N as calculated in FFDRG

## SAS® Example

### Using PROC POWER to calculate sample size when comparing two binomial proportions

An investigator wants to compare an experimental therapy to placebo when the response is success/failure via a two-sided, 0.05 significance level test and 90% statistical power. She knows from the medical literature that 25% of the untreated patients will experience success, so she decides that the experimental therapy is worthwhile if it can yield a 50% success rate. With equal allocation, $$n_2 = \dfrac{(2)(1.96 + 1.28)^2{0.375(1-0.375)}}{(0.25)^2} = 79$$. Thus, the investigator should enroll $$n_1 = 79$$ patients into treatment and $$n_2 = 79$$ into placebo for a total of 158 patients.

With an unequal allocation ratio of $$AR = 3, n_1 = 168$$ and $$n_2 = 56$$. Again, notice that the allocation ratio of AR = 3 yields a total sample size larger than that for the allocation ratio of AR = 1 (224 vs. 158).

This is a program that illustrates the use of PROC POWER to calculate sample size when comparing two binomial proportions.

***********************************************************************
* This is a program that illustrates the use of PROC POWER to         *
* calculate sample size when comparing two binomial proportions.      *
***********************************************************************;

proc power;
twosamplefreq groupweights=(1 1) groupps=(0.25 0.50) alpha=0.05 power=0.9
test=Fisher sides=2 ntotal=.;
plot min=0.1 max=0.9;
title "Sample Size Calculation for Comparing Two Binomial Proportions (1:1 Allocation)";
run;

proc power;
twosamplefreq groupweights=(1 3) groupps=(0.25 0.50) alpha=0.05 power=0.9
test=Fisher sides=2 ntotal=.;
plot min=0.1 max=0.9;
title "Sample Size Calculation for Comparing Two Binomial Proportions (3:1 Allocation)";
run;

SAS PROC POWER for Fisher’s exact test yields $$n_1 = 85$$ and $$n_2 = 85$$ for $$AR = 1$$, and $$n_1 = 171$$ and $$n_2 = 57$$ for $$AR = 3$$.

## Try it!

What would be the sample size required to have 80% power to detect that a new therapy has a significantly different success rate than the standard therapy success rate of 30%, if it was expected that the new therapy would result in at least 40% successes? Use a two-sided test with 0.05 significance level.

The answer is a total of 752 subjects.

Here is the output you should have obtained from SAS ...

Fixed Scenario Elements

The POWER Procedure
Fisher's Exact Conditional Test for Two Proportions

Fixed Scenario Elements
Distribution Exact conditional
Method Walter's normal approximation
Number of Sides 2
Alpha 0.05
Group 1 Proportion 0.3
Group 2 Proportion 0.4
Group 1 Weight 1
Group 2 Weight 1
Nominal Power 0.8
Computed N Total
Computed N Total
Actual Power N Total
0.801 752

# 6a.8 - Comparing Treatment Groups Using Hazard Ratios

6a.8 - Comparing Treatment Groups Using Hazard Ratios

For many clinical trials, the response is time to an event. The methods of analysis for this type of variable are generally referred to as survival analysis methods. The basic approach is to compare survival curves.

With an event time endpoint, it is mathematically convenient to compare treatment groups (and curves) with respect to the hazard ratio. The survival function for a treatment group is characterized by $$\lambda$$, the hazard rate. At time t, $$\lambda(t)$$ for a treatment group, is defined as the instantaneous risk of the event (or failure) occurring at time t. In other words, given that a subject has survived the event up to time t, the hazard at time t is the probability of the event occurring within the next instant. You can think of the hazard as the slope of the survival curve.

The hazard ratio is defined as the ratio of two hazard functions, $$\lambda_1(t)$$ and $$\lambda_2(t)$$, corresponding to two treatment groups. Typically, we assume proportional hazards, i.e., $$\Lambda= \dfrac{\lambda_1(t)}{\lambda_2(t)}$$ is a constant function independent of time. The graphs on the next two slides illustrate the concept of proportional hazards.

A hazard function may be constant, increasing, or decreasing over time, or even be a more complex function of time. In trials in which survival time is the outcome, an increasing hazard function indicates that the instantaneous risk of death increases throughout the trial.

An example where the hazard function might be decreasing involves the disease ARDS (adult respiratory distress syndrome), whereby the risk of death is highest during the early stage of the disease.

A sample size formula for comparing the hazards of two groups via the logrank test (discussed later in the course) is expressed in terms of the total number of events, E, that need to occur. For a two-sided, α-level significance test with $$100 \left(1 - \beta \right)\%$$ statistical power, hazard ratio Λ, and allocation ratio AR,

$$E=\left( \dfrac{(AR+1)^2}{AR}\right) \dfrac{\left( z_{1-\alpha/2} + z_{1-\beta} \right)^2}{ \left(log_e(\Lambda) \right)^2}$$

NOTE! this formula above matches FFDRG text p. 185 simple formula, if it is assumed that all participants will have an event. However, we most often have censored data, which is a number of participants who do not experience the event before the trial ends.

Since we do not expect all persons in the trial to experience an event, the sample size must be larger than the required number of events.

Suppose that $$p_1$$ and $$p_2$$ represent the anticipated event rates in the two treatment groups. Then the sample sizes can be determined from $$n_2 = \dfrac{E}{(AR \times p_1 + p_2)}$$ and $$n_1 = AR \times n_2$$

If a hazard function is assumed to be constant during the follow-up period [0, T], then it can be expressed as $$\lambda(t) = \lambda = \dfrac{-log_e(1 - p)}{T}$$. In such a situation, the hazard ratio for comparing two groups is $$\Lambda = \dfrac{log_e(1 - p_1)}{log_e(1 - p_2)}$$.

A constant hazard rate, $$\lambda(t) = \lambda$$ for all time points t, corresponds to an exponential survival curve, i.e., survival at time $$t = exp(-\lambda t)$$.

Survival curves plot the probability of the event occurring to a subject over time.

## SAS® Example

### Using PROC POWER to calculate sample size when comparing two hazard functions

An investigator wants to compare an experimental therapy to placebo when the response is time to infection via a two-sided, 0.05-significance level test with 90% statistical power and equal allocation. He plans to follow each patient for one year and he expects that 40% of the placebo group will experience infection and he considers a 20% rate in the therapy group as clinically relevant.

If he assumes constant hazard functions, then

$$\Lambda= \dfrac{log_e(0.6)}{log_e(0.8)} = 2.29$$

Then the number of required events is

$$E = \dfrac{(4)(1.96 + 1.28)^2}{{log_e(2.29)}^2} = 62$$

and the sample sizes are

$$n_2 = \dfrac{E}{(AR \times p_1 + p_2)} = \dfrac{62}{(0.4 + 0.2)} = 104$$ and $$n_1 = 104$$

SAS Example - This is a program that illustrates the use of PROC POWER to calculate sample size when comparing two hazard functions.

***********************************************************************
* This is a program that illustrates the use of PROC POWER to         *
* calculate sample size when comparing two hazard functions.          *
***********************************************************************;

proc power;
twosamplesurvival groupweights=(1 1) alpha=0.05 power=0.9 sides=2
test=logrank curve("Placebo")=(1.01):(0.6) curve("Therapy")=(1.01):(0.8)
groupsurvival="Placebo"|"Therapy" accrualtime=0.01 followuptime=1 ntotal=.;
plot min=0.1 max=0.9;
title "Sample Size Calculation for Comparing Two Hazard Functions (1:1 Allocation)";
run;

proc power;
twosamplesurvival groupweights=(1 3) alpha=0.05 power=0.9 sides=2
test=logrank curve("Placebo")=(1.01):(0.6) curve("Therapy")=(1.01):(0.8)
groupsurvival="Placebo"|"Therapy" accrualtime=0.01 followuptime=1 ntotal=.;
plot min=0.1 max=0.9;
title "Sample Size Calculation for Comparing Two Hazard Functions (3:1 Allocation)";
run;

Additional comments on this program: Note the curve statements indicate points on the survival curves. In this example, at the end of study, at time 1.01 (followup plus accrual in SAS), the proportion in the placebo group without an event is 0.6 and the proportion remaining the therapy group is 0.8.

SAS PROC POWER for the logrank test requires information on the accrual time and the follow-up time. It assumes that if the accrual (recruitment) period is of duration $$T_1$$ and the follow-up time is of duration $$T_2$$, then the total study time is of duration $$T_1 + T_2$$. It assumes, however, if a patient is recruited at time $$\dfrac{T_1}{2}$$, then the follow-up period for that patient is $$\dfrac{T_1}{2 + T_2}$$ instead of $$T_2$$. This assumption may be reasonable for observational studies, but not for clinical trials in which follow-up on each patient is terminated when the patient reaches time $$T_2$$. Therefore, for a clinical trial situation, set accrual time in SAS PROC POWER equal to a very small positive number. For the given example, SAS PROC POWER yields $$n_1 = 109$$ and $$n_2 = 109$$.

SAS notes for PROC POWER for survival

# 6a.9 - Expanded Safety Studies

6a.9 - Expanded Safety Studies

Expanded Safety (ES) trials are phase IV trials designed to estimate the frequency of uncommon adverse events that may have been undetected in earlier studies. These studies may be nonrandomized.

Typically, we assume that the study population is large, the probability of an adverse event is small (because it did not crop up in prior trials), and all participants in the cohort of size m are followed for approximately the same length of time. Under these assumptions, we can model the probability of exactly d events occurring based on a Poisson probability function, i.e.,

$$Pr \left[ D = d \right] = ( \xi m)^d exp(- \xi m)/d!$$

where $$\xi$$ is the adverse event rate.

The cohort should be large enough to have a high probability of observing at least one event when the event rate is $$\xi$$. Thus, we want

$$\beta = Pr \left[D \ge 1 \right] = 1 - Pr \left[ D = 0 \right] = 1 - exp(-\xi m)$$

to be relatively large. With respect to the cohort size, this means that m should be selected such that

$$m = -log_e(1 - \beta)/ \xi$$

## Example

Suppose a pharmaceutical company is planning an ES trial for a new anti-arrhythmia drug. The company wants to determine the cohort size for following patients on the drug for a period of two years in terms of myocardial infarction. They want to have a 0.99 probability $$\left(\beta = 0.99 \right)$$ for detecting a myocardial infarction rate of one per thousand ($$\xi$$ = 0.001).

m= -ln(0.01)/0.001

This yields a cohort size of m = 4,605.1. Rounded up, 4606.

(note the $$\beta$$ value in this problem is a probability, not quite the same as $$\beta$$ that we use in calculating power)

# 6a.10 - Adjustment Factors for Sample Size Calculations

6a.10 - Adjustment Factors for Sample Size Calculations

When calculating a sample size, we may need to adjust our calculations due to multiple primary comparisons or for nonadherence to therapy or to consider the anticipated dropout rate.

• If there is more than one primary outcome variable (for example, co-primary outcomes) or more than one primary comparison (for example, 3 treatment groups), then the significance level should be adjusted to account for the multiple comparisons in order not to inflate the overall false-positive rate.

For example, suppose a clinical trial will involve two treatment groups and a placebo group. The investigator may decide that there are two primary comparisons of interest, namely, each treatment group compared to placebo. The simplest adjustment to the significance level for each test is the Bonferroni correction, which uses $$\dfrac{\alpha}{2}$$ instead of $$\alpha$$.

In general, if there are K comparisons of primary interest, then the Bonferroni correction is to use a significance level of $$\dfrac{\alpha}{K}$$ for each of the K comparisons. The Bonferroni correction is not the most powerful or most sophisticated multiple comparison adjustment, but it is a conservative approach and easy to apply.

In the case of multiple primary endpoints, an adjustment to the significance level may not be necessary, depending on how the investigator plans to interpret the results. For example, suppose there are two primary outcome variables. If the investigator plans to claim “success of the trial” if either endpoint yields a statistically significant treatment effect, then an adjustment to the significance level is warranted. If the investigator plans to claim “success of the trial” only if both endpoints yield statistically significant treatment effects, then an adjustment to the significance level is not necessary. Thus, an adjustment to the significance level in the presence of multiple primary endpoints depends on whether it is an “or” or an “and” situation.

(A composite outcome, such as "time to stroke, MI or major cardiovascular event", is different from a co-primary outcome. In this situation, when the composite results in one statistical analysis, there is no need for adjustment.)

• Another consideration is nonadhereance to the protocol (noncompliance). All participants randomized to therapy are expected to be included in the primary statistical analysis, an intention-to-treat analysis. Intention-to-treat analysis will compare the treatments using all data from subjects in the group to which they were originally assigned, regardless of whether or not they followed the protocol, stayed on therapy,etc. Some participants will choose to withdraw from a trial before it is complete. Every effort will be made to continue obtaining data from all randomized subjects; for those who withdraw from the study completely and do not provide data, an imputation procedure may be required to represent their missing data in subsequent data analyses. Some participants assigned to active therapy discontinue therapy but continue to provide data (therapeutic drop-outs). Some on a placebo or control add an active therapy (drop-ins) and continue to be observed. The nonadherence (noncompliance) can lead to a dilution of the treatment effect and lead to lower power for the study as well as biased estimates of treatment effects.

Thus, a further adjustment to the sample size estimate may be made based on the anticipated drop-out and drop-in rates in each arm (See Wittes (2002). A similar formula is on p. 179 FFDRG.

$$N^∗= \dfrac{N}{((1- R_O - R_I))^2}$$ where N is the sample size without regard to nonadherence and N* is the adjusted number for that treatment arm.

$$R_O$$ and $$R_I$$ represent the proportion of participants anticipated to discontinue test therapy and the proportion in the control who will add or change to a more effective therapy, respectively.

• In other situations, an adjustment may be made to increase the sample size to account for the anticipated number of subjects who will drop-out of the study altogether so that there is sufficient power with the remaining observations to detect a certain difference. This adjustment is made by dividing the calculated sample size N by (1-W) where W is the proportion expected to withdraw.

$$N^{**}=\dfrac{N}{(1-W)}$$

Let's work an example.

#### Example

Suppose a study has two treatment groups and will compare test therapy to placebo. With only one primary comparison, we do not need to adjust the significance level for multiple comparisons. Suppose that the sample size for a certain power, significance level and clinically important difference works to be 200 participants/group or 400 total.

1. We plan an intention-to-treat analysis as our primary analysis and our concern is dilution of the true treatment effect due to these deviations from the assigned therapy. To adjust for noncompliance/nonadherence, we must estimate the proportion from the placebo group who will begin an active therapy before the study is complete. Let's estimate these 'drop-ins' to be 0.20. In the test therapy group, we estimate 0.10 will discontinue active therapy.

To adjust for noncompliance, we calculate $$N^{*} = 200/((1-0.2-0.1)^{2})$$ . N^{*} = 409/\text{ group}\) or 818 total. What an increase in sample size to maintain the power! (note whether I use n/group, 200/(0.49) or total n, 400/(0.49) I will get the same sample sizes. Just remember what your N represents. If there is any fraction at the end of sample size calculations, round UP to the next number divisible by the number of treatment groups.)

2. Suppose instead of dilution of treatment effect in an ITT analysis, our concern is the proportion of subjects who will leave the study without providing a key observation on the primary outcome. If we anticipate a 15% rate of discontinuing before this measurement, we may want to increase the sample size accordingly. 200/0.85= 236/group (rounded up).

These are relatively simple calculations to introduce the idea of adjusting for noncompliance, multiple comparisons or the withdrawal rate. More complicated processes can be modeled.

Finally, when estimating a sample size for a study, an iterative process may be followed (adapted from Wittes, 2002)

1. Determine the null and alternative hypotheses as related to the primary outcome.
2. What is the desired type I error rate and power? If more than one primary outcome or comparison, make required adjustments to Type 1 error.
3. Determine the population that will be studied. What information is there about the variability of the primary outcome in this population? Would would constitute a clinically important difference?
4. If the study is measuring time to failure, how long is the followup period? What assumptions should be made about recruitment?
5. Consider ranges of rates or events, loss to follow-up, competing risks, and noncompliance.
6. Calculate sample size over a range of reasonable assumptions.
7. Select a sample size. Plot power curves as the parameters range over reasonable values.
8. Iterate as needed.

Which of these adjustments (or others, such as modeling dropout rates that are not independent of outcome) is important for a particular study depends on the study objectives. Not only must we consider whether there is more than one primary outcome or multiple primary comparisions, we must also consider the nature of the trial. For example, if the study results are headed to a regulatory agency, using a primary intention-to-treat analysis, it is important to demonstrate an effect of a certain magnitude. Adjusting the sample size to account for non-adherence is sensible. On the other hand, in a comparative effectiveness study, the objective may be to estimate the difference in effect when the intervention is prescribed vs the control, regardless of adherence. In this situation, the dilution of effect due to nonadherence may be of little concern.

As we noted beginning this lesson, sample size calculations are estimates! When stating a required sample size, always state any assumptions that have been made in the calculations.

# 6a.11 - Summary

6a.11 - Summary

In this lesson, among other things, we learned to:

• Identify studies for which sample size is an important issue
• Estimate the sample size required for a confidence interval for p for given $$\delta$$ and $$\alpha$$, using normal approximation and Fisher's exact methods
• Estimate the sample size required for a confidence interval for μ for given $$\delta$$ and $$\alpha$$, using normal approximation when the sample size is relatively large
• Estimate the sample size required for a test of $$H_0 \colon \mu_1 = \mu_2$$ to have $$(1 - \beta)\%$$ power for given $$\delta$$ and $$\alpha$$, using normal approximation, with equal or unequal allocation.
• Estimate the sample size required for a test of $$H_0 \colon p_1 = p_2$$ for given $$\delta$$ and $$\alpha$$ and $$\beta$$, using normal approximation and Fisher’s exact methods
• Use a SAS program to estimate the number of events required for a logrank comparison of two hazard functions to have $$(1 - \beta)\%$$ power with given $$\alpha$$
• Use Poisson probability methods to determine the cohort size required to have a certain probability of detecting a rare event that occurs at a $$\text{rate} = \xi$$.
• Adjust sample size requirements to account for multiple comparisons and the anticipated noncompliance rates.

 [1] Link ↥ Has Tooltip/Popover Toggleable Visibility


text:
Opened 7 years ago

Closed 7 years ago

#6562 closed enhancement (fixed)

Reported by: Owned by: Ryan J Ollos Shun-ichi Goto normal TicketBoxMacro normal 0.11

Description

Trac 0.12 allows revision info for plugins and macros to be displayed on the About page. For single file macros, revision info is made available by including the following two lines below the section of import statements:

revison="$Rev$"
url="$URL$"

and applying the following svn:keywords properties:

svn propset svn:keywords "LastChangedRevision HeadURL" <filename>.py

For example, see changelogmacro/0.11/ChangeLogMacro.py@7473:9-10.

comment:1 Changed 7 years ago by Shun-ichi Goto

Resolution: → fixed new → closed

(In [7506]) Add plugin module info supported by Trac 0.12. close #6562.

comment:2 Changed 7 years ago by Shun-ichi Goto

Thanks, I've updated like you said.

comment:3 Changed 7 years ago by Ryan J Ollos

Thanks, I'll test it out with 0.12 to see if it works with the new version. Have you tested with 0.12dev?

comment:4 Changed 7 years ago by Shun-ichi Goto

I've tested and fixed some more (r7508). Now it should work with 0.12dev.

comment:5 Changed 7 years ago by Ryan J Ollos

Thanks. I tested it out and it is also working well for me with Trac 0.12dev-r9098.

Modify Ticket

Change Properties

format: plain


text:
# Momentum is a cotangent vector?

Imagine we have a particle described by $x \in M$, where $M$ is some manifold, then it is very intuitive I think that a velocity is an element of the tangent space at $x$, so $x' \in T_{x}M.$ Thus, by definition of the tangent bundle, we have $(x,x') \in TM$.

Now, in classical mechanics we learn that the conjugated mometum $p(x,x') = \partial_2L(x,x')$ and now I read that this guy is an element of the cotangent space, but I have no idea why.

I mean, to be in the cotangent space, you need to take elements in the tangent space of $x$ which are velocities as arguments and only depend linearly on them. Although it is clear that p takes velocities as arguments which is alright, it is not clear to me at this moment why this should happen linearly. Is this an additional (physical) input at this point or is there a mathematical argument why the momentum is now a linear map?

• – Qmechanic Apr 18 '15 at 15:07

I) Disclaimer: In this answer we will use the (traditional) physicist's definition of tensors using indices and their transformation properties under coordinate transformations. Moreover, let us suppress time dependence $t$ for simplicity.

II) Let the manifold $Q$ be the configuration space. The Lagrangian $L:TQ\to \mathbb{R}$ transforms as a scalar

$$L~~\longrightarrow~~ L^{\prime}~=~L, \tag{1}$$

the velocity $v^i$ transforms as a vector

$$v^i~~\longrightarrow~~ v^{\prime j}~=~\frac{\partial q^{\prime j}}{\partial q^i}v^i,\tag{2}$$

the Lagrangian canonical momentum

$$p_i~:=~ \frac{\partial L}{\partial v^i}\tag{3}$$

transforms as a covector

$$p_i~~\longrightarrow~~ p^{\prime}_j~=~\frac{\partial q^i}{\partial q^{\prime j}}p_i,\tag{4}$$

under general coordinate transformations

$$q^i~\longrightarrow ~q^{\prime j}~=~ f^j(q)\tag{5}$$

in the configuration space $Q$. Eq. (4) follows from the chain rule.

III) A point in the tangent bundle is of the form

$$(q,v)~\in~TQ,\qquad v~=~v^i \frac{\partial}{\partial q^i}.\tag{6}$$

Note that the velocity $v$ is an independent variable, which transforms as a vector (2) under general coordinate transformations (5) in the configuration space $Q$.

IV) The Lagrangian canonical momentum (3) can be viewed as a section

$$TQ ~\ni~ (q,v) ~\stackrel{p}{\mapsto} ~(q,v; p_i\mathrm{d}q^i)~\in~T^{\ast}TQ \tag{7}$$

in the bundle $T^{\ast}TQ \to TQ$.

V) Finally, let us for completeness & comparison mention the Hamiltonian canonical momentum (also denoted $p$) in the case where the phase space $M$ is the cotangent bundle $M=T^{\ast}Q$. In the case $M=T^{\ast}Q$, the Hamiltonian canonical momentum $p$ is an independent variable, which transforms as a covector (4) under general coordinate transformations (5) in the configuration space $Q$. A point in the tangent bundle is of the form

$$(q,p)~\in~T^{\ast}Q,\qquad p~=~p_i\mathrm{d}q^i.\tag{8}$$

• I am pretty sure that you are aware of the definition of the cotangent space which says that a cotangent space contains linear functionals on the tangent space. My question is: Why is the momentum a linear(!) functional? - See the wikipedia reference for a proper definition en.wikipedia.org/wiki/… I really want to see it according to this definition. – Solid State Physicist Apr 16 '15 at 22:56
• I mean, maybe your answer is perfectly correct, but I am assuming this definition of an element in the cotangent space and would therefore prefer to see it accordingly. – Solid State Physicist Apr 16 '15 at 22:59
• @Real Analysis: Concerning linear functionals: If you are familiar with how $\mathrm{d}q^i\in T^{\ast}Q$ can be viewed as a linear functional, then you can identify the momentum as the linear functional $p_i\mathrm{d}q^i\in T^{\ast}Q$. – Qmechanic Apr 16 '15 at 23:07
• but I think that is not what the literature always tells us: See also this accepted answer on mathoverflow claiming that $p$ is in some cotangent space mathoverflow.net/q/16460 . – Solid State Physicist Apr 16 '15 at 23:09
• Is the disclaimer because you don't want to bother, or because something actually breaks down when you relax the conditions? – ACuriousMind Apr 17 '15 at 22:37

The momentum is a covector because it is a gradient, and gradients are always covariant. It does what it says on the tin. However, you are right that this is a subtle point and it's not particularly clear at first sight.

For a lagrangian of the form $L=T-V$ with $V$ independent of $\dot q$, the canonical momentum is given by $$p=\frac{\partial L}{\partial \dot q}=\frac{\partial T}{\partial \dot q}.$$ This derivative measures how much $T$ changes with respect to small changes in $\dot q$, when these changes are small enough that a linear approximation to $T$ suffices. This is exactly the linearity of $p$ as a functional of $\dot q$.

This means that $p$ is a functional over increments in $\dot q$ rather than a functional over $\dot q$ itself. This is of course correct: if you have a configuration space $Q$, then lagrangian mechanics takes place in $M=TQ$, which is the space of all configurations $q$ and the corresponding velocities $\dot q$. Hamiltonian mechanics, on the other hand, takes place in $T^*M$ - the space of linear forms over $TM$. Note here that $TM=TTQ$ is, precisely, the space of increments in velocity (along with the velocities themselves as increments in the position.)


text:
# S. Hassi

Search this author in Google Scholar

Articles: 6

### Compressed resolvents of selfadjoint contractive exit space extensions and holomorphic operator-valued functions associated with them

Methods Funct. Anal. Topology 21 (2015), no. 3, 199-224

Contractive selfadjoint extensions of a Hermitian contraction $B$ in a Hilbert space $\mathfrak H$ with an exit in some larger Hilbert space $\mathfrak H\oplus\mathcal H$ are investigated. This leads to a new geometric approach for characterizing analytic properties of holomorphic operator-valued functions of Krein-Ovcharenko type, a class of functions whose study has been recently initiated by the authors. Compressed resolvents of such exit space extensions are also investigated leading to some new connections to transfer functions of passive discrete-time systems and related classes of holomorphic operator-valued functions.

### Generalized zeros and poles of $\mathcal N_\kappa$-functions: on the underlying spectral structure

Methods Funct. Anal. Topology 12 (2006), no. 2, 131-150

Let $q$ be a scalar generalized Nevanlinna function, $q\in\mathcal N_\kappa$. Its gene alized zeros and poles (including their orders) are defined in terms of the function's operator representation. In this paper analytic properties associated with the underlying root subspaces and their geometric structures are investigated in terms of the local behaviour of the function. The main results and various characterizations are expressed by means of (local) moments, asymptotic expansions, and via the basic factorization of $q$. Also an inverse problem for recovering the geometric structure of the root subspace from an appropriate asymptotic expansion is solved.

### Generalized resolvents and boundary triplets for dual pairs of linear relations

Methods Funct. Anal. Topology 11 (2005), no. 2, 170-187

### Operator models associated with singular perturbations

Methods Funct. Anal. Topology 7 (2001), no. 3, 1-21

### Generalized resolvents of symmetric operators and admissibility

Methods Funct. Anal. Topology 6 (2000), no. 3, 24-55

Let A be a symmetric linear operator (or relation) with equal, possibly infinite, defect numbers. It is well know that one can associate with A a boundary value space and the Weyl function M(λ). The authors show that certain fractional-linear transforms of M(λ) are identified as Weyl functions of extensions of A, and vice versa. This connection is applied to various problems arising in the extension theory of symmetric operators. Some new criteria for a linear operator to be selfadjoint are established. When the defect numbers of A are finite the structure of all selfadjoint extensions with an exit space is completely characterized via a pair of boundary value spaces and their respective Weyl functions. New admissibility criteria are given which guarantee that a generalized resolvent of a nondensely defined symmetric operator corresponds to a selfadjoint operator extension.

### Operator models associated with Kac subclasses of generalized Nevanlinna functions

Methods Funct. Anal. Topology 5 (1999), no. 1, 65-87


text:
# Tag Info

Accepted

### Is Farady's law valid for conducting loop only?

Faraday's law is for a mathematical loop, even with nothing there.
• 1,104

### Can the induced voltage in an inductor coil reverse the direction of curent?

You can't assume the current will increase at a rate of $2\text{ A/s}$ to find the induced emf because the inductor voltage (or alternatively, the induced emf) will oppose the current flow, and ...
• 7,257
1 vote
Accepted

### Would a ring magnet around a copper tube be slowed?

In a word, yes. Eddy currents are induced in conductors exposed to changing magnetic fields. Those currents specifically counter the change in magnetic flux the conductor is experiencing. That's Lentz'...
• 282
1 vote

### Is Farady's law valid for conducting loop only?

You can apply Faradays law for any loop as long as it is complete
1 vote

### Inductance in AC circuit

Why does an inductor oppose a change in current and not voltage? You have got already some good answers. Here is yet another intuitive way to understand it. The magnetic field energy carried by the ...
• 25.7k
1 vote
Accepted

### Inductance in AC circuit

Why does an inductor oppose a change in current and not voltage? Because induced electric field (which causes induced EMF) is associated with changing current (accelerated motion of electric charges),...
• 27.8k
1 vote

### Inductance in AC circuit

In a simple circuit where there is a changing voltage source with an inductance and a resistance in series, the same current flows through every device, so it is clear to say that the back emf opposes ...
• 996
1 vote

### Why is the equation of the induced current in the secondary coil in a transformer true?

In an ideal transformer in which no current is being drawn from the secondary, the current in the primary is determined by the inductive reactance of the primary and is out of phase with the voltage, ...
• 11.4k
1 vote

### How long of a wire do i need to construct a solenoid of $N$ turns?

aaand i am stuck here, i will maybe need to put into an integral, but not sure how yet, i will try and update if i come any further. some pointers/comments/feedback would be appreciated You need to ...
• 6,956

Only top scored, non community-wiki answers of a minimum length are eligible


text:
Paper ID: 1056 Title: End-to-end Learning of LDA by Mirror-Descent Back Propagation over a Deep Architecture
Current Reviews

Submitted by Assigned_Reviewer_1

Q1: Comments to author(s). First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. (For detailed reviewing guidelines, see http://nips.cc/PaperInformation/ReviewerInstructions)
=== Update after rebuttal ===

I thank the authors for a comprehensive rebuttal and extra experiments. It has addressed most of my concerns, and I have updated my score. The authors should make sure to properly tone down the claims about improved training for LDA (vs. sLDA), as well as mention the perplexity experiment. It seems to me that we do not really understand very well what is happening in these models at this stage; this perplexity experiment is just scratching the surface (and should be presented as such). I am also a bit puzzled by the use of alpha = 1.001 (vs. exactly 1 e.g.). As a prior, there is almost no difference between the two as a prior; but I can see that it changes a lot the MAP. In particular, using alpha = 1 (uniform distribution), the MAP is just maximum likelihood and could have zero entries; whereas any alpha > 1 creates a barrier in the objective function and prevents any theta_k to be too close to zero.

This being said, I think this paper makes interesting contributions, and so even though more careful experiments are needed to understand better what is happening, this can be left as future work and people can already build on the ideas introduced here for topic modeling.

[# of dominant topics experiment] The "# of dominant topics" experiment is interesting and should be included in the final version. As a note, a prior with alpha = 1.001 with K=100 gives on average about 60 dominant topics as a prior (defined as the number of topics to include to get 90% of the mass when sorted in decreasing order). This becomes 45 for alpha = 0.5 and 15 for alpha = 0.1 (which shows that these alpha are still too big to be used when K = 100). This experiment indicates that given the topic learned, the likelihood is still enforcing somewhat sparse topic proportion (I think that for an alpha so close to 1, the MAP should not be too different than the maximum likelihood solution). Though at K=100, one can already see a big difference in sparsity between BP-LDA alpha = 1.001 and Gibbs-LDA alpha=0.5.

[perplexity experiment] The authors should also report the perplexity for BP-LDA with alpha=0.5 and alpha=0.1 to control the difference in alpha. They should also try more topics. The reason that the perplexity is becoming worse with more topics is that alpha should change according to the number of topics. In my experience, properly estimating alpha yields a perplexity more robust to the number of topics (and can even get a better fit with more topics). To get some ball park, the alpha's learned by maximum likelihood using variational inference on standard text document datasets are much smaller than these values. For example, on the NIPS dataset from http://ai.stanford.edu/~gal/data.html, they are of the order of 1e-3 for a K=200 topic model. Finally, as I said in my previous review, it would interesting in future work to propagate the backprop to updating the alpha's as well...

=== end of update ===

This paper makes an interesting contribution to the supervised topic modeling literature. The idea of using backpropagation by unrolling a few steps of an iterative procedure (which computes a fixed point; or maximize some objective; or do some message passing) has been used before in several areas (as the cited [16]), but as far as I know, it has not been used for supervised topic modeling, so this is a fresh outlook, and I like it. It is also interesting that they are able to get significantly better results than linear regression (I hope this is ridge regression!), as usually supervised topic models struggle to improve over standard discriminative approaches. Their results in Figure 2b) where they basically do the same as logistic regression (within the variation) is more typical (though the other supervised topic model approaches do not do well on this one for some reasons).

* Missing experiment:

The reason that I am not putting a higher rating is because I think an important experiment is missing given that the authors claim that they their approximate MAP method to learn the parameters of the unsupervised LDA model is "outperforming previous learning methods". The classification / regression experiment is not meaningful for comparing unsupervised LDA learning techniques (it is great for sLDA, and the results are impressive; but unsupervised LDA is meant for an *unsupervised* evaluation!). The authors should thus also report the average test set per-word log-likelihood for LDA with the topic parameters learned by BP-LDA vs. Gibbs-LDA on their dataset. Note that they are definitively not allowed to do the PLSA-like technique of maximizing the posterior over theta on the test data to evaluate its likelihood (this is cheating!) -- they should just use the usual marginalization over theta for LDA. I suggest they use the code from http://homepages.inf.ed.ac.uk/imurray2/pub/09etm/ "Evaluation Methods for Topic Models", Wallach et al. ICML 2009, with some of their best methods. The authors should report these results in their rebuttal -- I am quite curious to see whether the superior prediction performance came at the cost of worse text modeling accuracy. If the authors can report on these results, I am willing to increase my rating (irrespective of whether their perplexity results are good or not).

* Novelty claim correction:

Using convex optimization to do MAP over theta for LDA is not new: this was already proposed in "Complexity of Inference in Latent Dirichlet Allocation", Sontag & Roy, NIPS 2011 (see Section 3.1). They also had proposed to use the exponentiated gradient algorithm to do the convex optimization, which is equivalent to mirror descent with KL divergence, i.e. yields exactly the same update as (12). The authors should properly correct this novelty claim and mention this prior work.

* Using alpha > 1 is not a good text model:

I also do not think that using alpha > 1 is a good idea for proper text modeling. It might be fine to get good classification / regression performance (it gives more dense features to the regressor); but if one would also evaluate the generative likelihood for the document, it won't be as good. All the work on topic modeling that I know always found that the individual hyperparameters of the Dirichlet prior over theta should be < 1 to give better perplexity (especially if a large number of topics is used). This was also mentioned in Section 3.2 of the [Sontag & Roy 2011] paper; see also Section 3 of

"On Smoothing and Inference for Topic Models", Asuncion et al. UAI 2009. I am willing to bet that if the authors do the perplexity experiment, they could get better results by using alpha < 1 (at least for the Gibbs sampler).

Also, note that it was also mentioned before that estimating the hyperparameters of the Dirichlet prior over theta (with asymmetric components) made a big difference for text modeling; see "Rethinking LDA: Why Priors Matter", Wallach et al., NIPS 2009. This was for unsupervised LDA. I am not sure whether it also could make a difference for sLDA -- but given that the authors have now an efficient backpropagation framework to learn all the parameters in a discriminative fashion; it might also be interesting to also backpropagate the gradient to estimate the alpha_k hyperparameters as well.

Using alpha > 1 means that a sampled theta is not sparse: this means that a specific document contains all topics with some probability, which certainly seems like a bad generative modeling assumption! I understand that the authors made this assumption because they wanted to claim that their MAP inference over theta was convex. On the other hand, given that they only do a small number L of iterations (and so they certainly do not optimize to convergence), and that the objective is not convex anyway in U and Phi, I do not think that having the inner MAP inference non-convex would be such a big problem. Using a finite number of iterations L is a way to lower bound the smallest value that theta could take in any case (as was suggested in Section 3.2 of [Sontag & Roy 2011]), and so the problem of negative infinities would operationally be avoided. They also seem to report better results for smaller alpha; so I would be curious to see Figure 4 to also report results for alpha < 1.

- Correction line 076: "this paper is the first work to perform a fully end-to-end discriminative training of LDA" -- it should be "of sLDA". They certainly do not do discriminative training of LDA: for LDA, they do an approximate MAP for Phi by removing the marginalization over theta that would normally be required by the model and replacing it with an approximate MAP over theta. This is still generative training, and so should not be called discriminative. In some sense, by doing MAP for theta, they are doing a kind of regularized PLSA model (probabilistic latent semantic analysis), which was the non-Bayesian precursor to LDA. In PLSA, there is a fixed theta per document, found by maximum likelihood. Here, they do regularized maximum likelihood for theta with the Dirichlet prior acting as a regularizer. It is presented as an approximation to LDA; but really, I would see it more as just regularized (potentially supervised) PLSA.

- Lines 120-122: the authors should clarify that Blei and McAuliffe had argued that using bar{z} instead of theta to tie the words to the variable to predict y was a better idea given their previous experience with two-signals modeling. I understand that the authors' framework could not handle bar{z} as the input variable for the distribution on y, as it is discrete, justifying their modification, but it is worthwhile to clarify this point. In their experiments, when they talk about "sLDA", do they mean the approach from [3] which used bar{z} as the input variable and also variational inference?

- Figure 3 a): are the error bars coming from different folds? What about the variation arising from just different random initialization for this non-convex problem? (Same thing for the sampling approach). Also, what about using a l2-regularizer on U to avoid overfitting for their method?

- Lines 396-399: do they use the same number of steps L when computing the test features; or they do full MAP for it?
[light reviewer]

quality:

6

(out of 10) clarity: 8 originality: 8 significance: 8

A very interesting paper. The exact MAP inference is not novel though (see below). And since they included their approximate MAP learning for unsupervised LDA as a contribution, they need to include a standard perplexity evaluation for it.

Submitted by Assigned_Reviewer_2

Q1: Comments to author(s). First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. (For detailed reviewing guidelines, see http://nips.cc/PaperInformation/ReviewerInstructions)
==update after rebuttal=== The authors do a good job performing new experiments which respond to most of my concerns and I have bumped up my score. The practical performance of \alpha < 1 is reassuring, and the effect of topic sparsity for \alpha = 1.001 is an interesting and new (to my knowledge) result, and should be presented in the final version with additional discussion. ==end update===

The authors derive a MAP objective for LDA, which for settings of Dirichlet hyperparameter \alpha > 1, is strictly concave and can be efficiently found, as opposed to the Bayesian posterior which is intractable. To handle the simplex constraints, the authors use a mirror descent algorithm with an entropic Bregman divergence which admits closed form updates. They formalize a fixed number of mirror descent iterations (later they show that the algorithm is not sensitive to this number) as a "layered architecture" and derive a backpropgation algorithm through this MDA algorithm. The backpropgation algorithm can then be applied to both unsupervised and supervised LDA objectives. The layered architecture representation is a useful abstraction makes it clear that any subsequent differentiable likelihood function can be used to connect the results of inference to observed data and obtain an efficient end-to-end algorithm. They evaluate their method against competing state of the art methods on two datasets, using both supervised and unsupervised variants and outperform the other methods in both predictive error and running time.

The paper is clearly written and easy to follow, including both derivations and experimental methodology. The introduction does a good job of situating the work among the rest of the literature. If anything, citations to related work are excessive and distracting, e.g. there are two MedLDA citations [20,21] with the same authors.

The AMR experiments are well done and easy to understand, particularly the runtime comparison

The main weaknesses I see in this paper which should be addressed in the rebuttal are, in priority order:

(1) In the MultiSent task, there is barely any improvement from Logistic regression. This means there is only one experiment (AMR) which convincingly shows that supervised LDA with the authors MAP-inference algorithm offers a significant boost in predictive performance. The authors should demonstrate the significance of their algorithm by obtaining a larger delta against bag-of-words regression on a larger number of datasets.

(2) The method requires a Dirichlet concentration of \alpha > 1, but this does not yield sparse distributions over topics for each documents, which is generally desirable in LDA. An experimental comparison sparsity for this method with \alpha > 1 and other methods for \alpha < 1 would be helpful, as well as an analysis of the impact of sparsity on predictive error. In addition, the experiments aggressively trimmed the vocabulary, which may mitigate the problem of non-sparse document-topic vectors. More experiments should be done where vocabulary size is varied.

(3) Compared to Bayesian methods, MAP may be prone to overfitting. The main regularization technique is to set the Dirichlet concentration parameters \alpha and \beta to be small positive values to encourage sparsity. Unfortunately this method limits choices here. How else can we mitigate overfitting in this model?

The authors propose a MAP inference method for LDA, relying on a convex objective function that can be solved with mirror descent. The inference method can be used to learn parameters for both supervised and unsupervised LDA and outperforms the existing state of the art in terms of both test error and runtime.

Submitted by Assigned_Reviewer_3

Q1: Comments to author(s). First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. (For detailed reviewing guidelines, see http://nips.cc/PaperInformation/ReviewerInstructions)
This paper proposes a novel inference procedure for doing MAP in supervised LDA. It seems like the main novelty of the paper is a procedure for learning the model in a (fully) discriminative fashion. The proposed approach yields empirical advantages compared to previous supervised LDA approaches on two tasks.

Overall I find this to be an interesting paper which proposes a novel way of training (supervised) topic models.

Some questions and comments: - Paper title: It would be worthwhile to emphasize that the paper is about

supervised LDA (even though "back prop" is a hint). I would also point

out that the approach proposes MAP inference. - In [3] the authors indicate that having the response variable be

dependent on the realized topic proportions (z) is advantageous. Is

there anything in your derivation that prevents you from doing the same?

It seems like you should be able to marginalize out theta. - Your approach requires the Dirichlet hyper-parameters to be greater than

1. Could you elaborate on the implication of this (e.g., on the sparsity

of the recovered topics and topic proportions)? It also appears that

competing methods such as sLDA are typically learned with alpha set to

values smaller than 1 (e.g., 1/K where K is the number of topics). This

should be compared to experimentally. - In the experiments I am not sure I see the advantage of reporting the

results on the smaller AMR dataset. I suggest only reporting results on

the full dataset (7.9M docs). Further, I think it's essential to report

the results of all baselines on the full dataset (it that's not possible

perhaps consider explaining why). - Overall the experiments seem to show that: 1) joint learning is

advantageous; 2) BP-LDA actually does better than vanilla inference

procedures for LDA. - Regarding 1) It would be interesting to further explain why joint

learning provides such an advantage. For example, are the learned topics

much different in joint methods vs. the others? My understanding is that

if one used BP-sLDA's topic proportions as covariates into linear

regression one could match BP-sLDA's results. Is this correct? My

intuition is that as the number of topics increase (it seems like using

100 topics for 7.9 million document may not be enough) BP-LDA and

BP-sLDA should do about the same. Have you noticed that in practice? - Regarding 2) Do you have an explanation as to why MAP (BP-LDA) does

better than fully-Bayesian inference (Gibbs-LDA)? Is it simply that it

finds a better local optimum? If so, as future work, it may be

interesting to explore how the ideas developed in this paper could be

used to optimize the variational objective. - Section 5.4 (computation time): It's useful to report computation time.

Comparing to other methods is also nice but the comparison seems unfair

as their code base is different.

Other comments: - In equation 11 what is \theta_d? - In the introduction readers may be led to believe that LDA is always

used for supervised task. I believe that is short-selling topic models.

It would be good to emphasize that the authors are interested in

versions of supervised topic models. - line 066: variable bound, did you mean 'variational'? - In your experiments did linear/logistic regression have access to all

words or only the top 5K? - The BP acronym is often used to denote belief propagation.

After rebuttal:

- Thanks for the rebuttal. It would be worthwhile to incorporate some of these points into the paper (specifically the extra dataset and the unsupervised results).

- In your rebuttal you wrote: "Joint learning enables the model to extract features most relevant to prediction, usually outperforming unsupervised learning + a separate."

classifier. "

I understand that. I think the size of the gap between joint training and unsupervised+classifier is still noteworthy. It would be good to provide an exploration of the topics.

- Thanks for your explanation of why you conditioned the response on the topics (and not z).
Good novel ideas, could have an impact. Experimental section is not completely convincing.

Submitted by Assigned_Reviewer_4

Q1: Comments to author(s). First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. (For detailed reviewing guidelines, see http://nips.cc/PaperInformation/ReviewerInstructions)
The paper proposed a new learning algorithm for supervised LDA, which is a hybrid of generative and discriminative methods.

The authors decoupled the parameter learning problem of supervised LDA into two optimization problems described in Eq. (3) and Eq. (4).

The proposed method iteratively compute $\Phi$ and $\theta_{d,L}$ and estimates $y_d$ using the learned $\theta_{d, L}$ by map inference.

The reviewer did not get the reason why the method computes $\hat{\phi}$ because $\hat{\phi}$ is never used to update $\theta_{d,L}$ and $\Phi$.
It is one of my light review papers. The paper proposed a new hybrid method for supervised LDA and empirically shows that it performs better score than existing methods.

Submitted by Assigned_Reviewer_5

Q1: Comments to author(s). First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. (For detailed reviewing guidelines, see http://nips.cc/PaperInformation/ReviewerInstructions)
Summary: The paper proposes a learning method for both unsupervised and supervised LDA, using exact MAP inference with mirror descent algorithm and backpropagation, instead of the traditional variational inference and Gibbs sampling approaches. Experimental results on predicting product review ratings (both regression and binary classification problems) show improvements over existing baselines.

The paper presents an interesting approach to learning traditional unsupervised/supervised LDA. I was particularly impressed with the scalability of the method, shown in Figure 5 where the training time of the proposed method on 7.9M documents is relatively comparable with that of the existing sLDA and MedLDA on 79K documents. However, besides that, I find that the arguments for using a fully discriminative training by MAP inference are not persuasive enough and are not supported well enough with empirical evidences. Here are my more detailed comments

One major concern I have with the proposed learning algorithm is that it only works for the case where \apha is greater than 1, as mentioned in section 3.1. I think this is a big limitation of this approach since in many settings, one desired property of LDA is its sparsity which is achieved by setting \sum_k \alpha_k < 1.

Second, in the experiments, the paper does not include details about the values of hyperparameters. Specifically, from my personal experience, \gamma plays a very important role in the prediction performance of sLDA. So, how do you choose \gamma in your experiments? Similar question applies to linear regression. I would also be good to include the performance of SVM.

In addition, I find Sections 3 and 4 describing the approach quite difficult to follow. Figure 2 is not particularly helpful. I would suggest providing a pseudo-code for the algorithm, in which some main steps can be explained in more details like in Section 3.1 and 3.2.

=== After rebuttal === I thank the authors for addressing my concerns and providing additional experimental results. I have ajusted my score accordingly.
The paper presents an interesting scalable learning method for both unsupervised and supervised LDA using exact MAP inference. However, I think that the proposed method has to trade-off an important property of traditional topic models (i.e., sparsity) while experimental results lack a lot of details to justify its effectiveness.

Author Feedback
Author Feedback
Q1:Author rebuttal: Please respond to any concerns raised in the reviews. There are no constraints on how you want to argue your case, except for the fact that your text should be limited to a maximum of 5000 characters. Note however, that reviewers and area chairs are busy and may not read long vague rebuttals. It is in your own interest to be concise and to the point.
We thank the reviewers for the valuable feedback. Below we address their concerns. Our revision will incorporate all points below plus other minor issues.

#alpha>1 vs alpha<1#
First, as pointed out by Reviewer 4, our method can also handle alpha<1 except that, in this case, the inference is no longer convex and hence no global optimal MAP inference is guaranteed as for the methods prior to this work. We did not pay much attention to alpha<1 earlier because we wanted to focus on convex inference. Based on all reviewers' feedback, we did new experiments and provide the resulting pR^2 on AMR (7.9M) below. They are not as good as our reported scores with alpha>1, but still significantly beats baselines.
#topics: 5 10 20 50 100
BP-sLDA (alpha=0.5): 0.488 0.548 0.575 0.571 0.574
BP-sLDA (alpha=0.1): 0.441 0.558 0.572 0.569 0.570

Second, our main contribution is to develop mirror-descent BackProp to improve prediction performance of supervised LDA. For fair evaluation, we first built strong baselines whose performance is tuned carefully (& indeed this led to alpha<1 for conventional topic models). Then the experimental results show that our method with alpha>1 still significantly outperforms all baseline topic models. To corroborate further, we carried out new large-scale experiments with BP-sLDA on AMR (7.9M) with #topics=200&alpha=1.001. We obtain pR^2=0.74, which even significantly beats a 200-unit neural network (pR^2=0.64) with the same model size.

Finally, when alpha is close to 1 (e.g. 1.001), the topic distribution can still be made reasonably sparse, as the probability simplex constraint on theta is akin to L1 regularization. To quantitatively demonstrate this, we carried out new experiments which gave the average # dominant topics (which add up to 90% probability) on AMR:
#topics: 5 10 20 50 100
BP-sLDA (alpha=1.001): 2.1 3.0 3.8 4.5 6.9
BP-LDA (alpha=1.001): 1.5 2.4 3.6 6.0 8.7
Gibbs-LDA (alpha=0.5): 1.6 1.7 1.9 2.1 2.3
Gibbs-LDA (alpha=0.1): 1.5 1.7 1.8 2.1 2.2

To Reviewer 1:
In Fig.3b (MultiSent), AUC=90.4% for LR, and AUC=91.3%, 91.2%, 91.4%, 90.8% for BP-sLDA with #topics=5, 10, 20, 50, 100. Note 1% absolute AUC improvement over 90.4% is significant (10% relative improvement). In revision, we will produce better visualization with error bars more clearly shown. In any case, BP-sLDA significantly beats prior-art topic models (AUC<80%).

Following your suggestion, we conducted a new binary text classification experiment with highly promising results on a large-scale proprietary dataset for business-centric applications (1.2M documents & 128K vocab), to be released in full after successful patent filing. In this new task, BP-sLDA (200 topics) achieves AUC=92.2% & error rate=15.2%, while LR has AUC=90.5% & error rate=17.1% (11% relative error rate cut). The gain is consistent with what was reported in the original draft, addressing your concern.

We set AMR's vocabulary size to 5k to be consistent with [18]. The pR^2 for full 701K-vocab AMR (7.9M) are:
#topics: 5 10 20 50 100
BP-sLDA (alpha=1.001): 0.633 0.677 0.672 0.682 0.684
Linear Regression: 0.403
even better than our reported results (5K-vocab AMR, 7.9M) in Table 1, showing that our method can effectively handle overfitting.

To Reviewer 2:
We make y depend on theta instead of z because it leads to a differentiable cost trainable by BackProp & SGD (z: discrete).

Joint learning enables the model to extract features most relevant to prediction, usually outperforming unsupervised learning + a separate classifier.

We conjecture that high performance of our method comes from better local optimum. Theoretical justification is our future work.

Running other LDA baselines on AMR (7.9M) takes too long (estimated to be ~10,000hrs shown in Fig.5).

To Reviewer 3:
We'll release codes with config files including all hyper-parameters.

In our model, gamma=1. After tuning, we found that performance is insensitive to gamma when gamma>=1.

The revised version will add the pseudo-code.

SVM: AUC=89.3% on MultiSent. SVR: pR^2=0.385 on AMR(7.9M).

To Reviewer 4:
Thanks for the reference [S&R] on convex inference & mirror-descent, which we will cite properly in the revised paper. To clarify, our contribution is to integrate BackProp with MDA to improve prediction performance, while [S&R] studies inference alone.

sLDA is from [3].

L is the same for training & test.

We appreciate your willingness to increase the rating. Indeed, our new experiment on per-word log-likelihoods (using [Wallach]'s code) confirms your bet:
#topics: 5 10 20
BP-LDA (alpha=1.001): -6.83 -7.12 -7.41
Gibbs (alpha=0.5): -6.17 -6.18 -6.21
Gibbs (alpha=0.1): -6.15 -6.13 -6.21

To Reviewer 5:
By \hat{\phi}, I assume you meant \tilde{\Phi}. We decouple learning into supervised & unsupervised parts, which learn \Phi & \tilde{\Phi} separately.

To Reviewer 6:
See our responses in section "alpha>1 vs alpha<1".


text:
# Question on a paper of Schoen and Yau

I am trying to understand the paper "Conformally flat manifolds, Kleinian groups and scalar curvature" by Schoen and Yau. In P.56, it says:

This implies that $\partial M$ has a zero $q$-capacity, and a standard result [AM] then implies that the Hausdorff dimension of $\partial M$ is less than or equal to $n-q$...

The reference [AM] refers to the paper "Bessel potentials, inclusion relations among classes of exceptional sets" of Adam and Meyers published in Indiana Univ. Math. J. (1973) 22, 876-905. I tried to read the paper of Adam and Meyers, but I did not know which theorem Schoen and Yau referred to. Also, I wonder if there are some easy ways to prove that the above claim of Schoen-Yau, since it was claimed to be standard.