Some Math Graphics

Dean Anton Sherwood has lots of good math graphics at http://www.ogre.nu/doodle/#chainmail. Here's one.

Annulus

An annulus is the region lying between two concentric circles in 2-space-- a ring.

Lipschitz continuity

From Wikipedia:

...Lipschitz continuity, named after Rudolf Lipschitz, is a smoothness condition for functions which is stronger than regular continuity. Intuitively, a Lipschitz continuous function is limited in how fast it can change; a line joining any two points on the graph of this function will never have a slope steeper than a certain number called the Lipschitz constant of the function....

* The function f(x) = x^2 with domain all real numbers is not Lipschitz continuous. This function becomes arbitrarily steep as x goes to infinity. It is however locally Lipschitz continuous.

* The function f(x) = x^2 defined on [ − 3,7] is Lipschitz continuous, with Lipschitz constant K = 14.

Elasticities in Regressions. (update of old post)Here are how to calculate elasticities from regression coefficients, a note possibly useful to economists who like me keep having to rederive this basic method:

1. The elasticity is (%change in Y)/(%change in X) = (dy/dx)*(x/y).
2. If y = beta*x then the elasticity is beta*(x/y).
3. If y = beta* log(x) then the elasticity is (beta/x)*(x/y) = beta/y.
4. If log(y) = beta* log(x) then the elasticity is (beta*y/x)*(x/y) = beta, which is a constant elasticity.
   (reason: then y= exp(beta*log(x)), so dy/dx = beta*exp(beta*log(x))*(1/x) = beta*y/x.)
5. If log(y) = beta*x then the elasticity is (beta* y )*(x/y) = beta*x.
   (reason: then y = exp(beta*x), so dy/dx = beta*exp(beta*x) = beta*y.)

6. If log(y) = alpha + beta*D, where D is a dummy variable, then we are interested in the finite jump from D=0 to D=1, not an infinitesimal elasticity. That percentage jump is

   dy/y = exponent(beta)-1,

   because log(y,D=0) = alpha and log(y, D=1) = alpha + beta, so

   (y,D=1)/(y, D=0) = exp(alpha+beta)/exp(alpha) = exp(beta)

   and

   dy/y = (y,D=1)/(y, D=0) -1 = exp(beta)-1

   This is consistent, but not unbiased. We know that OLS is BLUE, unbiased, as an estimator of the impact of the dummy D on log(Y), but that does not imply that it is unbiased as an estimator of the impact of D on Y. That is because E(f(z)) does not equal f(E(z)) in general and that ultimate effect of D on y, exp(beta)-1, is a nonlinear function of beta. Alexander Borisov pointed out to me that Peter Kennedy (AER, 1981) suggests using exp(betahat-vhat(betahat)/2)-1 as an estimate of the effect of going from D=0 to D=1, as biased, but less biased, and also consistent .

Partial Identification and Chi-Squared Tests

I heard Adam Rosen give his paper, "Confidence sets for partially identified parameters that satisfy a finite number of moment inequalities." It stimulated some thoughts. (Click here to read more.)

1. Suppose we wanted to estimate means of X and Y,(μ(x) μ(y)). Our theory says that they are distributed independently, bounded by [0,1]. But we only have data on X.

My maximum likelihood estimator will be a point estimate for μ(x) and an interval for μ(y). I have partial identification. If the sample mean of x were .6, my estimate would be (.6, [0,1]).

If I had a prior on μ(y), I could use that. Maximum likelihood or any kind of minimum distance-MOM estimator would leave every value in [0,1] equally good as an estimate of μ(y).

Another example would be if we wanted to estimate the mean of X+Y, μ(x+y), but only had data on x. If the sample mean of x was .6, our estimate for μ(x+y) would be the interval [,6, 1.6].

We would also have partial identification in a model in which y = αx1 + βx2 but x1 and x2 were endogenous and we had an instrument for x1 but not for x2.

2. Suppose we have partial identification, and our estimation has yielded us a best-estimate interval for the single parameter theta, which is thetahat = [5,10]. Our null hypothesis is that &theta &ge 6. Do we reject it?

We want to construct a confidence set C such that if we repeat the procedure, &alpha = 5% of the time we will wrongly reject the null when it is true:

(1) Prob(&theta -hat is in C)|&theta &ge 6) = .05

C will be a set of intervals.

But that probability in (1) is ill-defined, because C will differ depending on whether &theta =.6, 7, 9, 26, or whichever value greater than 6 we might pick. So we'll be conservative, making it hard to reject the null, and pick the value of &theta for which C is biggest. That kind of conservatism problem arises even in the simplest frequentist inequality null-- the problem is that the null is not "simple".

A nice thing about the chi-squared test is that it avoids having to define C for the &theta -hat space. Instead, we just find the scalar chibarredsquared statistic, a function of the interval, and look at the confidence interval for that test statistic. This is what chi-squared tests do in general--- they transform a multi-dimensional acceptance region into a one-dimensional acceptance interval. For example, we could use a Chi-squared test (or its close relation, an F-test), to test whether the pair of numbers (α, β) was close enough to (0,0).

Here, though, it's especially neat because we're not just doing an R-n to R mapping: we're mapping from a set in (R, intervals on R) to R. An interval on R can be reduced to its pair of endpoints, but even then our mapping wouldn't be as simple as a mapping from three real numbers to one.

Case Control Studies and Repeated Sampling

A standard counterintuitive result in statistics is that if the true model is logit, then it is okay to use a sample selected on the Y's, which is what the "case-control method" amounts to. You may select 1000 observations with Y=1 and 1000 observations with Y=0 and do estimation of the effects of every variable but the constant in the usual way, without any sort of weighting. This was shown in Prentice & Pyke (1979). They also purport to show that the standard errors may be computed in the usual way--- that is, using the curvature (2nd derivative) of the likelihood function. (Click here for more)

This, I was skeptical of. If the constant is misestimated, how can you deduce the variance of the disturbance term, and if you can't deduce that, how can you deduce the standard error of any of the coefficients? Nowhere have I seen a clear demonstration or an intuition for the result, so I thought there might be a crucial unnoticed mistake in the math somewhere, as is not unknown in famous papers (e. g. Hotelling on location, Tullock on overdissipation, Viner on average cost curves, and Rothschild-Stiglitz on risk).

Since I did not follow all the steps of the Prentice-Pyke proof and so did not know of any error in what they did, I tried doing a Monte Carlo study which seemed to confirm my intuition.

Since then, however, I have seen where my Monte Carlo study went wrong, and now I believe Prentice and Pyke. Some details are instructive.

1. An intuition-- a bit shaky, I think, but better than nothing (let me know if it's false). Suppose that a coefficient is estimated correctly by some estimator. We want to estimate the estimator's standard error, to know how variable the estimate would be if we repeated the estimation with different disturbances. For this, we need to know how noisy the data is. We do not need to know how noisy the data in the whole population is, however, just how noisy in the kind of sample we draw. If our procedure is to draw a biased sample, then we need to know what will happen in other biased samples, not in the population. It is okay to use the sample for this purpose. In using a standard error, we are not generalizing anything to the population (not estimating goodness of fit, for example), we are just generalizing to repeated samples.

2. How to think about repeated sampling and how to do a Monte Carlow study. What I did was to construct a population of 60,000 data points, drawing X from a uniform distribution on [0,1] and a disturbance epsilon from a logit density with an α "constant" coefficient of -4 and a β X coefficient of 0. If α + epsilon < 0 then Y=0; if α + epsilon >= 0 then Y = 1. That yields 1,039 points with Y= 1, about 1.7% of them.

Our estimation procedure is to combine two random samples of 1,000 observations with Y=0 and 1,000 observations with Y=1 and do a logit estimate of alpha and beta. We would expect the estimate of alpha to be wrong-- not close to 0.017-- and the estimate of beta to be right-- close to 0.000-- since we have a large enough sample that consistent estimates ought to be close to the true parameters.

The maximum likelihood estimate would give us standard errors based on the second derivative of the likelihood function or on bootstrapping. In repeated sampling, we would expect the standard deviation of the alpha estimates not to be close to the average of the estimates of its standard error. The question to be investigated is whether the the standard deviation of the beta estimates is close to the average of the estimates of its standard error.

So far, so good. Where I made my mistake, I think, is in the definition of "repeated sampling". Ordinarily in frequentist thinking, in repeated sampling we keep the X values the same in each sample, and we draw new disturbances, which combine with the fixed X's to give new Y's. That also amounts to conditioning on the X's, though we wouldn't have had to condition the X's, since our estimator should work fine even if we changed the X's in each sample too. (If we did change the X's, though, that change the information content in each sample--- a sample in which X only varied between .3 and .4 would have less information and yield worse estimates than one with X varying widely between .02 and .94. So in small samples, especially, we'd have to make some allowance for that.)

Here, though, we can't keep the X's fixed. If we did, then although our first sample would have 1,000 observations with Y=1, our succeeding samples would have about 34. We wouldn't be using the case-control method.

So what we have to do is to think about repeated samples with 1,000 Y=0 observations and 1,000 Y=1's. Turning our usual thinking upside down, we need to keep the Y's fixed, draw new disturbances, and let the X's vary. This is especially hard to think about here, because knowing Y and epsilon does not tell us X-- remember, Y is coarse and contains less information than alpha + beta*X + epsilon,and beta is zero here too, making things even worse.

The best way to proceed is to think about repeating the entire scientific procedure, including the sampling as well as the estimation. The way I did this was to take 100 n=2000 samples from the 60,000-point population, each time combining equal-sized subsamples with Y=0 and with Y=1.

Recall, however, that there are only 1,037 Y=1 values in the entire population. Thus, my repeated sampling had to be with replacement, and was using the same Y=1 observations over and over. It is OK to use the same X values repeatedly, but these observations also had the same epsilon values each time, so the samples are not independent in the way needed for the law of large numbers to work. The standard errors computed by maximum likelihood came out wrong--- not equal to the standard deviation of the estimates, but that is to be expected when the draws are not independent.

Realizing this, I also tried doing the procedure with 100 n=200 samples instead of 100 n=2000 samples. I still used sampling with replacement, but now there was less overlap between replacements, less dependence between samples. And now the estimated standard errors were close to the standard deviations.

This, I expect is what would happen if I did the kind of repeated sampling that is our thought experiment for the kind of real studies that use the case-control method. That thought experiment is to take repeated draws of 60,000-point populations, with the same X's each time but with different epsilons and hence Y's. Each of the 100 Monte Carlo samples would be from a different population draw.

Is Not Necessarily Equal To

At lunch at Nuffield I was just asking MM about some math notation I'd like: a symbol for "is not necessarily equal to". For example, and economics paper might show the following:

Proposition: Stocks with equal risks might or might not have the same returns. In the model's notation, x IS NOT NECESSARILY EQUAL TO y.

Click here to read more

I can't say x \neq y there, because maybe x=y, maybe not, depending on the parameter values.

If anybody knows of such a symbol, please let me know. In fact, just a symbol for IS NOT NECESSARILY might be useful, since we could combine it with other symbols, e.g.

x IS NOT NECESSARILY = y

Horse heights ARE NOT NECESSARILY > cow heights

The biased estimator IS NOT NECESSARILY \sim N(0,1).

OCTOBER 7. Stimulated by a comment below: The modal logic symbols might work well. Wikipedia's "Modal logic" says: "The basic modal operators are usually written (or L) for Necessarily and (or M) for Possibly."

Maybe I could say " x ≠ y" for "x is not necessarily equal to y".