Applying Regression Blog

The Ghost Map

2008-01-20T04:44:00.001Z

If you're anything like me, then when you learned about Jon Snow and the Broad Street Pump, you learned something like this:

There was a cholera epidemic in an area of Soho in London in 1854.
Jon Snow was a doctor there, who drew a map of where the deaths were, and realized that water coming from the Broad Street pump was the source.
Upon realizing that, he removed the handle from the pump, to stop people using it.
Snow showed that cholera was an infectious disease.
He thereby also founded the discipline of epidemiology.

Steven Johnson has written a book, called The Ghost Map, named after the map that was drawn by Snow, all about the incident. (Johnson also wrote Everything Bad is Good for You).

The first thing you learn from the book is that the popular account of the Broad Street pump incident is wrong in many, many ways. The events happened in almost the reverse order. Epidemiology existed before the incident (not long before, and Snow was a founder). Some people thought that cholera was infectious before this incident. Most didn't. Most didn't think it was infectious afterwards either. Snow removed the pump handle - but before drew the map. He didn't even draw the best map - someone else drew a better improved version later on.

The book also goes into detail about what was known about the minor details of people's lives, where they worked (this was important, because this affected what they drank) where their relatives lived, but sometimes details are lost - no one knows the name of the first victim. It examines the epidemic on a broader, political scale - why had things got to the state where it was possible for a cholera epidemic to occur. It examines how the situation arose that cholera could spread - the city got too big, the carts that had to carry the, errrmmm... , waste had too far to go, and removing the 'night soil' became too expensive; people filled their cellars with shit instead. It looks at what happened as a result - sewage systems removed sewage from people's houses, and dumped it in the river.

So why am I writing about it here? It shows how three things that are required for statistical analysis to have an effect come together. First, there was a theory - the theory could be tested by the collection of data. Snow (and others) spend a long time collecting that data, finding who had died, where they had lived, what they did, but the theory was needed to guide the data collection. Second, the data had to be presented clearly - in this case on the map, which was a form of bar chart, with a bar on each house. (Nowadays we'd call that a Geographical Information System (GIS).) Third, the consumers of research had to be persuaded - in this case, the government departments who needed to build sewers to improve public health.

Finally, the book is extraordinarily well written. It manages to keep the text flowing without becoming turgid, and without deviating from the facts, here's a brief example: "Word of the outbreak had traveled through the wider city and beyond. The chemist's son who had enjoyed his pudding days before on Wardour Street died on that Sunday at his home in Willesden."

There's more information in the John Snow entry on Wikipedia.

Effect Sizes

2007-11-17T04:56:00.000Z

A Davies sent a message to the psych-methods list:

I have carried out three hierarchical regression analyses (linear forattitudes and intention, and logistic for behaviour). Within these regressions I have controlled for past behaviour and baseline psychological variables, such that I want to know what the effect of the intervention isonce these variables are controlled for in the early steps.

I am familiar with using means and standard deviations to calculate cohen’s d, but am unsure how to obtain these values from a regression analysis? Are there other approaches that can be used from data analysed using hierarchical regression? I understand that the R2 value can be used but as I understand it that is an indicator of the effect of adding the variable to the regression. I've also read about using the t-test statistic but when I have used it it seems to indicate a large effect of the intervention (d=.45) when actually the intervention is only marginally significant (p<.07) as a predictor of change. Does anyone have any suggestions?

To answer this question, we should think about what an effect size is, and what it's for. Sometimes the units of measurement are meaningful. If eating a tomato every day is associate with living (on average) two years longer, then there's no need to turn this into an effect size. You understand what two years means. It makes sense. You don't need an effect size, and you can easily compare it to other studies that examined the influence of eating a carrot every day.

In psychology, and other social sciences, we often have measures that are not meaningful. If one study has looked at the efficacy of Prozac for depression, and found that Prozac was associated with a difference of 8 points on the CESD; and a second study has looked at the efficacy of CBT and found a difference of 4 points on the PHQ, it's hard to compare those two effects.

The solution is an effect size, something like Cohen's d. Cohen's d is very simple. It's the difference between the two groups, divided by the standard deviation (that's a pooled standard deviation, so it's a tiny bit fiddlier, but not much).

If the SD of the CESD was 16, then 8 is half of that, so the effect of Prozac was d = 0.5. If the SD of the PHQ was 8, then 4 is half of that, so the effect of Prozac was d = 0.5, and the two effects were the same.

And as long as we have a dichotomous predictor, we are happy.

However, when we move to regression and multiple regression, it gets trickier, and we need something different. There are several things that we can use.

The first is the standardized effect, what SPSS calls beta. If you've only got one predictor, it's the correlation. Correlations are nice. We understand correlations. We might not like the standardized effect for a couple of reasons. One of them is that it destroys the units. If we have units we are interested in, then the standardized effect hides them. If the standardized effect of the relationship between tomatoes eaten per day and longevity is moderate (say 0.3) I might go and eat a lot of tomatoes. However, you might then tell me that eating one more tomato per day increases longevity by 12 seconds. That's pretty poor. If I only knew the standardized effect, that would be hidden from me. If your predictor is dichotomous, then the standardized effect is very silly.

The second choice is a partially standardized effect. Here, you standardize only the outcome variable, and keep the predictor unstandardized. This effect is the difference, in standard deviation units, associated with a 1 unit change in the predictor. If your predictor is dichotomous, that's Cohen's d. If it's not dichotomous, it's analogous to Cohen's d.

The third choice if you're using hierarchical regression is the change in R2. You ask how much additional variance is explained by the predictor (or predictors) that you added to the model at each step.

In the case of logistic regression, things get harder (as a general rule, everything is harder with logistic regression). Every sortware program gives you the estimate of the logistic regression as output, but not every package gives the Odds Ratio (sometimes called Exp(B)) unless you ask for it. (In Stata, for example, you need to use the , or option; in R, SPSS and SAS it's automatic).

You should always present the odds ratio, because it makes more sense than the estimate. But it doesn't make a lot of sense.

We can't standardize our outcome, because it's dichotomous. We could standardize our predictor, if that made sense, and present the partially standardized OR.

However, odds ratios are funny. They're funny because people don't know how to interpret them, so you should give them help. And you give them help by converting the parameter estimates into probabilities, then choose sensible values for the covariates, and calculate the probabilities associated with them.

I'm going to use the auto data in Stata to demonstrate this (I'll give the Stata code at the end). I'm going to regress foreign (a dichotomous variable indicating whether a car was made in the USA or not), on price (in 1000$) and mpg.

Here's the Stata output:

---------------------------
foreign | Coef.
-------------+-------------
price | .2660188
mpg | .2338353
_cons | -7.648111
---------------------------

Of course, we need to get the odds ratios instead of the linear estimates.

----------------------------
foreign | Odds Ratio
-------------+--------------
price | 1.30476
mpg | 1.263436
----------------------------

The two variables are in real units, so there's no need to standardize them. A price increase of $1000 is associated with the odds of a car being foreign increasing by 1.3 times (these are odds ratios, so they are multiplicativel; holding mpg constant) and one more mpg is associated with odds of being foreign 1.26 times higher. But what does that mean, in terms of the probability (because that's what we're interested in) of a car being foreign.

Let's compare two cars. One that costs $4000, and one that costs $5000 (these aren't new data). We can calculate the probability that each of those cars is foreign, and that will give us an effect size.

We just need to plug our numbers into the regression equation. But hold on, we need numbers for mpg. Let's pick a low value, say 14. (I'm not going to go through all the calculations, because it will take too long. If you're not familiar with how this is done, you can either believe me, or look it up.)

Our $4k which does 14 mpg car has a 0.035 (3.5%) probability of being foreign.
Our $5k which does 14 mpg car has a 0.045 (4.5%) probability of being foreign.

So adding $1k to the price of a car increases the probability it's foreign by 1%.

But what if we chose a different number of mpg? Let's use 25.
Our $4k which does 25 mpg car has a 0.32 (32%) probability of being foreign.
Our $5k which does 25 mpg car has a 0.38 (38%) probability of being foreign.

Which means that at this range, adding $1k has increased the probability by 6%. That's rather a large difference.

You need to calculate those probabilities, and you need to calculate them at appropriate values of the other covariates, in order to ensure that the reader can interpret your regression. However, it's a difficult issue, and you'll want to read more. Two good references are:

Regression Models for Categorical and Limited Dependent Variables. by J Scott Long. (There's a 2nd edition of this book out, which I haven't seen, but it's published by Stata Press, and therefore might be Stata specific).

Data Analysis Using Regression and Multilevel/Hierarchical Models , by Andrew Gelman and Jennifer Hill.

(Tip of the Hat: Some of my thinking on this has been influenced by Thom Baguley, and thanks to Greg Meyer for a correction).

Stata Code
sysuse auto
replace price = price / 1000
logit foreign price mpg
logit foreign price mpg, or

Start them young

2007-07-05T17:24:00.000+01:00

Sean Carey teaches a course called Introduction to Regression, at the Essex Summer School in Social Science Data Analysis. Here's a picture of his offspring that he uses in his initial presentation. I liked it, so I ~~stole~~ ~~borrowed~~ ~~used~~ displayed it here. (Notice that the error term is missing, or the hat on the y)

Historical Regression

2007-06-01T05:47:00.000+01:00

There's been a story on the news recently that a report suggests that childhood obesity is increasing because more mothers are working, and not staying at home with their children. It's been reported in a few places, one of them is here. It says

"We saw that start to happen. We could track childhood obesity. There's a direct correlation," said Terry Mason, Chicago's public health commissioner."

This might well be true, or it might not (that's the sort of thing that other blogs can worry about), they mentioned correlation, so we're going to talk about it on this blog.

So, a direct correlation, eh? And what's the sample size here? Let's see, well, we've got before, and we've got after. So we'll give them N=2. That's not a lot, is it? In fact, any two measures (which change) measured before and after some time have to have a correlation of r=+1, or r=-1. They don't even give a specific date to it - it's 'the 1980s'. As far as that evidence goes, you might as well say that it was Ronald Reagan getting elected, the release of Windows version 1.0 (anyone remember that? It had a clock that was pretty cool, but nothing else), or the fact that W Germany made it to the world cup finals 3 times (I'm including 1990, 'cos we're allowed to be a bit fuzzy here.)

In other words, if there WASN'T a direct correlation, we should be surprised.

This is an example of a more general problem (well, two more general problems).

The first is inspecting the data, and then making up a theory about the causal relationships that exist. But you've cheated, you looked at the data. Another example of this sort of thing is the clustering of leukaemia cases around nuclear power stations. When people first theorised this, they first looked at the data, and they said let's choose leukaemia (not any other sort of cancer) amongst children (we think of leukaemia as a children's disease, because that's what makes the news, but it's not necessarily), and let's make the children aged, errmm, less than 3, and make it, errrmmm, 5 miles from a nuclear power station. No! 10 miles! Ah look, we've got a cluster!" Obesity is a problem now, but they identify the root of this in the 1980s - 20 years ago, so any event between now and then would appear to do the job.

The second (related) problem is that of the sample size. Any time you say "Y happened, because X happened before" where X and Y are one off national events, you have an N of 2, and you can't say anything. Donohue and Levitt famously argued in 1999 that legalising abortion had decreased murder rates 20 (or so) years later (it's famous because they discuss it in Freakonomics). But they've got an N of 2. Murder rates are unstable, they weren't going to stay the same, so there was always going to be a correlation. This is discussed in a paper by Ted Goertzel called Myths of Murder and Multiple Regression (and if I ever write a paper with a title that good, I'll die happy). If you want to make this sort of statement, you need to have a bigger sample - you need to have abortion criminalised again, a few years later, because the murder rate should then rise, then ban it, and it should drop again. Alternatively, you need to have abortion legalised in different countries at different times, and then see what happens to the murder rate in those countries, the same period afterwards.

Social scientists do have a habit of being able to make up a plausible sounding theoretical explanation of any result that they happened to find. What if Donohue and Levitt had found that murder rates went up, as a result of abortion being legalized? Do you think they could have made up a plausible sounding theory to account for that too? (Go on, you try. I bet you can.) Well, it happens that other researchers have found that, by controlling for a couple of other variables, legalizing abortion really did make the murder rate go up.

It's unlikely that this will ever replicate, at least enough times to get a sensible sample, and it's unlikely that there will be changes in the workforce that mean that most mothers no longer work, so we'll never know who (if anyone) is correct. And a hallmark of science is that it has to be able to be wrong - for a theory or a finding or a fact to be considered to be scientific, we must be able to state what evidence would make us say "Oh, that was wrong then". For statistical analysis of this sort of result, there isn't anything that would enable us to reject the theory, and if we can't reject it, we might as well rely on psychoanalysis. ("You hate your father? That's because of repressed sexual urges. You love your father? That's because of unrepressed sexual urges.")

Stepwise regression

2007-05-04T22:55:00.000+01:00

There has been a little exchange on the SAS-L list about stepwise regression, and whether you should use it.

Jerry Davis (defending stepwise regression, in applied settings) wrote:
Stepwise selection methods are just statistical tools and like any tool, may be used inappropriately. I doubt if Walmart gives a rip about multicollinearity, they just want better predictions. If stepwise does the job, so be it.

Peter Flom wrote:
The analogy to a tool is common.
Stepwise methods ARE like a tool. They are like a hammer whose head flies off at frequent but irregular intervals.

Mediation

2007-05-03T15:01:00.000+01:00

The latest issue of Psychological Methods (Volume 12, issue 1) has two (count them! Two!) articles on mediation and moderation.

The first, by Jeffrey Edwards and Lisa Schurer, "Methods for Integrating Moderation and Mediation: A General Analytical Framework Using Moderated Path Analysis" (link is to the abstract only) is about how to most appropriately combine analysis on moderation and mediation. The article looks a bit tricky - there are lots of equations, but they are not nasty hard equations, and working through them gets what you want.

The second is by Scott Maxwell and David Cole, Bias in Cross-Sectional Analyses of Longitudinal Mediation. is possibly more important, and is about whether mediation analyses are biased, if everything is measured cross-sectionally (i.e. at the same time). The short answer is yes.

This is problematic for many researchers, who measure three things, and then carry out a mediation analysis to show that X causes M and M causes Y. Almost every effect that we are interested in turns out to be biased if you don't measure the effects longitudinally.

Guide to R for SPSS and SAS Users

2007-02-01T03:51:00.000Z

Bob Muenchen, who works in the Statistical Consulting Centre at the University of Tennessee, has put together a guide to using R, for SPSS and SAS Users. I've gone on about R before, on this blog for SEM, here and here; and I've written about how to do regression in R.

The guide is 91 pages long, and focuses on data manipulation more than analysis. Getting info on data analysis in R is pretty straightforward, getting information on manipulation is harder, so this is a welcome addition.

Canonical Correlation

2007-01-26T04:06:00.000Z

Catherine Day sent an email to the psych-postgrads list, which said:

Sorry to bombard you with yet another statistical problem but I'm desperate! I have 2 sets of latent variables (1 set measuring taste preference and another set measuring personality dimensions).
I'm looking at the relationship between taste preference and personality and understand that canonical correlation is the appropriate analysis. The problem is nobody at Sheffield Hallam has performed one before and we do not have the add-on package for SPSS that does this.
Does anyone know how to do this type of analysis or can point me in the
direction of some training?

I replied:

Canonical correlation is one possibility. It's kind of atheoretical though - that is, if you know what the latent variables are, you probably shouldn't use it. You might want to use structural equation modelling or partial least squares analysis instead (although both of those are pretty fiddly, and you shouldn't use them unless you really, really have to).

Just to check: are you modelling at the item level? If you are, I'd consider summing to the scales (maybe factor analysing first) and then doing regression.

There's a book on canonical correlation, in the Sage Little Green Books series. I think that Tabachnick and Fidell's book 'Using multivariate statistics' cover it as well.

Finally, you have got the add on, you just don't realise it. It's an SPSS syntax file, called 'canonical correlation.sps', and you'll find it in the SPSS folder (c:\program files\SPSS, if you're using windows with a locally installed version).

To use it, you type syntax (into the syntax editor) like this:

INCLUDE 'c:\Program Files\SPSS11\Canonical correlation.sps'.
CANCORR set1 = y1, y2, y3/
set2 = x1, x2/.

Where x1 and x2 are predictors, and y1 and y2 are outcomes (just keep going until you've got them all). (I think I've got my x and y the right way around - I don't have SPSS here, so I'm guessing a bit.)

SEM on the cheap

2007-01-07T01:53:00.000Z

Helen M sent a message to the psych-postgrads list, because she didn't have the AMOS package for SEM. And it ain't cheap.

I sent the following reply:

First, there is a program called Mx, which is free, and available from http://www.vcu.edu/mx/ - it was designed for twin studies, but can do any kind of structural equation modelling. It has a path diagram input, but it needs tweaking in syntax, which is a little fiddly - until you get used to it.

Second, there is a program call R, which is also free - it's available from www.r-project.org. R can read an SPSS file, and can do everything you'll ever want to do - it can also read an SPSS file. R comes in packages, and one of the packages is called sem (note - lower case), it's written by John Fox, and can do, SEM (upper case).
It's syntax based, but it's pretty easy - you write y <- x to regress y on x, and y <-> x to correlate y with x. R will download the sem package itself, if you ask it nicely, but the sem home page is here: http://socserv.mcmaster.ca/jfox/Misc/sem/index.html . Also, if you use R, you make life easier for yourself with a program called JGR,
pronounced, JaGuaR - which you can find here: http://rosuda.org/JGR/
(JGR stands for Java GUI for R (hmmm... I'll explain an acronym with
three more abbreviations).

Third, precisely for people in your situation, you can 'rent' a copy
of LISREL, which will do SEM for you - LISREL has something of a bad
reputation, because it was the first package out there, and so people
remember that, but it's much easier nowadays. It costs $65 feeble
American dollars, which works out at about £33 nowadays - or for a
year, it's $100. http://estore.e-academy.com/index.cfm?loc=estore/soft_browse/soft_display_product&ID_Product=525

Fourth, if you've got access to Stata, there's a free Stata add on
called gllamm (generalized linear latent and multilevel models - or
something like that), it's free and it's available from
www.gllamm.org. It can do SEM, but it doesn't seem that
straightforward to me (which means I tried for 5 minutes and gave up).

Fifth, if you're doing something very straightforward, and you're not
bad at Excel, you can do SEM using Excel - although if you want to do
anything more complicated than a one factor CFA, you're in trouble.
And even if you only want to do that, you're in a bit of trouble.
I'm only saying it 'cos I wrote a paper that tells you how to do it.
And no one ever does it. Not even me.

Also, you're going to want some help when you make trivial mistakes
which will drive you insane to spot (comma in the wrong place - things
like that). You can email this list, which might help, the
psych-methods list (on jiscmail as well), semnet - although you will
immediately get bogged down with arguments about arcane details of
your models. If you use sem, you can ask the r-users list (you'll
find stuff on the R web site about that - you get an awful lot of
email on it though).

I'm assuming you've got a book, so I won't go into that now.

Mediation in cluster trials

2006-12-20T06:06:00.000Z

Shocking I know, but I've found out that some people who read this blog don't regularly read Multivariate Behavioral Research. If you're one of those people you won't know that the most recent issue (vol 41, number 3) has a paper about mediation effects in cluster randomized trials. (A cluster randomized trial is one where people are randomized in clusters, such as classes, or schools, or households).

Here's the abstract.

SEM in R (for free)

2006-07-26T10:19:00.000+01:00

One of the problems with trying to learn about structural equation modelling (SEM) is that one needs to have access to a relatively expensive program (Mplus, AMOS, Lisrel, EQS), or one needs to use a free program, such as Mx.

Mx is a great program - it can do all sorts of amazingly clever things, but even the authors would admit that it isn't the world's easiest program to use - in fact, it's possibly got the steepest learning curve of any SEM program.

A solution to this is the package sem, written by John Fox, using R. R is a free and open source program which can read an SPSS data file, and sem is a package (add-on) to it. Learning sem (the package) has always been a bit tricky, because there is a bit of a lack of material to help - the help files (like pretty much all R help files) tell you how everything works, but don't really tell you how to do stuff. That's all been solved by a paper, authored by Fox, in the most recent issue of the journal Structural Equation Modeling (a right riveting read, I'll tell you) (Vol 13, Number 3, pages 465-486). Anyway, it's a nice introduction, and you can read it here.

More on multilevel in SPSS

2006-06-07T12:16:00.000+01:00

Dan Wright sent an email to the psych-methods list, with some websites that he gives to students interested in multilevel models in SPSS. They are here:

http://www.spss.ch/upload/1107355943_LinearMixedEffectsModelling.pdf
http://www.mlwin.com/softrev/reviewspss.pdf
http://epm.sagepub.com/cgi/reprint/65/5/717
http://www.sussex.ac.uk/Users/danw/masters/advanced%20stats/multilevel%20exercise.htm
http://www.ats.ucla.edu/stat/examples/alda/

Huber-White estimates in SPSS

2006-04-13T12:03:00.000+01:00

Statistical tests usually assume that the measurements are independently and identically distributed (i.i.d.). This means that if you know the residual from one person, you should not be able to make any prediction about the residual from any other person. One occasion where this is violated is in cluster randomised trials. In a cluster randomised trial, we don't randomise individually, instead we randomise in groups, or clusters. This happens, for example, when we randomise a hospital to a group, but measure the effects on patients, or if we randomise a teacher to a group, and measure the effect on children.

Cluster randomised trials are becoming more common in medical, and other research, and there is increasing recognition that we need to analyse them in appropriate ways. This can be done in a number of ways. One approach that is commonly used is the Huber-White sandwich estimator, implemented in Stata as 'robust estimates'. It's also possible to use multilevel modelling in SPSS (although it's a bit fiddly - you really need to use syntax) or generalised estimating equations, which will be available in SPSS 15.

However, it's possible to get the same results using the SPSS complex samples procedure, although it's a bit fiddly. I'll demonstrate this using some data that were the basis of a paper by Richards, et al, from the BMJ.

When people telephone their doctor's surgery out of hours, they usually get through to a member of staff at the surgery. In this study, some weeks this remained the case, some weeks they were put through to NHS Direct . One of the outcomes was how much it then cost to treat the person. The assumption of i.i.d. is violated - a person phoning in the same week is likely to be more similar in cost to a person phoning in another week. (For example, if there is a flu bug going around, a lot of people may phone the doctor, to be told that they need to go to bed and take paracetamol, which is cheap. If there is a very high pollen count in a week, people may be hospitalised with asthma, which is expensive.) We don't need to know if this happened, we just need to know if it might have happened.

(Although we could check using the intra class correlation - ICC, sometimes called the intra-unit correlation. I did, and it was 0.02. Not very high, but big enough to matter, as we shall see).

I first did the analysis with SPSS, ignoring the clustering of the data. Here, I find that that the difference between the groups was £2.40 per person, with 95% confidence intervals £0.58 to £4.22, p =0.010, with NHS direct costing more. Thus, we would conclude that there is evidence that people who telephone NHS direct cost more to treat (in total) than people who telephone the surgery.

I then did the analysis with Stata, requesting robust standard errors, with week as a clustering variable - this gives Huber White sandwich estimators. The syntax for this is horribly easy:

regress totcost group, robust cluster(week)

The difference between the two groups is the same - NHS direct cost £2.40 more (which is good, if that wasn't the case, then we would have done something wrong). However, the 95% confidence intervals are now £5.65 to -£0.85, p = 0.141. Ignoring the clustering would have meant that we risked an inflated type I error rate.

I can also do this in SPSS (although it's not widely known, and the SPSS website support doesn't tell you this). We have to use the Complex Samples procedure, which appeared in SPSS 13.0. (And you have to have this module - it's an 'optional extra', as is much of SPSS).

First, we need to create a variable which is full of 1s. The syntax is:

compute constant = 1.

Or you can use the menu commands.

We then choose Analyse, Complex Samples, Prepare for Analysis.

Select 'Create a plan file' (which is the default), click 'Browse' and type a file name (I'll call it example, adn SPSS will add '.csaplan' to it). Click Next.

Choose the constant variable as the Sample Weight (so that this is always 1 - we want everyone to be sampled) and in the cluster variable, put (surprisingly) the cluster variable - this is week. Click Next.

We want sampling with replacement. This is the default on the next screen, so click next.

Then click Finish. SPSS produces a chunk of syntax which looks like this:

CSPLAN ANALYSIS
/PLAN FILE='example.csaplan'
/PLANVARS ANALYSISWEIGHT=constant
/PRINT PLAN
/DESIGN CLUSTER= week
/ESTIMATOR TYPE=WR

Next, we run the analysis. To do this, select Analyse, Complex Samples, General Linear Model.

In the Plan File, we choose the file that we just created, and then click continue.

Put the outcome into the dependent variable box, and the predictor into the covariate box (even though our predictor is categorical I prefer to put it into covariate, because you never know how SPSS is going to code it.)

Click on the statistics button, and ask for model: parameters, standard error, confidence interval and t-test (why oh why oh why would you not want these?). Then click continue and OK.

We get the following chunk of syntax:

CSGLM totcost WITH group
/PLAN FILE = 'example.csaplan'
/MODEL group
/INTERCEPT INCLUDE=YES SHOW=YES
/STATISTICS PARAMETER SE CINTERVAL TTEST
/PRINT SUMMARY VARIABLEINFO SAMPLEINFO
/TEST TYPE=F PADJUST=LSD
/MISSING CLASSMISSING=EXCLUDE
/CRITERIA CILEVEL=95.

The estimate for the difference is £2.40, with 95% CIs £5.65 to -£0.85, and p = 0.141. Which is the same as we got in Stata.

If the residuals are independently distributed, but are not identically distributed (and are still normal) this means that there is heteroscedasticity, and the correction can still be used. In this case, repeat the above procedure, but just don't put anything into the clusters box.

(Veronica Morton helped me with this.)

Interpreting Interactions

2006-03-13T20:35:00.000Z

Daniel Bauer and Patrick Curran have written a paper called Probing interactions in fixed and multilevel regression: Inferential and graphical techniques in the journal Multivariate Behavioral Research which describes a method for probing interactions between continuous variables.

I'll illustrate this in SPSS, using the data file which is used to illustrate interactions in the book Applying regression and correlation. You can get it here (in SPSS format). The dataset comprises the number of books (on statistics) that students read, the number of lectures that they attended, and the final grade they got. To spoil the ending, both number of books they read and number of lectures they attended were positive predictors of final grade. However, in addition, there is an interaction between books and grade.

To discover this, we standardise the two predictor variables and multiply them together to create the interaction variable. We then regress grade onto the standardised books, attendance and the interaction variables.

The results are shown in the following table:

Variable	B	Std. Error	p	Lower 95% CI	Upper 95% CI
Zscore(books)	5.951	2.403	.018	1.076	10.825
Zscore(attend)	5.702	2.404	.023	.827	10.578
Interaction	4.503	2.140	.042	.162	8.844

It's pretty clear that books has an effect, and attendance has an effect. We can interpret these in terms of standard deviations - we predict that a person who reads 1 SD more books will (controlling for attendance and the interaction term) achieve a grade that is 5.95 points higher. We predict that a person who attends 1 SD more lectures will achieve a grade that is 5.70 points higher.

And we predict something from the interaction term, but we aren't exactly sure what yet.

One way to see what we do predict from the interaction term is to 'pick a point'. Choose arbitrary values for attendance (say) plug them into the equation and see what happens.

Let's do that first. Because we don't like stuff to be hard, we will pick a low, medium and high number of attendances. The easiest way to do this is to use z-scores -1, 0 and 1, respectively.

Then, our equation looks like this:

Pred Grade = books * 5.95 + attendance * 5.70 + interaction * 4.50

We can expand this - because we know that the interaction is equal to books * attendance

Pred Grade = books * 5.95 + attendance * 5.70 + books * attendance * 4.50

Now we just stick -1, 0 or 1 into attendance, for a low medium and high attender.

Low (-1)
Pred Grade = books * 5.95 + -1* 5.70 + books * -1 * 4.50

Notice a couple of times we multiply a value by -1. Multiply those out.

Pred Grade = books * 5.95 + -5.70 + books * -4.50

Now notice that we have boos * 5.95 + books * -4.50 - that makes books * (5.95 - 4.50) which makes books * 1.45

Pred Grade = books * 1.45 + -5.70

And, as we aren't interested in the constant, we can lose anything that's not multiplied by books.
Pred Grade = books * 1.45

Same again for medium (this one's easier, because we just multiple by zero a lot).
Pred Grade = books * 5.95 + 0* 5.70 + books * 0 * 4.50
Pred Grade = books * 5.95

Same again for high:
Pred Grade = books * 5.95 + 1* 5.70 + books * 1 * 4.50
Pred Grade = books * 10.45

So, if you don't attend many lectures, reading another SD of books is going to help you a bit, but only a bit - your predicted grade only goes up by 1.45. If you are a moderate attender, you increase your predicted grade by 5.95 for the same number of books, and if you are a high attender, your predicted grade increases by 10.45 for the same number of books. (Notice that we are always saying predicted grade - we don't know that this is the cause).

This is all well and good, but we might want stuff like p-values on these, or standard errors, or confidence intervals. We might say that for the low attender the relationship isn't even statistically significant - reading more books doesn't help (because the low attenders might read the wrong books). This is where Bauer and Curran's paper comes in.

I'm not going to go into the nitty gritty. I'm just going to show what you can do with it. I'm going to use some calculators that you can find here. (But in the future, it looks like they might be here, and I'll probably forget to change this.)

The most exciting piece of output is this graph (and the tools on the web let you draw it; click on the graphs for bigger, clearer images).

The graph shows, on the x-axis, the (z-score of) the number of lectures attended by the student. On the y-axis is the simple slope of the (z-score of) books read. We evaluated that slope at -1, 0 and +1. We could do that by looking at those positions on the x-axis, tracking up to the black like, and reading off the values on the y-axis. The values appear to be where we said they would be - I've drawn in the lines on the next chart - remember our earlier calculations showed that they should be at 1.45, 5.95 and 10.45.

However the more exciting thing is the red lines. These show the confidence intervals around the slope. We said that if lecture attendance was low, we might expect that the effect of reading more books was not statistically significant. This does appear to be the case - the confidence intervals of the slope include 0 when the number of lectures attended is low.
Finally, the vertical dotted line shows the (z) value of lectures at which the p-value of the slope becomes statistically significant. This line appears at -0.21. In other words, if the z-score of the number of books read is below -0.21, we do not have evidence that reading more books will help improve the person's predicted grade.

Correlation and causation

2005-12-19T14:19:00.000Z

A nice example of the failure to distinguish between correlation and causation on the badscience.net website. The Daily Mail (which I'll decline to link to) says that moderate drinkers are thinner than non-drinkers, therefore drinking makes you thin.

It's possible to think of a very large number of reasons that this relationship might exist. My top choice would be age - older people tend to drink less (drinking peaks at around age 21 - you'll have to take my word that I have read that somewhere reputable). Older people are more likely to be overweight than younger people. Maybe they've got their causation reversed - fatter people are less likely to drink than thinner people, because they are trying to lose weight.

It's also possible to think of one very good reason that this is not a causal relationship. Take two identical people, make them do exactly the same things. Make them eat the same foods. Make one of them drink water. Make one of them drink wine. The one that drinks wine will become heavier than the one that doesn't - it's the laws of physics.

However, when someone makes a causal statement based on a correlation, it's not for us, the readers, to find reasons it may not be true. It is for them to present a convincing argument. Saying "smokers get more cancer, because more smokers get cancer than non-smokers" is not a convincing argument, until a lot more evidence is collated. This is discussed in statistics as principled argument by Robert Abelson, and also in Applying regression and correlation, by me and Mark Shevlin.

Freakonomics

2005-11-08T22:07:00.000Z

I started to read the book Freakonomics, a book that's caused quite a stir (although it passed me by). The very first example it gives is that of a paper by Donohue and Levitt in which they claim that legalising abortion in the early 1970s cut crime in the 1990s - the type of person that has an abortion is the type of person who has children who then commit crimes. Basically, the criminals of the future were never born, so Donohue and Levitt claim.

Goertzel has used this very example in a paper, published in the Skeptical Enquirer of Econometrics as Junk Science. Donohue and Levitt used multiple regression, to find the effects of legalising abortion, controlling for other variables. But then N is, effectively, 1. Abortion was only legalised once - and that means that the correlation with any other event is 1.00, and the data are unanalysable.

As Geortzel says in his paper: "[this is] a way to model an inadequate body of data. Econometrics cannot make a valid general law out of the historical fact that abortion was legalized in the 1970s and crime went down in the 1990s. We would need at least a few dozen such historical experiences for a valid statistical test."

Apart from that, I rather liked the book. It did have some interesting examples of multiple regression, although it didn't really label them as such.

SPSS User Group Meeting

2005-08-25T10:22:00.000+01:00

The SPSS User Group Annual Conference will be held in York in November this year. Diana Kornbrot will be talking about presenting results in logistic regression.

Longitudinal multilevel models in SPSS

2005-08-24T21:28:00.000+01:00

In Version 11, SPSS added a multilevel modelling (they call it mixed models) procedure. I find the menus completely incomprehensible - I've tried intelligently selecting the appropriate options, and I've tried randomly guessing, and neither gives me the right answer.
Instead, you have to use syntax - and so here, to get you (and me) started, is the syntax for longitudinal multilevel models in SPSS.
I'm assuming you've a variable called time, which represents, well, time. A variable called outcome, which represents the outcome, and a variable called subject, this is (you've guessed it) the subjects.

Your data will look something like:

Time Outcome Subject

1 10 1
2 12 1
3 15 1
1 11 2
2 15 2
2 14 3
3 17 3

...

3 18 200

If you only want random intercepts:

MIXED
outcome with time
/random = intercept | subject (id)
/print = solution.

For random intercepts and slopes, with a zero correlation between slope and intercept):

MIXED
outcome with time
/fixed = time
/random = intercept time | subject (id)
/print = solution.

(Note that the (id) specifies the form of the correlation matrix - id is short for identity.

And for random slope, random intercepts, and correlation between slope and intercept:

MIXED
outcome with time
/fixed = time
/random = intercept time | subject (id) covtype (un)
/print = solution.

The covtype (un) is unstructured.

A much more detailed and useful guide to mixed models in SPSS, written by Alastair Leyland, can be found here.

Extrapolating too far

2005-08-10T22:35:00.000+01:00

I came across a great example of extrapolation beyond the data, on the medstats mailing list.

It's about how to cook a turkey, by dropping it out of the window. Lots of times. I did a bit of googling, but I couldn't find any other reference to it. The link in medstats is here. If you know more, email me.

Incidentally, the Journal of Irreproducible Results became the Annals of Improbable Research.

Website updates

2005-08-09T13:45:00.000+01:00

Some updates have been made to the website. The front page http://www.jeremymiles.co.uk/ has been updated. Errors and omissions has been updated. Further reading has been updated, with a couple of new books, and new editions of some books.

Introducing the Blog

2005-08-09T11:17:00.000+01:00

This is the Applying Regression blog, which is going to contain stuff about updates to the book Applying Regression and Correlation, by Jeremy Miles and Mark Shevlin.

More information on the book.