Online Workshop:

Effectively Dealing with Missing Data without Biasing your Results

Learn the new approaches to missing data so you can confidently analyze your data with no loss of power, accurate p-values, and unbiased results.

If you have any experience with missing data, you know it really messes analyses up:

• You lose power when cases get dropped.
• Results become biased when representative cases aren't included.
• Standard errors and p-values become too small if you lose cases or estimate missing values.

But there is something unique about the way it messes analyses up.

It’s not a data issue like skewness or outliers that you can just ignore (whether you should or not).

Ignoring missing data still means choosing a method of dealing with missing data–-you’re just using the default. The default method in most statistical software is listwise deletion--drop any case with any value missing.

Depending on which statistical software you’re using and the patterns and percentage of missing data, the default may be a perfectly acceptable way of dealing with the missing data.

Or it may be the worst possible option.

And in data analysis, it’s always better if you understand the defaults, know what they’re doing in your data set, and decide for yourself if it’s the best approach.

Up until about 10 years ago, there weren’t many other options.

There was listwise deletion and there was imputation (inserting estimates for the missing values).

Listwise deletion can decimate your sample, your power, and bias results.

Imputations solved the power issue, but most of the imputation methods were pretty sketchy, and they biased both overall results and p-values even worse.

It was a “damned if you do, damned if you don’t” kind of situation.

But it’s different now.

In August 1999, just a month after I started at the Statistical Consulting office at Cornell, I saw a talk by Joe Schaefer at the Joint Statistical Meetings about multiple imputation. I was blown away. It seemed too good to be true–it solved pretty much all of the problems with missing data.

So I read all that I could, attended a week-long mini-class, and tried it all out.

At that time, you had to use special stand-alone software to implement it, and all the ones I tried were a bit clunky to use.

Luckily, statistical software has caught up. And in that time, a few new studies have shown that some of the restrictive assumptions of multiple imputation aren’t as restrictive as they at first seemed. So multiple imputation is easier and more accurate than ever.

It’s also become clear that some of those old methods aren’t always as horrible as they seemed–there are some situations when listwise deletion works just fine.

But it pays to know the difference, and how to implement not just multiple imputation, but maximum likelihood approaches, which also give great outcomes and are quite a bit easier to use.

That is what you'll learn in this workshop--the issues involved in missing data, indepth understanding of the approaches and how to implement them, and the steps to diagnose the best approach in your situation.

In this workshop, you will learn:

Module 1: Missing Data–The Problem and Basic Solutions

Part 1. What is Missing Data?
Part 2. Missing Data Mechanisms
Part 3. The Four Main Approaches
Part 4. Complete Case Analysis
Part 5. Imputation

In this first module, you'll get the big picture. The real issues, causes, and the solutions.

You learn step by step what the different mechanisms are--exactly how random the missingness is, and how that affects your results.

You'll get an understanding of where missing data fits in to an analysis strategy and its relationship to other types of problem data--censoring, truncation, and other partial information.

And finally, we'll explore two traditional, simple techniques for dealing with missing data--complete case analysis and imputation. They do work in some situations, but they're disasters in others. You will learn how to tell the difference, and how to use them well.

Module 2: Multiple Imputation

Part 1: What is Multiple Imputation: The Concept
Part 2: When to Use it
Part 3: How to Do it, Step-by-Step, in SPSS and SAS

Mulitiple Imputation is a godsend in some really hairy missing data situations. Even with up to 50% of data missing, it can give you unbiased parameter estimates, standard errors, and full power.

But it has to be done well, and that's not always easy. It requires a solid imputation algorithm and model.

This module will teach you, in detail, how to build an imputation model, how it differs from your analysis model, and what to do with the resulting imputed data.

Module 3: Multiple Imputation in Practice--Special Cases

Part 1: Multiple Imputation for Categorical Variables
Part 2: Imputation of the Dependent Variable
Part 3: The Role of Interaction Terms and Transformations in Imputations
Part 4: Imputing Scales or Scale Items

Multiple Imputation is very simple if only one predictor variable is missing data, it is highly correlated with other variables, and if it is continuous and normally distributed.

But real data is never so clean.

Luckily, multiple imputation can handle a lot of mess. So in this module, we'll explore how to do mulitple imputaton in many messy situations.

So you will know how to make solid analysis decisions even with messy data.

Module 4: Maximum Likelihood and NonIgnorable Missing Data

Part 1: Maximum Likelihood Approaches
Part 2: Non-Ignorable Missing Data

Multiple Imputation isn't the only game in town. There are a number of Maximum Likelihood techniques for running models that have all the advantages of Multiple Imputation without the hassle of imputing anything.

You may already be using some of them. And if you're running linear models, you can take advantage of these techniques right as you run your models. No extra steps required.

It's actually quite easy to do. But it only works for linear models.

So in part 1 you'll learn what maximum likelihood estimation is, the types of analyses for which it works, and the exact steps to implement it.

Then in part 2, we'll briefly discuss the approaches available for non-ignorable missing data. This is where you really have to make some crazy assumptions because the approaches require you to know something about the missing values.

Module 5: Missing Data Diagnosis

Part 1: Decision Factors in Choosing an Approach
Part 2: Missing Data Diagnosis, Step-by-Step
Part 3: Conclusions

Part of the reason it is so hard to learn how to deal with missing data is that the right approach depends on how much data are missing, patterns of missing data, why the data are missing, and how you will use the data in analysis.

These all vary in different types of research.

Learning how to analyze the patterns and reasoning for choosing an approach may be the most important part of the workshop.

This is actually the first step in dealing with missing data, but we save it for last so you have a clear picture of what your options are once you do the diagnosis.

So in this module, you'll learn, in detail, how to analyze the data and the patterns of missingness to figure out the most likely mechanism, the effects of the missing data, and the best way to proceed in dealing with it.

This workshop is for you if you:

• Have struggled with the devastating loss of power that comes from missing data
• Realize that listwise deletion and mean imputation don't usually work well, and are looking for a better way
• Have heard about the amazing miracle of multiple imputation and want to learn what it is and how to do it
• Have struggled with using multiple imputation and realize that it can be quite difficult to implement well. You want to know when is it really necessary, and when (and how) can you use Maximimum Likelihood instead, which is both simple and powerful

Prerequisites:

• You will get the most out of the workshop if you have had at least two statistics classes and at least two years experience in data analysis.
• You use SAS or SPSS. You are welcome to use another software package, but I am only familiar with using Missing Data procedures in these software packages. I know that R, S-Plus, and Stata have good multiple imputation capabilities, and I have used these packages before, so I'm mildly familiar with them, but I don't know the specifics.
  
  Important: SPSS can only do multiple imputation in version 17.0 and higher, but there is a work-around for earlier versions, which I will show you. For any of the SPSS work, you will need to have the missing values add-on module. If you have it, "Missing Values" will appear in your Analyze menu. If you don't and are employed by a university, you can get a one-year license for Windows or Mac to the full SPSS suite, including all their modules at On the Hub. Note that the Grad Pack does NOT contain the Missing Values Module, but the Faculty Pack does.
  
  
• We will use AMOS for the Full Information Maximum Likelihood. AMOS now comes bundled with most versions of SPSS. No prior experience using AMOS is necessary. Full Information Maximum Liklihood can also be run in Lisrel and MPlus, but I do not have experience with those programs.

The Instructor:

Karen Grace-Martin is a statistical trainer and consultant and an expert on missing data, SPSS, and SAS.

She has guided and trained researchers through their statistical analysis for over 15 years. Her focus is on helping statistics practitioners gain an intuitive understanding of how statistics is applied to real data in research studies.

Comments from past workshop participants:

We've designed the format to support your learning and convenience:

Web based online workshop: The workshop consists of five modules.

Each week, we'll release one module of training materials, then follow up with a live Question & Answer session at the end of the week.

You can master that material and have all your questions answered before moving on.

For each module, you'll get:

1. Training Materials

• one or more training videos to download and watch on your own time (and keep forever). The videos are screen shots of what is happening on my computer screen. They include both slide presentations as I go over the concepts and SPSS and SAS demonstrations.
  
  
• pdf handouts of the presentation slides for note-taking
  
  
• exercises to practice what you've learned, with syntax and answers to check your work.
  
  
• written, searchable pdf enhanced transcripts of all the videos. So if you learn better by reading, or just want to look something up, you have the material in the format that works best for you. These enhanced transcripts are edited for easier reading and contain slides and screen shots, so you get all the visuals.
  
  
• recommended resources and references to cement and expand your learning.

2. Live Q&A Webinars

About a week after we release the training materials, we'll meet in a live Q&A webinar.

You attend over the internet. Audio is through either your computer speakers/microphone or by telephone. Ask questions out loud or write it into the chat.  You'll be able to see my screen, so I can show you how to do something or we can review the output.

I highly encourage you to attend the Q&A Sessions live, but if you have to miss one (it happens) they will be recorded and made available within 24 hours on our workshop website. Just submit a question before a Q&A and I'll cover it in the session. Where do you submit it? On our...

Private workshop website

We've also created a workshop site that you become a member of. For a year.

The site is our home-base for the workshop. It's where you'll find everything you need to support your learning of each module:

• SPSS, SAS, and excel data files. You get full access to use the data to try everything I demostrate in the workshop and try things on your own.
  
  
• SPSS and SAS syntax code to run and explore all of the examples yourself. You won't learn it unless you try it. So I'm giving you my syntax so you can see exactly how I got all the output I'm showing you.
  
  
• PDF handouts of the presentation slides, which will be available ahead of each session, on which you can take notes.
  
  
• Video Screenshot recordings of each workshop session made available within 48 hours. So you can review the material right away, a year later, or while doing the exercises. Or if you need to miss one (or all!) of the live sessions, you can still participate on your own schedule.
  
  
• Exercises.  (Yes, homework!).  You really need to practice this stuff. Get your hands dirty. So we're giving you the data to try it on your own. But don't worry--you won't be stuck on some coding error that won't work. I'm also giving you the syntax I used to do the exercises and the answers.
  
  
• A place to submit written questions between sessions. So as you're reviewing videos afterward (or if you missed a live session), just submit a question. I'll answer it there if I can, or if it's something I need to show you, I'll answer it in the next session.
  
  
• Video Recordings of the Q&A sessions. If you have to miss one, no problem. Just submit your questions ahead of time and I'll answer in the Q&A. Just watch it later.
  
  
• A list of helpful resources and suggestions for further reading. There are definitely some good books and articles out there on this topic, and you will learn the techniques better with some background reading. I'm selecting the sections that best explain each topic, and I'll make the list available when you first register, so you have time to interlibrary loan them.
  
  
• Bonus videos. I've included a number of videos from some webinars on relevant topics to help jog your memory or clear up misunderstandings. Included are:
  • What Happened to R squared?: Assessing Model Fit for Logistic, Multilevel, and Other Models that use Maximum Likelihood
  • The 11 Steps to Performing any Statistical Model
  • Random Intercept and Random Slope Models
    

But the biggest benefit of the membership site is you have help for a year.

Travelling to a live workshop is expensive. Not just the workshop itself, but the flight, the hotels, the meals. Plus you have to take a few days off work.

But I always found the worst part about travelling to a live workshop is that you learn so much all at once, then go home to your busy life. You have to catch up on everything you just missed. You don't have a chance to implement what you've learned right away. Then you lose it.

And you're on your own. When you do go to implement it, new questions come up. But the instructor is no longer available. And it doesn't feel right to send an email.

But on our workshop website, you get support for a year.

Watch the videos when it's convenient. Watch them again when a new issue comes up.

Ask questions as they come up for you.

And as we add resources, and answer more questions, you have access to the updated material.

So you really learn it. Inside out.

Cool, huh?

It's all the advantages of a live workshop without the disadvantages.

The Details:

Registration for this workshop is closed. If you'd like a notice when we offer this workshop again, join our Advance Discount List by filling in your name and email address below.