Simon Cheng

Assistant Professor

Department of Sociology

University of Connecticut

Paper

Cheng, Simon and J. Scott Long.  forthcoming.  “Testing for IIA in the Multinomial Logit Model.”  Sociological Methods & Research.  PDF file

Abstract 

The multinomial logit model is perhaps the most commonly used regression model for nominal outcomes in the social sciences. A concern raised by many researchers, however, is the assumption of the independence of irrelevant alternatives (IIA) that is implicit in the model. A variety of tests have been developed to test IIA and two recent articles by Fry and Harris (1996, 1998) have explored the statistical properties of these tests. In the current article, we undertake a series of Monte Carlo simulations to evaluate the three most commonly discussed tests of IIA. The results suggest that the size properties of the IIA tests depend upon the data structure for the independent variables. With some structures, the size properties are reasonable, while in others they are extremely inflated. These findings are consistent with our earlier impression that, even in well-specified models, the IIA tests often reject the assumption when the alternatives seem distinct and that they do not reject IIA when the alternatives can reasonably be viewed as close substitutes. We conclude that tests of the IIA assumption that are based on the estimation of a restricted choice set are unsatisfactory for applied work. 

Files for MC simulation analyses, performed in Stata 8.

§        iiaado.zip: contains the ado files used by Stata to run the simulations.  These files should be placed in the same folder as the do files below so that Stata can load them as needed.

§        iiado.zip: contains the Stata programs (.do files) used to generate data, run the simulations, and summarize the results.

§        iialog.zip: contains the Stata output logs (.log files) used in the paper.

 

Documentation note:  With the exception of 11, all syntax files must run in sequence.

 

§         Generate raw data

 

   01_109_rawgen.do // Create data set 109_raw == paper data # 1

   02_110_rawgen.do // Create data set 110_raw == paper data # 2

   03_111_rawgen.do // Create data set 111_raw == paper data # 3

   04_112_rawgen.do // Create data set 112_raw == paper data # 4

   05_101_rawgen.do // Create data set 101_raw == paper data # 5

   06_107_rawgen.do // Create data set 107_raw == paper data # 6

   07_103_rawgen.do // Create data set 103_raw == paper data # 7

   08_106_rawgen.do // Create data set 106_raw == paper data # 8

 

§         Check raw data sets; must run 01-08 first

 

   11_rawcheck.do

 

§         MC Simulation: with fixed df for SH; must run 01-08 first

 

♦    Reported in the paper, resample if singular.

   21_109_monte.do // Sample size = 150 250 350 500 1000 2000

   22_110_monte.do // Sample size = 150 250 350 500 1000 2000

   23_111_monte.do // Sample size = 150 250 350 500 1000 2000

   24_112_monte.do // Sample size = 150 250 350 500 1000 2000

   25_101_monte.do // Sample size = 150 250 350 500 1000 2000

   26_107_monte.do // Sample size = 150 250 350 500 1000 2000

   27_103_monte.do // Sample size = 150 250 350 500 1000 2000

   28_106_monte.do // Sample size = 150 250 350 500 1000 2000

 

♦    Supplementary analysis, resample if singular.

   35_101_monte.do // Sample size = 500 1000 2000 3000 4000 5000

   36_107_monte.do // Sample size = 500 1000 2000 3000 4000 5000

   37_103_monte.do // Sample size = 500 1000 2000 3000 4000 5000

   38_106_monte.do // Sample size = 500 1000 2000 3000 4000 5000

 

♦    Supplementary analysis [p.13 in "IIA_NoName.pdf"], resample if singluar

   395_101_monte.do // Sample size = 1000 2000 4000 6000 8000 10000

   396_107_monte.do // Sample size = 1000 2000 4000 6000 8000 10000

   397_103_monte.do // Sample size = 1000 2000 4000 6000 8000 10000

   398_106_monte.do // Sample size = 1000 2000 4000 6000 8000 10000

 

§         Merge results from simulations, with change in df for small-hsiao test

 

♦    Merge results from simulations 21 to 28, with change in df for SH test

   41_merge.do 

 

♦    Renorm mtt tests; empirical critical values by data structure

   42_mergeMTT.do

 

♦    Merge results from simulations 35 to 38, with change in df for SH test

   43_mergeLarge.do

 

·    Supplementary analysis

 

♦    Merge results from simulations 1k-10k, with change in df for SH test

♦    Supplementary analysis [page 13 in "IIA_NoName.pdf"]

   44_mergeLarge2.do 

 

§         Other statistics reported in the paper, pp.10 & 11 in [IIA_NoName.pdf]

 

   50ReportedStatistics.do

 

§         Plot the results

 

·    Presented in the paper

 

♦    HM % reject; data with x2 as continuous

♦    Generate [51_hmreject_x2con.wmf]

♦    Figure 1 in the paper

   51_hmreject_x2con.do

 

♦    SH % reject; data with x2 as continuous

♦    Generate [52_shreject_x2con.wmf]    

♦    Figure 2 in the paper

   52_shreject_x2con.do

 

♦    HM & SH % reject; severe size distortion

♦    Generate [53_hmshreject7_103.wmf]

♦    Figure 3 in the paper

   53_hmshreject7_103.do

 

♦    SH % reject; data with x2 as dummy

♦    Generate [54_shreject_x2dum.wmf]

♦    Figure 4 in the paper

   54_shreject_x2dum.do

 

·    Supplementary analysis not presented

 

♦    SH % reject; data with x2 as dummy, N = 500 to 5000

♦    MC with resampling until nonsingular sample

   55_shreject_x2dum.do

 

·    Supplementary analysis [page 13 in "IIA_NoName.pdf"]

 

♦    SH % reject; data with x2 as dummy, N = 1,000 to 10,000

♦    MC with resampling until nonsingular sample

   56_shreject_x2dum.do

Last modified on April 18, 2006

 

*-*