Simon Cheng

Assistant Professor

Department of Sociology

University of Connecticut


Paper

Cheng, Simon and J. Scott Long. forthcoming. Testing for IIA in the Multinomial Logit Model. Sociological Methods & Research. PDF file


Abstract

The multinomial logit model is perhaps the most commonly used regression model for nominal outcomes in the social sciences. A concern raised by many researchers, however, is the assumption of the independence of irrelevant alternatives (IIA) that is implicit in the model. A variety of tests have been developed to test IIA and two recent articles by Fry and Harris (1996, 1998) have explored the statistical properties of these tests. In the current article, we undertake a series of Monte Carlo simulations to evaluate the three most commonly discussed tests of IIA. The results suggest that the size properties of the IIA tests depend upon the data structure for the independent variables. With some structures, the size properties are reasonable, while in others they are extremely inflated. These findings are consistent with our earlier impression that, even in well-specified models, the IIA tests often reject the assumption when the alternatives seem distinct and that they do not reject IIA when the alternatives can reasonably be viewed as close substitutes. We conclude that tests of the IIA assumption that are based on the estimation of a restricted choice set are unsatisfactory for applied work.


Files for MC simulation analyses, performed in Stata 8.

        iiaado.zip: contains the ado files used by Stata to run the simulations. These files should be placed in the same folder as the do files below so that Stata can load them as needed.

        iiado.zip: contains the Stata programs (.do files) used to generate data, run the simulations, and summarize the results.

        iialog.zip: contains the Stata output logs (.log files) used in the paper.

 


Documentation note: With the exception of 11, all syntax files must run in sequence.

 

         Generate raw data

 

01_109_rawgen.do // Create data set 109_raw == paper data # 1

02_110_rawgen.do // Create data set 110_raw == paper data # 2

03_111_rawgen.do // Create data set 111_raw == paper data # 3

04_112_rawgen.do // Create data set 112_raw == paper data # 4

05_101_rawgen.do // Create data set 101_raw == paper data # 5

06_107_rawgen.do // Create data set 107_raw == paper data # 6

07_103_rawgen.do // Create data set 103_raw == paper data # 7

08_106_rawgen.do // Create data set 106_raw == paper data # 8

 

         Check raw data sets; must run 01-08 first

 

11_rawcheck.do

 

         MC Simulation: with fixed df for SH; must run 01-08 first

 

    Reported in the paper, resample if singular.

21_109_monte.do // Sample size = 150 250 350 500 1000 2000

22_110_monte.do // Sample size = 150 250 350 500 1000 2000

23_111_monte.do // Sample size = 150 250 350 500 1000 2000

24_112_monte.do // Sample size = 150 250 350 500 1000 2000

25_101_monte.do // Sample size = 150 250 350 500 1000 2000

26_107_monte.do // Sample size = 150 250 350 500 1000 2000

27_103_monte.do // Sample size = 150 250 350 500 1000 2000

28_106_monte.do // Sample size = 150 250 350 500 1000 2000

 

    Supplementary analysis, resample if singular.

35_101_monte.do // Sample size = 500 1000 2000 3000 4000 5000

36_107_monte.do // Sample size = 500 1000 2000 3000 4000 5000

37_103_monte.do // Sample size = 500 1000 2000 3000 4000 5000

38_106_monte.do // Sample size = 500 1000 2000 3000 4000 5000

 

    Supplementary analysis [p.13 in "IIA_NoName.pdf"], resample if singluar

395_101_monte.do // Sample size = 1000 2000 4000 6000 8000 10000

396_107_monte.do // Sample size = 1000 2000 4000 6000 8000 10000

397_103_monte.do // Sample size = 1000 2000 4000 6000 8000 10000

398_106_monte.do // Sample size = 1000 2000 4000 6000 8000 10000

 

         Merge results from simulations, with change in df for small-hsiao test

 

    Merge results from simulations 21 to 28, with change in df for SH test

41_merge.do

 

    Renorm mtt tests; empirical critical values by data structure

42_mergeMTT.do

 

    Merge results from simulations 35 to 38, with change in df for SH test

43_mergeLarge.do

 

    Supplementary analysis

 

    Merge results from simulations 1k-10k, with change in df for SH test

    Supplementary analysis [page 13 in "IIA_NoName.pdf"]

44_mergeLarge2.do

 

         Other statistics reported in the paper, pp.10 & 11 in [IIA_NoName.pdf]

 

50ReportedStatistics.do

 

         Plot the results

 

    Presented in the paper

 

    HM % reject; data with x2 as continuous

    Generate [51_hmreject_x2con.wmf]

    Figure 1 in the paper

51_hmreject_x2con.do

 

    SH % reject; data with x2 as continuous

    Generate [52_shreject_x2con.wmf]

    Figure 2 in the paper

52_shreject_x2con.do

 

    HM & SH % reject; severe size distortion

    Generate [53_hmshreject7_103.wmf]

    Figure 3 in the paper

53_hmshreject7_103.do

 

    SH % reject; data with x2 as dummy

    Generate [54_shreject_x2dum.wmf]

    Figure 4 in the paper

54_shreject_x2dum.do

 

    Supplementary analysis not presented

    SH % reject; data with x2 as dummy, N = 500 to 5000

    MC with resampling until nonsingular sample

55_shreject_x2dum.do

 

    Supplementary analysis [page 13 in "IIA_NoName.pdf"]

 

    SH % reject; data with x2 as dummy, N = 1,000 to 10,000

    MC with resampling until nonsingular sample

56_shreject_x2dum.do


Last modified on April 18, 2006

 


*-*