Differential item functioning analysis of the herrmann brain dominance instrument jared a. Differential item functioning responsive translation. Assessment developers design and construct questionnaires or tests including sets of items that measure, for example, cognition, personality traits, or political views. Implications for fairness issues in test development are also discussed. A general framework and an r package for the detection of. Item response theory detects differential item functioning between healthy and ill children in qol measures. An r package for rajus differential functioning of. Differential item functioning analysis with ordinal logistic. Item response theory detects differential item functioning between healthy and ill children in qol measures michelle m. Differential item functioning dif of sf12 and qlesq. Differential item functioning analysis with ordinal logistic regression techniques difdetect and difwithpar paul k. Judicious application of this methodology by the researchers, however, requires an.
An r package for rajus differential functioning of items and tests framework. Average item scores for subgroups having the same overall score on the test are compared to determine whether the item is measuring in essentially the. Differential item functioning analysis of the herrmann brain. A powerful methodology for identifying biased items is the differential item functioning, or dif, study. White males were the reference group, and the focal groups were. Once differently functioning items are identified, adjustments can be made. We analyzed 95 cognitive reading items, administered to students in 29 european countries. Differential item functioning dif is an important issue of interest in psychometrics and educational measurement. Software for analyzing differential item functioning using the mantelhaenszel and standardization procedures. As is commonly done, the total score was used as the matching. An item displays dif when test takers possessing the same amount of an ability or trait, but belonging to different subgroups, do not share the same likelihood of correctly answering the item. Starting from a framework for classifying dif detection methods and from a comparative overview of the most traditional methods, an r.
Differential item functioning analysis of the herrmann. Gibbons, phd, lance jolley, ms, and gerald van belle, phd introduction. The multipleindicator multiplecause mimic model was utilized to assess differential item functioning dif, comparing english versus french and versus dutch patient responses separately. Differential item functioning dif has been widely used in healthcare, business management, and educational measurement. Recommendations for conducting differential item functioning.
The use of new item formats, new item types, and new test. As demonstration, a dataset from a 40item math test in a statewide testing program is analyzed by two widely. The results of dif studies can be followed by investigations into the sources of, and. Sep 01, 2017 this is the webpage for the handbook on differential item functioning. Table 30 supports the investigation of item bias, differential item functioning dif, i. Traditional methods for evaluating differential item functioning do not control for potentially important pretest differences between groups, which may confound results. Maybe one could use them in modeling project planning where there are many components whose development influence the development of oth.
Differential item functioning dif detection rates were examined for the logistic regression and analysis of variance anova dif detection methods. A handbook on the theory and methods of differential item functioning dif. Pdf an introduction to differential item functioning. A nonparametric bayesian diagnostic for detecting differential item functioning in irt models mark e. The comparability of english, french and dutch scores on the. The mh dif method was conducted using the differential item functioning analysis system difas computer program penfield, 2005.
Differential item functioning dif occurs when examinees with the same ability have different probabilities of endorsing an item. Potential sources of differential item functioning in the adaptation of tests paula elosua and alicia lopezjauregui department of psychology university of the basque country, spain this report shows a classification of differential item functioning dif sources that have an effect on the adaptation of tests. This study focuses on the use of multipleindicator. Researchers are examining a restricteduse dataset from the early childhood longitudinal study, kindergarten class of 199899. Differential item functioning dif is the preferred psychometric term for what is otherwise known as item bias. A variety of statistical procedures have been developed to assess dif in tests of dichotomous hills, 1989. This article provides a brief primer overview of differential item functioning dif analysis. Current issues 951 people who answered the item correctly at the ability level m, and the proportion of people who answered the item correctly at the ability level m, respectively. The majority of items displayed little or no differential item functioning dif. The purpose of the present analysis is to use differential item functioning dif to identify differences in the performance of native and immigrant students in pisa 2009 that can be directly related to their responses to particular items. Research seeks to boost stem proficiency in latino youth. Differential item functioning magnitude and impact measures. Shuang wang financial data analyst, cloud data developer. In studying dif, members of the focal group and the reference group should be comparable.
Partnered with differential to engage their community through the worlds first spiritualfitness app. Differential item functioning analysis with ordinal. When items function differently for different populations, it is referred to as dif. Dif analysis investigates a differential characteristic of a test item between subpopulations of examinees and is useful in detecting possibly biased items toward a particular subpopulation. Figure 1 displays a scatterplot for the males and females item difficulties.
The comparability of english, french and dutch scores on. Differential item functioning dif is when a test item favors or hinders a characteristic exhibited by group members of a testtaking population. As far as i know, there is no application of differential equations in the discipline of software engineering. Partnered with differential to drive sales through a custom platform tailored to their sales team and process. Assessing dif helps to understand the nature of instruments, to assess the quality of a measure and to interpret results. Differential item functioning dif has been increasingly applied in fairness studies in psychometric circles. Differential item functioning magnitude and impact. Differential item functioning analysis of the herrmann brain dominance instrument jared andrew lees brigham young university provo follow this and additional works at.
This study aimed to examine whether the items of the quality of life enjoyment and satisfaction. Chi square procedures test a hypothesis, but do not produce a parametric measure of the amount of dif exhibited by the studied item. X fits an item response model when x are item scores e. If dif is found for many items on the test, the final test scores do not represent the same. Item response theory irt methodologies were employed in order to examine the myersbriggs type indicator mbti for differential item functioning dif on the basis of crossed gender and ethnicity variables.
Logistic regression modeling as a unitary framework for binary and likerttype ordinal item scores. The preferred, more neutral, term is differential item functioning dif. An introduction to differential item functioning analysis eric. Differential item functioning between ethnic groups in the epidemiological assessment of depression. A unidimensional factor model showed good fit in all samples.
This issue, known as test bias, has been the subject of a great deal of recent research, and a technique called differential item functioning dif analysis has become the new standard in psychometric bias analysis. Pmc free article broekman bfp, nyunt sz, niti m, jin az, ko sm, kumar r, ng tp. Partnered with differential to automatically track strokes and deliver insights so their customers can improve. Shuang wang senior software engineer at coupang my team is hiring greater seattle area. Software for the computation of the statistics involved in item response theory likelihoodratio tests for differential item functioning, 2001, unpublished manuscript to complete dif analyses. In brief, differential item functioning dif occurs when groups such as defined by gender, ethnicity, age, or education have different probabilities of endorsing a given item on a multiitem scale after controlling for overall scale scores. For example, englishtospanish translations of math wordproblems or. Item response theory, classical test theory, differential item functioning, factor analysis, multilevel modeling, latent growth analysis, latent profile analysis. Differential item functioning differential item functioning dif analysis can be used to examine whether items function similarly across different groups and identify items that appear to be too easy or difficult after controlling for the ability levels of the compared groups. Modifying measures based on differential item functioning. Evaluation of two types of differential item functioning.
Nonetheless, if the differential item functioning is a result of language choices that provide an advantage to one subgroup over another, or if the item is found to measure something other than what was intended, etc. Differential item functioning research papers academia. If they dont, thats indicative of differentialitem functioning the items dont function the same in both languages. A biased item deviates from the item response theory irt models used in naep, because the probability of doing well on the item depends not only on what the examinee knows and can do and on the item as reflected in the item parameters, but also on a characteristic of the item that is unrelated to the construct being measured. Item response theory detects differential item functioning. Differential item functioning dif analysis is a key component in the evaluation of the fairness and validity of educational tests. If the factor bringing about such a difference is not part of the construct of focus in the test, then the test would be biased. Differential item functioning dif occurs when an item on a test or questionnaire has different measurement properties for 1 group of people versus another, irrespective of mean differences on the construct. Lees department of instructional psychology and technology master of science differential item functioning dif is present when examinees who have the same level of a trait have a different probability of correctly answering a test item intended to. Thus, differentially functioning items elicit different.
We present an ordinal logistic regression model for identi. Dif occurs when examinees from different groups show differing probabilities of success on or endorsing the item after matching on the construct that the item is intended to measure notice that this is exactly the definition of mi applied to test items. Starting from a framework for classifying dif detection methods and from a comparative overview of the most traditional methods. Several methods have been proposed in recent decades for identifying items that function differently between two or more groups of examinees. The methods were applied to simulated data sets of varying test length 20, 40, and 60 items and sample size 200, 400, and 600 examinees for both equal and unequal underlying ability between groups as well as for both fixed and. Differential item functioning columbia university mailman. The rows in each group refer to the levels from lower to higher, with the fourth row indicating the sum of each ability level. Differential item functioning dif of sf12 and qlesqsf. Some of these procedures, such as the mantelhaenszel chi. Introduction essentially the same differential item functioning dif procedures have been used in practice since the late 1980s. Paper 29002015 multiple ways to detect differential item. Average item scores for subgroups having the same overall score on the test are compared to determine whether the item is measuring in essentially the same way for all subgroups.
Dif is a necessary but not sufficient condition for item bias. When all of the items in a passage were examined, none of the passages showed differential functioning at the passage level. This paper presents dfit, an r package that implements the differential functioning of items and tests framework as well as the monte carlo item parameter replication approach for producing cutoff points for differential item functioning indices. Differential item functioning is a statistical characteristic of an item that shows the extent to which the item might be measuring different abilities for members of separate subgroups. Naep analysis and scaling differential item functioning. Perhaps the item is tapping a secondary factor or factors overandabove the one of interest. Mar 07, 2018 nonetheless, if the differential item functioning is a result of language choices that provide an advantage to one subgroup over another, or if the item is found to measure something other than what was intended, etc. Part of theeducational psychology commons this selected project is brought to you for free and open access by byu scholarsarchive. Apr 12, 20 differential item functioning dif is when a test item favors or hinders a characteristic exhibited by group members of a testtaking population. Differential item functioning of the geriatric depression scale in an asian population. Differential item functioning dif refers to group differences in performance on a test item that cannot be explained by group differences in the construct targeted. Measurement invariance and differential item functioning.
659 362 1339 679 1542 533 827 39 795 1153 1505 1354 1146 840 125 1507 385 230 524 1233 987 620 929 540 946 1147 969 797 598 1150 447 503 986 81 483 1055