IPIP-NEO-120

The IPIP-NEO-120 is an optimized assessment of the five factor model of personality. It features the ability to measure six underlying facets for each of the five traits. The assessment strikes a nice balance when compared to other five factor assessments in terms of brevity and breadth while maintaining solid validity and reliability (see below). The test was created using items from the open source project called the International Personality Item Pool (IPIP).

Validity

In test theory, validation refers to the level of evidence that supports the ability of users to draw useful, accurate, and truthful inferences/interpretations from a measure. In other words, validity refers to an instrument measuring what it says it measures and the ability of results to generalize to real-world outcomes. Three types of validation include:

Content validity

Ability to infer from test measures to larger construct domains; the degree to which the instrument measures all facets of what it says it measures

According to Johnson (2014), the IPIP-NEO-120 is derived from the larger IPIP-NEO-300 item pool (see Goldberg, 1999) and was reduced to top four items for each facet measure using the following techniques:

Reliability: kept items with highest alpha coeffecients: mean α of .84 (traits) and .68 (facets)
Rational-Intuitive:

Replace similar/duplicate items
Keep items that are similar to NEO-PI-R
Remove any legally questionable items

The same study confirms that factor component loadings of the IPIP-NEO-120 are representitive of the five major personality traits.

Criterian validity

Ability to make inferences from test scores to another real-world behaviors

IPIP-NEO-120

Five Factors (OCEAN Traits)

Individual, Interpersonal, and Social Life Outcomes

As summarized in Soto (2019), effect sizes between the five factors and important life outcomes are generally between r = .20 - .40 (p < .05 - .01).

Construct validity

Ability to make inferences from test scores to various psychological constructs (e.g., characteristics grouped as personality traits) ; the overall confidence that a test measures what it claims to measure

The five factor theory was popularized by Norman (1963) as an "adequate taxonomy of personality attributes" - it factor analyzed scores from various personality trait scales into five distinct factors. Later, a test called the NEO was created from this earlier body of work and used the familiar OCEAN trait categories: McCrae & Costa (1985) provided validation data for their NEO assessment. Later, Costa & McCrae (1995) provided validity for six facet constructs within each major trait in the NEO-PI-R assessment tool.

Kajonius & Johnson (2019) document factor-analytic results of the IPIP-NEO-120 based on a large internet sample in the United States (n = 320,128). This analysis confirmed a five trait structure with corresponding facets: Root Mean Square Error of Approximation (RMSEA) results for the five scales were 0.06 or less for all except Extraversion which was reported at 0.07. Further, IPIP-NEO-120 scales are highly correlated with scales from longer-form assessments of the five factors (see the test characteristcs table below).

Reliability

Reliability refers to the consistency and repeatability of test scores or measurements. It answers the question, how likely is an individual to obtain the same score if the same measure is repeatedly administered to them?

Inter-rater reliability

Assesses the degree of agreement between two or more raters in their appraisals.

Many studies have explored the extent that different observers ratings are different than self-ratings when evaluating the five factors. In particular, Kim et al. (2019) conducted a meta analysis of over 150 studies examining the degree that big five ratings from friends, family, colleagues, and strangers would differ from self-ratings. Overall, no significant differences were found (average effect δ = −.038) indicating good consistency between self and other ratings across various methods of assessment.

Test-retest reliability

Assesses the degree to which test scores are consistent from one test administration to the next.

McCrae et al. (2010) indicates that the big five personality traits generally have strong retest stability when readministered to the same participants after 1 week (retest rs = .91-.93) and over the course of 6 years (retest rs = .87-.93). Trait consistency for Neuroticism, Extraversion, Openness, and Agreeableness peaks in middle age whereas Conscientiousness continuously stabilizes over time. Trait scores are also influenced by life events (Specht et al., 2011).

Inter-method reliability

Assesses the degree to which test scores are consistent when there is a variation in the methods or instruments used.

As demonstrated in Johnson (2014), convergent / scale correlations between IPIP-NEO-120 and other methods of measuring the five factors are relatively high:

NEO PI-R rs = .76 to .87
BFI rs = .33 to .57 (p < .05)
MiniMarkers rs = .31 to .49 (p < .05)
Acquaintance ratings rs = .25 to .49

Internal consistency reliability

Assesses the consistency of results across items within a test.

The alpha coefficient is a measure of reliability. It increases with fewer measurement errors of an assessment's items and when items are measuring a single construct.

Johnson (2014) reports alpha coefficients from a large internet sample (n = 619,150) of users who completed the IPIP-NEO-120:

Neuroticism: α = .90
Extraversion: α = .89
Openness to experience: α = .81
Agreeableness: α = .86
Conscientiousness: α = .90
Facets: mean α = .75; α range = .63 (Liberalism) to .88 (Cautiousness)

Supporting Test Characteristics

Scale Names		Number of Items			Mean Item Intercorrelations			Coefficient Alpha			Scale Correlations
IPIP	NEO	NEO (+ / -)	IPIP 300 (+ / -)	IPIP 120 (+ / -)	NEO	IPIP 300	IPIP 120	NEO	IPIP 300	IPIP 120	IPIP 300 vs NEO	IPIP 120 vs NEO
Neuroticism		27+21=48	33+27=60	17+7=24	0.22	0.22	0.24	0.93	0.93	0.88	.89 [.95]	.87 [.97]
Anxiety	Anxiety (N1)	4 + 4 = 8	5+ 5=10	4+0= 4	0.38	0.34	0.39	0.83	0.83	0.71	.76 [.92]	.76 [.99]
Anger	Angry Hostility (N2)	5 + 3 = 8	5+ 5=10	3+1= 4	0.34	0.43	0.47	0.8	0.88	0.77	.77 [.92]	.71 [.90]
Depression	Depression (N3)	6 + 2 = 8	7+ 3=10	3+1= 4	0.42	0.44	0.51	0.84	0.89	0.8	.81 [.94]	.76 [.93]
Self-Consciousness	Self-Consciousness (N4)	5 + 3 = 8	6+ 4=10	3+1= 4	0.26	0.28	0.3	0.74	0.8	0.63	.73 [.95]	.60 [.88]
Immoderation	Impulsiveness (N5)	4 + 4 = 8	5+ 5=10	1+3= 4	0.24	0.25	0.36	0.72	0.77	0.69	.74 [.99]	.65 [.92]
Vulnerability	Vulnerability (N6)	3 + 5 = 8	5+ 5=10	3+1= 4	0.35	0.33	0.38	0.79	0.82	0.7	.78 [.97]	.74 [.99]
Extraversion		29+19=48	36+24=60	18+6=24	0.15	0.16	0.18	0.89	0.92	0.84	.89 [.98]	.85 [.98]
Friendliness	Warmth (E1)	6 + 2 = 8	5+ 5=10	2+2= 4	0.35	0.41	0.45	0.8	0.87	0.77	.76 [.91]	.68 [.87]
Gregariousness	Gregariousness (E2)	4 + 4 = 8	5+ 5=10	2+2= 4	0.35	0.27	0.28	0.8	0.79	0.6	.78 [.98]	.73 [.99]
Assertiveness	Assertiveness (E3)	4 + 4 = 8	5+ 5=10	3+1= 4	0.33	0.35	0.44	0.8	0.84	0.75	.81 [.99]	.73 [.94]
Activity Level	Activity (E4)	5 + 3 = 8	5+ 5=10	3+1= 4	0.27	0.2	0.35	0.74	0.71	0.68	.72 [.99]	.63 [.89]
Excitement-Seeking	Excitement-Seeking (E5)	6 + 2 = 8	8+ 2=10	4+0= 4	0.2	0.27	0.34	0.65	0.77	0.67	.67 [.95]	.59 [.89]
Cheerfulness	Positive Emotions (E6)	4 + 4 = 8	8+ 2=10	4+0= 4	0.37	0.3	0.39	0.81	0.81	0.71	.77 [.95]	.69 [.91]
Openness To Experience		24+24=48	28+32=60	12+12=24	0.17	0.17	0.19	0.91	0.92	0.85	.87 [.95]	.84 [.96]
Imagination	Fantasy (O1)	3 + 5 = 8	6+ 4=10	4+0= 4	0.36	0.32	0.37	0.81	0.82	0.7	.74 [.91]	.69 [.92]
Artistic Interests	Aesthetics (O2)	5 + 3 = 8	5+ 5=10	2+2= 4	0.41	0.38	0.4	0.84	0.85	0.72	.80 [.95]	.76 [.98]
Emotionality	Feelings (O3)	5 + 3 = 8	5+ 5=10	2+2= 4	0.28	0.3	0.34	0.75	0.81	0.67	.71 [.91]	.65 [.92]
Adventurousness	Actions (O4)	3 + 5 = 8	4+ 6=10	1+3= 4	0.18	0.25	0.33	0.64	0.77	0.66	.72 [.99]	.62 [.95]
Intellect	Ideas (O5)	5 + 3 = 8	5+ 5=10	1+3= 4	0.39	0.39	0.47	0.82	0.86	0.78	.81 [.96]	.75 [.94]
Liberalism	Values (O6)	3 + 5 = 8	3+ 7=10	2+2= 4	0.31	0.39	0.47	0.78	0.86	0.76	.71 [.87]	.63 [.82]
Agreeableness		26+22=48	24+36=60	7+17=24	0.14	0.13	0.16	0.89	0.9	0.81	.84 [.94]	.76 [.90]
Trust	Trust (A1)	5 + 3 = 8	6+ 4=10	3+1= 4	0.43	0.32	0.38	0.84	0.82	0.7	.78 [.94]	.73 [.95]
Morality	Straightforwardness (A2)	3 + 5 = 8	2+ 8=10	0+4= 4	0.26	0.24	0.29	0.74	0.74	0.62	.65 [.88]	.54 [.80]
Altruism	Altruism (A3)	5 + 3 = 8	5+ 5=10	2+2= 4	0.26	0.26	0.33	0.72	0.77	0.65	.68 [.91]	.54 [.79]
Cooperation	Compliance (A4)	3 + 5 = 8	3+ 7=10	0+4= 4	0.25	0.22	0.26	0.72	0.72	0.56	.72 [.99]	.62 [.98]
Modesty	Modesty (A5)	4 + 4 = 8	4+ 6=10	0+4= 4	0.26	0.25	0.31	0.73	0.76	0.63	.71 [.95]	.64 [.94]
Sympathy	Tender-Mindedness (A6)	6 + 2 = 8	4+ 6=10	2+2= 4	0.17	0.23	0.35	0.61	0.75	0.68	.62 [.92]	.55 [.85]
Conscientiousness		28+20=48	31+29=60	11+13=24	0.19	0.23	0.2	0.91	0.92	0.84	.85 [.93]	.80 [.92]
Self-Efficacy	Competence (C1)	5 + 3 = 8	6+ 4=10	4+0= 4	0.26	0.28	0.26	0.71	0.79	0.57	.68 [.91]	.59 [.93]
Orderliness	Order (C2)	3 + 5 = 8	5+ 5=10	1+3= 4	0.28	0.33	0.44	0.74	0.83	0.76	.77 [.98]	.68 [.91]
Dutifulness	Dutifulness (C3)	6 + 2 = 8	5+ 5=10	2+2= 4	0.24	0.19	0.19	0.68	0.71	0.47	.60 [.86]	.53 [.94]
Achievement-Striving	Achievement Striving (C4)	5 + 3 = 8	7+ 3=10	2+2= 4	0.26	0.28	0.35	0.72	0.79	0.68	.71 [.94]	.57 [.81]
Self-Discipline	Self-Discipline (C5)	4 + 4 = 8	5+ 5=10	2+2= 4	0.35	0.37	0.34	0.81	0.85	0.66	.77 [.93]	.72 [.98]
Cautiousness	Deliberation (C6)	5 + 3 = 8	3+ 7=10	0+4= 4	0.23	0.25	0.38	0.7	0.76	0.7	.69 [.95]	.61 [.87]
Mean for Facet Scales		4 + 4 = 8	5+ 5=10	2+2= 4	0.3	0.3	0.36	0.76	0.8	0.68	.73 [.94]	.66 [.91]

Adapted from the IPIP website

Notes

The numbers under the "(+ / -)" heading indicate the number of positively and negatively items in the scale.
Values in brackets are corrected for unreliability, using a different sample.

References

Costa, P. T., & McCrae, R. R. (1995). Domains and facets: Hierarchical personality assessment using the Revised NEO Personality Inventory. Journal of Personality Assessment, 64(1), 21-50. http://doi.org/10.1207/s15327752jpa6401_2

Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality Psychology in Europe, Vol. 7 (pp. 7-28). Tilburg, The Netherlands: Tilburg University Press. https://ipip.ori.org/A%20broad-bandwidth%20inventory.pdf

Johnson, J. A. (2014). Measuring thirty facets of the Five Factor Model with a 120-item public domain inventory: Development of the IPIP-NEO-120. Journal of Research in Personality, 51, 78-89. http://doi.org/10.1016/j.jrp.2014.05.003

Kajonius, P. J. & Johnson, J. A. (2019). Assessing the Structure of the Five FactorModel of Personality (IPIP-NEO-120) in the Public Domain. Europe's Journal of Psychology. 15 (2), 260-275. https://doi.org/10.5964/ejop.v15i2.1671

Kim, H., Di Domenico, S. I., & Connelly, B. S. (2019). Self–other agreement in personality reports: A meta-analytic comparison of self- and informant-report means. Psychological Science, 30(1), 129-138. http://doi.org/10.1177/0956797618810000

McCrae, R. R., & Costa, P. T. (1985). Updating Norman's 'adequacy taxonomy': Intelligence and personality dimensions in natural language and in questionnaires. Journal of Personality and Social Psychology, 49(3), 710-721. http://doi.org/10.1037/0022-3514.49.3.710

McCrae, R. R., Kurtz, J. E., Yamagata, S., & Terracciano, A. (2011). Internal consistency, retest reliability, and their implications for personality scale validity. Personality and Social Psychology Review, 15(1), 28-50. http://doi.org/10.1177/1088868310366253

Norman, W. T. (1963). Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings. The Journal of Abnormal and Social Psychology, 66(6), 574-583. http://doi.org/10.1037/h0040291

Soto, C. J. (2019). How Replicable Are Links Between Personality Traits and Consequential Life Outcomes? The Life Outcomes of personality Replication Project. Psychological Science, 30(5), 711-727. https://doi.org/10.1177%2F0956797619831612

Specht, Jule & Egloff, Boris & Schmukle, Stefan. (2011). Stability and Change of Personality Across the Life Course: The Impact of Age and Major Life Events on Mean-Level and Rank-Order Stability of the Big Five. Journal of Personality and Social Psychology. 101. 862-882. https://doi.org/10.1037/a0024950