Content Validity: How to Calculate CVI and CVR

Content validity is the degree to which the items of a questionnaire or test adequately represent the construct they are supposed to measure. You assess it before collecting your main data, by asking a panel of subject-matter experts to rate each item and then summarizing their agreement with indices such as the content validity index (CVI) and Lawshe's content validity ratio (CVR).

If you are developing a new scale, adapting an existing one to a new population, or translating an instrument, reviewers will usually expect a content validity step in your methods section. This guide shows how to run that step, how to calculate each index by hand, and how to report it.

What content validity is (and is not)

Content validity asks two questions about an instrument:

• Relevance: Is each item relevant to the construct?
• Representativeness: Do the items, taken together, cover the full domain of the construct without important gaps?

It is a judgment-based form of validity evidence. No respondent data are involved; the evidence comes from experts. That makes it different from:

• Face validity, which is an informal impression (often from lay people) that the test "looks like" it measures the right thing. It is not a quantified expert judgment.
• Construct validity, which is tested with respondent data, for example through factor analysis, convergent and discriminant validity, and correlations with related measures.
• Reliability, which concerns consistency of scores, typically Cronbach's alpha for internal consistency.

A scale can be reliable but have poor content validity (it consistently measures only a narrow slice of the construct), so these checks complement each other.

Step 1: Define the construct and assemble the panel

Before experts can judge relevance, they need a clear definition. Give each expert:

1. A written conceptual definition of the construct and, if relevant, its dimensions.
2. The list of items, grouped by dimension.
3. A rating form with clear instructions and space for comments.

Who counts as an expert? People with demonstrable knowledge of the construct: researchers who have published on it, experienced clinicians or practitioners, and sometimes members of the target population (for example, patients for a patient-reported outcome measure). Describe their qualifications in your paper.

How many? A widely cited recommendation (Lynn, 1986) is a minimum of three experts, and panels of roughly 5 to 10 are common in practice. More experts reduce the chance that agreement occurs by luck, but very large panels are rarely necessary.

Step 2: Collect relevance ratings

The most common approach for the CVI uses a 4-point relevance scale:

A 4-point scale avoids a neutral midpoint. For calculation, ratings are dichotomized: 3 or 4 counts as relevant, 1 or 2 as not relevant.

Step 3: Calculate the item-level CVI (I-CVI)

The I-CVI is the proportion of experts who rated an item as relevant:

I-CVI = number of experts rating the item 3 or 4 / total number of experts

Recommended cut-offs (Lynn, 1986; Polit & Beck, 2006):

• With 5 or fewer experts, the I-CVI should be 1.00 (all experts agree).
• With 6 or more experts, the I-CVI should be at least .78.

Items below the cut-off are revised or removed, and expert comments usually tell you why.

Step 4: Calculate the scale-level CVI (S-CVI)

There are two ways to summarize the whole scale, and they answer slightly different questions.

S-CVI/Ave (averaging method): the mean of the I-CVIs across all items. A common standard is S-CVI/Ave of .90 or higher (Polit & Beck, 2006).

S-CVI/UA (universal agreement): the proportion of items that received a relevant rating (3 or 4) from every expert, that is, items with I-CVI = 1.00. A common standard is .80 or higher. S-CVI/UA is stricter and drops quickly as you add experts, because one dissenting rating removes an item from the "universal" count.

Polit and Beck (2006) recommend reporting S-CVI/Ave as the main scale-level index; many authors report both.

Worked example: I-CVI and S-CVI

Suppose you develop an 8-item scale measuring "perceived workload" and ask 6 experts to rate relevance on the 4-point scale.

I-CVI. Item 3: 5 of 6 experts rated it 3 or 4, so I-CVI = 5/6 = .83. Item 5: 4 of 6, so I-CVI = 4/6 = .67.

With 6 experts the cut-off is .78. Items 3 and 7 (.83) pass. Item 5 (.67) fails and should be revised or dropped, guided by the comments from experts 1 and 4.

S-CVI/Ave. Sum the I-CVIs: 1.00 + 1.00 + .833 + 1.00 + .667 + 1.00 + .833 + 1.00 = 7.333. Divide by 8 items: 7.333 / 8 = .92. This meets the .90 standard.

S-CVI/UA. Items 1, 2, 4, 6, and 8 have universal agreement: 5 of 8 items, so S-CVI/UA = 5/8 = .63. This is below .80.

This pattern is common: S-CVI/Ave looks acceptable while S-CVI/UA does not. After revising item 5, many researchers send the revised items back to the panel for a second round and recalculate.

Optional: adjusting for chance agreement (modified kappa)

The I-CVI does not account for the possibility that experts agree by chance. Polit, Beck, and Owen (2007) proposed a modified kappa for each item:

• Probability of chance agreement: Pc = [N! / (A! × (N − A)!)] × 0.5^N, where N is the number of experts and A is the number rating the item relevant.
• Modified kappa: k* = (I-CVI − Pc) / (1 − Pc)

For item 5 (N = 6, A = 4): Pc = 15 × 0.015625 = .234, so k* = (.667 − .234) / (1 − .234) = .57. For items 3 and 7 (A = 5): Pc = 6 × 0.015625 = .094, k* = (.833 − .094) / (1 − .094) = .82. Items with I-CVI = 1.00 have k* = 1.00.

Polit and colleagues used the evaluation criteria of Cicchetti and Sparrow: k* above .74 is excellent, .60 to .74 good, and .40 to .59 fair. Item 5 is only fair, which agrees with the I-CVI verdict.

Lawshe's content validity ratio (CVR)

Lawshe (1975) proposed a different approach that asks whether each item is essential. Each expert classifies an item as "essential", "useful but not essential", or "not necessary". The CVR is:

CVR = (ne − N/2) / (N/2)

where ne is the number of experts who rated the item "essential" and N is the total number of experts.

CVR ranges from −1 to +1:

• CVR = +1: all experts rated the item essential.
• CVR = 0: exactly half did.
• Negative CVR: fewer than half did.

Example. A panel of 10 experts; 9 rate an item as essential. CVR = (9 − 5) / 5 = .80. If 7 rate it essential, CVR = (7 − 5) / 5 = .40.

Critical values: the idea

A positive CVR alone does not prove agreement beyond chance. With a small panel, a majority can arise by luck. Lawshe therefore published a table of minimum CVR values that depend on panel size: the smaller the panel, the higher the CVR an item needs to be retained.

The logic is a one-tailed binomial test. If experts were choosing "essential" at random with probability .50, how many "essential" votes would you need before that outcome becomes unlikely (p below .05)? For a panel of 10, getting 8 or more "essential" votes by chance has a probability of about .055, while 9 or more has a probability of about .011. So 9 of 10 is required, which corresponds to a critical CVR of .80. With 5 or 6 experts, only unanimous agreement (CVR = 1.00) clears the bar.

Later authors (Wilson, Pan, & Schumsky, 2012; Ayre & Scally, 2014) found inconsistencies in the original table and recalculated critical values from the exact binomial distribution. If you use CVR, cite the table you used and state the panel size.

CVI or CVR? CVI (relevance) is more common in nursing and health sciences; CVR (essentialness) is common in education and personnel selection. Some studies report both, using CVR to decide which items to keep and CVI to evaluate the retained items.

How to report content validity (APA 7)

State who the experts were, what scale they used, which indices you computed, the cut-offs, and what you changed. Example:

If you ran a second round, report the post-revision indices as well. A table listing each item's I-CVI (and modified kappa, if computed) is often placed in a supplement or appendix.

Common mistakes

• No construct definition for experts. Ratings of relevance are meaningless if experts imagine different constructs.
• Using a 5-point scale with a midpoint and then guessing how to dichotomize it. Decide the dichotomization rule in advance and report it.
• Reporting only S-CVI/Ave when individual items are weak. A good average can hide one or two poor items; always inspect I-CVIs.
• Treating content validity as sufficient. It is one source of validity evidence. After data collection you still need reliability and construct validity analyses.
• Ignoring comments. The qualitative feedback usually explains why an item failed and how to fix it.
• Confusing CVR with CVI. They use different rating questions (essential vs. relevant) and different formulas. Do not apply CVI cut-offs to CVR values.

After content validity: the next checks

Once you collect responses with the finalized instrument, the usual sequence is:

1. Internal consistency for each construct, typically Cronbach's alpha. You can check a scale quickly with the Cronbach's alpha calculator, and the reliability analysis guide explains interpretation.
2. Factor structure (exploratory or confirmatory factor analysis).
3. Convergent and discriminant validity using AVE, composite reliability, the Fornell–Larcker criterion, and HTMT.

FAQ

What is a good content validity index?

For individual items, an I-CVI of 1.00 with 3 to 5 experts, or at least .78 with 6 or more experts. For the whole scale, S-CVI/Ave of .90 or higher and S-CVI/UA of .80 or higher are the commonly cited standards (Polit & Beck, 2006).

How many experts do I need for content validity?

At least three. Panels of 5 to 10 are common. Keep in mind that the I-CVI cut-off and Lawshe's critical CVR both depend on panel size, so report how many experts you used.

Is content validity the same as face validity?

No. Face validity is an informal impression that a test appears to measure what it should. Content validity is a structured, quantified judgment by experts about the relevance and coverage of the items.

Can I establish content validity with a statistical test on my survey data?

No. Content validity comes from expert judgment about the items, collected before the main study. Statistical analysis of respondent data supports other forms of validity evidence, such as construct validity.

Should I report CVI or CVR?

Use the one that matches your rating question and your field's conventions. CVI is based on relevance ratings; CVR is based on whether experts consider an item essential. Reporting both is acceptable if you collected both types of ratings.

From validated instrument to results section

Once your instrument is finalized and the survey data are in, Data2Paper's Generate Paper can take the CSV or Excel file and draft a full paper, including reliability and validity analyses, tables, and interpretation. You can preview the first half for free and pay $19.90 to unlock the full draft if it is useful. It is an AI-generated draft, so check the numbers and citations before you submit anything.