An Introduction to Data Analysis & Presentation
Prof. Timothy Shortell, Department of Sociology, Brooklyn College


Before we begin discussing some of the important measurement issues, we have to be familiar with the vocabulary of quantitative analysis. Let's look at some important contrasts:

A theoretically defined collection of elements to be studied.

A subset of elements, taken from a population, from which measurements will be made.

Random Sample:
A sampling technique in which each element in a population has an equal, non-zero, chance of being selected into the sample.

Convenience Sample:
A set of sampling techniques in which a researcher draws a sample by taking advantage of existing groups or easily accessed respondents. For example, a researcher might study the students in an Introductory Statistics course. A convenience sample is not necessarily representative of the intended population.

Numeric representation of a characteristic of a population.

Numeric representation of a characteristic of a sample.

A characteristic that can take on more than one value.

A characteristic that takes on only one value in a given sample or population.

When we speak of a variable, we are referring to a characteristic of the objects (people, organizations, movements, countries, etc.) we are studying. A variable consists of two or more values. Values are the particular qualities the variable may have. For example, gender is a variable with two values, male and female.

When we talk about statistical relationships, we mean relationships between variables, not just some of its values.

In order to understand the usefulness of quantitative analysis, we have to be familiar with the kinds of information that a quantitative variable may possess.

At the nominal level, a variable has no numeric qualities at all. The values of a nominal variable are simply labels. Religious affiliation, for example, is a nominal variable with values such as Protestant, Catholic, Jewish, or none. These are labels that describe kinds of religious organizations.

The values of nominal variables are unordered. For this reason, they may be presented in any order. No mathematical operations (such as adding or multiplying) are valid for nominal variables. They are, for this reason, of rather limited usefulness in quantitative analysis.

In ordinal variables, the values are ordered along some continuum; they represent rank. For example, the variable church attendance might have the values weekly, monthly, a few times a year, or never. These values are ordered from more frequent to less frequent attendance. We know that someone who says they attend weekly attends more frequently than someone who says a few times a year.

What we don't know with ordinal data, however, is the amount of difference between the values. In this example, we don't know how much more often a person who attends weekly attends than some one who attends a few times a year.

Ordinal variables have more numerical qualities than nominal variables, but we cannot treat the values as if they were numbers. Some mathematical operations are valid with ordinal data, but some are not.

Only interval level variables are fully numeric. The values of interval level variables are real numbers, with both order and magnitude information. Examples of interval variables include age, household income (in dollars), occupational prestige scores, number of foreign films seen last year, and so forth. Because mathematical operations are meaningful with interval level data, they are generally the most useful kind of data for quantitative analysis.

Quantitative data is useless if we have not measured our concepts in a reliable and valid way. Good measurement is the foundation of sociological research. When we design a research project, we usually begin by asking questions about the possible relationships among concepts of interest to us. For example, we might wonder if religious affiliation and religious commitment are related to political view. Religious affiliation, religious commitment and political view are concepts. Before we can collect data to try to answer this research question, we have to transform these concepts into measurable items. The process of transformation from concept to measurement is called operationalization.

When we try to operationalize a concept, we find it can be a surprisingly difficult task, even for some ordinary, familiar concepts. The first discovery we are likely to make is that concepts are multidimensional; that is, they have more than one kind of meaning. If we want to operationalize ideology, for example, we quickly realize that American ideology has more than one dimension. Ideology might include the dimensions of political liberalism, economic liberalism, foreign policy liberalism and social liberalism, among others.

The multidimensionality of concepts requires us to use more than one item to measure a concept. If we were going to measure ideology, we would need items measuring political liberalism, economic liberalism, foreign policy liberalism, social liberalism, and so forth.

When we measure concepts, we need to be able to assess how well we have operationalized them. This entails two related properties of a measurement: reliability and validity.

When we measure something, we want to be sure that, if we were to measure the thing a second time, we would end up with the same result. If we generate the same result upon repeated measurements of an item, we say it is reliable. Reliability is consistency in measurement.

Reliability is often inversely related to precision. Let's consider an example. If we want to measure household income, if we ask our respondents to estimate their income to the nearest dollar, we would generate a relatively precise measurement. On the other hand, most people probably don't know their household income that precisely, and so, our measurement would probably not have good reliability.

Instead, say we were to measure household income by asking respondents to pick one of the following categories: less than $25,000, $25,000-$40,000, $40,001-$75,000, $75,001 or more. Since most people can probably pick the appropriate category for their household income, we would have good reliability. This measurement, though, is much less precise.

When we measure a concept, we need to think about how much precision is useful. We don't want to design items that are more precise than is necessary, because they might not be reliable.

We also have to assess the validity of our measurements. Validity concerns the accuracy of our measurements. We want to be sure that we are measuring what we think we are measuring. It is a challenge to create measurements that are fully valid.

We assess validity in four basic ways. The first is face validity. If a measure, on the surface, seems to be measuring what we want to measure, we say it has face validity. For example, if we want to measure church attendance, the following item has face validity:
How often do you typically attend religious services in a month?
but this item does not:
How often have you been to St Patrick's Cathedral in New York?

If an item has face validity, we next assess its content validity. As we already noticed, our sociological concepts are usually multidimensional. To fully measure these concepts, we have to measure all the dimensions. If we succeed at this, we say the measurements have content validity. If we measure only some of the dimensions of a concept, we have failed to achieve content validity.

If our measure has content validity, we check to see if it is related to other concepts we've measured. Construct validity is determined by examining the relationship between our measurement and other measurements in our data that we expect, on the basis of theory, to be related. For example, if we wanted to measure religiosity, we would check to see if our measure of it is related to church attendance, since the literature in this area suggests that religiosity is related to attendance.

I. Discuss the following survey items in terms of their reliability:

Q1. Please indicate the extent to which you agree or disagree with the following statement: "I would agree to an increase in my taxes if the extra money is used to prevent environmental pollution." Do you
1. strongly agree
2. agree
3. neither agree nor disagree
4. disagree
5. strongly disagree

Q2. All in all, how would you describe your state of health these days? Would you say it is
1. Very good
2. Good
3. Fair
4. Poor
5. Very poor

Q3. Generally speaking, would you say that most people can be trusted, or that you can't be too careful in dealing with people?
1. Most people can be trusted
2. Can't be too careful

Q4. Some people feel they have completely free choice and control over their lives, and other people feel that what they do has no real effect on what happens to them. Please use the scale to indicate how much freedom of choice and control you feel you have over the way your life turns out.
1. None at all
10. A great deal

Q5. How would you describe the place you currently live. Would you say it is
1. Large city
2. Small city
3. Suburb
4. Town
5. Rural area

Q6. There has been a lot of discussion these days about multiculturalism. Please indicate the extent to which you agree or disagree with the following statement: "English should be the official language of our government." Do you
1. strongly agree
2. agree
3. neither agree nor disagree
4. disagree
5. strongly disagree

Q8. According to the U.S. tax code, corporations can write off interest payments for certain kinds of debt. The more money they borrow, the less they pay in taxes. Some economists argue that this encourages growth, and therefore, job creation. Others say that it an incentive to seek short-term profits through mergers and acquisitions rather than long-term growth through improvements in technology and productivity. On the following scale, which comes closest to your opinion about the effect of the tax write-off for corporate debt?
1. Encourages job creation
7. Encourages short-term profit seeking

II. For each of the following concepts, devise a face valid survey item:
a) alienation
b) political efficacy
c) prejudice
d) prestige.

III. Compile a list of topics or dimensions that you think are necessary to validly measure the following concepts:
a) creativity
b) social class
c) life satisfaction
d) political participation

IV. Identify a variable that you could use to assess the construct validity of the following concepts:
a) authoritarianism
b) liberalism
c) extroversion
d) delinquency

All materials on this site are copyright © 2001, by Professor Timothy Shortell, except those retained by their original owner. No infringement is intended or implied. All rights reserved. Please let me know if you link to this site or use these materials.