How is interpretive quantitative research helpful?

Even if employing interpretive quantitative measures does not have the downside of leading to a confusing free-for-all, we can still ask, along with Stam, whether there is something to be gained by using numbers in our investigations. As I pointed out in my position paper, I agree with researchers who embrace positivism about some of the useful features of quantitative measures and quantitative research procedures in general (e.g., they enhance our ability to investigate group differences without being unduly influenced by dramatic instances of a phenomenon). I want to mark out an additional basis for appreciating what quantitative methods have to offer.

In my position paper, I argued that quantitative research procedures can make a special contribution because they require us to concretely specify our ideas about psychological phenomena. I endorsed such measurement procedures as relational coding, which could be called ‘‘soft’’ measurement, but I also discussed how what could be considered ‘‘strong’’ measures and related quantitative procedures (e.g., coding discrete behaviors, conducting experiments) also offer useful ways to concretely specify phenomena of interest—although I argued for reconceptualizing these methods as interpretive procedures and recognizing that they do not exhaustively specify the constructs and processes under investigation. Now, I want to extend my analysis of the ways such ‘‘apparently strong’’ measures and procedures can be extremely helpful. To begin with, ‘‘apparently strong’’ procedures can be highly informative about particular situations that are of interest in connection with particular applied problems.

For example, consider Wood’s (e.g., Wood & Middleton, 1975) paradigm for examining how mothers scaffold their children’s attempts to learn how to build a block puzzle, which I referred to in my position paper. That paradigm includes a clearly delineated procedure for identifying the specificity of parental bids at guiding a child. Although the goal is to explore a relational process (i.e., do mothers home in and out contingently as a function of the child’s moment-to-moment success), the specificity measure does not rely on relational coding. Instead, each bid is coded based on its own properties. Investigating parent–child interaction in this specific situation has been shown to have applied utility. In a study I conducted (Westerman, 1990), assessments of maternal behavior in the context of Wood’s paradigm discriminated between mother–preschooler dyads with and without compliance problems. In an experimental study, Strand (2002) found that teaching mothers to home in and out when they show their children how to build Wood’s puzzle leads to greater child compliance in a separate context.

‘‘Apparently strong’’ quantitative methods also can lead to the discovery that specific, concrete forms play a role in many situations, not just the original measurement context. For example, Strand (2002) found that the specificity scale was useful when applied to a task other than Wood’s block puzzle. Similarly, we might find that measures which were initially employed in particular structured observation contexts, say a measure of verbal aggression based on decibel levels or, more likely, a measure of activation in a certain part of the brain, identify specific concrete forms that play a particular role quite generally. Merleau-Ponty (1962) used the term ‘‘sediment’’ to refer to concrete forms of this sort. Sediment often plays a part in psychological phenomena, and ‘‘apparently strong’’ quantitative procedures can be very helpful because they enable us to learn about these aspects of practical activity.

Two qualifications are in order, however. First, even when one aspect of a phenomenon of interest typically takes a specific concrete form, we need to recognize that it is part of a larger, meaningful process. For example, even if Wood’s specificity scale worked in all contexts—which is extremely unlikely—it would be crucial to appreciate the role that the specificity of maternal directives plays as part of doing something, that is, teaching a child. It is not specificity per se, but the modulation of maternal efforts as a function of the child’s success at what he or she is doing that is crucial. The second qualification is that there are always limits to the ways in which specific concrete contents function in a particular manner. It is useful to discover that a certain area of the brain typically functions in a particular way as part of what a person is doing, but another area might play this role under particular circumstances, perhaps due to brain plasticity. ‘‘Apparently strong’’ quantitative studies can be helpful here, too, because they are useful for marking out the relevant limits.

Such research has other benefits that can be considered the flipside of the advantages I have mentioned so far. Studies employing ‘‘apparently strong’’ quantitative procedures can help us understand psychological phenomena in terms of richly generative principles, because quantitative measures such as discrete behavior codes provide concrete examples of meaningful constructs and quantitative procedures like experiments constitute concrete examples of meaningful processes. For example, research employing Wood’ paradigm suggests the general principles that ‘‘homing in and out’’ is a crucial feature of parenting and that this process refers to modulating the specificity of parental bids. ‘‘Apparently strong’’ quantitative methods are well suited for investigating these claims. We might well find, for example, that Wood’s specificity scale itself is relevant only in a few contexts, but that some other concrete characterization of modulating specificity is on the mark quite generally. Alternatively, we might find that, however understood, modulation of specificity has limited relevance, but that ‘‘homing in and out’’ captures an important process if we define it in concrete ways that share in common contingently providing more or less help. ‘‘Apparently strong’’ quantitative research is very useful for learning about general principles because these principles are concretely meaningful; they are not abstract ideas. Even if we somehow knew beforehand that a given principle was true (which, of course, is never the case), we would not know what it actually means because there is no transparent mapping from the principle to concrete events. ‘‘Apparently strong’’ quantitative research procedures would help us greatly in this hypothetical situation, and they help us all the more in real research situations in which we simultaneously must learn the principles and what they mean concretely.

4. It’s ‘‘good’’ quantitative research and it’s interpretive

Studies by Fischer and his colleagues (e.g., Fischer, 1980; Fischer & Bidell, 1998) and Dawson (2006) have investigated development in a wide range of domains, including among many others, understanding of social interaction concepts such as ‘‘nice’’ and ‘‘mean,’’ skills in mathematics, and understanding ‘‘leadership.’’ This research has provided a great deal of support for a clearly delineated 13-level developmental sequence in complexity ranging from reflexive actions to understanding principles. Dawson et al. (2006) claimed that it demonstrates the value of ‘‘strong,’’ positivist quantitative methods, which, they say, are excluded in the approach to quantitative research I offered in my position paper. In particular, they argued that their work provides a ‘‘developmental ruler’’ that represents a universal, content-independent measure of increasing hierarchical integration.

Do their examples show that Stam was off the mark when he argued that, although such research is highly desirable, it is something that we see rarely if at all in the field? Do the examples demonstrate that my approach fails to incorporate an important range of research efforts?

In fact, I believe that this research offers us excellent examples of ‘‘good’’ quantitative research. I disagree with Dawson et al.’s characterizations of their own research, however. As I see it, the research in question is a fascinating example of one of the situations I described earlier: the case in which initial findings in a particular domain or a few domains suggest a general principle. In particular, in this situation, the general principle is the developmental sequence of hierarchical integration. There is a real risk here (given our philosophical tradition) of imagining that this sequence is a fully abstract, reified structure that ‘‘lies behind’’ concrete phenomena and failing to recognize the ways in which interpretation enters into the research.

The studies by Fischer, Dawson, and their colleagues employ measures that are extremely useful, but not ‘‘strong’’ in the positivist sense marked out by classical notions of measurement or Stam’s idea about measures that ‘‘refer back to some concrete feature of the world.’’ Consider examples from Dawson’s (2006) LecticalTM Assessment System. In that system, a child’s understanding is said to be at the level of single representations if the child offers a statement like ‘‘Camping is fun’’ in an assessment interview. By contrast, the child’s understanding would be at the higher level of representational mappings if he or she employed an expression describing a ‘‘linear’’ relationship, such as ‘‘If you don’t do what your father tells you to do, he will be really mad at you.’’ But determining the level of such responses is by no means a transparent process. For one thing, there is no one-to-one relationship between developmental level and form of speech. A child might say, ‘‘If I go camping, I have fun’’ and still be at the level of single representations, if the statement really boils down to ‘‘Camping is fun’’ because the child cannot actually coordinate relevant single representations in a mapping relationship. Dawson (2006) herself noted that meaning is ‘‘central’’ to the scoring procedure and gave an example concerning the interview question, ‘‘Could you have a good life without having had a good education?’’ In this example, a rater found it difficult to score a response that included the word ‘‘richer’’ because it was not clear whether this word referred to having more money or having a life with broader/deeper significance.

Dawson (2006) claimed that the ‘‘developmental ruler’’ provides a way to ‘‘look through’’ content in assessments of structure, but in my view, the brief remarks I have just offered point to a crucial sense in which hierarchical integration is a concretely meaningful idea. When we apply the developmental ruler to a new domain, we have to discover not only whether the developmental sequence holds in that domain but also what counts as single representations, representational mappings, and so forth in this context. The ‘‘ruler’’ provides us with valuable ideas about how to think about complexity, but in itself it is empty. To use it, investigators have to proceed with the crucial steps of designing an assessment procedure and preparing scoring manuals for each domain. These steps reflect the investigators’ rich appreciation of the concretely meaningful practices in the domain, including what kinds of connections can obtain within this range of phenomena. This rich appreciation is largely prereflective understanding. Hence, the procedures and scoring manuals for each domain play a truly central role that is not ‘‘given’’ by the general principles. Furthermore, they do not offer exhaustive concrete specifications of the phenomena of interest. Raters have to draw on their own prior familiarity with the way things work.

Some further comments are in order concerning the fact that most or all of the studies under consideration were based on assessing individuals’ developmental level in a structured interview or by means of some other similar structured procedure. These assessments provide measures that have considerable precision. Moreover, they unquestionably tap important skills. Nevertheless, we should recognize that such investigations differ from other possible studies that would examine what an individual does when he or she is engaged in ongoing activities. For example, consider ‘‘leadership,’’ one of the research areas discussed by Dawson et al. Dawson (2006) described a carefully developed system for assessing a subject’s level of understanding this concept from responses during an assessment interview. But instead of proceeding this way, an investigator could examine what a subject does upon encountering a particular item while going through his or her ‘‘inbox’’ or when a subordinate asks the subject a specific question in a naturally occurring situation. Thinking about such in situ examples makes it clear that we could not possibly map what a person might do in such situations onto the developmental sequence without drawing on rich prior familiarity with the relevant practices. Therefore, these examples underscore the role played by interpretation. Looking at this matter the other way, the in situ examples help us to see how much actually is involved when we do use the structured assessment procedures that these investigators have so successfully developed. A great wealth of interpretive appreciation of the phenomena is concretized in those measurement procedures.

The in situ examples also raise a new issue: is the 13-level sequence relevant for some or all naturally occurring situations, or is its relevance limited to the kind of skills that can be assessed in the particular ways typically employed in the research in question, which could be called skills at understanding of a more reflective sort? I am not asserting that the complexity sequence would not hold for a broad range of skills involving in situ behavior. I only wish to point out that the sequence might be limited in these ways. The work is interpretive. It is based on procedures that provide concrete examples of certain meaningful phenomena. Therefore, we can ask whether the assessments made in these investigations actually serve as concrete examples of clearly in situ psychological phenomena and, more generally, we can ask what is the range of phenomena that are successfully tapped by the structured assessments. None of this is to argue against the value of this research. It is possible for raters to draw upon their prereflective understanding and employ the carefully developed manuals and the developmental model to assess complexity levels. Furthermore, it is of great interest that research efforts along these levels have demonstrated that the developmental sequence holds in many different areas when skills are assessed using the kinds of procedures that have been employed. In sum, I believe that the research by Dawson, Fischer, and their colleagues represents examples of excellent, ‘‘apparently strong’’ quantitative research.