Theory, Method, and Practice in Comparative and International
University of Maryland
Revised, April 2008
I have been very fortunate in my professional life in many
ways. One way is that I get paid, in part, to reflect on some of
the significant issues of our day. Over the years, my views have
evolved, shaped by what I have seen, discussed, experienced, read,
and studied. I wish to take the opportunity afforded by this
Presidential Address to reflect on what I believe and how I came to
believe it. I hold strong views about our field of comparative and
international education, views that often clash with the dominant
modes of theory, method, and practice. In this address, I wish to
explain what I see as fundamental flaws in some of these dominant
modes of thinking and explore what some of the alternatives to those
views are. I will begin by saying a few words about how I see the
field of comparative and international education and then discuss
theory, method, and practice in turn.
the past sixty years, we have seen enormous development of the field
of comparative and international education (CIE), from one that was
a focal point for a relatively small number of academics and
practitioners to one that extends to large numbers of people engaged
in grassroots organizations; nongovernmental organizations (NGOs);
field projects; universities; and local, national, and international
agencies. In the 1950s, CIE was a small field, relatively closed and
cloistered, composed of a range of historical and cultural studies
to ‘traveler’s tales.’ The 1960s were a watershed, epitomized by
Harold Noah and Max Eckstein’s (1966) book, The Science of
Comparative Education. During the 1960s, the borders of
comparative education came down and along with them much of the
introspective discussion that had tried, rather unsuccessfully, to
come up with definitions to focus and limit the field. Instead, the
borders were opened, especially to all the social sciences.
Moreover, the 1960s and 1970s saw ferment within and across the
social sciences about the validity of the modernization narrative
and the associated debates about the meaning and practice of
development. The resulting paradigmatic controversies also became
an integral part of the CIE field.
This openness of CIE to the theories, methods, practices,
debates, and controversies from many fields continues to form the
most vital part of our field today. And it is this integral and
necessary permeability that forms both a great opportunity and a
great challenge to CIE. Harold Noah once argued at a CIES meeting
that the comparative advantage of comparative educators was our
focus on education. I respectfully disagreed at the time and still
do. The principal comparative advantage of comparative education is
that the field is literally constituted by border crossings, and
comparative educators, by necessity, roam far beyond education.
In my view, no other disciplinary or professional field has such a
broad, interconnected vantage point from which to view the dilemmas
of our time. Education is the anchor that focuses us, but we know –
and have to know – significantly more in some ways, for example,
than economists or other social scientists, even about their own
fields, about the research methodologies used, about policy and
practice. For example, a comparative educator today needs to
understand economics’ human capital theory, but also its complement
in sociology, modernization theory, as well as the critique of both
(Arnove and Torres 2007; Ginsburg and Price-Rom 2008).
has yielded a serious dilemma to those of us involved in university
education. I feel no other field has to develop the breadth of
knowledge that CIE requires for a student to become an educated,
reflective practitioner and/or researcher.
Nowhere is this challenge more apparent than in deciding what core
knowledge should be in CIE graduate programs. I have spent many
years at Florida State University and the University of Maryland
engaged in explicit discussions with colleagues of what this core
knowledge should include. The most recent effort, a few years ago,
yielded this list: students should know something about
anthropology, economics, political science, and sociology, as well
as debates within and across these fields; development theory and
practice; globalization; postmodernism and the other posts-;
feminism/gender and development; race and ethnicity/multiculturalism/diversity/identity; current education and
development issues; the nature and history of the CIE field;
critical pedagogy; policy studies, analysis, and planning;
environmental issues; international institutions; the role of NGOs,
civil society, and social movements; participatory perspectives;
community development; the history and philosophy of education; the
arts and comparative literature; psychology.
This is obviously a tall order. Of course, how much a
student learns about any of these areas is variable and depends on
specific decisions we make in our core required courses, of which
presently the University of Maryland has three. We do try to cover
something about each topic above (except for the last three
categories). There are at least two things that give coherence to
our teaching this array of topics. One is that all issues are
relevant to the central dilemma of our time (‘What to do about
poverty, inequality, and development?’) and for our field (‘What is
education’s role in all this?’). A second is that a central tenet
of my view of the field is that all knowledge is seriously
contested. Therefore, the debates surrounding each topic mentioned
above become focal points in the curriculum.. Since the nature of
the debates on each topic is similar, the core courses have a
coherence that can turn out students with a remarkable ability to
integrate a lot of diverse issues.
I did not knowingly become a part of this CIE field until
1980 when I took a job in a CIE program at Florida State
University. Prior to that I had done my graduate work at Stanford
University in economics and public policy, with a concentration in
the economics of education, and had taught for seven years, partly
in an educational policy program at Cornell and partly in a public
policy program in Brazil. During that time, I saw my main
professional identity as an economist of education concerned with
public policy. However, I felt very much at home in CIE from my
first exposure to it at FSU and in CIES, when I attended my first
meeting in March 1981, held at FSU.
One reason I felt so at home in CIE was that economics had a
prominent place in it at that time. But even more so, I felt at home
because, from the 1970s onward, economics, and associated human
capital and modernization perspectives were hotly contested within
CIE. And from my earliest work in this profession, as a graduate
student, to my work today, teaching, writing, and engaged in
practice, this contestation has remained central to my understanding
of the dilemmas we face in theory and practice. In this section, I
discuss some of the key issues and occurrences that have influenced
my thinking about theory, especially economic theory.
bulk of my graduate education focused on neoclassical economic
theory – what I see as the fictional account of the efficiency and
equity of free markets. But this was the Vietnam War era, the late
sixties to the early seventies, which threw even academia into
social upheaval. Thus, from the beginning I saw glimpses of
competing stories, competing social explanations. The economics
faculty had John Gurley, a neo-Marxist economist, and the education
faculty had Martin Carnoy and Henry Levin, two strong critics of
neoclassical economics. Many of the economics students were
dissenters from the orthodoxy, and, at one point, we even got
together and raised enough money to hire Joan Robinson, a famous
critic of neoclassical economics, as a visiting professor for a
These critics exposed me to alternative theories. But
equally influential were my early misgivings about the neoclassical
story. It became clear in my graduate school courses that
neoclassical economic theory had fundamental inconsistencies that
rendered it problematic at best, absurd at worst. Chief among these
inconsistencies is that in order for a market system to be
efficient, you need a number of assumptions to hold (mainly, profit
and utility maximization, perfect market competition, and perfect
knowledge). Yet these assumptions can never hold completely in
practice. That might be OK if coming close to these assumptions was
good enough to get you close to efficiency. However, that is not
the case. About three years into my graduate education, one of my
classes spent a day on ‘Second Best Theory,’ which asks, ‘If we
don’t live in the first-best world of the assumptions of perfect
competition, but have, let’s say, one imperfection in an otherwise
perfect world (e.g., one monopoly), what are the results?’ (Rakowski
1980). It turns out that Second Best Theory says the results of
one imperfection ripples through the market system, so the outcome
is not efficient nor close to efficient.
This means that Adam Smith’s dictum that the invisible hand acts in
the social interest is totally false in a real world economy rife
with multiple deviations from free market theory.
I remember one professor saying in class that we have ‘workable
competition’ in the real world, but when asked what that was and how
it related to the numerous benefits of perfect competition that they
had spent years teaching us, he was at a loss, as are all
early on, it has seemed to me that the concept of efficiency, which
is thoroughly embedded in the concept of perfect competition, was
empty in the real world and, therefore, calls for improving
educational or economic efficiency were devoid of meaning.
The great feat of neoclassical economics has been to convince people
that there is a vantage point separable from distributional
concerns, where one could look to see if the system as a whole is
better off, defined as efficiency, where the decisions to produce an
array of goods and services could be made in the interests of all
irrespective of how little one had. However, if prices are not
defined according to the exact dictates of free market theory, then
private profitability or social rate of return analysis tell us
nothing about the comparative social advantage of producing, say,
more yachts instead of rice and beans, or more primary education
instead of higher education. Without the justification provided by
neoclassical economic theory, all markets become just a convenience
but have no legitimacy as acting in the social interest (Klees
1993). Given the great inequalities it yields, I believe that
someday our market system of wage labor will be considered as
illegitimate as slavery is now considered. The fact that the market
pays someone $1.50 for a day of backbreaking labor while others get
millions of dollars for their white collar labor is akin to a form
of slavery for which no one takes responsibility and which is
disguised by the rhetoric of freedom.
Given my misgivings about neoclassical economic theory, my
exposure to some alternative theoretical frameworks, and the
contestation in the fields of education and economics in the 1970s,
it is not surprising that much of my early research focused on what
I came to view as the paradigm wars (e.g., Ashby et al. 1980; Klees
1986; Klees and Wells 1983). These were not some abstract
theoretical debates either. My consulting work was focused on
specific education innovations, most often in developing nations,
and it was clear to me that how I evaluated any real world program
or policy depended completely on what theoretical framework I
adopted. If I started from what we might call right-of-center
politics, as does neoclassical theory, I got a very different
picture than if I started from left-of-center politics, as does what
I call political economy (see discussion below under Alternative
Theories). Therefore, what I did in most of my consulting work and
research work is examine from, what I call for convenience here,
Left and Right perspectives – topics such as educational technology,
nonformal education, educational access and quality, vocational
education, vouchers, education and work, etc. Moreover, I found
that these same underlying Left/Right debates were not limited to
the fields of education and economics. In doing work that related
education to other fields of study, I started to find these same
debates – in health and nursing, agricultural and rural development,
communications, sociology, and other social sciences.
then and still see now these paradigm debates as fundamental to our
social understandings. In the early eighties, I pointed out how I
thought Mark Blaug (1975) was wrong in his famous review of the
history of economics (Klees and Wells 1983). Blaug (1975) concludes
his paper by rejecting Thomas Kuhn’s (1962) understanding of
paradigm change, which said that paradigm conflicts are resolved,
not by internal rules of evidence, but by the nature of the social
context in which these conflicts are embedded. For Kuhn’s “external
history” approach to be valid in economics, Blaug (1975, p. 431)
said there must be instances of “(1) internally consistent, well
corroborated, fruitful, and powerful scientific ideas which were
rejected at specific dates in the history of a science because of
specific external factors, or (2) incoherent, poorly corroborated,
weak scientific ideas which were in fact accepted for specific
external reasons.” While Blaug (1975, p. 431) could think of “no
unambiguous examples of either (1) or (2) in the history of
economics,” a political economist’s response would be obvious. The
best example of (1) is the general rejection of the political
economics paradigm due to the domination of a capitalist
system-generated worldview; and the best example of (2) is the
acceptance of the completely unrealistic framework of perfect
competition and efficiency because it conforms to capitalist
important point to make about these paradigm debates is that they
were generally not direct debates. That is, in practice, at least
in the U.S., neoclassical economics has so dominated theory and
practice that the vast majority of research offers only one view,
containing few critiques or alternatives.
Similarly, in education, despite having long-term critics like
Apple, Arnove, Bowles, Carnoy, Gintis, Kelly, Kozol, Levin, Masemann,
McLaren, Stromquist or Weis, the vast majority of educational
researchers still do not seem to be very aware of any challenges.
There are exceptions, but still, critiques and alternatives are
mostly seen only in the writings of critics because it is the
critics who are forced to examine and confront dominant views.
Mainstream educational research, of course, has some internal
debates, but my point is that it is not forced to engage with these
The Labor Market
In the late 1970s and 1980s, there was a significant
theoretical debate in which mainstream researchers in economics and
sociology were forced to engage with their critics. The debate was
over what was called labor market segmentation.
In economics, this was about a challenge to the neoclassical idea
that there was just one big perfect labor market in which success
was determined by your individual human capital characteristics. To
the contrary, political economists and other critics of the
neoclassical story saw an imperfect labor market with fractures
(e.g., divisions into primary and secondary labor markets) and
structures (e.g., large firms, unions, sexism, and racism) that
greatly influenced whether an individual succeeded. In sociology,
this was about a similar challenge to the idea generated from the
dominant structural-functionalist theory and its derivative status
attainment theory that, like economics, argued that individual
success was determined by individual characteristics. To the
contrary, a growing wave of critical sociologists, often sharing a
conflict theory critique of structural functionalism, argued, like
political economists, that success in the labor market was greatly
determined by structural factors.
this debate interesting for many reasons. First, it was a major
debate in both the economics and sociology fields. Second, labor
market success, in terms of measures like income, status, and
mobility, is one of the most studied topics in both fields. Third,
the underlying debate was about one of the most important issues of
our times, the nature of poverty and inequality. The dominant views
in sociology and economics saw labor market inequality as resulting
from individual differences which led to policy recommendations to
improve individuals through education and health, sometimes
accompanied by a blame-the-individual-for failure attitude. The
critics saw the source of poverty and inequality as structural, less
amenable to simplistic solutions and more in need of systemic
reform. Fourth, the outcome of the debate in the two fields was
very different. The debate did not really challenge neoclassical
preeminence in economics but, in sociology, it overturned the
dominance of structural functionalism and status attainment theory.
other feature of this debate that interested me is that proponents
of the dominant perspective were forced to respond directly to
critics. The labor market segmentation debate was one of the rare
instances where we had an actual back and forth between scholars
subscribing to conflicting paradigms. However, it was depressing to
find that both sides used essentially the same, three-point
1. ‘You have no theory.’
2. ‘Your methods are wrong.’
3. ‘Even if we were to use some of your theory and methods, the
empirical results still confirm our original conclusions.’
Now these points
were not empty ones – both sides could explain why they thought the
opposition had flawed theory, methods, and results (though often
through creating straw persons). But nonetheless, there was
something sophomoric about the debate. Although no one went ‘Nyah,
Nyah, Nyah’ you got the feeling they were close. I do not mean this
reflection on the sophomoric nature of the labor market segmentation
debate to imply a ‘plague on both your houses’ conclusion. After
all, I am very much committed to one of the houses, as I explain in
the next section. And, of course, even if you accept my
characterization of this debate, I will leave it to you to decide
how typical this debate is. Unfortunately, I find it very typical.
For the most part, adherents of divergent paradigms do not confront
each other, but when they do, little productive engagement occurs.
Now, this may
simply be the necessary result of true paradigm clash, where
incommensurate theories and methods make scientific resolution of
differences an impossibility. But if that is so, it raises
fundamental questions about how we do research and how we make
decisions. It calls into question the scientific paradigm that
putatively underlies research findings and policy choices today. If
dominant approaches to theory and method cannot guide our policy
choices, then we need to broaden our purview. We need to consider
alternative theories, methods, and practices and need to embed these
considerations in legitimate political processes instead of
pretending they can be adjudicated through some technical,
scientific search for truth.
In economics, in most Western countries, neoclassical
economics predominates. This is true in many but not all developing
countries. This theoretical dominance is linked to the domination
of capitalism (Amin, 1998). Both have become even more entwined and
enshrined over the last twenty years or so, especially with the fall
of the Soviet Union. It has become commonplace for those on the
Right to proclaim that we have reached ‘the end of history,’ ‘the
end of ideology’ (Fukuyama 2006). But, of course, this is not true.
The Left, as a theoretical and practical enterprise, is vibrant,
very much alive and well across fields of study and across the
My disenchantment with neoclassical economics, modernization
theories, and related dominant approaches to understanding education
and development made me search for alternative theories. What I saw
first, in some of my graduate school economics courses, was the work
of U.S. political economists who drew on Marx and a group of
neo-Marxists. What I saw in my education courses with Carnoy and
Levin was how they and others used this framework to understand
education. Issues of class predominated, but while I was in
graduate school I read Simone de Beauvoir’s (1972) The Second Sex,
and that book and a growing leftist feminist movement greatly
influenced how I saw education and development. It was clear to me
that gender and patriarchy were as fundamental to understanding our
social world as class and capitalism. And it also became clear that
issues of race and ethnicity added another important dimension.
This was the time I started doing research that crossed fields and
found that these same critical issues raised in education and
economics were raised throughout the social sciences and applied
I see an even stronger and more remarkable confluence of critical
perspectives than ever before. While they cross disciplines and
applied fields, these alternative theories, such as the following,
are very much interrelated: dependency; world systems, critical,
neomarxist, progressive economic, economic reproduction, cultural
reproduction, resistance, feminist standpoint, gender and
development, socialist feminist, critical race, queer, intersection,
critical postmodern, poststructural, postcolonial, and critical
pedagogy. And this does not include all the related critical
theories within each social science and applied field.
I am not saying that these theories offer identical
perspectives, just that they share essential commonalities with
respect to addressing the two major questions social theories face:
‘How do we understand our social world?’ and ‘What can we do about
it?’ In terms of understanding, most fundamentally, all these
theories are focused on marginalization. They see the world as
composed of systems and structures that maintain, reproduce, and
legitimate existing inequalities. From these perspectives,
inequalities are not system failures, but the logical consequence of
successful system functioning. In terms of what to do, while most
of these theories recognize that reproduction is pervasive, they
also agree that there are serious challenges to reproduction. There
is general agreement that those challenges have two interrelated
sources. One is that the systems and structures that dominate are
not monolithic but are pervaded by contradictions, such as that
between the stated value of political democracy and the reality of
economic authoritarianism or that between the stated value of human
equality and the reality of systematic discrimination. The other is
a belief in human agency, in that oppression can be recognized and
fought individually and collectively.
these theories do not have a name in common, I sometimes call them
“critical theory,” “critical paradigm” or “political economy” and
will use these terms in this presentation to refer to this
intersection of these alternative theories. And, as I see it, it is
border crossing among these theoretical perspectives which forms the
terrain of what we might call critical comparative education today.
Here lies an incredible comparative advantage of our field – it
offers one of the few vantage points from which such an array of
alternatives is studied and therefore where their intersections
In this section
I focus on regression analysis, the statistical technique for
uncovering the causal impact of one variable on another. While some
might see this as technical and esoteric, regression analysis is the
dominant methodology for studying educational and social policy and
practice and it is important that all researchers and practitioners
recognize its fundamental flaws. As a graduate student in
economics, I was well-trained in these quantitative research
methods. This set of methods, involving causal modeling and
estimation via some form of regression analysis, was dominant in
economics at that time (and still is), and since then has become the
dominant method in many social science and applied fields, even
making inroads in anthropology and history. But, even in graduate
school, I had misgivings. The theory behind regression analysis
reminded me of the failure of neoclassical economic theory If the
assumptions of neoclassical economics held, then prices were
accurate indicators of economic value and the invisible hand of
capitalist markets worked. However, the assumptions were such that
they could never be true, and Second Best Theory was unforgiving;
under real world conditions, prices utterly lose their social
meaning. In regression analysis, one also begins with a few
assumptions about your model – that all variables are included,
measured correctly, and their functional interrelationships
accurately specified. If the assumptions hold true, then regression
coefficients are accurate estimators of causal impact. However, in
the real world there are always multiple failures of the
assumptions. Regression analysis theory does talk about the failure
of one assumption at a time, but offers no guidance as to how
inaccurate the resulting regression coefficients are under real
world misspecification conditions.
Despite my misgivings, some of my early research used
regression analyses and, to this day, I insist that students learn
regression analysis and still work closely with students who use it
in their research. Nonetheless, because of these misgivings, I
became very attentive to methodological debates. Part of my focus
has been on the flaws in regression analysis and part has been on
alternative approaches to research methods. In this section I
Anatomy of a
Upon taking a position at FSU in 1981, I was fortunate enough
to work with Sande Milton, a sociologist of education, who shared my
interest and misgivings about these kinds of quantitative research
methods. We found ourselves paying attention to the results of
regression analysis research. In particular, we found ourselves
looking at well-publicized findings in education and trying to
understand on what they were based and what they really said.
years of looking at particular findings, our conclusions were
dismal. We found that literatures using regression analysis
research generally found little in the way of consistent findings.
Those conclusions that were reached offered no consistent
information beyond what you would get from correlation data.
An example was Walker and Schaffarzick’s (1974) review of the
literature on the effectiveness of curricula. After an exhaustive
review of millions of dollars spent on curriculum research, they
came up with two conclusions that could be supported by this
research: 1) if the subject was in the curriculum, students learned
it better than if it were not and 2) if the subject were emphasized,
students learn it even better. While, these are not empty
conclusions, they offer little support for the benefits of a
supposed scientific method.
When, rarely, a particular consistent finding that could be
useful to policy is reported across studies, it often turns out that
the consistency is more in the eye of the beholder than in the
data. One of the few examples touted as a consistent finding was
the positive effect of textbooks on student achievement. Three
regression analysis based studies were singled out in major
literature reviews (Lockheed and Verspoor 1991; Fuller 1987) as
offering convincing research on this point. When I examined each, I
found fundamental flaws in the studies they cited. For example,
Lockheed, Vail, and Fuller (1986), looking at textbook use in
Thailand, used a very difficult-to-interpret textbook variable,
controlled for hardly any teacher or school variables, and actually
found that textbooks did not matter in some of their regressions.
Stephen Heyneman and colleagues (1984), looking at textbook use in
the Philippines, had no pretest measure, no school or teacher
variables were included, and the dummy variable on textbook usage
was very unclear. The third study reported the results of an
experiment that looked at textbook usage in Nicaragua (Jamison et
al. 1981). While, in theory, experiments might not need regression
analysis controls, in practice they mostly do because experimental
controls in field settings are always inadequate.
Dean Jamison and colleagues controlled for a few variables but again
left out almost all home and school variables. Moreover, the
textbook variable was not statistically significant in their
individual data (it was in their group averages) and only showed an
effect for 1st grade not 4th grade. I go back
to this old research because the effectiveness of textbooks is one
of the few consistent findings from this input-output research, yet
we see that the research is fundamentally flawed, subject to rampant
misspecification and contestable interpretations. I want to point
out that these are well-known, competent researchers. It is not my
contention that they did an especially bad job as much as that good
regression analysis practice is simply not possible.
found my observations about these methodological problems
convincing, they were not very likely to be considered convincing
evidence by researchers concerned with methodological issues. So
Sande Milton and I set out to develop a convincing argument using
what is probably the most studied topic using regression analysis,
the determinants of earnings. One of my greatest professional
regrets is that we never published the resulting paper. One of my
proudest moments is that when I sent the paper to Mark Blaug, the
preeminent economist of education, he said that while he was unhappy
with our conclusions, if he were editor of the Journal of
Economic Literature, the top economics journal for literature
reviews like this, he would publish it as is because the profession
needs to discuss it. However, Sande and I thought economists would
not be open to the argument, so we wound up sending it successively
to the top two sociology journals. Both came back to us with split
reviews, one saying, like Blaug, we need to publish and discuss this
and the other saying that we have known about these problems since
graduate school and the authors are throwing out the baby with the
bathwater. Unfortunately, Sande and I got tired of revising it
(Klees and Milton 1986).
This paper was a logical follow-up to the labor market
segmentation research I had been doing at the time since all of this
literature used regression analysis to “prove” their side of the
debate. The argument of the determinants of earnings paper was
straightforward. The three principal conditions necessary for the
regression coefficients of an “earnings function” to be accurate
estimators of true causal impact are very far from being fulfilled.
First, all relevant variables that may affect earnings can never be
included. Our theories literally posit dozens of variables, and
which variables are included in a particular regression study is
idiosyncratic. Examples of variables that some researchers have
considered relevant are: health status, years of schooling, quality
of schooling, type of schooling, cognitive ability, race, ethnicity,
religion, socioeconomic status, gender, immigration status, marital
status, participation in a union, job search, occupation status and
differentiation, labor market segment, firm and industry
characteristics, and many more. Second, we do not know the “right”
way to measure most of these variables. Measurement is ad hoc and
varies from study to study. Third, the functional interrelationship
between variables is not known. While it is common to use the
natural logarithm of income as the dependent variable, even
neoclassical economists admit the basis for doing so is very weak
and, in actuality, what would be needed is to specify some unknown
set of complex simultaneous equations filled with variables subject
to complex interactions.
result of this state of affairs is endless misspecification – by
necessity. Each researcher has an almost infinite array of choices
in how they specify the earnings function they estimate. Each
regression study is never a replication but always different from
others in many respects. The upshot is each regression study is
idiosyncratic. Since it is relatively easy to get significant
coefficients, especially with large data sets, everyone finds their
particular variable of interest to be significant. When there is
controversy, everyone finds empirical evidence to support their side
of the debate. Every segmentation theorist finds labor market
segments to be a significant factor in determining earnings and
other labor market outcomes, yet no neoclassical economist or
structural functionalist sociologist ever does.
Agriculture Production Function Studies
This dismal conclusion that the major social science tool for
empirical research may be a dead end is demonstrable in all its
uses. Perhaps the second most common use of regression analysis is
in education where it is used to estimate what are called
educational production or input-output functions. The dependent
variable usually studied is the score on some achievement test. The
three conditions for proper specification are again impossible to
fulfill. First, the array of potential independent variables is
huge, including, for example: socioeconomic status, gender, race,
ethnicity, age, homework effort, computer use in the home, previous
learning, ability, motivation, aspiration, peer characteristics,
teacher degree level, teacher practices, teacher ability, teacher
experience, class size, school climate, principal characteristics,
curriculum policies, to name a few. Second, there is no agreement
on how to measure most, if not all, of these variables. Third,
again the possible functional interrelationships are innumerable.
Contrary to the linear formulation usually run, recursive and
simultaneous equation formulations with an array of interaction
terms among the independent variables have been posited but little
Literally hundreds of these educational production function studies
have been done. Again, with such an infinite array of specification
choices, almost every study is unique and idiosyncratic. Eric
Hanushek (2004, 1979) has, over the long term, studied and
summarized the results of such studies. Not surprisingly, he and
others have found inconsistent results. However, he and the vast
majority of quantitative researchers cling to the hope that
improvements in models and data can eventually show some clear
results. To the contrary, I see the complete indeterminacy of this
form of research built into the very assumptions on which it is
At one time, I thought that perhaps regression analysis could
work under certain circumstances, such as when it was used to
examine input-output relations that were based on physical relations
more than human and social relations. However, when I got involved
in estimating agricultural production functions – the relation of
agricultural output to the array of inputs that produced it (Ashby
et al. 1980), I found there was no greater ability to fulfill the
conditions for proper specification here than anywhere else. First,
the array of potential agricultural inputs was enormous: from
amounts and type of seed and fertilizer used to the way in which
they were applied; amount and distribution of rainfall and sunshine;
farm size; the number, use, and education of farm workers; farm
management practices and the education of the farmer; access to
credit and extension services; and many more. Second, there was
again disagreement about how to measure inputs and outputs. Third,
there was disagreement about the nature of the functional
interrelationship between them. When I ran my data, I found that
with one specification some farm practices and characteristics were
significant and with a different specification others were, and
there was no theoretical or practical means of choosing among a very
large variety of alternative specifications, each with different
results. This is always the case in regression analysis.
In theory, when using regression analysis, you are supposed
to start with a complete model specification, and then take your
data and estimate it, a one-shot deal. Given the indeterminacy of
model specification, no one does that in practice. In his now
classic article, “Let’s Take the Con Out of Econometrics,” Leamer
(1983, p. 36) describes regression analysis in the real world and
The econometric art as it is practiced at the computer …
involves fitting many, perhaps thousands, of statistical
models….This searching for a model is often well-intentioned, but
there can be no doubt that such a specification search invalidates
the traditional theories of inference. The concepts of unbiasedness,
consistency, efficiency, maximum likelihood estimation, in fact, all
the concepts of traditional theory utterly lose their meaning by the
time an applied researcher pulls from the bramble of computer output
the one thorn of a model he likes best, the one he chooses to
portray as a rose.
question then becomes whether we have learned anything from all this
research? Most quantitative researchers would say they have, but I
believe that such learning, if examined, would turn out to be from a
subset of studies done from a perspective with which the researcher
agreed. As Leamer (1983, p. 37) put it: “Hardly anyone takes data
analyses seriously. Or, perhaps more accurately, hardly anyone
takes anyone else’s data analyses seriously.”
As above, this is the case with the LMS literature. Even
within paradigms, there is no replication. Hardly anyone ever uses
anyone else’s specification without “improving” on it, arguing
explicitly or implicitly that the previous study was incorrect.
These remarks do not imply that, at least within paradigms, there is
no cumulative learning from one another’s arguments. Such learning
does take place. However, the argument here suggests that
regression-based causal inference is simply an excuse for theorizing
but does not provide any valid evidence for it.
I believe that this type of quantitative methodology is a
dead end, no better than alchemy and phrenology, and someday people
will look back in wonder at how so many intelligent people could
convince themselves otherwise. This is not a problem that better
modeling and data can fix. But I should say that I do not see the
essence of the problem as quantification. Quantifying social
phenomena clearly has its limits and, at best, yields approximations
(Samoff 1991). But cross-tabulations and correlations are useful to
suggest interrelationships. As is well-known, however, any
associations found may be spurious and have a myriad of alternative
explanations. The problem is that the causal relations underlying
such associations are so complex that the mechanical process of
regression analysis has no hope of unpacking them. If we are
interested in looking at quantitative data, I am afraid we are stuck
with arguing from cross-tabulations and correlations. This is a
dismal prospect for most quantitative researchers who have spent
years becoming virtuosos at data analysis and see the implications
of my argument as essentially abandoning the research enterprise.
Fortunately for us, if not for them, the research enterprise is
alive and well, with a myriad of alternative methodologies with
which to investigate our educational and social world.
When I went to graduate school, introduction to research
methods courses focused almost exclusively on Campbell and Stanley’s
(1963) examination of the design and analysis of quantitative
experimental and quasi-experimental studies. This is still true
today within certain fields and university departments. However,
the past 30 years has seen a blossoming of alternative approaches to
research methods, especially in education, but in other fields as
well. Education has been at the forefront of such changes, largely,
in my view, because many of the changes were generated within the
field of program evaluation which grew, in large part, from the
educational evaluations that were mandated by the U.S. Congress in
the 1960s and 1970s. Many of those involved in evaluation fieldwork
simply found that the quantitative approach to research and
evaluation could not capture the experience of the programs they
were studying and drew upon other traditions, such as in sociology
or anthropology, or invented new approaches. In subsequent years,
these forays yielded a wide array of alternative methods for
research and evaluation (Mertens, 2004)
For a number of years, I have been fortunate to teach our
department’s Introduction to Research Methods course. While any
grouping of methods is somewhat arbitrary and their labeling always
problematic, the course is divided in three, focusing in turn on
quantitative/positivist methods, qualitative/ interpretive methods,
and critical/transformative methods. There is a large literature on
the qualitative/quantitative debate. Some argue too much has been
made of it, while others, whom I agree with, argue that there are
fundamental differences in outlook that need to be considered (Smith
and Hesushius 1986). Regardless, it is clear that there are lots of
qualitative alternatives to quantitative experimental and regression
analysis approaches, including: case study, ethnography, grounded
theory, phenomenology, narrative, and oral history, to name a few.
Additional methodological alternatives are offered by
critical/transformative perspectives which come out of the array of
theories in the critical paradigm discussed above. These
perspectives criticize the fundamental lack of objectivity of
positivist/quantitative research and qualitative/interpretive
research, arguing that there is no neutral research, and that too
often such studies are done in support of dominant interests.
Critical/transformative research takes an explicit position to work
in the interests of marginalized people. This includes research
under the labels of participatory, action, feminist, indigenous,
critical, critical ethnography, and critical race (Denzin et al.
2007; Mertens 2004).
Proponents of quantitative research recognize that some of
these alternative methods exist but usually, at best, relegate them
to the realm of generating ideas, not to the scientific process of
building knowledge of the social world. To the contrary, many
proponents of alternative methods argue that they are as or more
valid, reliable, and generalizable than quantitative.
For example, Miles and Huberman (1994, p. 434) argue:
Qualitative studies…are especially well-suited to finding
causal relations; they look directly and longitudinally at the local
processes underlying a temporal series of events and states, showing
how these led to specific outcomes, and ruling out rival
hypotheses. In effect we get inside the black box; we can
understand not just that a particular thing happened, but how and
why it happened.
arguments are made for the transferability and generalizability of
qualitative and critical research (Donmoyer 1990).
To conclude, I make a similar point here to that I made under
reflections on theory. If the dominant methodological paradigm is
bankrupt, as I argue, then there is no need for despair since there
are a plethora of alternative approaches that we can use to study
educational and development policy and practice.
There are few, if any, armchair comparative and international
educators. Ours is a field focused on practice and engaged in
practice. While my principal position has been as a university
professor, education policy and practice has always been a focal
point of my research and of my work as a consultant. I have worked
on several dozen projects in as many countries. Given my critique
of theory and method, it is not surprising that I have been and
continue to be deeply critical of educational and development policy
Central to my critique of development practice is my critique
of neoliberalism, which has dominated development discourse for over
a quarter of a century. Neoliberalism derives directly from
neoclassical economic theory. Neoliberals are fundamentalists.
They believe in a strict interpretation of the neoclassical bible of
perfect competition. They generally believe that free markets do
work in the social interest and that government does not.
It is neoliberals who have given us the Washington Consensus and
policies designed to curtail the state, deregulate, privatize, and
liberalize. These practices, embodied in the Structural Adjustment
Programs (SAPs) of the 1980s and 1990s, continue to today as part of
the Poverty Reduction Strategy Process (PRSP). The PRSP, which is
supposed to orient all work by the International Monetary Fund and
the World Bank, is a travesty of its intent; it is not
country-owned, it does not allow for serious consultation with civil
society, and its results look a lot like the bankrupt SAPs. I
believe that as a result of neoliberal-shaped globalization we have
had more than a quarter century of public policies and private
actions that have contributed to the further marginalization of
certain individuals, groups, and nations, while privileging others
Not all neoclassical economists are fundamentalists. Liberal
neoclassical economists dominated in the 1960s and 1970s. They
believe in the need for strong government intervention to correct a
world prone to unacceptable inequalities and market imperfections.
From a critical perspective, this is a vast improvement over a
neoliberal stance but does not go far enough, as I will discuss
As noted above, education policy and practice for the past
quarter century has been embedded in what some have called
neoliberal globalization (Carnoy 1999, 2000). In practice, this has
meant that education policy, as with economic and social policy,
experienced a significant and sustained shift from that of the 1960s
and 1970s (Klees forthcoming). Below, I briefly reflect on some of
these policies and practices.
Of course, learning outcomes are important. But the new
wisdom, especially of the last decade or so, is that they are all
that is important. It used to be that there was a strong focus on
educational inputs and processes – are there schools available and
functional, are trained teachers available, are class sizes
reasonable, are there sufficient books and other learning materials
available, is teaching learner-centered, are students actively
engaged, etc.? Today these questions are rarely asked or answered.
Attention to learning outcomes tends to lead to massive testing
programs (like the relatively useless No Child Left Behind in the
United States) – and little more. It seems that this is the era of
the cheap fix. Neoliberals want to measure learning outcomes, but
no one wants to spend the money it will take to improve learning
outcomes. As a wag pointed out, the current emphasis on testing for
achievement is like giving a sick patient a thermometer instead of
what is needed to cure the illness.
For the most part, this new emphasis on learning outcomes seems to
have gotten us harmful policies like large class sizes and the
hiring of untrained, paraprofessional teachers.
Vouchers, and Charters
Privatization has been the watchword of neoliberalism and has
gotten a lot of play in education. Voucher schemes, which spend
public funds to send some students to private schools, have been
tried in a number of countries with disappointing results. Most
studies show that student achievement is no higher in voucher
schools than in non-voucher schools (Klees forthcoming; Levin 1998).
Charter schools, which are public schools with diminished
regulation, have become quite popular in the U.S., but again the
research does not indicate charter school students have higher
achievement than non-charter school students (Imberman 2007).
Schools, Inc., a U.S. based private company provides a case in
point. It has been hired by many school districts to manage failing
schools. External evaluations report mediocre test results, with
Edison students doing no better than comparable students in public
schools (Levin 2006; Bracey 2002). However, while public schools
are trying to improve student achievement, they must also be
responding to the many other needs expressed by parents and
communities, e.g., providing counseling, health, welfare, policing,
and other services. Yet with all this pressure to serve so many
needs, public schools are still able to produce an achievement gain
equivalent to Edison’s – a company that was hired to focus on just
one outcome, achievement. Why would anyone think that the private
sector can offer anything to improve education generally?
Vouchers and charters will generally increase inequality, as
those who are more well-to-do are able to take greater advantage of
them. Vouchers and charters also leave a weakened and ignored
public school behind. Vouchers and charters are also another one of
neoliberalism’s cheap fixes. Neoliberal reforms, like
privatization, have often been management and governance reforms
which require little in the way of additional resources. They have
become the cheap magic bullet of the Right – you do not have to
worry about providing smaller class sizes, better technologies,
qualified and motivated teachers – you just change the organization
Merit pay – bonuses – for high-achieving teachers and schools
has received periodic attention. There is a simple logic to
rewarding superior performance. However, gains in student
performance may not be due to the particular teacher or the school.
Gain scores reflect the joint effect of many individual, home, peer,
and school influences. While, in theory, some sort of regression
analysis could control for other variables, in practice (as
discussed earlier) we have no ability to specify the analysis
correctly. Different controls lead to rewarding different teachers
and schools with no rational basis for choosing among them. Merit
pay is simply one example of what is wrong with the simplistic
neoliberal policy of tying resources to output. There is nothing
wrong with the concept, but the idea of a simple connection between
measurable inputs and outputs is wrongheaded (Klees forthcoming;
Rothstein 2008). If it was such a good idea, why, for example, is
there not a push to have merit pay schemes for World Bank staff
based on the percent by which their work reduces poverty?
Rewarding good teachers and helping problematic ones can best be
achieved through an evaluation process that reflects the
complexities of teaching and uses professional judgment, principally
by peers (Millman and Darling-Hammond 1990).
The neoliberal era has been characterized by the deprecation
of higher education. In the 1960s and 1970s, education policy paid
attention to the joint expansion of primary, secondary, and higher
education as essential for development goals. In the 1980s, this
changed sharply; primary education was touted, public higher
education was to be cutback and privatized, and secondary education
was put in a holding pattern (World Bank 1995).
Ostensibly, this was based on rate of return analysis that showed
higher returns to primary education. However, these rates of return
were based on both failed economic theory and flawed empirical
analysis, as described earlier, thus forming an illegitimate basis
for such important decisions (Klees forthcoming; Samoff 2007). Even
if you accept rate of return evidence, research in the late 1990s
concluded that we had been vastly underestimating the returns to
higher education by not taking into sufficient account the diffuse
social benefits which neoclassical economists call externalities.
This admission of error called into question 20 years of policies
that cut back higher education, harming individuals and nations,
making them less competitive in the world system. Political
economists would argue, however, that it is not rates of return but
the interest of the world system that led to these policies; that
is, these policies had the effect of making developing countries
more dependent on competing in terms of low wage labor, which was
desirable in the 1980s and 1990s, but now, given the high costs of
higher educated labor in richer countries, it is more useful to the
world system to have developing countries compete more in this
Women’s Movement of the 1960s and 1970s, fueled by a confluence of
liberal, neoliberal, and critical perspectives, has had far-reaching
social consequences. Among other things, more attention has been
paid to women and development. While the nature of this
consideration has been contested, in education, this has translated
into attention to girls’ and women’s education, particularly the
former (Diaw2005; Stromquist 1997). Expanding girls’ education has
been explicitly a part of Education for All (EFA) goals and the
Millennium Development Goals (MDGs). And, since 1990 when EFA was
initiated, there has been some progress in increasing girls’ access
to primary and secondary school (UNESCO 2003).
However, that progress has been slow and tortuous. EFA goals
on gender equity were postponed from 1995 to 2005, and then in 2005,
neither EFA goals nor their equivalent in the MDGs were met – nor is
it clear when they will be met. When we go beyond access to look at
quality issues, a much larger gender gap exists, as girls continue
to be disadvantaged in multiple ways in the educational process.
Nelly Stromquist (1997, p. 60) discusses the need to go even
further, beyond first and second level responses to access and
quality, to a third level response “that challenges the ideological
and material conditions that support women’s inferior position in
society.” However, progress in challenging the patriarchal nature
of most societies is slow, and neoliberal policies either do not
recognize the problem or avoid the issue. For example, girls’
education as it is practiced by most development agencies is
determinedly apolitical, often carefully divorcing itself from the
activities of the local and national women’s movement, thus
weakening its progressive potential (Stromquist et al. 2000).
EFA and the MDGs
As with girls’ education, other original EFA goals were
postponed. Also, the nature of the goals changed (Torres 2000).
Before the World Conference on EFA in Jomtien, Thailand in 1990,
World Bank staff told other participants in the EFA planning process
that the focus had to be on primary education; adult education could
not be included or the Bank would withdraw its support.
Nonetheless, at Jomtien, pressure by NGOs and others included
attention to adult education as part of EFA goals. In practice,
however, the attention was almost exclusively focused on attaining
universal primary education (UPE) and, in the Dakar meeting
revisions, and in the MDGs, UPE became the centerpiece (King 2007).
The original EFA goals envisioned UPE by 2000. In the
mid-1990s, it was clear this would not be achieved, and so the goal
was postponed to 2015 (as it is in the MDGs). However, it is clear
today to all analysts that under current efforts, the goal will not
be even close to being fulfilled in 2015, nor will any of the other
MDGs concerning poverty, health, gender equity, and the environment.
Yes, again, as with girls’ education, you can point to some
progress. But UPE goals and promises have been made every decade
since the 1960s, with some progress, yet no fulfillment.
So what is happening here? Hans Weiler (1984) called it
compensatory legitimation; more colloquially, I see it as a form of
‘good cop, bad cop.’ Shaky and poorly-performing economies,
increasing poverty and inequality, widespread conflicts, and the
equivalent of structural adjustment policies everywhere, all call
into question the legitimacy of the social order – this is the bad
cop. To compensate for this, actors in the world system of
neoliberal globalization must introduce polices, for example, EFA
and the MDGs, aimed at ameliorating some problematic conditions and
thus restoring legitimacy – this is the good cop. This argument
does not question the good intentions of the proponents of these
policies but does question their effects. Simply having these
policies may be sufficient for compensatory legitimation; fulfilling
them, judging by past experience, seems to be less important. This
does not have to be so. We could achieve UPE and the rest of the
MDGs in a very few years if we were willing to devote the kind of
resources and attention that go into current priorities, like the
been more than a quarter century since the Reagan/Thatcher tide
resulted in a neoliberal shift around the world. Based on
ideological beliefs, without convincing supporting evidence, the
array of economic and educational practices above were rapidly
introduced. Clearly, alternative practices are possible. In
education, we could pay attention to the provision of sufficient
educational inputs as well as the attainment of educational outcomes
and not restrict the latter to test scores. We could focus
attention on improving public schools instead of instituting various
forms of privatization. We could pay all teachers well and have a
professional peer evaluation system. We could use a vastly expanded
early childhood development system that truly tried to compensate
for the unequal starts that children get. We could reinvigorate
public higher education and make it the centerpiece of a
postsecondary education system. We could face head on the
challenges of a sexist world and situate our attention to the
education of girls within that. And, as I said above, we could
fulfill our rhetorical commitment to EFA and the MDGs.
There are two questions raised by these examples of
alternative practices. First, what will it take to make them a
reality? Second, from a critical paradigm perspective, do they go
far enough (Chan 2006)? Let us start with the latter. More
fundamental reforms of educational practice are needed. Some
examples: In education, perhaps the best developed critical
alternative approach is rooted in the work of Paulo Freire and
elaborated in an extensive literature on critical pedagogy (McLaren
and Kincheloe 2007). There also has been interesting work on the
implications of the global environmental crisis for a radical
ecological education (Orr 1991). Given the prevalence of war and
conflict, there has also been a burgeoning attention to the
substantial transformations needed to make peace education an
integral part of our K-16 curriculum (Lin et al. forthcoming).
Applying a human rights perspective, there has also been significant
work on the right to education and a rights-based education (Klees
and Thapliyal 2007; Tomasevski 2003).
From a critical perspective, we will not have successful
educational reform without fundamental social and economic
transformation. This means more than a rejection of neoliberal
policies and a return to the liberal, welfare state policies of the
past, although, given present circumstances, at least some critics
would welcome such a return. In practice, however, those policies
did not offer a serious challenge to poverty, inequality, and
marginalization. There is much analysis of what would constitute a
serious challenge. There is exciting work being done on the theory
and practice of alternative forms of social and economic life under
such topics as participatory and deliberative democracy (Fung and
Wright 1999); economic democracy (Alperovitz 2004; Hahnel 2005);
global governance (Riker 2005); feminism (Sanders 2002); human
rights (OHCHR 2002); alternative forms of globalization (Cavanagh
and Mander 2002); and ecological economics (Daly 1996; Daly and Cobb
From a critical perspective, transforming our education,
economic, and social systems are not straightforward. They cannot
simply be accomplished with the right policy but are results of
struggle (Ginsburg et al. 1991). The contradictions inherent in
today’s world system – most importantly, perhaps, the contradictions
between a rhetoric of fairness and equality and a reality that is
fundamentally unfair and unequal – open up spaces for action.
Critics place a lot of faith in human agency to take advantage of
those spaces – human agency being the individual and collective
transformative choices and actions of those who share a critical
perspective. At the individual level, for example, teachers can
sometimes offer an alternative education in their classroom, or
development agency staff can sometimes defy current policies.
Nongovernmental organizations (NGOs) have developed programs,
principally at a local level, that offer alternative approaches to
dealing with our most difficult education and development problems.
NGOs and other civil society organizations can collectively lobby to
challenge neoliberal education and development policies. And,
perhaps most promising, social movements – such as the women’s
movement, civil rights movement, alter-globalization movement,
landless movement, Dalit movement, the human rights movement,
children’s rights movement, and others – bring the power of
collective action to confronting the unfairness and irrationality of
Resistance and agency have changed laws and constitutions.
We have seen entire governments change, as in the pervasive shift to
the left in Latin America in countries like Argentina, Bolivia,
Brazil, Chile, Ecuador, Nicaragua, Uruguay, and Venezuela.
Even in the U.S.,
the pendulum may be swinging back to the liberal side, if a Democrat
gets elected President. While this still leaves us far from a
critical stance, it could open up influential global agencies like
the World Bank to some alternative perspectives.
Theory, method, and practice concerning educational, social,
and economic issues are hotly contested. Unfortunately, given the
unparalleled dominance of neoliberal globalization, in many national
and international settings, that contestation is hardly visible.
Even relatively mild forms of dissent are often met by disregard,
hostility, or derision. I remember being struck by George
Psacharopoulos telling me that some of his colleagues at the Bank
called Lester Thurow, the noted liberal neoclassical economist,
In this presentation, I have argued that dominant economic
and social theories are bankrupt and need to be replaced by the
array of theories that have been developed focusing on understanding
and overcoming marginalization. I have also argued that the
dominant, causal modeling, quantitative approaches to research have
been a dead end, and should be replaced by an array of
methodological alternatives, including many that depart from a
critical perspective. Finally, I have argued that our practices
have failed to achieve their stated goals; instead of leading to a
more effective and equitable system, they have been reproductive of
our unjust and unequal social order.
I learned early on that you had to take sides in these
debates. One of my earliest consulting projects was looking at the
effects of nonformal education and information provision to poor
Guatemalan farmers in the highlands. After our presentation of the
final report, representatives of farmer groups thanked me because I
was the only one who mentioned that the real problem was not
education and information, but that these farmers had been
marginalized by pushing them onto tiny plots of unproductive land.
Similarly, I have been thanked – and critiqued – many times for
raising such unpopular political issues. I say this not to praise
myself, I have not done much, but to underline what many of you
know. There is nothing neutral about anything we do. Every paper,
every project, every talk, and every class (if you are a teacher) is
embroiled in these controversies. In the interests of progress, it
should be incumbent on all of us – no matter what your position --
to recognize, acknowledge, and engage these debates.
Presidential Address delivered to the Comparative and International
Education Society annual meeting, New York City, March 17-21, 2008.
I would like to thank Susanne Clawson, Dave Edwards, Mark Ginsburg,
Jing Lin, Nelly Stromquist, Joel Samoff, and the students in my
“Alternative Education, Alternative Development” class for their
comments on an earlier draft. This paper will be published in the
Comparative Education Review.
Some scholars lament the absence of clear boundaries to the CIE
field (e.g., Epstein and Carroll 2005). While I understand the
difficulties this causes, as illustrated in the discussion of core
knowledge below, I think the advantages are many, as illustrated
throughout this address. Moreover, given the interdependent and
complex nature of today’s social problems, the boundaries are also
becoming much more ambiguous in many traditional disciplines.
The late George Papagiannis, my colleague for many years at Florida
State University, used to talk about how all our students needed to
be prepared to be “scholar-doers,” crossing that theory/practice
Below, I talk of the focus of my early research on what I call the
Left/Right debates, which sometimes became up-close and personal.
At the March 1981 CIES meetings, George Papagiannis had organized
the opening plenary to be a debate between Michael Apple and C.
Arnold Anderson. When, at their conclusion, I asked Anderson a
question that reflected those Left/Right debates, his response was
that I ‘needed to study more economics.’ As you can imagine, as a
new faculty member brought in to teach economics of education, my
students had a field day with that one.
Actually, there is no such thing as “close to” efficiency. The
system as a whole is either efficient or it is not, and when it is
not, one cannot say how inefficient is it (Rakowski 1980).
This sentiment attributed to Adam Smith misrepresents his work and
the degree to which he was very critical of free markets (Schlefer
It is not meaningless to call specifically for fewer dropouts or
improved learning, but that is quite different from a statement that
primary education is a more efficient investment than higher
While the market is a convenience that future, saner, societies may
continue to use for some purposes, it has two fundamental flaws that
render it problematic. First, it contributes to an abrogation of
social responsibility, as today, when market outcomes of horrendous
inequality or environmental destruction are seen as natural, not
anyone’s fault. Second, markets are fragile. For example, millions
of small decisions can contribute to economic or environmental
crises. See Hahnel (2005) for a discussion of alternatives.
In part, this reflects what I mean by the comparative advantage of
CIE. With my border crossing, what can be seen as an isolated
phenomenon of paradigm debate becomes a fundamental schism that
crosses all theory and practice.
Dani Rodrik’s (2007) new book illustrates the problem – recognizing
the challenges within neoclassical economics but not being able to
see anything outside. For him there is only “one economics.” See
Palley (2007) for a critique.
In countries where critical perspectives have reached majority
status, like Brazil, this is less true.
(Beck, Horan and Tolbert 1980; Cain 1976; Carnoy 1980; Hauser 1980).
I am sure this shift to the left in American sociology had many
causes, but a principal one from my perspective is that the younger,
critical sociologists (radicalized during the Vietnam War era) had
become professionals just as the field’s methods became more
quantitative, and therefore they were better-trained in these
methods. They literally out-regressed the old guard!
(e.g., Beck, Horan and Tolbert 1980; Cain 1976; Carnoy 1980; Hauser
While I focus on the intersection of this array of critical theories
and think it is important and useful to critical scholarship and
practice to do so, it is also important to recognize that each of
the theories mentioned above has its own traditions and foci, that
there are important differences between and among them, and that
they often are quite insular, not communicating with each other.
Regression analysis is supposed to unpack correlations and estimate
quantitative causal impact. In practice, variables are not
consistently significant, and the size of a coefficient, upon which
policy implications depend, varies considerably.
In theory, randomized experiments do give good evidence of cause and
effect. However, in practice, they do not for two reasons. First,
experiments cannot be carried out on most phenomena of social
interest. For ethical reasons, we cannot engage in long-term
experimentation with significant aspects of people’s lives. Second,
when we can do field experiments over the relatively short-term with
things like vouchers, class size, Head Start, or welfare, we no
longer have laboratory conditions. Control groups always turn out
systematically different than the experimental group, and the result
is we no longer have the ability to make clear inferences. Instead,
we must use regression analysis or its equivalent to control for
differences between the two groups. However, this becomes an ad hoc
exercise, even worse than the causal modeling approach where at
least you try to accurately specify the relevant intervening
variables and their interrelationships. The results, in practice,
are that you never get any more agreement on the results of
experiments than on those of regression analyses.
I should also point out that, of course, we know that textbooks can
contribute to student achievement (depending on what is in them, how
they are used, etc.), but my point is that this kind of research
cannot even capture whether, in a particular case, textbooks affect
achievement, let alone the more important policy question, how much?
These classifications are not clear and neat. For example, you can
do grounded theory or feminist research from positivist,
interpretive, or critical perspectives. To make things more
confusing still, critical paradigm researchers may avail themselves
of any method, including quantitative ones.
Given the problems with the positivist paradigm, there is
considerable literature examining alternative criteria for good
research for the interpretive and critical paradigms (Mertens 2005).
An irony is that it has been neoliberals in public service who have
been so vehement in attacking the idea of public service, arguing
that all government workers follow narrow self-interests. Yet I am
sure they think that they themselves are the exceptions, engaging in
true public service. But perhaps they do prove their point as their
policies have served to lead to a massive concentration of wealth,
enriching themselves and their cronies.
There is also the important issue of whether it is a good
thermometer; achievement testing only captures a very narrow range
of educational outcomes.
It should be noted that the studies of voucher critics and voucher
proponents generally use the type of multivariate regression
analysis I argued is suspect. Nonetheless, here and elsewhere I
will use it simply to make the argument in the proponents’ own
These reforms led in many countries to the proliferation of low
quality private postsecondary institutions, a cutback of lower and
middle class access to quality institutions, and a differentiated
system that exacerbated inequality (Klees forthcoming).
The original EFA goals talked of basic education, not primary
education, but this too got narrowed.
According to a recent estimate, 67 countries will not reach UPE by
2015 (Glewwe and Zhao 2006).
Such stories are commonplace about other noted economists like
Michael Piore (1983) and Joseph Stiglitz (2003).
[ go to the CIES Forum ]
Alperovitz, Gar. 2004. America Beyond Capitalism: Reclaiming Our
Wealth, Our Liberty, and Our Democracy. New York: John Wiley.
Amin, Samir. 1998. Spectres of Capitalism. New York:
Monthly Review Press.
Ashby, Jacqueline, Klees, Steven, Pachico, Douglas, and Wells, Stuart.
1980. “Alternative Strategies in the Economic Analysis of
Information/Education Projects.” In McAnany, Emile, ed.,
Communications in the Rural Third World: The Role of Information in
Development. New York: Praeger Publishers.
Arnove, Robert, and Torres, Carlos, eds. 2007. Comparative Education:
The Dialectic of the Global and the Local (3rd ed.). New
York: Rowan & Littlefield.
Beck. E.M., Horan,
Patrick, and Tolbert, Charles. 1980. “Social Stratification in
Industrial Society: Further Evidence for a Structural Alternative, A
Reply to Hauser.” American Sociological Review, 45: 712-719.
Blaug, Mark. 1975.
“Kuhn versus Lakatos, or Paradigms versus Research Programmes in the
History of Economics.” History of Political Economy, 7: 399-433.
Bracey, G. 2002.
The War Against American’s Public Schools: Privatizing Schools,
Commercializing Education. Boston: Allyn & Bacon.
Cain, Glen. 1976.
“The Challenge of Segmented Labor Market Theory to Orthodox Theory.”
Journal of Economic Literature, 14: 1215-1257.
T., and Stanley, Julian. 1963. Experimental and Quasi-Experiemntal
Designs for Research. Chicago: Rand McNally & Company.
1999. Globalization and Educational reform: What Planners Need to
Know. Paris: UNESCO. Available at: http://www.unesco.org/iiep
2000. “Globalization and Educational Reform.” In Karen Monkman and
Nelly Stromquist, eds. Globalization and Education: Integration and
Contestation Across Cultures. New York: Rowman & Littlefield.
Cavanagh, John, and
Mander, Jerry, eds. 2002. Alternatives to globalization: A better
world is possible. San Francisco: Berrett-Koehler Publishers.
2006. “Between Efficiency, Capability, and Recognition: Competing
Epistemes in Global Governance.”
Comparative Education, 43(3): 359-376.
Daly, Herman E., and Cobb, J. 1994. For the Common Good: Redirecting
the Economy Toward Community, the Environment, and a Sustainable Future.
Boston : Beacon Press.
Daly, Herman E. (1996) Beyond Growth: The Economics of Sustainable
Development. Boston: Beacon Press.
Denzin, Norman, Lincoln, Yvonna, and Smith, Linda. 2007. Handbook of
Critical and Indigenous Methodologies. Thousand Oaks, Ca.: Sage.
Diaw, Codou. 2005. “Gender and Education in Sub-Saharan Africa: The
Women in Development (WID) Approach and its Alternatives.” In A. Abdi
and A. Cleghorn, eds., Issues in African Education: Sociological
Perspectives. New York: Palgrave Macmillan.
Donmoyer, Robert. 1990. “Generalizability and the
Single-Case Study.” In W. Eisner and A. Peshkin, eds., Qualitative
Inquiry in Education. NY: Teachers College Press.
Epstein, Erwin and Carroll, Katherine. 2005. “Abusing Ancestors:
Historical Functionalism and Postmodern Deviations in Comparative
Education,” Comparative Education Review, 49(1): 62-88.
Fukuyama, Francis. 2006. The End of History and the Last Man.
NY: Simon and Schuster.
Fuller, Bruce. 1987. "What Factors Raise Achievement
in the Third World?" Review of Educational Research, 57: 255-292.
Fung, Archon, and
Wright, Erik Olin. 1999. Experiments in Empowered Deliberative
Democracy. www.ssc.wisc.edu/ wright/deliberative.html.
Ginsburg, M., Cooper, S., Raghu, R., and Zegarra, H.
1991. “Conceptual Issues in ‘Educational Reform’: Ideology, the State,
and the World Economic System.” In M. Ginsburg (ed.) Understanding
Educational Reform in Global Context: Economy, Ideology, and the State.
New York: Garland Publishers.
Ginsburg, Mark, and Price-Rom, Alison. 2008. “Comparative and
International Education.” In E. Provenzo, ed.
Encyclopedia of the Social and Cultural Foundations of Education. Thousand Oaks, CA: Sage.
Glewwe, P., and Zhao, M. 2006. “Attaining Universal Primary Education
Schooling by 2015: An Evaluation of Cost Estimates.” In Cohen, J.,
Bloom, D., and Malin, M. Educating All Children: A Global Agenda.
Cambridge, MA: American Academy of Arts and Sciences.
Hahnel, Robin.(2005. Economic Justice and Democracy: From Competition
to Cooperation. NY: Routledge Press.
Hanushek, Eric. 1979. “Conceptual and Empirical Issues in the Estimation
of Educational Production Functions.” Journal of Human Resources,
14 (3): 351-88.
Hanushek, Eric. 2004. “What If There Are No ‘Best Practices’?”
Scottish Journal of Political Economy, 51 (2):156-72.
Hauser, Robert. 1980. “On ‘Stratification in a Dual Economy’ (A Comment
on Beck, Horan, and Tolbert 1978).” American Sociological Review,
P., Jamison, Dean, and Montenegro, Xenia. 1984. "Textbooks in the
Philippines: Evaluation of the Pedagogical Impact of a Nationwide
Investment." Educational Evaluation and Policy Analysis, 6
Imberman, Scott A. 2007. “Achievement and Behavior in Charter Schools:
Drawing a More Complete Picture.” Occasional Paper No. 142. National
Center for the Study of Privatization in Education, Columbia University,
New York, NY.
Jamison, Dean, Searle, Barbara, Galda, Klaus, and Heyneman, Stephen.
1981. “Improving Elementary Mathematics Education in Nicaragua: An
Experimental Study of the Impact of Textbooks and Radio on
Achievement.” Journal of Educational Psychology, 73 (August):
King, Kenneth. 2007. “Multilateral Agencies in the Construction of the
Global Agenda on Education.” Comparative Education, 43
Klees, Steven. 1986. “Planning and Policy Analysis in Education: What
Can Economics Tell Us? Comparative Education Review, 30
Klees, Steven. 1993. “Resource Allocation and Social Choice in
Education: The Need for Alternative Criteria and Processes.” Washington,
DC: World Bank, Higher Education Group.
Klees, Steven. 2007, forthcoming. “A Quarter Century of Neoliberal
Thinking in Education: Misleading Analyses and Failed Policies,”
Globalization, Societies, and Education.
Klees, Steven, and Milton, Sande. 1988. “Inferences from Regression
Analysis: The Case of Earnings Functions.” Paper Presented at the
Annual Meeting of the Comparative and International Education Society,
Toronto, Canada, March 13-16, 1986.
Klees, Steven, and Wells, Stuart. 1983. “Economic Evaluation of
Education: A Critical Analysis in the Context of Applications to
Educational Reform in El Salvador.” Educational Evaluation and Policy
Analysis, 5 (3): 327-346.
Klees, Steven, and Thapliyal, N. 2007. “The Right to Education: The Work
of Katarina Tomasevski.” Comparative Education Review,
Leamer, Edward E. 1983. "Let's Take the Con Out of Econometrics,"
American Economic Review 73
Levin, Henry. 1998. “Educational Vouchers: Effectiveness, Choice, and
Costs.” Journal of Policy Analysis and Management, 17: 373-392.
Levin, Henry. 2006. “Our Schools in the Year 2030 Forum: How Will They
Be Different?” Education Next. Available at:
Lin, J., Brantmeirer, Ed., and Bruhn,
C., eds. (forthcoming). Transforming Education for Peace: Educators
as Peace Makers. Greenwich, CT: Information Age Publishing.
Lockheed, Marlaine E.,
Stephen C. Vail, and Bruce Fuller. 1986. “How Textbooks Affect
Achievement in Developing Countries: Evidence from Thailand.” Educational
Evaluation and Policy Analysis, 8 (Winter): 379-392.
Lockheed, Marlaine E., Adriaan M. Verspoor, et. al.
1991. Improving primary education in developing countries. Washington,
DC : Oxford University Press.
McLaren, Peter, and Kincheloe, Joe, eds. 2007. Critical Pedagogy:
Where Are We Now? New York: Peter Lang.
Merten, Donna. 2004. Research Methods in Education and Psychology:
Integrating Diversity with Quantitative and Qualitative Approaches
(2nd ed.). Thousand Oaks, CA: Sage.
Miles, M., and Huberman, A. 1994. Qualitative Data Analysis (2nd
ed.). Newbury Park, CA: Sage.
Millman, Jason, and Darling-Hammond, Linda.(1990. The New Handbook of
Teacher Evaluation: Assessing Elementary and Secondary School Teachers.
Newbury Park, CA: Sage.
Noah, Harold, and Max
A. Eckstein. 1966. "A Design for Teaching 'Comparative Education'."
Comparative Education Review 10 (October): 511-513.
OHCHR. 2002. Draft Guidelines: A Human Rights Approach to Poverty
Reduction Strategies. New York: United Nations.
Orr, David. 1991. “What is Education for: Six Myths about the
Foundations of Modern
Education, and Six New Principles to Replace Them.”
The Learning Revolution (IC#27), p. 52.
Palley, Thomas. 2008. “Breaking the Neoclassical Monopoly in Economics.”
Available at: www.thomaspalley.com
Piore, Michael. 1983. “Labor Market Segmentation: To What Paradigm Does
It Belong?” American Economic Review, 73 (2): 249-253.
Rakowski, James. 1980. “The Theory of the Second Best and the
Competitive Equilibrium Model.” Journal of Economic Issues, 14:
Riker, James. 2005. “Promoting Visions and Strategies to Advancing
Global Democracy.” Unpublished paper. The Democracy Collaborative,
University of Maryland, College Park, MD.
Rodrik, Dani. 2007.
One Economics, Many Recipes: Globalization,
Institutions, and Economic Growth. Princeton, NJ:
Princeton University Press.
Rothstein, Richard. 2008. “Holding Accountability
to Account: How Scholarship and Experience in Other Fields Inform
Exploration of Performance Incentives in Education,”
National Center on Performance Incentives Working
Paper 2008-04. Washington, DC: Economic Policy Institute.
Samoff, Joel. 2007. “Institutionalizing International Influence.” In R.
Arnove and C. Torres (eds.), Comparative Education: The Dialectic of
the Global and the Local, 3rd ed.. New York: Rowan and
Samoff, Joel. 1991. “The Façade of Precision in Education
Data and Statistics: A Troubling Example from Tanzania,” Journal of
Modern African Studies, 29 (4): 669-689.
Saunders, K. (ed.). 2002. Feminist Post-Development
Thought. NY: Zed.
Schlefer, Jonathan. 1998. “Today’s Most Mischievous Misquotation: Adam
Smith Did Not Mean What He is Often Made to Say,” Atlantic Monthly,
121 (3): 16-19.
Smith, John, and Heshusius, Louis. 1986. “Closing Down the Conversation:
The End of the Quantitative-Qualitative Debate among Educational
Inquirers.” Educational Researcher,15 (1): 4-13.
Stiglitz, Jospeh. 2003. Globalization and Its Discontents. New
York: W.W. Norton.
Klees, Steven, and Shirley Miske. 2000. “USAID Efforts to Expand and
Improve Girls’ Primary Education in Guatemala.” In Cortina, Regina, and
Stromquist, Nelly P., eds. Distant Alliances: Promoting Education
for Girls and Women in Latin America. New York: RoutledgeFalmer.
1997. Increasing Girls' Participation in Basic Education.
Institute for Educational Planning.
Tomasevski, Katarina. 2003. Education Denied: Costs and Remedies.
London & New York: Zed Books.
Torres, Rosa-Maria. 2000. One Decade of Education for All: The
Challenge Ahead. Paris: UNESCO.
UNESCO. 2003. EFA
Global Monitoring Report 2003/4, Gender and Education for All: The Leap
to Equality. Paris: UNESCO.
Walker, D., and Schaffarzick, J. 1974. “Comparing Curricula.” Review
of Educational Research, 44 (1): 83-111.
Weiler, Hans. 1984. “The Political Economy of Education and
Development.” Prospects, 19(4):468-477.
World Bank. 1995. Priorities and Strategies for Education.
Washington, DC: World Bank.