Research rich with description is the new standard

The greatest flaw of traditional social science research is that it’s not specific enough.

However, that doesn’t make the research invalid, said Lee S. Shulman, at the plenary session on the second day of the ISSOTL 2013 conference held at the Raleigh Convention Center and hosted by Elon University’s Center for Engaged Learning.

Shulman, President Emeritus of the Carnegie Foundation for the Advancement of Teaching and Charles E. Ducommun Professor of Education Emeritus at Stanford University, delivered the speech titled “Situated Studies of Teaching and Learning: The New Mainstream.”

More than 600 professors, college administrators and students from around the world registered for ISSOTL 2013, an annual program of the International Society for the Scholarship of Teaching & Learning, of which several Elon University professors are active members. Shulman urged those in attendance to find a way to “repair this imbalance in the world” of research.

 

​

Shulman, whose work examined the quality of teachers and teaching from elementary school through professional and graduate school, talked about the limitations of what he referred to as the “gold standard” in research. “It gains its reputation, it gains its standing by studying people in situations that are unlike anything in the real world, therefore, they can be generalized to everything in the real world,” he said.

While powerful ideas are often the product of such work—and he wasn’t suggesting they be disregarded—studies that delve into the particulars and are rich with description are equally valuable. He said that his hope was that “we recognize that generalization-oriented studies in their place are valuable. They are in no way more valuable than situated investigations that don’t make a claim for generalizations but make the claim for getting smarter about a piece of the world we care about when we need to be smarter about it.”

With a combination of examples and a healthy sense of humor, Shulman illustrated why situated research, such as studies of teaching and learning, is the “new mainstream” and should be seen as the gold standard for educational scholarship.

Shulman referenced a recent clinical trial regarding the benefits of using a certain drug to treat brain tumors. The results of the trial, which included double-blind experimental studies, random assignments and the highest research standards, indicated that the drug didn’t have any significant benefits in treating a specific type of brain cancer.

“Except it did have benefits,” Shulman said. “If you looked at hundreds and hundreds of cases, there were patients whose brain tumors went into remission when they took this medication.”

What the study said was that “the drug wasn’t ineffective. It just wasn’t universally effective” so it was labeled as not having significant benefits.

“But notice what is being said,” Shulman pointed out. “Real research—the kind of research that makes it into journals—is research that intends to make statements about something that is true in general for the most part. It seeks generalized statements. It seeks universal statements.”

But, as a result of that study, researchers can now ask different questions and focus on patients who had positive outcomes. Shulman cited it as an example of how generalized studies alone aren’t always useful.

“Most of the knowledge we gain is not universal,” he said. “It is not general. It is specific to particular situations and that’s not trivial. To know the world as it really is, that’s no minor accomplishment.”

Shulman also emphasized that generalizations don’t last forever. “I don’t care how strongly you feel about them or how strong you feel the evidence is. They decay.”

Doing a study once and publishing it isn’t enough when the world is constantly changing and technology is evolving.

“What does it mean to teach inquiry skills to students who can call up Wikipedia in 10 seconds? What’s the most important thing to teach them?” Shulman asked.

 

The norm used to be to teach students how to find information, but it’s no longer the case. In today’s world, students must be taught “how to dig their way out of a flood of information and make some sense out of it,” Shulman said. “It all changed because the context changed, the students changed. That makes a difference.”

President Leo M. Lambert introduced Shulman and expressed his gratitude to the international society and its members for strengthening teaching and learning in classrooms and institutions around the world.

“It’s a terribly important contribution, especially at a time when state governments like North Carolina’s are retreating on what historically had been near sacred commitments to higher education,” Lambert said. “…  I cannot think of a time when higher education opportunities mattered more in the world. So much hangs in the balance that will undoubtedly be decided by the future leaders in our classrooms today. We must remember our business is not awarding diplomas. It is human transformation.”

​

Visit issotl13.com for more information on the conference.