This is not a typical research paper. This is a paper about research. In 1994, four fairly recent doctoral graduates of The University of Georgia were discussing the possibilities of publishing on the World Wide Web. We established Instructional Research Online (InTRO) as a repository of research papers, media, and other artifacts related to research in Instructional Technology. Defining a direction for InTRO was, and remains an iterative process. The website (available: http://intro.base.org) includes sections on reviews of books and media, original research, links to resources in instructional technology, and interviews with research leaders. One feature that has helped define InTRO and its purpose has been the section on interviews with research leaders.
These interviews are informal conversations with people who conduct research in the field of instructional technology. The purpose of the interviews is to gain insight into the processes and issues of research in the field of instructional technology by conversing with people who have significant experience engaging in research in and tangential to our field.
Our practice of interviewing research leaders may have grown out of a doctoral seminar in research at the University of Georgia taught by Tom Reeves. The culminating assignment of the course was to focus on a research leader in instructional technology and to read from that person’s body of research. Analyzing this body of work, you could begin to see what somebody’s research agenda was, how their research had remained focused on a single area, shifted slightly, or shifted radically. The final phase of this assignment was to contact this person and interview them. All of the information was assimilated and presented back to the class.
The practice of interviewing research leaders was a valuable one. We may have learned as much from talking to these people personally as we did by reading journal articles authored by them. We certainly came to realize that people who are regarded as leaders in research in our field can also be pleasant people who are interested and willing to talk about their work. Much of what was learned about research in this class was through contact with people on a personal level. One reason for this may be that much of what people know is not set forth in print. It is informal knowledge, or patterns of understanding that may not be documented, codified, or shared formally.
Most of us will recognize that there are times when we learn as much working on projects as we learn sitting in classes. For example, while one may study front end analysis it is not until one does an actual analysis that one realizes the practical application of gathering information about an instructional environment. Work in practical settings helps inform, reinforce, and extend what we study. One reason for this is that when we work on projects we work with people who know more than we do, and we are able to tap into the informal body of knowledge that they hold. This is true in the study of research as well. Informal conversations about research are important in helping form opinions on research, and in helping people figure out how to conduct research.
In our interviews we were interested not only in what kind of research these people were doing, but we were also interested in what advice they might have to people in the field interested in conducting research. It remains our contention that there is much to be learned from people who conduct research. Fittingly enough, our first research leader interview was with Tom Reeves.
Selection of Leaders
In every field there are individuals who are regarded as leaders or, if you will, bellwethers for the field. Their work is generally recognized as important because it not only informs the field, but it also shapes the direction of the field. Their work may not always be agreed with, but it is recognized as important and significant. It helps to challenge what we know in order that we may ultimately know more.
The selection of leaders for our interviews is an ongoing process; we continue to conduct and publish interviews. Our original selection of leaders to interview was certainly based on their contributions to the field, but other factors may have played a role in their selection as well. The InTRO editorial board started as a team of young researchers. While we did the initial screening without any formal selection criteria, we did work to select a balance of perspectives. It was and remains a goal to include people from different institutions and different research areas. We have included not only people conducting research in academe, but also people conducting research in industry.
We tended to interview people who had a significant number of publications, and who had been in the field for more than 10 years. We looked to people who were active in professional organizations such as AECT, and we have striven to include a diversity of views from a diverse group of people. Additionally, there is a fair amount of self-selection among our interviewees as a number of our requests for interviews have been declined. Consequently the people represented in these interviews do represent good, interesting, and challenging perspectives, though they may not represent every perspective at this point. It should be stated that the InTRO editorial board is always open to suggestions on research leaders to interview.
Analyzing the Interviews
The purpose of this paper is to identify and report on an analysis of the published interviews. Interviews are conducted via email. Typically, we review a researcher's work, and then email them a set of questions based on that review that we think will be of interest to our audience. Once we receive the researcher's response we then send a set of follow-up questions. We compile these with the original responses, ask the interviewee to review it, and then post it on InTRO.
The following analysis represents a unique opportunity for the reader. All of the data we have available to us is available to you. This analysis represents what was said during the course of the interviews, and unlike other types of interviews used in research, we were not able to go back and ask for clarification after analysis had begun. This is not, in the strictest sense, a research article. It is a synthesis of comments, suggestions, and other important points made by the interviewees. We make this distinction because we feel that there are conclusions that we think are appropriate, but are simply not borne out in the interviews. There are conclusions that could be drawn in this paper based on personal experience and the interviews, but we shall refrain from making those in this paper.!! We wish to concentrate on setting forth only what can be drawn from the interviews themselves. There are obviously many interpretations that are possible based on the interviews. This paper and the accompanying presentation represents one. We invite you to draw others for yourselves. And with that invitation we also extend to you the invitation to formalize your interpretations and submit them to InTRO for publication.
How to identify a problem: A question of questions
The journey of a thousand miles is begun with a single step. This may be true, but if you are unclear on the direction in which your first step should be, your journey may not go that well. This is the problem many people have in conducting research: it is hard to identify a unique question. The interviews provide some help in this area.
Gaining confidence
Nearly everyone working on research issues has faced the problem of identifying an area in which to do research. There is no shortage of important questions to ask, or important issues to study according to the interviewees, but they do recognize that people find it hard to get started in research. Most people who find it hard to identify a problem may not suffer from a lack of ideas, but rather from a lack of confidence that their ideas were good and viable. It is suggested by our researchers that one thing that has made them "good" at research has been believing that their ideas were important and of interest. They have learned to analyze questions, and to "trust their instincts" when looking at problems in the field. One thing that has helped them "trust their instincts" has been the involvement of a good mentor or good colleagues who can serve as a sounding board for idea. Mentors and colleagues can help people gain support for their ideas and give them the encouragement they need to pursue them.
Reviewing the literature
There are many ways that a researcher can gain support for their ideas. One is by careful consideration of the existing research. One sound and practical method suggested on how to identify a research area is to think about the areas that you want to know something about and go read about them. As you read you may find "holes" in the literature, that is to say areas in the literature where significant questions are going unanswered. If you want an answer to a question, but there exists no answer in the literature, then that is an area of research to pursue. The "holes" are where you should be working. Reading and understanding the literature is critically important, as is the ability to read and synthesize large bodies of research. One researcher feels that the work they have done in writing literature reviews is the most difficult, but possibly the most important work that they have done.
Finding studies and other articles that support your research ideas is only one reason to consider the literature carefully. Finding absolutely no consideration of your ideas in the literature is a fairly compelling indication that a pressing need for your work exists. Again, this is looking at the holes in the literature and focusing your work to fill those holes. As you begin to do research, your work begins to help answer important questions, but may not be able to answer every question. We are reminded by many research leaders that few individuals will make a difference with one piece of work. Collectively we contribute to the body of literature in the field. The knowledge base grows as more people ask and answer questions. The combined total of all of this asking and answering is what we know in the field of instructional technology. It is contributing to this body of knowledge that is important.
Finding questions in real world environments
Still, many people are simply trying to get an idea started. Reading the literature and finding holes in the problem may seem overwhelming to some. Beyond being overwhelming, it may not be practical to gain all questions from the literature. Looking at naturally occurring environments is one way to find out what questions are out there. One piece of advice is for researchers to go to educational environments, such as schools, and observe people using an educational product. We should look to see where they are having problems, what is making sense to the user, and what is confounding the user. Noting these problems can provide excellent sources for ideas in research.
Combining observations and literature can help inform your reading, and help to isolate problems that literature can then be associated with. It is not only the observations of people using other people’s products that can lead to interesting research projects, but it is also observing people working with products that we develop that can inform research questions.
Instructional technologists are involved in a number of efforts. In addition to reading, we may be working on media design and development projects. Practical experience, especially for people new to the field, provide us with activities that enable us to gain a better insight into the way the field works, and what it means to work in the field. Many of the researchers interviewed here point to the close bond between research and development. Problems encountered during a development project often lead to interesting research questions. Leaders who have worked in industry state that their experience in project work has been the driving force in many of their research activities. This points towards the important relationship between research and development.
The relationship between research and development
Instructional Technology is an active field, and it is important to remember that in the final analysis it is our job to make a positive impact on the way that people learn. Saettler (1990) reminds us that technology in our field is defined quite broadly. We work with hard technologies, sometimes referred to as "things that plug in," but we also work with soft technologies, or "advances in thought." Always, we are looking forward. Though much of what we do is practical in nature, even studies in seemingly esoteric areas such as post modernism are used to inform our work. In our field, research certainly influences practice. However, this is a reciprocal arrangement because often times in instructional technology practice influences research.
Blurring lines
Most research leaders find that the boundaries between work interests and research interests blur significantly. Many researchers with advanced degrees have worked for years before returning to study for an advanced degree and pursuing a career that involves research. Because of this, many of their initial research questions came from their work experiences. As they continued to do research, they also continued to do work in the field. Some of that work may have been in the design and/or the development of educational programs. For many of our interviewees, they were involved in significant evaluation activities. While the type of work may vary, what remains constant is the connection between work experiences and research projects. Many researchers may serve dual roles. They may design the environment conduct research on both the process of building the environment and the ultimate value of the finished environment. This indicates a very strong connection between research and development, and suggests that it is not only common for researchers in instructional technology to also be designers and developers of instructional products, but that it is also highly desirable.
One researcher recommended that working on projects was so important that young academics should do it for free if need be. The practical applications of this work can greatly improve your skills as a researcher and keep you in touch with what the field is doing. So while most leaders in research tend to credit the real life experience they have, they recognize it as not only important in giving them experience, but in keeping them current as well.
Connecting research and practice
Combing research and development may be desirable, but what can be difficult is making the results of your research valuable to others. We face a problem with research in our field according to our researchers. Chiefly we don’t connect practitioners to research as seamlessly as we might desire. It is often assumed that the role of a researcher is to tell the practitioner what to do. However, people who are working in the field everyday have many ideas that can make interesting research questions, or may have answers to many significant questions in our field. In instructional technology, many active researchers are also practicing professionals, but the connection between practice and research should be made stronger. It has been suggested by some leaders that our research efforts should be focused on making information available to practitioners in schools, industrial, or military applications. Our research, and the results of our research, need to be more accessible to practitioners. Our research, and our field, needs to make a greater impact on the environments in which we work; our research results need to be more practical.
The most important piece of work
Finding interesting work and letting it guide your research can make your research not only more valuable to the field, but more interesting to yourself as well. Many of our interviewees are remembered for a "seminal work" or a particular theory or line of inquiry that they have been working on for a period of time. However nearly all of them expressed that the most interesting thing that they had done was what they were working on now. This indicates a passion for research, and a commitment to new ideas. It would be hard to be involved in a project if you were not interested in it. Therefore one recommendation made is that you work to find projects that can hold your interest.
Doing research on your work is a piece of advice that is often heeded by many people. Working in the field is both professionally and personally rewarding. Because many of the researchers point out that we need to be studying naturally occurring environments to see how people learn, working in environments becomes even more critical. Finding a naturally occurring environment in which you can work is not easy. Establishing yourself by working on a project, or in an educational environment can make research populations available to you.
Choosing a mentor
There is much to know about research. There is the problem of identifying questions, seeking funding, managing the process, and ultimately disseminating your work and writing for publication. One of the most important things a novice researcher can do is to seek out a mentor to help them learn and master this process. The definition of mentor here is not limited to the relationship between a student and a professor in graduate school. A novice researcher is well advised to seek out others in the field who can help them work in the area of research. Many leaders advise not only the novice to search out mentors, but also the mentors to treat their apprentices in a collegial manner.
The role of the mentor
Mentors can do a lot for an apprentice. Beyond simply teaching them how to do things, they can introduce them to people in the field, or involve them in research and writing projects. Apprentices gain valuable experience in writing for publication from their mentors. But ultimately what the mentor can offer to an apprentice is confidence, and confidence is one of the most important skills one can develop as a researcher. To submit your work to the rigors of the referee process, one needs to have a bit of an ego, and a thick skin. Mentors can help novice researchers develop these in the right amount and appropriate manner. One researcher relates a story of how they were "cooked" good by a discussant at a conference. The presentation went abysmally, but ultimately the paper was reworked and published. The research, ideas, and the paper had value, but that value wasn’t recognized during the first presentation of the idea. It was not until sometime later that the paper made it into publication. The point is that simply because one person does not like or agree with a research idea it does not mean that the idea should not be pursued. Mentors can help apprentices see this and gain the skills and confidence they need to continue working when an idea is rejected.
The role of the apprentice
An apprentice can offer much to the mentor as well. Often times, it is an apprentice that gets their mentor involved in a particular project or line of research. One suggested reason for this is that mentors work with more and more apprentices. While a mentor may have spent their time reading and writing in the past, more and more of their time is spent working with more and more people. It becomes a challenge to stay abreast of everything that is going on in the field. Often times it is the apprentice, who can focus their efforts and energies more narrowly, who can point the mentor in a new direction. The point here is one of collegiality. It is generally recognized that people working together can bring diverse views, skills, and ideas to a project. The same is true of the relationship between the mentor and the apprentice. Care should be taken in choosing a mentor. You should choose a mentor who not only publishes, but who publishes with their apprentices. Novice researchers, or people in their first academic position, should choose a mentor where they work who can help them find resources, locate outlets for publishing, and help them learn to write grants.
Most people recognize that the mentor has much to offer the apprentice. It is suggested here that the apprentice also has much to offer the mentor. This reciprocal relationship promotes not only a greater understanding of the field to the novice researcher, but also helps keep the mentor current in working with others in the field.
One caveat-one of our interviewees points out that there simply may not be enough mentors to go around. Mentoring takes a lot of time and entails a special relationship between the faculty and students. Given the fairly large student faculty ratio in many of our programs it may not be possible for every student to have a mentor. In fact, it may be the case that if your are fortunate enough to have a mentor, then you are fortunate indeed.
A question of methods
The issue of whether to do qualitative or quantitative research is one that most researchers are faced with. Historically, much of the research that was done in education in general and instructional technology specifically, was quantitative in its planning and execution. For many years, faculty supervising research were trained solely in statistical research methods. Today, more and more researchers are looking to alternative research methodologies such as qualitative research.
Research traditions
Much of our research methodology has been derived from the hard sciences. There are many reasons for this. Historically the field of education has constructed controlled environments in which to test single or multiple variables. Most often these were contrived environments, and if you look at research in the field of instructional technology you will see that most of the research in the past several years still follows this paradigm (Reeves, 1995).
Many researchers interviewed here state that their initial research involved isolating single variables and working with them in a controlled environment. One reason for this may be that many of our research leaders originally were psychologists, or educational psychologists trained in the dominant research paradigm of the times. The dominant research paradigm for many years has been quasi-experimental studies. People researched in a manner in which they were taught to research. Some leaders suggest that much of their initial research agenda, while interesting to them, was also driven by the tenure and promotion process. A combination of history and environment has driven the field for years in the types of methods and research tools it uses. Currently, most leaders are recognizing that there are more tools available to researchers today and are seeking to use these tools in research.
Mixed methods
A recurring message from nearly all our research leaders is to avoid the qualitative versus quantitative debate entirely. This debate has been characterized as "mindless" by some people. Our research history and much of our current research recognizes the need to isolate variables. However much of our current research also realizes that variables in the real world are not really controllable, and they rarely behave like the variables that can be controlled in the world of physical science. Most of the people interviewed here work with the philosophy that in order to answer certain questions you will need certain sets of tools, and therefore suggest that researchers study and develop skills in both qualitative and quantitative research methods. When an environment presents itself you will then be ready to study it regardless of what questions may need to be asked, or what tools are needed to answer the questions.
Additionally, some of the leaders interviewed here suggest that beyond qualitative and quantitative research methods, there exists the need to consider issues in yet more divergent research paradigms. Among these are the methods associated with post modernism, semiotics, and historical research. Additionally there are at least two researchers who make the argument that our research not only needs to be more responsive to practitioners, but that it needs to be more responsive to society as well. There is small, but growing call in the field for socially responsible research in instructional technology.
Disseminating research
Research is done to inform practice, and the results of research are meant to be shared with the larger community. Disseminating research begins with having questions that need answering, and culminates in the publication of that research.
The research agenda
Traditionally research is disseminated in two formal ways: the conference presentation and the journal article. Informally the results of research can be spread through informal conversation among colleagues. People need to share the results of their research so that they can not only make sense out of what they are doing, but also so that may gain insight into the next phase of the research, or the next study in a long term research agenda. Many people speak of research as a pipeline. You conduct work on an idea that gets proposed to a conference. The initial paper is done and presented at a conference. Based on reactions and feedback at the conference the paper is reworked for submission to a refereed journal. While you are working on one project, you should be considering other issues that you can work on later. Work that can be proposed to conferences the next year will keep you busy after one piece goes in to print. The goal is to keep the pipeline full at all times, meaning that there should always be a project to work on. For those in a tenure earning position, it should be noted that the tenure process is too short to begin work in distinctly different areas, but that rather you should focus your work within a common framework. This common framework or area of research is often referred to as a research agenda.
For most researchers, the research agenda will start with the dissertation or thesis and move forward from there. While many researchers can and do make dramatic shifts in their research interests, most tend to work in abroad area such as evaluation, computer-based learning environments, electronic performance support systems, cross cultural research, or the processes associated with instructional design to name but a few. The key point is that by staking out an area in which to work a researcher does a couple of significant things. They focus on key problems and become recognized as a person who can inform this small area of the field. Taken in concert with other people working in either similar areas, or in completely different areas, a fabric is woven that constitutes the body of research in the field. By establishing a research agenda, or a line of inquiry, a researcher gets to the business of making a contribution by advancing the knowledge of the field. As you become recognized as an expert in that area, you can begin to help others conduct research in that area, and you can begin to make connections between your research area and other research areas. For example a person with significant research experience in distance education could team up with a person with significant experience in hypertext to conduct research in web-based instruction. The two areas inform each other, and can be used to move forward into a new area. The two areas do not represent mere appurtenances to each other, but rather represent unique applications of divergent areas.
Writing research
People in the "real-world" are interested in our work, but they find it hard to put their hands on the information they need easily. This is suggested by more than one researcher. Our field is filled with three letter acronyms (TLA’s) that are often seen as cliquish in nature and make it sometimes more difficult for people to use the results of our research than we would intend. While it raises the question of "who do you write for, it may also suggest that rather than saying things to sound intellectual, we should simply say what we mean. We should avoid confusing language, or, as the bumper sticker says: "Eschew obfuscation."
It is a recurring theme in nearly all of the interviews that one of the biggest problems we face as a field is in having our work make a difference in the life of the individual, and that one of the ways in which we can achieve this is through the dissemination of our research. Clarity of writing is stressed, as is the age-old soul of wit, brevity.
Conferences and journals
Conference presentations and journal articles remain the two most common ways to disseminate research. While not mentioned in the interviews directly, it is well known that the lag time between writing and publication can be fatal to a new idea: by the time the new idea gets published the idea is already generally recognized as being obsolete. Most of us will have to begin planning what our conference presentations will be so long before the conference even begins that some ideas may be found to be inappropriate by the time the conference arrives.
One reason that it can take so long for research results to be published is the referee process. It does take time for busy people to read their colleagues’ work carefully and make decisions on its appropriateness for publication. Web-based publishing has been mentioned by more than one researcher as an alternative to the traditional paper based journals. While changing the medium itself is not likely to yield dramatic results, it is possible that it could speed up both the publication speed and help increase access to the published research.
Conclusions
Hannafin and Hannafin (1995) warn that in the future IT might "continue its emphasis on training practitioners, but will be unable to forge its own destiny and advance its own research." (P. 320) It is important that we pay greater attention to research and to training researchers if we are to avoid this somewhat dire consequence. Becoming a researcher is not the same thing as learning to do research. We can learn research methodology in our classes on quantitative and qualitative analysis. We can learn to do literature reviews through research seminars. But knowing these things does not guarantee that one will be able to put them together in an intelligent, scholarly, and creative fashion. Only by working with those among us who are already doing quality research, can we began to discover how to synthesize these elements in such a fashion as to do quality research ourselves. There may not be enough mentors to go around. But we hope that through the interviews with the research leaders on InTRO we can assist those mentors among us in reaching and in teaching enough of us to make a difference.
Hannafin, M.J. & Hannafin, K.M. (1995). The status and future of research in instructional design and technology revisited. In G. J. Anglin, (Ed.). Instructional technology: Past, present and future. Englewood, CO. Libraries Unlimited. (pp. 314-321).
Reeves, T. C. (1995).Questioning the questions of instructional technology research. Instructional Technology Research Online. [Online] Available: http://intro.base.org.
Saettler, P. (1990).The evolution of American educational technology. Englewood, CO. Libraries Unlimited.
Seels, B. B, & Richey, R.C. (1994). Instructional technology: The definition and domains of the field. Washington, DC. The Association for Educational Communications and Technology.