Mitchell, 'Research Methods Using Computer Networks', Arachnet Electronic Journal on Virtual Culture v2n04 (September 27, 1994) URL = http://hegel.lib.ncsu.edu/stacks/serials/aejvc/aejvc-v2n04-mitchell-research The Arachnet Electronic Journal on Virtual Culture __________________________________________________________________ ISSN 1068-5723 September 27, 1994 Volume 2 Issue 4 EJVCV2N4 MITCHELL Research Methods Using Computer Networks Tony Mitchell Department of Chemistry St. Cloud State University St. Cloud, MN 56301 tmitchel@tigger.stcloud.msus.edu Marcin Paprzycki Department of Mathematics and Computer Science The University of Texas of the Permian Basin Odessa, TX 79762 paprzycki_m@gusher.pb.utexas.edu George Duckett Deakin University, Burwood Campus Faculty of Education 221 Burwood Highway Burwood, Victoria 3125 Australia gduckett@deakin.edu.au INTRODUCTION The collection of data has always been a major difficulty in doing survey-based research. The advent of global computer networks connecting millions of people opens new possibilities and creates new problems for such research. In this paper we wish to first address means of doing collaborative research using computer networks and, second, describe a method for collecting and disseminating data through computer networks. DISTANCE-BASED COLLABORATION In 1991, two of the authors of this paper (Mitchell and Paprzycki) began a collaborative research project dealing with the use of computer networks in education (1,2). Since both were faculty members at the same institution (UTPB), this effort was done in the "traditional" manner of collaboration. Because the collaborative work dealt with computer networks, it seemed logical to use that medium as a means of exchanging information (manuscript drafts, notes, references, etc.) when one of the authors (Mitchell) moved to Minnesota. Three papers (3, 6, 7) and two notes (4, 5) as well as a number of presentations were prepared using this method of collaboration. We are not alone in using computer networks for collaborative research. There have been at least four other articles prepared using this approach (8, 9, 10, 11). Holderness noted that collaborative research through computer-based communication is rapidly increasing and no longer limits small groups to working at one site (12). This is also illustrated by the location of the researchers in this group (Australia, Texas, Minnesota with a fourth member in North Carolina). Holderness, however, takes the view that such approaches are somewhat restrictive because not all individuals have access to computer networks. This is a rather unfortunate view because it reinforces the stereotype that users of computer-based communications must also be computer programmers and that, only through computer programming, is an individual able to develop skills in this area. Second, Holderness does not take into account the rapid changes in this area of communication. The large number of local and commercial bulletin board systems (BBS) show that access is not limited to select individuals. Many systems are designed to help the non- programmer make an efficient use of the system (much in the same manner prevailing personal computers no longer require the mastery of obscure commands). Some of these BBS' can facilitate the exchange of messages locally or connect the user to large networks, e.g., Compuserve, Delphi, America Online and others. In addition, there are a number of states which have created networks for use by high school and elementary teachers (1). The rapid growth of electronic media for communication is also evident in the number of discussions which center on how to cite electronic communications in manuscripts and the rise in the number of electronic journals (a rise forecasted in 1, 2). Put together, these developments have made the computer-based exchange of ideas extremely popular and led to situations similar to McLuhan's vision of a global vision (for example, the idea of electronic town-hall meetings presented during the 1992 U.S. Presidential elections). Following the publication of the East-West Conference materials Duckett joined our group to undertake further study in the area of computer literacy. During the latter part of 1992 and through 1993, our group communicated almost exclusively through e-mail and used selected software to transfer data between the three locations in order to develop two surveys, determine who and how would receive the surveys, distribute the surveys, collect the data and prepare an initial report (18). In view of the distances between the three sites (Minnesota, Texas, and Australia), this approach was the only advisable means of completing this research. An iterative method was developed to facilitate the collaboration process. Essentially, one of the three authors would draft a part of the manuscript and send it to the others for comment, corrections, and additions. The next in line would add his comments, make corrections and/or additions and send it to the third author who would then do likewise and return the manuscript to the first author. This process would then be repeated until a satisfactory result was obtained. It was by this method the computer literacy surveys for 1992 and 1993 as well as this particular manuscript were prepared. DESCRIPTION OF RESEARCH In the remainder of this paper, we would like to describe how we used computer-based communications to initiate and complete two research projects. The first of these projects, an outgrowth of the development of a computer literacy course at St. Cloud State University, began in January, 1992. The course was developed to help prospective elementary and secondary teachers become computer literate. The research study resulted when we attempted to determine what were the basic computer literacy requirements in the United States. This study involved several computer-based discussion groups (listservs) related to computers and education. The approach used in this study was similar to the Delphi method developed by the RAND corporation for the United States Air Force in the early 1960's (13). The Delphi method uses multiple mailings to groups of individuals and is designed to obtain a consensus about a selected topic. Following the initial mailings, the results of previous mailings are included in the process. Initially, messages were sent to five listservs (AETS, CNEDUC, DEOS, DTS, and ERL) asking for information about computer literacy requirements for prospective teachers in the various states and how such requirements (if any) were met. The second mailing was sent to the CTI listserv in addition to the previous five lists to elicit additional information, comments, and to check for accuracy. Two additional mailings were made to the above lists and to the NEWEDUC and IPCT lists as well. The results of this first study have been summarized in (6). The aim of the second study was to further specify the computer literacy requirements for pre-service teachers (as they exist currently and as they should be in the future). In addition, because previous work had indicated that there was a lack of consensus about the definition for computer literacy, the second study attempted to define computer literacy in terms of pre-service teacher education. While developing the survey, a list of seventy listservs related to computers and education was developed for a list of all education related listservs (14). This new list represented a broader distribution than the first survey. Having completed the initial data collection, we are currently conducting a second survey to confirm the results of the first study as well as obtain additional information. PRACTICAL CONSIDERATIONS There are a number of issues that need to be addressed when collecting survey data by using computer networks. The first is related to the hardware involved. When engaged in distance collaboration, it helps if the platforms, as well as the software, used by collaborators are compatible. When a variety of materials are exchanged, it makes it easier when they can be viewed in a finished, formatted form. Otherwise, transferring information must be done in a straight text form with limited formatting. A word of caution is in order when information is transferred using e-mail. Some mailers have difficulty with non-ASCII characters. There are two ways around this problem. When both sides of the transfer are UNIX environments, the "uuencode" command can be used. Otherwise, the use of ASCII based characters is strongly recommended. In addition to being aware of the systems other members of the collaboration group are using, it is also important to be aware of any changes in the systems which might occur during the research period. During the development and release of the 1993 surveys, computers serving the Australia and Minnesota sites independently suffered problems with the software used to support e-mail. In the case of the SCSU computers, the situation was a routine change in software and communication with SCSU resumed within the week. The problem at the Australian site was more complicated, involving both software and hardware, and took longer to resolve. It is not known how these crashes, which occurred during a time of maximum communication between sites and when a high volume of responses was expected, affected the response rate. It can be however assumed that they could have, as we do know for a fact that there were e-mail messages that were never delivered due to the mailer problems. While it is impossible to predict major system failures, such as the Australian problem, one should check with system administrators concerning routine system changes, such as what occurred at SCSU. Knowing when the system may be inactive can help in laying out the schedule for any communication which may need to be done. DISTRIBUTION OF SURVEY Survey research must focus on two problems, the size of the survey instrument and the distribution of the instrument. While size is a critical issue in any survey research, it is especially critical when using e-mail. If the instrument is too long, then people will not be ready to spend the required time answering all of the questions. This could result in partially completed surveys as well as a reduction in the total number returned. As it pertains to our current research, we decided to split the instrument into two parts. Part one was distributed to individuals via the chosen listservs and contained a place when they could indicate if they were willing to participate in the second part of the study. The second part was then sent to them electronically (where possible). How the instrument is distributed and problems related to this distribution is also of concern. While we sought to distribute the survey instrument via selected listservs, it has been suggested that this method encourages duplication since many individuals are members of more than one listserv. As it is possible to obtain a list of the members for a given listserv, it has been suggested that those seeking to conduct surveys should do so and thus contact individuals directly. Based on our experiences, we feel that this method, while eliminating duplication, is not a practical approach for distribution. During the 1992 study, the third mailing was done in this manner. The questions and received responses to the first two mailings were sent directly to the members of the identified listservs. However, in doing so, a header of some 250 lines was produced. The result was that many individuals did not see the message (such a lengthy header would take two or three screens to clear before the reader finds the real message). One individual responded that the amount of "garbage" present in the mail header was enough to make him not participate. One could also create a "list" of lists and send the survey questionnaires and materials out in bulk. This may also not be the best approach. Many listservs require that an individual be a member of the listserv before submitting or disseminating information through the list (this is especially true for moderated lists). This would eliminate any gains made through a bulk distribution approach. Second, as indicated by the comments from one postmaster, even using a bulk method does not eliminate the lengthy header. Additionally, the number of surveys currently passing through both Internet and Bitnet is increasing and lengthy headers could lead to the perception that such surveys are noise and spurious efforts rather than serious scientific efforts (15). One solution to the distribution problem might be the development of a specific listserv which would facilitate the processing of a large number of messages. This would enable messages to be archived by researchers and retrieved by interested individuals. It would also create a shorter message for specific listservs. The development of such a listserv would, by design, require the cooperation of local computer services and the availability of the appropriate software. The recent increase in the use of GOPHER software offers another possibility for the transfer of information. The results of the current study suggest a return to the original process of submitting inquires to a limited number of listservs. since the life of a questionnaire is rather short (7 - 10 days), it would be possible to first identify those listservs where the interest matches the central research question. The survey could then be sent to members of those selected listservs. After a stated period of time, the results would be tabulated and returned to the original listservs along with additional listservs for comments and additional responses or information. This also validates the effort by returning the summary of results to the original participants. Appendix A contains additional information we feels needs to be considered when using computer networks for research purposes. METHODOLOGICAL ISSUES One important question is "Will using computer networks work as a research tool?" There are a number of issues related to that question which must be addressed. The major benefit to using computer networks is that questionnaires can be distributed to specific groups in a short period of time, and depending on network configurations, more rapidly than traditional methods. Quite often, such research is discouraged because there is no assurance that the survey materials will returned in a timely manner. Using computer networks makes such return easier to accomplish. In addition, assembling the data as a database and processing the date can be easily automated. The basic problem, inherent in all survey research, is the small rate of return. This problem is amplified when computer networks and listservs are used. Given that one individual may be a member of several listservs, it would be difficult establishing the exact number of questionnaires that were distributed. How this factor is controlled is still being considered. As it was discussed above one possible solution is the use of a Delphi-type approach (5). After distributing the initial materials and waiting for sufficient responses (indicating in the materials what deadlines apply), the compiled data will be returned to the initial survey population (i.e., listservs), as well as others that may be applicable, asking for corrections, comments, and additional responses. If individuals are interested in participating in the study, they can contact the researchers individually to obtain survey materials. This procedure can be repeated as needed/determined in advance to increase the number of responses and study possible changes in response patterns. It should be noted that this approach is best suited for research attempting to arrive at a consensus about a given question. If one is simply obtaining data without seeking to arrive at a consensus, this approach should be modified (for instance, by not distributing the previous results, which could influence responses). It should also be pointed out, that taking into account membership in a given list may change over time, repeated mailings to a selected list can increase the number of participants as well as allow the study of changes in response patterns. One criticism of using computer networks to do survey research is that it results in "scatter shot" surveys and, as such, is not a wise approach. We would argue that, if the survey is specific in nature, the listservs chosen with the same specificity, and there is an understanding of how and when the materials are to be distributed and collected, the survey would not be of the "scatter shot" variety. A second criticism might be that by conducting a survey using computer networks, one is eliminating a portion of the population (in a similar way to that suggested by Holderness (12)). This is an important point. People without access to computer networks in general or the specific listserv where the message is being distributed would, in all likelihood, not receive the message. We feel that, as the number of individuals using networks increases (16), and with the survey targeted to a specific population by the choice of listserv(s), the probability of missing a portion of the population will decrease proportionally. This brings into question the issue of specificity. If the questions used in the survey deal with the use of computer networks and/or other computer usage related issues, then that portion of the population who do not use computer networks are not likely to be interested in the study and would not like respond (or even be capable of responding). As has been previously stated, one must take care in designing a survey which meets the needs of the targeted population. A careful selection of listservs is advisable in order to obtain as large a sample population as possible. There is at least one current research project dealing with the informational structure of discussion groups (17). The outcome of this research may provided addition information about the effectiveness of computer-based research techniques. CONCLUSION What advantages are there in using computer networks for research? For surveys dealing with computer literacy, such as ours, computer networks would seem to be a natural medium as the people who can be reached in this manner may be considered computer literate. Second, because of the globality of computer networks, a multi-cultural dimension is added to the survey population. For surveys in general, the two most obvious ones are speed and access to a large number of participants from a wider sample area (i.e., the world) in a shorter period of time. If the goal of the study is to gain a specific set of information, the computer networks allow a rapid means of accessing such information. If the goal of the survey is in doubt, unclear, or does not involve issues addressed by selected listservs, then the results gained using computer networks will be no better than other survey methods. ACKNOWLEDGEMENTS We wish to thank Professor Diane Kovacs and Professor Gwen Pearson for their useful comments in the preparation of this manuscript. APPENDIX A The following comments may be of use to those contemplating using computer networks as a means of collecting research data. 1. Check with system administrators prior to the initiation of any major data collection activities to see if any maintenance or software changes/upgrades are scheduled and if they will interfere with the processing of e-mail during the project's activity. 2. Identify the lists best suited for the intended study (see reference 14). If the list is moderated, it may be advisable to contact the moderator/list owner privately to discuss the materials to be distributed. Some moderators/owners are very protective of their lists and, unless you contact them in advance with a justification for the study, may reject it out of hand. Otherwise, it will be necessary to include the time spent allowing the moderator to review the materials in the overall time frame of the project. 3. Sending a mass distribution through e-mail requires careful thought and planning, especially where the number of addresses included in a single broadcast is considered. Trying to send all of the messages in one try could cause confusion if the messages start to bounce back as undeliverable or rejected. It becomes very difficult to determine just which got through and which did not. If you are unable to interrupt the returned mail messages, you may be sending duplicate e-mail, causing postmasters, list moderators, and list owners a great deal of anxiety. This in turn could cause them to delete your message. 4. If you wish to use a direct mail approach for the members of a given listserv, it is possible to obtain a list of members from the records section of the listserv. Unless specifically requested by an individually, the electronic address of that individual is included in this file when the subscription begins. 5. To eliminate confusion and anxiety, for all parties, we recommend the following procedures when sending a message to a large number of lists: a. Subscribe to the list you are interested in. Many lists require that you be a member of the list prior to sending information over the list. We recommend that you become a member; if you are not, there is the possibility that your message will be rejected by the server prior to any review by the moderator/list owner. By subscribing to the list in question, even if it is only for the time of the study , you will know that the submitted materials have been distributed. b. Prior to the distribution of materials contact the list moderator. If the moderator will not allow the message to be distributed, ask for permission to send a short message asking interested individuals to contact you directly. c. Titles/subject lines are important! Use the title/subject line to draw attention to your message. Many people skim titles and, if the title is uninteresting or missing, will simply delete the message without reading it. d. Include a short description of what does the message contain, so the uninterested individuals can discard it without further reading. e. Because of how systems are set up, your message may not appear immediately after submission. Before moving on to the next step, wait until you receive confirmation (through a receipt of the message) that the message has been distributed. f. Wait a few days to see if anyone replies to the original message before unsubscribing the list. (It is a common error for newcomers to lists to send their replies to the listserv rather than to the requesting individual.) g. Do not assume that, because your e-mail address is included in the header or footer of the message, the receiver will see it or receive it; include the appropriate return addresses in the body of the message. Some software strip headers and footers from the original message. Also, remember that some lists are not accessible because you do not have access to the appropriate gateway. This is especially true in the case of international mail; you may need to find someone who can post the original message and forward the responses back to you. h. Ensure that the returns are sent to the person responsible for the collection and recording of the data (if more than one person is working on the project). i. When using word processing software to develop the instrument and other materials, use basic text characters. The translation of some text may become garbled or otherwise unintelligible to some software. We found that the document should be formatted to a column width of 70 characters with a return at the end of each line. This allows for variations in receiving software and reduces the number of problems in reading the document. REFERENCES 1. Tony Mitchell and Marcin Paprzycki, "An Overview of Computer Networks in Education: Computer Networks and Network Services", Proceedings of the Second Annual South Central Small College Computing Conference, Journal of Computing in Small Colleges, 6(5), 1991, 1. 2. Tony Mitchell and Marcin Paprzycki, "Computer Networks in Education", Journal of Reading, 35(3), 1991, 264. 3. Marcin Paprzycki and Tony Mitchell, "An Overview of Computer Networks in Education: Using Computer Networks in the Classroom", Proceedings of the Fifth Annual Southeastern Small College Computing Conference, Journal of Computing in Small Colleges, 7(3), 1992, 27. 4. Tony Mitchell and Marcin Paprzycki, "Teaching New Technologies", Proceedings and Abstracts of the East-West Conference on Emerging Computer Technologies in Education, Moscow, 1992, 140. 5. Marcin Paprzycki and Tony Mitchell, "How Computer Networks Will Influence Education", Proceedings and Abstracts of the East-West Conference on Emerging Computer Technologies in Education, Moscow, 1992, 145. 6. Tony Mitchell and Marcin Paprzycki, "Teaching New Technologies: Developing a Computer Literacy Course for Prospective Teachers", Proceedings of the Sixth Annual Southeastern Small College Computing Conference, Journal of Computing in Small Colleges, 8(3), 1993, 65. 7. Tony Mitchell and Marcin Paprzycki, "Using Technology in the Classroom: Today and Tomorrow", in: Carey, D., Carey, R., Willis, D. A., Willis, J. (eds.), Technology and Teacher Education Annual -- 1993, Charlottesville, VA, 1993, 219. 8. Paul Oppenheimer and Ed Zalta, "On the Logic of the Ontological Argument", Philosophical Review, Atascadero, Ridgeview, 1991, 5. 9. Jean W. Pierce, Gene Glass, and Michael Young, "The Education Research List (ERL-L) on the INTERNET", submitted to Educational Researcher. 10. Hope Tillman and Sharyn Ladner, "Special Librarians and the Internet", in press. 11. Diane Kovacs, Kara Robinson, and Jeannie Dixon, " ", Journal Scholarly E-Conferences on the Academic Networks: How Library and Information Science Professionals use Them. _Journal of the American Society of Information Scientists_ Forthcoming 12. Mike Holderness, "A Conspiracy of Silent Communication", Times (London) Higher Education Supplement - appeared on the ARACHNET listserv 3 March 1993. 13. L. A. Melton and others, "A Study of Undergraduate Education in Chemistry by the Delphi Technique", Journal of Chemical Education, 54, 1977, 640. 14. This list of listservs is available through FTP from nic.umass.edu. The file is located in pub/ednet as educatrs.lst. The author of this list is Prescott Smith, pgsmith@educ.umass.edu. 15. Diane Kovacs, private communication, dkovacs@kentvm.kent.edu. 16. see for example "The Internet Domain Survey", January, 1993, from the Network Information System Center (available by FTP from ftp.nisc.sri.com in pub/zone). 17. Project H -- an international project studying the informational structure of electronic discussion groups. For further information, contact Sheizaf Rafaeli, Hebrew University of Jerusalem, Israel (sheizafr@shum.cc.huji.il) or Fay Sudweeks, University of Sidney, Australia (fays@archsci.arch.su.edu.au). 18. Marcin Paprzycki, Tony Mitchell and George Duckett, "Using Computer Networks to Study Computer Literacy," in: Willis, J., Robin, B., Willis, D.A., (eds.), Technology and Teacher Education Annual -- 1994, AACE, Charlottesville, 1993, 91, abridged version of this text appears in: Computer on Campus; National Conference -- Conference Proceedings, University of South Carolina, 74. Tony Mitchell Assistant Professor of Chemistry St. Cloud State University St. Cloud, MN 56301 tmitchel@tigger.stcloud.msus.edu Table of Contents _______________________ The Virtual Square ================== edited by James Shimabukuro (jamess@uhunix.uhcc.Hawaii.Edu) The Virtual Square is devoted to non-refereed opinion columns and essays. Articles and Sections of this issue of the _Electronic Journal on The CYBERSPACE MONITOR ====================== ________________________________________________________________________ The _Electronic Journal on Virtual Culture_ may be retrieved by gopher to refmac.kent.edu under "Electronic Journals" or via anonymous ftp to byrd.mu.wvnet.edu or via e-mail message addressed to LISTSERV@KENTVM or LISTSERV@KENTVM.KENT.EDU (instructions below) or GOPHER gopher.cic.net Papers may be submitted at anytime by email or send/file to: Diane K. Kovacs- Editor-in-Chief, _Electronic Journal on Virtual Culture_ ejvcedit@kentvm.kent.edu _________________________________ *Copyright Declaration* Copyright of articles published by Electronic Journal on Virtual Culture is held by the author of a given article. If an article is re-published elsewhere it must include a statement that it was originally published by Electronic Journal on Virtual Culture. The EJVC Editors reserve the right to maintain permanent archival copies of all submissions and to provide print copies to appropriate indexing services for for indexing and microforming. _________________________________ ____________________________ GOPHER Instructions ____________________________ GOPHER to refmac.kent.edu 70 Electronic Journals/ ____________________________ Anonymous FTP Instructions ____________________________ ftp byrd.mu.wvnet.edu login anonymous password: users' electronic address cd /pub/ejvc get EJVC.INDEX.FTP get filename (where filename = exact name of file in INDEX) quit _______________________________ LISTSERV Retrieval Instructions _______________________________ Send e-mail addressed to LISTSERV@KENTVM (Bitnet) or LISTSERV@KENTVM.KENT.EDU Leave the subject line empty. The message must read: GET EJVCV2N3 CONTENTS Use this file to identify particular articles or sections then send e-mail to LISTSERV@KENTVM or LISTSERV@KENTVM.KENT.EDU with the command: GET where is the name of the article or section (e.g., author name) and is the V#N# of that issue of EJVC.