Guidelines for Evaluation of each Questionnaire Item
Following are the questionnaire items and the characteristics of Web-based instruction that prompted its creation. If there are no characteristics stated for an item, the item itself was a characteristic of Web-based instruction. The characteristics of Web-based instruction were gleaned from a literature review and a pilot study.
You might want to use the characteristics that shaped each questionnaire item to evaluate your Web-based course!
Appearance of Web Pages
1. The font (type face, size, and style) used on the Web pages detracts from the content.
Suggestions:
Word processor fonts smaller than 12 points or HTML fonts smaller than H5 should be a avoided. A smaller font size would be illegible on high resolution screens.
Avoid all capital letters.
Do not use more than two fonts on a Web page.
Use the default font type if at all possible because a different font type specified in the Web page may not be available on visitors' computers, thus rendering information illegible.
Use only serif or sans-serif type face for text. Most people prefer reading serif type face, but on a computer screen it is extremely difficult to read when small. Therefore, use serif type face for big text, but sans-serif type face for small text. Avoid script or other decorative type face because they are difficult to read unless extremely large.
Text should be left-justified for easier scanning.
2. The Web pages appear lifeless and dull.
Suggestions:
Paragraph breaks, headings, blank lines, horizontal bars, bulleted lists, color, highlighting, bold print, images, relegating information to other pages, etc., should be used to minimize high text density.
3. The Web pages are dominated by overly bold graphics or text.
Suggestions:
Static images should be pleasing to the eye and not overwhelm the viewer.
A primarily three-dimensional hyperspace should be avoided.
4. The color scheme of course documents interferes with text comprehension.
Suggestions:
The use of background patterns or colors that interfere with the readability of the text should be avoided.
Text colors should be selected so that pages are readable when copied to black and white displays (e.g., default in Microsoft Word) or black and white printers.
Text colors must contrast sharply with the background.
5. The layout of the Web pages is uncluttered.
Suggestions:
Paragraph breaks, headings, blank lines, horizontal bars, bulleted lists, color, highlighting, bold print, images, relegating information to other pages, etc., should be used to minimize clutter. Although such features should not be used to the extent to where they contribute to clutter.
The home page should primarily contain links to other areas of the Web site keeping explanatory comments to a minimum.
Avoid all capital letters.
Allow ample "white space" to avoid overloading the user with too much information at one time.
Web pages should look the same on any computer (e.g., fonts, colors, page layout).
Keep in mind that Web pages look different on various monitors depending on their size and picture resolution (e.g., layouts may look cramped, users may have to use horizontal scrollbars to see all parts of the page). Web page width should be designed to adjust to various screen resolutions or monitor sizes.
6. The Web pages are overcrowded with hyperlinks.
Suggestions:
Avoid making a link every time a keyword of another Web page is mentioned in the text.
Intrapage links should be avoided.
7. The Web pages contain unnecessary animated or blinking graphics.
8. A considerable number of pictures or animations that are supposed to be on the Web pages are missing.
Suggestions:
The students must be told to enable their browsers to accept JavaScript and Java files.
Hyperlinks and Navigation
9. The hyperlinks are clearly identifiable on the Web pages. Note: Hyperlinks are buttons, graphs, or phrases that connect one Web page with another.
10. Important information is easy to find on the Web pages.
Suggestions:
If feasible, information should be presented in order of importance.
The students must be told to enable their browsers to accept JavaScript and Java files.
Let the user know what is new on the Web site.
All Web pages should state the date on which they were last updated.
Specify tables as percentage of available space because fixed width tables may cause info show up beyond the right margins of the screen. It will also be chopped off when printed.
11. The hyperlinks tell me clearly what information I am connecting to.
Suggestions:
Hyperlinks, whether text, buttons, or images, should provide a hint of the content of the page they connect to. Textual descriptions should be clear and concise.
Linking images or icons should display a distinctive feature of the page they link to.
12. It is easy to locate a particular Web page from any other Web page.
Suggestions:
All hyperlinks should connect to existing Web pages.
Linking images and labels should be consistent. The same image or label should always connect to the same Web page.
A title should appear on all Web pages in the heading and with the "title" HTML tag.
All internal pages of a Web site should contain a link to the home page on top and bottom on the right side.
All Web pages internal to a Web site should have common headers, footers, and navigational controls.
13. The layout of the course Web site is clear to me.
Suggestions:
All Web pages making up a Web site should have a common look and feel.
Users should be warned of links that launch applications (e.g., video, audio, etc.) or open a new browser window.
Describe the components that make up the course and where to find them (e.g., instructor notes, e-mail tool, bulletin board feature, assignments, course-related Web-sites, etc.).
14. The buttons in the WebCT course management system clearly tell me what function they perform.
Technical Issues
15. The following online course media require an unreasonably long time to load to my home computer:
a. Video Presentations
b. Audio Presentations
c. Pictures or Animations
d. Interactive Computer Video Conferencing (CUseeMe, etc.)
Suggestions:
Provide thumbnail images for video clips and large static images.
To reduce down load time, use video clips less than one minute in length, or use print or audio narration together with pictures or slide shows.
Video or audio presentations, as well as interactive computer video conferences must be adapted to average modem speeds (28.8 Kbps to 56.6 Kbps).
File size should be offered if an audio, video, or image file to be downloaded is larger than 65 KB to warn of possibly lengthy download times.
An effort should be made to allow viewing of an entire Web page in at most three consecutive screens (14-inch monitor) with browser in default setting. If necessary, link to another Web page for further discussions.
Text documents should download in ten seconds or less since most users cannot keep their attention focused longer.
16. The technical quality of the following online course media is poor:
a. Video Presentations
b. Audio Presentations
c. Pictures or Animations
e. Interactive Computer Video Conferencing (CUseeMe, etc.)
Suggestions:
Video file size should not be curtailed excessively because it might make motion and sound intelligible.
Video or audio presentations, as well as interactive computer video conferences must be adapted to average modem speeds (28.8 Kbps to 56.6 Kbps).
Static images should be pleasing to the eye and not overwhelm the viewer.
Static images should be displayed with a text HTML tag, that is, an alt tag, so that they are decipherable by screen readers used by vision-impaired users.
Online Applications
17. Overall, the following ONLINE applications are easy to use:
a. Video Player
b. Audio Player
c. Interactive Computer Video Conferencing
d. Chat Rooms
e. Bulletin Board
f. Private E-Mail
g. White Board
h. Tutorials
i. Simulations
j. Plug-ins
Suggestions:
Novice Web users need to be instructed on how to use a Web browser, a search engine, or other applications pertinent to the course.
Students must receive explicit directions on where to find and how to install plug-ins.
Class Procedures and Expectations
18. I know exactly what actions to take in the event of technology-related problems.
Suggestions:
Novice Web users need to be shown how to recognize and deal with Internet connection problems. Be aware of firewalls!
The students must be told to enable their browsers to accept JavaScript and Java files.
Technical jargon should be defined.
Discuss procedures to follow when the network crashes while taking online quizzes.
19. In the beginning of the semester, I was given enough time to become familiar with the technology.
20. I am told exactly how to turn in each assignment.
Suggestions:
Explain what to do to avoid losing messages during electronic transmission.
21. I am given reasonable alternatives to scheduled "fixed time" activities (chats, tests, field trips, etc.).
22. The grading procedures are clearly stated.
Suggestions:
The instructor should describe in detail how to select a leader, the role of the leader, and how grades pertaining to team tasks are assigned to individual members.
23. The directions for completing assigned tasks are confusing.
Suggestions:
Explain how and by when to complete all tasks listed on weekly assignment pages.
24. The due dates and deadlines are clear to me.
Suggestions:
Describe where and how assessment will taking place (e.g., online, with proctor, etc.).
25. In the beginning of the semester, I was told exactly what is expected of me as a student in an Internet course (learning style, academic and technical requirements, etc.).
Suggestions:
Describe the computer hardware and software necessary for the course.
Explain the computer literacy necessary for the course.
List required academic prerequisites.
Discuss the learning style of a successful distance education student.
Make recommendations pertaining to good study techniques.
Content Delivery
26. The course content is delivered with appropriate media. Note: Media includes printed materials, audio, video, pictures, animation, etc.).
Suggestions:
The media (e.g., text, video, sound, static images, animation) used in the course must be sufficient to facilitate comprehension of the subject matter.
The instructor notes should be helpful in explaining and expanding the subject matter.
In addition to the textbook, distance instructors must develop their own instructional materials to simulate the presence of a human guide and teacher.
A textual presentation alone will eventually lead to boredom and decreased motivation. Therefore, color, static images, animation, video, sound, and interactivity via the JavaScript or Java programming languages.
Video and audio presentations, static images and animations, and interactive computer video conferences should only be used if they are helpful in explaining and expanding the subject matter.
27. The instructor provides enough examples to allow me to better understand the subject matter.
28. The assigned tasks increase my comprehension of the subject matter.
29. I am given useful resources for extra practice or for expanding my knowledge (online tutorials or libraries, content-related Web sites, etc.).
Suggestions:
Link the lessons to real-life work.
Opportunities must be provided for enrichment and remediation.
30. The instructional methods used in this course help me learn the subject matter. Note: Instructional methods may include lectures, case studies, discussions, group work, etc.
Suggestions:
The students should be told the purpose of the lesson.
The students should be told what they have to know by the end of the instruction.
The students should link new information with related information already stored in long-term memory.
The information must be sufficiently current to meet the student's need.
There should be no obvious gaps or omissions in the coverage of the subject matter.
Group discussions must be relevant to the acquisition of knowledge.
Group size must also be taken into account if the learner-learner interaction is to be successful. Both in synchronous and asynchronous communication, large groups can be overwhelming for the participants and might lead to information overload. Five to ten participants per group are sufficient when conducting chat sessions or interactive computer video conferences. Asynchronous communication, on the other hand, can often facilitate the interaction between twenty or more participants, particularly in the case of individual or group presentations. However, in certain instances, a smaller group size is also advisable in asynchronous communication, specifically when students are required to post papers for discussion or are asked to collaborate on assignments.
Activities should not become overwhelming for the students.
31. The assessment activities (tests, quizzes, essays, presentations, etc.) contribute to my knowledge of the subject matter.
Suggestions:
The students must get the chance to apply the new information.
Assessment must be relevant to the material learned.
Students must be tested to find out to what degree they have internalized the skill.
32. The materials used to present the subject matter reflect the personal touch of the instructor.
Suggestions:
The instructor should use colloquial language and a lively writing style both in notes pertaining to the subject matter and in communication.
Instructor and Peer Interaction
33. The instructor communicates with me in a thoughtful manner.
Suggestions:
Online communication should be warm, responsive, empathetic, and considerate.
Never publish private e-mail without permission.
Do not type everything in all caps. It is rude and is like shouting. However, it is acceptable to capitalize one or two sentences for emphasis.
Every message should be acknowledged so the sender knows that it was received.
Avoid sarcasm and insults. Negative comments sound worse in written messages than in face-to-face conversation.
Use emoticons such as the smiley and winky, that is, :-) or ;-), to show how to convey intended humor or to tease in a nonthreatening way.
34. The messages from the instructor are clear to me.
Suggestions:
Messages should be kept brief and to the point.
If responding to a message, quote relevant passages or summarize it for those who may have missed it.
Use space to break up paragraphs to improve readability of messages.
Describe how special symbols (e.g., mathematics, foreign language) must be typed in an environment that does not allow their real representation.
Avoid unorganized and cluttered bulletin boards that don't allow students to find important information.
35. The instructor uses an informal conversational style (uses humor, is folksy, etc.).
Suggestions:
The instructor should use colloquial language and a lively writing style both in notes pertaining to the subject matter and in communication.
The instructor shows humor.
36. The instructor encourages proper communication among students (teaches Internet etiquette or behavior during discussions, etc.).
The instructor must set guidelines that govern the behavior of teams.
Provide a protocol, such as "…" and "over" to indicate that a chat room participant has finished (over) a comment or has more to say (…). Without such a protocol chats can be confusing and chaotic.
37. The instructor confirms in a timely manner that assigned tasks have been received.
Suggestions:
Respond to students within 24 hours.
38. I can count on the instructor to clear up quickly any confusion that I may have with a topic.
Suggestions:
Explicit advice and suggestions to the student as to what to do and what to avoid, what to pay particular attention to and what to consider.
39. The instructor makes an effort to ask me how I am doing.
Suggestions:
Instructors should send 2-3 weekly messages questioning students understanding with respect to lectures, handouts, discussions, assignments, directions for sending assignments, software, sending attachments, how to format messages, code of conduct, plagiarism, netiquette, online tone, etc.
40. I am encouraged to get in touch with the instructor when questions or concerns arise.
41. The instructor responds to my messages in a timely manner.
Suggestions:
Timely, clear, relevant, and diplomatic feedback from the instructor must be given.
Respond to students within 24 hours.
42. The instructor is difficult to reach when WebCT (author note: this was the LMS used in 2000) is unavailable.
Suggestions:
All Web pages should at least provide the name, E-mail address, and phone number of a contact person.
Inform students on how and during what times to contact the instructor.
43. The instructor's participation in mandatory discussions (in chat rooms, on the bulletin board, etc.) is poor.
Suggestions:
Instructors should exhibit a high level of participation in bulletin board and chat room discussions.
44. I am encouraged to communicate with my peers.
Suggestions:
Instructors may promote interaction by challenging their own entries, ask questions by using a pen name, or remove the name from a question sent to the instructor's e-mail address then share the question on the bulletin board.
Instructors may also present questions to students and require responses on the bulletin board or in private e-mails, as well as ask different students to present items of interest to the class (e.g., technology, troubleshooting, subject matter, study, or Web resources tips).
Students should be enticed to exchange private e-mails in order to continue to discuss an assignment, to share information, or to study for tests.
Depending on the nature of the class, students could also be encouraged to give each other useful feedback on their work, such as in English composition courses, or to collaborate with students from similar courses, such as in laboratory courses required in the study of biological, physical, or computer science. In general, courses in mathematics, the sciences, art, and music do not lend themselves well to the discussion format. However, instructors may still initiate discussions by requiring students to pose questions about the material to other students.