|
Task #1
|
#2 |
#3
|
#4 |
#5 |
#6 |
#7 |
#8 |
#9 |
#10 |
Web Manual
Task 1:
 Find
the Santa Ana College Library Web page.
FINDINGS
5 of 10 participants easily
completed Task 1 within 0.5 minutes time
-
These subjects went directly to
the Library Web site (Click on Students, Click on Library, Labs &
Bookstore, then Nealley Library).
3 of 10 participants completed
Task 1 within 1-3 minutes time
2 of 10 participants did not
complete Task 1 within the time allotted
RECOMMENDATIONS
Present a recommendation and
supporting documentation requesting that a link to the Library be added to
the Santa Ana College Home page to Dean Breeden and subsequently to Maria
Sugranes/SACTAC. There is an
overwhelming amount of documentation to support this recommendation.
Many colleges and universities have a link from their home page.
Another second option would be to propose separating the word
Library from the Library, Labs & Bookstore phrase, and listing Library
singularly. In addition,
suggest moving the Library link higher up the Students list for greater
visibility.
Starting
Point:
http://www.sac.edu/students/library/nealley/
Task
2:
You are looking for books on capital punishment.
Using the Library Web site, find a recent book written between
1995-2002 on this topic.
FINDINGS
4
of 10 participants easily completed Task 2 within 2 minutes time
These
subjects went directly to the Library Catalog.
3
of 10 participants completed Task 2 within 3-5 minutes time
-
One
subject went directly into the Library Catalog, but required
additional time to type & scroll.
-
One
subject chose the Library Catalog because it was second on the list,
after Library Information.
-
One
subject tried Electronic Resources and Information Competency
Tutorials first before going into the Library Catalog.
3
of 10 participants could not complete Task 2 within time allotted
-
One
subject tried the campus Search link.
The subject considered the Library catalog to be a shopping
catalog.
-
Once
subject tried Academic Search and then MAS Ultra.
-
One
subject tried WWW Search Engine – Google, at three minutes was
prompted to go back to the Library Web page.
This person would have continued in Google.
Card
sort observations (selected observations)
Students:
-
Associated
finding books under the Library Catalog category, 9 of 10 students in
the matching exercise. Other
categories that students associated with books included:
Other Libraries and Search.
-
Will
try to search the SAC site and the Internet for books, etc.
RECOMMENDATIONS
In
Task 2, 4 of 10 participants did not choose Library Catalog to find a
book. If the term Library
Catalog is used, a list of what can be found using the Catalog must be
given either as text on the Library Web page or visible when a mouse over
is used. If a change of label
is considered, i.e., Find Books, then all items that can be found using
the Library Catalog must be listed, e.g., Find Books, Journals, Tapes,
Course Reserves, etc. A FAQ or help document could reinforce and improve
Library Catalog use.
The
“SAC” search option needs to be moved to the bottom of the screen and
labeled as Search SAC Web or SAC Search to differentiate it more clearly.
Task 3:
You are looking for a CD that was recommended to you. Find out if the Library has a copy of the CD, Guitarra:
the guitar in Spain by
Julian Bream.
FINDINGS
4 of 8 participants easily
completed Task 3 within 2 minutes time
-
One subject tried Journal and
Newspaper Index first and then went into the Library Catalog to search
by title.
-
One subject debated on using the
sound recording limit. This
person was the only one to mention the limit.
-
Two subjects tried Electronic
Resources and then went into the Library Catalog. One searched by author, initially typing in the first name,
followed by the last name. The
other searched by title.
2* participants completed Task 3 within 2 minutes time, but
were prompted – data removed from sample denominator.
-
One subject tried to search the
author’s name as a title search, then they an author search by
typing the name in by first name, followed by last name.
*This person was prompted which affects this particular
finding.
-
One subject tried searching the
title as a subject search. *This
person was prompted to try to search by title which affects this
particular finding.
3 of 8 participants completed
Task 3 within 3-5 minutes time
-
One subject used the Library
Catalog directly, but tried to search the entire title as a subject
search. When this did not
work, they tried an author search by typing the name in by first name,
followed by last name.
-
One subject used Advanced
search.
-
One subject tried Electronic
Resources, then Library Technology, then WWW Search, then Library
Instruction, and finally Library Catalog.
The subject tried to search Guitarra as a subject search, then
tried Julian Bream as an author search by typing the name in by first
name, followed by last name.
1 of 8 participants could not
complete Task 3 within time allotted
-
One subject tried Library Technology Program and then tried
Library Catalog. The
subject typed in as a title search: the format, the title/subtitle,
the author and punctuation (C D. Guitarra: the guitar in Spain by
Julian Bream). The search
yielded no results. The
subject did not consider other search options or limits.
Card sort observations (selected
observations)
Students:
-
Do not associate finding journal titles,
videocassettes, audiocassettes, and CDs with the Library Catalog.
-
Think that video, audio, and CDs should come
under the Electronic Resources category, 5 of 10 students.
-
Rarely change the field to search in or specify
a limit.
RECOMMENDATIONS
Only 2 of 10 participants went
directly to the Library Catalog to try to find the CD.
Of these two people, only one was successful.
The students do not know that they can find CDs using the Library
Catalog. See Recommendations under Task 2.
Task 4:
You need to write a one page summary of a journal article from the journal, Exceptional Children. Find out if
the Library subscribes to,
FINDINGS
6 of 10 participants easily
completed Task 4 within 2 minutes time
-
One subject chose the Journal and Newspaper Indexes and
then went to Library Catalog. Searched
by Journal Title, because they had seen this option before.
-
One subject debating between
Journal and Newspaper Indexes, but went to Library Catalog and used
Advanced Search. They
typed in the journal title, but left the default to search all of
these (title, author, etc.).
-
One subject went directly into
Library Catalog and selected journal title, but had a typographical
error and thought initially that SAC Library did not own the journal.
-
Three subjects went directly
into Library Catalog and selected journal title and easily located the
journal.
3 of 10 participants
completed Task 4 within 3-5 minutes time
-
One subject went into Journal
and Newspaper Indexes, then Academic Search and typed in journal title
in the Find box. After
locating an article the subject made reference to the note in the
record indicating that the title is held at SAC.
The subject said that they do not use the computer, but go to
the Periodicals Department to see what journals SAC Library has.
When shown how to search for a journal title using the Library
Catalog, the subject stated they wouldn’t have thought to try the
Library Catalog.
-
Two subjects went into Journal
and Newspaper Indexes, then Academic Search and typed in the journal
title in the Magazine (title) box.
1 of 10 participants could not
complete Task 4 within time allotted
Card sort observations (selected
observations)
Students:
-
Do not associate finding journal titles,
videocassettes, audiocassettes, and CDs with the Library Catalog.
-
Associated Journal Titles with the Journal and
Newspaper Indexes category in the matching exercise, not the Library
Catalog.
RECOMMENDATIONS
Interestingly, 9 of 10
participants completed the task but only 6 of them used the Library
Catalog. Using Academic
Search or asking a Librarian/inquiring at the Periodicals Department does
technically complete the task.
See Recommendations under Task
2.
Task 5:
Your instructor has placed a book on reserve in the Library.
Using the Library’s Web site, find out what books are on
reserve for one of the Counseling classes.
Where is the reserve section in the Library?
FINDINGS
3 of 10 participants easily
completed Task 5 within 2 minutes time
-
One subject didn’t know where the Reserve Section was but
knew that they could search in the computer to find what was on
reserve because they had learned that in one of the Library classes. The subject went into the Library Catalog, clicked on Course
Reserve and chose the pull-down menu to search by course.
-
Two subjects went directly to
Library Catalog, clicked on Course Reserve and chose the pull-down
menu to search by course. They
knew this because a librarian had shown them how to do this.
3 of 10 participants
completed Task 5 within 3-5 minutes time
-
One subject went directly into
the Library Catalog, but required additional time to read help section
at the bottom of the page, to type and scroll.
-
One subject chose the Library
Catalog and used Advanced Search.
The subject typed in “books on counseling 116”, followed by
“books on counseling 125”. Next
they clicked on Course Reserve and looked under instructor for the
word, counselor. Then
they tried department, counseling.
-
One subject tried Library
Information, then Library Instruction, and finally Library Catalog.
After clicking on Course Reserve, they chose Department and
selected Counseling. The
subject stated that they would find a librarian to tell them where the
Reserve Section was.
4 of 10 participants could not
complete Task 5 within time allotted
-
One subject tried a Basic Search in the Library Catalog.
They had been shown last week how to do this, but couldn’t
remember how.
-
One subject didn’t know what
Course Reserve meant. After
the moderator explained about Reserves, the subject stated that it is
better to just ask the librarian.
-
One subject stated that they
would just go to the Front Desk and give them the instructor’s name
to get the book. When
asked what they would do if they were checking from home, the subject
stated that they would try anything, like the Library Catalog.
-
One subject said that they would
go to the Front Desk and ask. When
asked what they would do if they were searching from home, the subject
stated that they would try more searching, like Library Instruction,
Staff Directory, Library Catalog for author/title.
Card sort observations (selected
observations)
Students:
-
Some know what is Course Reserve, some do not,
3 of 10. One person
thought it was the course description, one person thought it was
registering for a course, and another thought it was reserving a room
for a class. Some of
those that did know what Course Reserves do not associate finding that
information through the Library Catalog.
RECOMMENDATIONS
Some participants know what
Course Reserve means, some do not. Some
know that the items are at the Front Desk but do not know how to access
what is on Reserve using the Library Catalog.
The problem is similar to the other Catalog related tasks.
A separate link to Course Reserve might be helpful, adding Course
Reserve to what you can find using the Library Catalog, and adding a note
to the FAQ or help documentation.
Task 6:
Your instructor has told you to go to the Library’s Web site to search the Academic Search database for journal
articles on racial profiling. Find
and enter the Academic Search database and
FINDINGS
4 of 10 participants easily
completed Task 6 within 2 minutes time
-
These subjects went directly to
Journal and Newspaper Indexes, then Academic Search. Each used Basic Search (default) and entered the search
words, racial profiling. Each
scrolled down the results list and made mention of either the email
button, print button, or both buttons.
Two participants went into the full-text of the selected
article.
4 of 10 participants completed
Task 6 within 3-5 minutes time
2 of 10 participants did not
complete Task 6 within the time allotted
-
One student started with the
Library Catalog, Advanced Search.
Then they talked about trying Electronic Resources and
Information Competency Tutorials.
Finally they used Journal and Newspaper Indexes.
One student initially tried Journal and Newspaper Indexes and
then backed out to try the Library Catalog.
After leaving the Library Catalog they studied the WWW Search
heading and then went back into Library Catalog.
RECOMMENDATIONS
8 of 10 participants completed
the task within 5 minutes, but 4 or half of these people didn’t go
directly to Journal and Newspaper Indexes.
The use of a different label or adding explanatory information may
help. In the card sorting
exercise, the majority of participants identified that finding journal
articles, newspaper articles, and magazine articles were all under the
Journal and Newspaper Indexes heading.
Card sort observations (selected
observations)
Students:
-
7 of 10 knew what a periodical is, 3 did not.
-
6 of 10 associated the word, index, as the
index in the back of a book. 3
of 10 were aware of periodical indexes.
1 person had no idea was an index is.
-
In the matching exercise, all students were
able to successfully associate the majority of terms related to the
Journal and Newspaper Indexes category.
Examples include: periodical
indexes, journal articles, magazine articles, and National Newspaper
Index. The one term not
associated was, Academic Search.
Task 7:
You need to use the correct citation style for your research paper’s bibliography. Using
the Library’s Web site, find a Library Web document that has style
manual information that you could use.
FINDINGS
0 of 9 participants did not
complete Task 7 within the time allotted
-
The moderator explained to each
participant what a style manual was and clarified any questions that
the participant had about the task.
One participant had written a research paper and had used the
MLA Style Manual. Three
participants initially tried the Information Competency tutorials,
another three initially tried the Library Catalog. One person tried
Journal and Newspaper Indexes, then Google.
One person tried Electronic Resources but went into
Biographical Sources. A
few of the participant comments include:
-
“This is not an electronic
resource”.
-
“I would have never” looked
there. “I think of
videos, materials in the Library” under that heading.
-
Electronic Resources and
Pathfinders are “very vague for me”.
Card sort observations (selected
observations)
Students:
-
Do not know what a style manual is, 8 of 10
students.
-
5 of 10 students associate Electronic Resources
with videos, CDs, audiocassettes, etc. Another person thought this category included Google and
Yahoo. Another person
thought the category was about electronics.
-
There were 2 students who knew what a style
manual is and 1 student who knew what type of information would be
found under the Electronic Resources category.
-
None of the 10 students were able to clearly
articulate what a Pathfinder is.
RECOMMENDATIONS
Participants
tried many different headings to try to complete the task.
Participants do not recognize that the Library has guides on
various topics, including style guides, for their use.
The label of Electronic Resources and Pathfinders does not have
meaning for them. The card sorting matching exercise demonstrated that 5 of 10
participants thought that Electronic Resources included videos, CDs, etc.
A FAQ
with library terminology may be helpful to provide definitions for
commonly used jargon, such as bibliography, style manuals, etc.
The use
of Electronic Resources and Pathfinders should be discontinued and
implementation of a new system of categories/headings/labels needs to be
considered. A system that is
more descriptive and intuitive for the students.
With a mouse over feature, an expanded list of contents for each
heading or label would help users locate the appropriate link.
After a new system of categories is developed, another card sorting
or matching exercise could be conducted.
Task 8:
You are having difficulty accessing the journal and newspaper indexes from home.
Using the Library Web site, where can you find
FINDINGS
1 of 10 participants easily
completed Task 8 within 0.5 minutes time
One participant scrolled down the page and went directly to
the Connect From Home link.
4 of 10 participants completed
Task 8 within 1-2 minutes time
Two participants knew to scroll down to find the link
because they had been shown in a Library Instruction course/workshop.
Two participants went directly to Journal and Newspaper Indexes
first and then returned to the Library Web page and scrolled down the page
to the link.
5 of 10 participants did not
complete Task 8 within the time allotted
One participant scrolled down
the Library Web page and “missed” the Connect From Home link.
One person clicked on the SAC HELP option at the top of the screen
under the graphical (picture) banner.
When prompted to return to the Library Web page, they eventually
found the link. Their comment
was, “you know what, you can’t see that link on this screen”.
Another participant commented, “It’s not very noticeable.
I’ve been here many times but I have never noticed it”.
RECOMMENDATIONS
The
Connect From Home link needs to be move to the top portion of the screen,
not below the fold/scroll line. The
general SAC HELP link is confusing. The
global site links should be at the bottom of the screen and prefaced by
the designation, SAC. Library-specific
help documentation is needed.
Task 9:
How would you describe your
overall experience in finding the information requested in this study on the Library Web
site?
Frustrating
(10 point scale)
Satisfying
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
8
|
9
|
10
|
|
2
|
|
1
|
|
2
|
3
|
|
1
|
|
1
|
FINDINGS
-
I could not figure out if
Library Catalog is the right one.
I got frustrated and would just give up.
-
some were hard, some were easy,
some were in the middle
-
some were frustrating, some were
easy
3 of 10 found the overall
experience frustrating (1-3 on the scale).
5 of 10 found the overall experience with some frustrating/some satisfying
(4-7 on the scale).
2 of 10 found the overall experience satisfying
Task 10: (Optional)
List two things that
you liked about our Library Web site.
FINDINGS
-
I like the library catalog, you
find most of the things there.
-
What I liked … anything?
No.
-
Way to find the books.
-
Can get to other places besides
the library’s web site.
-
Journal & Newspaper Indexes I
use the most. Like
finding articles in newspapers and journals … cannot print in the
Library, so I email it to myself.
-
I like Nealley Library
Information, it tells me something.
-
I can find my book and get the
information, if I could find it.
-
Information Competency – I did
it twice and some of the questions I had to ask a friend.
List two things that need
improvement on our Library Web site.
-
Don’t like left menu, not
descriptive enough, I would have to go one by one.
-
No graphics to “grab my
attention”, it’s only three different colors.
-
“mouse over” menu so
you don’t have to click on it to find out what’s there.
-
It would be better if there were
fewer words. “There are too many words/letters, it
distracts
me”. Google is popular because it is very simple, not like Yahoo.
-
Left menu – letters too small,
and the categories … are not used often by students.
-
Reduce the numbers of categories.
-
Library Catalog should be more
emphasized since I learned from this test that I can find so much
using it.
-
Different places to find the book
in the library … confusing.
-
Frustration with the message –
results with no hits.
-
I’ll find many titles and
don’t know what to choose.
-
I have to scroll down to find
things. Sometimes I
don’t think about scrolling down and thought there’s no more …
-
Journal or Electronic, I don’t
know what they are trying to tell me.
-
I don’t like the basic search.
It doesn’t make any suggestion when I did wrong.
Limit, what does that do?
Miscellany
Card sort observations (selected
observations)
Students:
1.
Have a fairly good idea as to what information they can
find under the Library Information heading.
2.
Under utilize the Library Catalog because they mostly
associate it with finding only books.
3.
May not know what a journal index is, but are able to find
journal and newspaper articles because they see the words, journal and
newspaper in the Journal and Newspaper Indexes heading.
4.
Rarely/Never use the resources under the Electronic
Resources and Pathfinders heading. 2
of 10 students indicated that they would find Recommended Web Sites by
Department under the Electronic Resources and Pathfinders heading.
5.
Are not sure what Information Competency is, unless they
have been required to complete the tutorials.
6.
Know that Yahoo and Google can be found under the WWW
Search Engines. Some students
think that everything is on the Internet.
7.
Some know some of the information that may be found under
the Library Technology Program heading, but some think that LIS 103
information can be found there too.
8.
4 of 10 students associated Bibliographic Instruction as
Library Instruction in the matching exercise.
9.
5 of 10 students thought that Academic Search would be
found under the general search button.
10.
Many students liked the mouse over option to present the
contents of each heading.
Draft submitted: 26 November 2002
Approved by Web Team: 18 December 2002
Approved by Library
Administrator:_________________________
Page
URL: http://www.sac.edu/students/library/nealley/usability/usabilityrpt.htm
Web page submitted by Linda Martin
|
