Craigslist Marketplace Website
The Craigslist website functions as a buying and selling marketplace. Craigslist includes a "create a posting" feature which allows users to list items for sale.
COURSE:
Usability Testing/Interaction Design
MY ROLE:
UX Researcher
TEAM:
Just me!
TIMELINE:
6 weeks
Project Overview:
THE IDEA: What is the usability of creating a posting on Craigslist?
THE PREDICTION: Craigslist users will make more errors during task completion than the average error rate across all sites.
Part 1:
I designed a test plan to conduct a formative, moderated in-person usability test.
CONTEXTUAL RESEARCH
What is an error?
Any slip the user makes will be counted as an error. Slips are defined as any unintended action a user makes while trying to do something on an interface, even though the goal is correct.
(“The Design of Everyday Things”, Norman 1988)
​
What is the average user error rate?
The average number of errors per task across many consumer and business websites is 0.7 errors.
(“The Practical Guide to Usability Testing”, Sauro 2010)
​
PARTICIPANT RECRUITMENT
Sample Size Calculations
Through iterations, a sample size of fourteen participants was calculated to achieve 80% power.
​
Formula:
n = [(za + zb)2 * s2]/d^2
One-tailed z-score values:
za = .95 = 1.64
zb = .80 = .84
Standard Deviation from Pilot Study: 1.41
Initial sample size calculation:
d = 1
n = [(1.64 + .84)2 * (1.41)2]/1^2 = 12.23
Iterations:
t(11) = [(1.80 + .84)2 * (1.41)2]/1^2 = 13.86 ~ 14
t(13) = [(1.77 + .84)2 * (1.41)2]/1^2 = 13.64 ~ 14
​
Estimated sample size: 14
Sampling
Participants were recruited using convenience sampling.
DESIGN AND PROCEDURES
Data and Variables
Quantitative data was collected and the number of errors per task was analyzed across users.
​
Task Description
Participants who passed a screening were shown an interactive prototype of Craigslist’s homepage. Participants were asked to create a posting for a hypothetical product: a microwave in used condition to be sold for $50, pickup only, in the West Los Angeles area.
Data Analysis
A one-sample one-tailed T test was used to analyze user error rate. The analysis compared the present group’s average user error rate during task completion to the average user error rate during task completion across all sites.
Part 2:
I conducted the usability test and analyzed findings.
PROTOTYPE
Interactive Prototype
I built an interactive prototype of the "create a posting" flow on Craigslist, which users utilized during the task.
RESULTS
Notable Findings
12 out of 14 users made at least one error while attempting to create a posting.
Projected that an average Craigslist user creating a posting is likely to make between 1.03 and 3.32 errors
(95% Confidence Interval)
​
Only 3 out of 17 users made errors after they had located the "create a posting" or "post" button.
Most errors were made during step one of the task: finding a means to create a posting.
The average Craigslist user made 2.8 errors while attempting to create a posting.
Users commit 1.5 standard deviations more errors when using Craigslist than when using other websites (95% confidence; t-test)
The number of errors users made while attempting to create a posting ranged from 0 to 7.
The number of errors that most commonly occurred was 2 errors.
​
Analysis
A one-tailed one-sample t-test revealed that, users commit significantly more errors when using Craigslist (M = 2.9), 95% CI [1.03 - 3.32] than when using other websites (M = 0.7), t(16)= 4.1, p < .001, d = 1.6. This effect is large as the difference between errors committed on Craigslist and errors committed on all other sites was over one and a half standard deviations. There is a 95% likelihood that an average user on Craigslist will make between 1.03 and 3.32 errors, on average, when attempting to create a posting.
Qualitative Insights:
Comments from user interviews following task completion:
"I didn't even notice the (create a posting) button..." (male; 24 years old)
"...there were so many options. I just looked for something to do with a microwave" (female; 22 years old)
"...I would've gotten it if they made it more obvious" (male; 26 years old)
"Why did they made it look different than the rest of the buttons" (male; 18 years old)
"It was so hard to see. It's too small" (female; 48 years old)
User Insights:
Users were observed to make errors for different reasons:
6 out of 17 users reported searching for the button in the middle section of the homepage
3 out of 17 users were observed to search for words related to "microwave" amongst the listings for sale
2 out of 17 users reported noticing the "create a posting" button, but choosing a different button instead
Why did users make errors?
Users averted attention to the center of the homepage, rather than where the “create a posting” button was located.
​
Users misunderstood listings that were for sale, as categories for creating postings.
​
Overall, as an interface, Craigslist is designed such that goal of purchasing listings is prioritized over the goal of creating postings. User attention is therefore more readily drawn to purchasing a listing than to creating a posting.
Cognitive Principles
Why didn't users avert attention to the “create a posting” button?
​
1) Users perceive items with similar appearances as belonging to the same group
-
Every button on Craigslist utilizes blue text against a white background, except the "create a posting" button, which utilizes purple text against a yellow background
-
The "create a posting" link was not as readily perceived as a button, as it employs a different appearance than every other button on the page
-
​
2) Users more readily perceive items that employ greater color contrast
-
All of the links to listings for sale utilize blue text against a white background, whereas the "create a posting" link utilizes purple text against a yellow background
-
The "create a posting" employs less contrast compared to the listings for sale, this explains why users reported not noticing the button
-
​
3) Users perceive items that are larger to be more important than items that are smaller
-
The links to listings for sale take up the majority of the page, whereas the goal of creating a posting is given one single link on the entire page
-
The "create a posting" button is perceived as less important than the listings for sale, explaining why users reported searching for words related to "microwave" in the listings for sale
-
​
Part 3: Future Directions
RECOMMENDATIONS
Recommendations
A Craigslist homepage redesign should equally prioritize the goals of creating a posting and purchasing a listing.
​
Qualitative findings suggest the redesign should employ greater color contrast and information size for the "create a posting" button.
​
The redesign should clarify the purpose of the different information sections on the page.
Mockups
I created high-fidelity mockups on Figma which implemented the recommendations in the redesign.​
​
The redesign should clarify the purpose of the different information sections on the page.
Current Design:
High-Fidelity Mockup #1:
High-Fidelity Mockup #2:
Future Directions & Limitations
A future study should assess average number of errors in alternate designs.
​
A future study should assess satisfaction of task completion in each design.
​
A future study should utilize a more representative sample.
Due to limitations of budget in this study, participants were recruited using convenience sampling.
​
​