Feedback Systems for Carnegie Mellon Transit
Team: Dylan Smith, Eric Guo, Swarnakshi Kapil, Asit Parida, Wuyang Wang
(September 2018 - December 2018)
User-centered (ux) Research & Design, Web
Since 2014, ridership for CMU transit has been dropping every year, without a clear reason why.
My team and I found that there was an absence of official forms of feedback and communication regarding CMU transit services. To solve this, we created a feedback pathway between Carnegie Mellon University students and student advocates from the Graduate Student Assembly via intensive research and iterative design.
With our solution, any rider of CMU transit with a mobile browser can provide feedback in under 30 seconds.
Rapid prototyping and Ideation
Parallel & Experience Prototyping
Ridership for Carnegie Mellon University transit services (Shuttle & Escort) has decreased by 40,000 riders (~10%) in the last 4 years. Our team was asked by the Graduate Student Assembly (GSA) of Carnegie Mellon to find a reason why.
According to the Graduate Student Assembly, one of the parties involved with CMU transit, current user data is only “fueled by opinions” and “anecdotal evidence.”
We found that students have a multitude of good and bad ride experiences, but often don’t share their problems as there are no official means of communication beyond catch-all emails.
Using the insights we found about when and where users want to give feedback, we created a simple web-based feedback page scannable via a QR code on the back of every shuttle seat. If the user does not have a QR code scanner or does not want to use one, we have also included a short URL. Using this method, users are able to give feedback in under a minute or less. We were told by our client representative that due to the low cost of implementation and the viability and need, that they would be very interested in developing our solution further.
After intensive evaluative research and multiple rounds of boring-to-zany-to-ridiculous ideation and voting, we found that there are no official pathways for feedback and communication about CMU transit services. We decided to move forward with this idea, as it had not been addressed before. We also confirmed with a representative of the Graduate Student Assembly that this information would be valuable.
In order to shape and guide our focus, we created the following research questions:
Are students satisfied with the services they have now?
What improvements could be made?
Why do some students not use the existing shuttle (day) and escort (night) services, and what can be done to increase usage?
contextual inquiries, interviews, & surveys
Our team split up to perform contextual inquiries, semi-structured interviews, and surveys on stakeholders involved.
Through our research, we found out the following insights:
Students often just complain to each other about bad ride experiences; they don’t reach out to the school unless in extreme situations.
When students do reach out, it’s often to University Police, as they are in charge of the transportation department. There is no designated feedback pathway for them to use.
The GSA body is very interested in a continuous stream of feedback, as such data in the past “has only been piecemeal.”
think-alouds and usability audit reviews (uars)
We performed Think-Alouds over 8 hours using competitor apps in order to see user pain points when giving feedback on mobile.
Through our Think-Alouds, we found some critical insights:
Users want feedback systems in general to be as painless as possible.
Our participants noted that they would often feel “too busy” to give feedback if it wasn’t an extremely easy process/hardly used any cognitive resources.
Feedback systems need to have both buttons to press (easy - low cognitive attention), as well as an information box for more detailed feedback (requires more attention - higher cognitive attention). This was helpful when designing critical UI elements of our artifact.
Users may have a specific mental model for feedback similar to other apps at their disposal (i.e. such as in ride sharing apps, food delivery apps, etc.)
Users think that acknowledgement of feedback is necessary.
e.g. a “Thank you for your feedback” text box after input
To bring credibility and documentation to each problem found in the Think-Aloud process, we created Usability Audit Reviews (UARs): heuristic evaluations to determine impact, frequency, persistence, and weight. All of these factors helped give us a gauge on what to prioritize in terms of usability when ideating for our future scenarios and designs.
walking the wall
In order to synthesize our findings prior to creating scenarios and ideas that addressed user needs, we performed an activity called Walk the Wall. Prior to this activity, we took all of our current research findings and placed them up sequentially on the wall. We then took 15 minutes of complete silence to put down different problems, ideas, and opportunities. We put these notes on Post-its and placed them on various spots on the wall. We then reviewed these notes to sync up and explore the most common user scenarios.
We used the scenarios from the Walk the Wall activity to create several different storyboards for an activity developed at Carnegie Mellon, called Speed Dating. More information about Speed Dating can be found here. We were told to specifically create a mixture of realistic and extreme scenarios, as the goal was to provoke user reaction. For example, we purposefully created some awkward feedback scenarios, such as giving your bad feedback verbally to the driver’s face. The purpose of Speed Dating isn’t to see how much users like one idea over another, but rather explore how these scenarios can uncover hidden user needs.
We split up into teams of 2 (one facilitator, one notetaker) and talked to a combination of different stakeholders, but focused mainly on graduate students who take CMU transit. We found a number of valuable insights:
Paper forms are “old” and “outdated,” and can cause extra work for staff
Users were concerned about drivers reading negative feedback
Users want the least amount of effort to give feedback: “People are really busy and think that there’s no time [for giving feedback]”
Once we’d collected enough data, we used our latest Speed-Dating and Walking the Wall research to create initial screens.
In order to test out our prototypes, we used several evaluative methods of research.
5 second test
With this method, we attempted to replicate how our solution will used in its original context. By using the 5 second test, we were be able to tell if our design was clear enough to our target users in the small time frame they will have to understand and use it.
The insights we received were:
Imagery stood out more to the participants than the text did (the shuttle icon and the presence of 4 options).
Users could tell that the screen was to give feedback. However, certain participants thought the feedback was for an app experience, not a ride experience. This could be because a) the test was not done in context and b) the verbiage was just “how was your experience,” not, “how was your ride experience?”
Users expressed giving stars to be an easier way to provide feedback.
Pre-filled issues were easier for users to fill and recall vs. just writing out the issue themselves.
final design artifacts
QR CODE & HARD-CODED WEBSITE
The insights from previous iterations suggested a physical design artifact as well as a coded version of the feedback document page, in order to start generating actual data for the stakeholders.
We connected the two designs with a QR code, which individuals can scan to access the feedback page. Through testing, we found that most individuals would prefer to give feedback during the ride or immediately after, suggesting that the physical poster should be inside buses, and not at stops.
Our website is live! Scan the QR code to try out our system now!
Based on our client feedback regarding the ease and low cost in implementing our solution, we consider this project a success. We learned a lot in the process, and got to experiment with several research methods (some successful, and some not). We addressed a real user need that was confirmed to be relevant and valuable. In future iterations, we would like to pilot our QR code in several shuttle and escort buses, and share this data with GSA in an easy-to-read format. We would also like to include driver feedback in the process, in order to give them a voice in the overall experience as well.