edited.jpg

Feedback Systems for Carnegie Mellon Transit

Team: Dylan Smith, Eric Guo, Swarnakshi Kapil, Asit Parida, Wuyang Wang
(September 2018 - December 2018)

User-centered (ux) Research & Design, Web


UCRE_Step1.png

overview

Since 2014, ridership for CMU transit has been dropping every year, without a clear reason why.

My team and I found that there was an absence of official forms of feedback and communication regarding CMU transit services. To solve this, we created a feedback pathway between Carnegie Mellon University students and student advocates from the Graduate Student Assembly via intensive research and iterative design.

With our solution, any rider of CMU transit with a mobile browser can provide feedback in under 30 seconds.


 

ROLES

User Researcher
UX Designer
Visual Designer

TOOLS USED

Rapid prototyping and Ideation
Contextual Inquiries
Think-Alouds
Surveys
Parallel & Experience Prototyping
5-second Test

SOFTWARE USED

Sketch
Indesign
Photoshop
Invision
Figma
Tableau

 

PROBLEM

shuttle.png

Ridership for Carnegie Mellon University transit services (Shuttle & Escort) has decreased by 40,000 riders (~10%) in the last 4 years. Our team was asked by the Graduate Student Assembly (GSA) of Carnegie Mellon to find a reason why.

According to the Graduate Student Assembly, one of the parties involved with CMU transit, current user data is only “fueled by opinions” and “anecdotal evidence.”

We found that students have a multitude of good and bad ride experiences, but often don’t share their problems as there are no official means of communication beyond catch-all emails.


Solution

Step 1: Scan QR Code

Step 1: Scan QR Code

Step 2: Rate your CMU ride

Step 2: Rate your CMU ride

Step 3: Submit Feedback

Step 3: Submit Feedback

Using the insights we found about when and where users want to give feedback, we created a simple web-based feedback page scannable via a QR code on the back of every shuttle seat. If the user does not have a QR code scanner or does not want to use one, we have also included a short URL. Using this method, users are able to give feedback in under a minute or less. We were told by our client representative that due to the low cost of implementation and the viability and need, that they would be very interested in developing our solution further.


PROcess

FOCUS

After intensive evaluative research and multiple rounds of boring-to-zany-to-ridiculous ideation and voting, we found that there are no official pathways for feedback and communication about CMU transit services. We decided to move forward with this idea, as it had not been addressed before. We also confirmed with a representative of the Graduate Student Assembly that this information would be valuable.

 

GENERATIVE RESEARCH

In order to shape and guide our focus, we created the following research questions:

  1. Are students satisfied with the services they have now?

  2. What improvements could be made?

  3. Why do some students not use the existing shuttle (day) and escort (night) services, and what can be done to increase usage?

process

contextual inquiries, interviews, & surveys

Our team split up to perform contextual inquiries, semi-structured interviews, and surveys on stakeholders involved.

Through our research, we found out the following insights:

Participant filling out survey

Participant filling out survey

  1. Students often just complain to each other about bad ride experiences; they don’t reach out to the school unless in extreme situations.

  2. When students do reach out, it’s often to University Police, as they are in charge of the transportation department. There is no designated feedback pathway for them to use.

  3. The GSA body is very interested in a continuous stream of feedback, as such data in the past “has only been piecemeal.”

think-alouds and usability audit reviews (uars)

We performed Think-Alouds over 8 hours using competitor apps in order to see user pain points when giving feedback on mobile.

Through our Think-Alouds, we found some critical insights:

  1. Users want feedback systems in general to be as painless as possible.

    • Our participants noted that they would often feel “too busy” to give feedback if it wasn’t an extremely easy process/hardly used any cognitive resources.

  2. Feedback systems need to have both buttons to press (easy - low cognitive attention), as well as an information box for more detailed feedback (requires more attention - higher cognitive attention).  This was helpful when designing critical UI elements of our artifact.

  3. Users may have a specific mental model for feedback similar to other apps at their disposal (i.e. such as in ride sharing apps, food delivery apps, etc.)

  4. Users think that acknowledgement of feedback is necessary.

    • e.g. a “Thank you for your feedback” text box after input

To bring credibility and documentation to each problem found in the Think-Aloud process, we created Usability Audit Reviews (UARs): heuristic evaluations to determine impact, frequency, persistence, and weight. All of these factors helped give us a gauge on what to prioritize in terms of usability when ideating for our future scenarios and designs.

walking the wall

In order to synthesize our findings prior to creating scenarios and ideas that addressed user needs, we performed an activity called Walk the Wall. Prior to this activity, we took all of our current research findings and placed them up sequentially on the wall. We then took 15 minutes of complete silence to put down different problems, ideas, and opportunities. We put these notes on Post-its and placed them on various spots on the wall. We then reviewed these notes to sync up and explore the most common user scenarios.






Speed dating

We used the scenarios from the Walk the Wall activity to create several different storyboards for an activity developed at Carnegie Mellon, called Speed Dating. More information about Speed Dating can be found here. We were told to specifically create a mixture of realistic and extreme scenarios, as the goal was to provoke user reaction. For example, we purposefully created some awkward feedback scenarios, such as giving your bad feedback verbally to the driver’s face. The purpose of Speed Dating isn’t to see how much users like one idea over another, but rather explore how these scenarios can uncover hidden user needs. 

We split up into teams of 2 (one facilitator, one notetaker) and talked to a combination of different stakeholders, but focused mainly on graduate students who take CMU transit. We found a number of valuable insights:

prototyping

Once we’d collected enough data, we used our latest Speed-Dating and Walking the Wall research to create prototypes.

evaluative research

In order to test out our prototypes, we used several evaluative methods of research.

5 second test

experience prototyping pt. 1 (failure)

experience prototyping pt. 2 (Recalibration & success!)




final design artifacts

hard-coded website

 

Speed dating

prototyping

Once we’d collected enough data, we used our latest Speed-Dating and Walking the Wall research to create prototypes.

evaluative research

In order to test out our prototypes, we used several evaluative methods of research.

5 second test

experience prototyping pt. 1 (failure)

experience prototyping pt. 2 (Recalibration & success!)



final design artifacts

hard-coded website

Screen Shot 2019-02-04 at 9.39.58 PM.png

Our website is live! Scan the QR code to try out our system now!


symposium poster

UCRE-final (1)-min.png

reflection

Pleased as punch! Even if our poster was a little bit crooked…

Pleased as punch! Even if our poster was a little bit crooked…

Based on our client feedback regarding the ease and low cost in implementing our solution, we consider this project a success. We learned a lot in the process, and got to experiment with several research methods (some successful, and some not). We addressed a real user need that was confirmed to be relevant and valuable. In future iterations, we would like to pilot our QR code in several shuttle and escort buses, and share this data with GSA in an easy-to-read format. We would also like to include driver feedback in the process, in order to give them a voice in the overall experience as well.