User Study

On 4th November, I conducted an usability test on the website https://www.udacity.com/. For this purpose, I recruited two students from the University with diverse backgrounds. User 1 is a female Civil Engineering student who is fairly proficient with computers and has used sites like Coursera before. User 2 is a Computer Science student who is highly proficient student and uses Lynda on a daily basis. [1][2]

I recruited two students because I believe that Udacity is a very well designed website and it would be better to get opinions from two people rather than a single user. Also, because of it’s brilliant interface, I feared that the user would be able to complete all the tasks in very less time. Also, I asked both the users to sign the “consent to record” form which I had used in my previous assignment as well which looks like this. This test was also conducted in the study room of my apartment and the session was recorded in Quick Time player. This time I have avoided to post images of the session as I believe it would compromise on the anonymity of the user.

untitled-8

Usability Test and tasks [3][4][5]

11/04/2016

Researcher: Smit Desai

 

 

Task Timestamp Observation Category

(Error, success…)

Severity

(1-5)

1. What are your initial observations about the website? What is this website about? Is it visually appealing? User 1: 10:00 pm

User 2: 11:00 pm

 

 

 

User1: Liked the visual appeal of the website. Really liked the design and the layout of the home page. But could not figure out what my classroom is. Did not watch the video but guessed what nanodegree is.

User 2: Noted that the website “looks really good”. It is interactive and contains lots of information. But could not figure out what nanodegree is. Also, did not watch the video.

1. Error

2. Error

4
2. Could you please make an account on this website without using any of the social networking plug ins?  

User 1: 10:06 pm

User 2: 11:08 pm

 

 

User 1: Easily made the account.

User 2: Did not quite figure out that the user had to go to my classroom to make the account. But did not take long to guess it.

1. Success

2. Success

5
3. Could you look for free courses offered by Nvidia?

 

 

 

User 1: 10:13 pm

User 2: 11:14 pm

 

 

User 1: Easily found the course using filters

User 2: Easily found the course using search option in the catalog menu. But noted that there needs to be a search option on the home page.

1. Success

2. Success

4
4. Can you register and start the free course found in the previous task? Also, how long will it take to complete the course?  

User 1: 10:15 pm

User 2: 11:17 pm

 

 

User 1:  Was able to register quite easily. But could not find the information needed for the timeline.

User 2: Registered quite easily. Played around with the interface and really liked the layout.

1. Error

2. Success

5
5. Could you please post a question on the said course forum?

 

User 1: 10.20 pm

User 2: 11.21 pm

 

User 1: Did it easily. But did hover around the round icons. Found it difficult to come to the home page from forum.

User 2: Done without a problem. Found it difficult to come to the home page from the forum.

 

 

 

1. Success

2. Success

 

 

3
6. Imagine that you work for Google and your boss has assigned you a task of hiring someone with Data Analytics skills. You don’t have much time to conduct interviews, so you have resorted to help from Udacity to hire someone. Where on the website would you go to accomplish this task? User 1: 10.22 pm

User 2: 11.24 pm

 

User 1: Could not figure out where to go. Went to the contact page. Then played around with random links. And eventually gave up.

User 2: Again could not quite figure out what to do and how to do things. Tried to look for student information in the individual courses. Again, after trying for a long time, gave up.

 

 

1. Error

2. Error

4

 

 

7. There is a job opening of an IOS engineer that your friend told you would suit your interest. Go ahead and apply for that position. User 1: 10.28 pm

User 2: 11.30 pm

User 1: Was able to find the job quite easily. Went through all through all the right links.

User 2: Again, could not really find the jobs option in the footer menu. Searched for it. And eventually figured it out.

 

 

 

 

1. Success

2. Success

 

4
8. Sign up for a nanodegree course. User 1: 10.35 pm

User 2: 11.36 pm

User 1: Was able to do so quite easily. Went through all the right links.

User 2: Did it easily. But did not like the fact that the amount to be paid was at the very end where credit card information was needed.

 

 

 

1. Success

2. Success

 

 

5

The tasks and their actions have been defined in the table above. I think this time I was more clear about what exactly needed to be done. My tasks are better phrased and also I have built up scenarios in a more believable way. I had to repeat my tasks the last time when I conducted the test or phrase it in a better way, but this time the problem did not arise. I could conduct the test in a much better manner.

Interesting and Surprising Points [5][7]

What I found the most interesting was the fact that none of the user watched the video about nanodegree on the home page. Sure, it is a very beautifully designed video which clearly explains what the nanodegree program is about but it is of no use if people do not decide to watch it. I am not stating that they should get rid of the video, I am just reinstating the fact that they need a little blob which conveys what the nanodegree program is about.

Another interesting point is a very general point. The home page of the website is a single page website where you have to scroll down and down to view the options in footer. But when there is so much scrolling involved, one might never know that there is a footer section. In this case, the footer section has lots of important links which might be useful for the users and employers alike. Both of my users were not able to navigate through the footer which I thought they would.

What the users didn’t do and why?[6][8]

Again the fact that the users did not watch the video was very interesting to me. They just assumed the meaning of nanodegree. User 1 was able to guess it correctly while User 2 could not do so. Also, I thought the User would have a problem with the usage of the word catalog but it was surprising that the users were quite familiar with that term. May be because the term is also used on Coursera that this problem did not arise. Another interesting thing that the user did not do was to use the filter options when they were trying to search for the Nvidia course. That is how I did it, but the users were more comfortable just searching for the course.

Also, Scott Berkun notes that it is important to know who the customer is in http://scottberkun.com/essays/22-the-list-of-reasons-ease-of-use-doesnt-happen-on-engineering-projects/ [9]. Here the customers are mostly students. And students normally like knowing the price of things before they get interested. But the price is only displayed to the customer at the very end. I am giving the website the benefit of the doubt and not calling it a dark pattern  but it might be. (http://www.theverge.com/2013/8/29/4640308/dark-patterns-inside-the-interfaces-designed-to-trick-you) [9].

Redesign Speculations, list of flaws and solutions[10]

  1. Payment information: I believe that this is a pretty big problem with this site. Not knowing what you are paying for could be a deal breaker for a lot of people (especially students). This could be solved by simply putting a payment option in the below shown image part of the website. android_basics_nanodegree_by_google___udacity

 

2. Search Box: The catalog button contrary to my beliefs turned out to be easily discernible. But the users did point out the need to have a search button on the website. So, just using a Coursera like Search+Catalog option would be a good idea.

coursera___online_courses_from_top_universities__join_for_free

3. Scrolling: I like the fact that they have all of their information on the home page. But what it is doing is introducing a lot of scrolling to the home page. I believe that this is a very heavy trade off for a website which has a lot of interesting links in the footer section. Nielsen talks about an “aesthetic and minimalist” design [9]. I believe that should be followed in this case.

4. Sign up/Sign in: Udacity has the sign up/sign in option in the my classroom area of the website. I think it is a pretty standard practice to introduce those options on the home page. So, I would get onto that as soon as possible.

Overall, it is a very well designed website but such minor fixes would improve it exponentially. Many of my earlier criticisms have been completely revamped in the website’s latest iteration but these fixes still need to be addressed.

 

 

 

 

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s