Hey, Kyle, I'm Jason. Hi, nice to meet you. Nice to meet you. I'll be your moderator for the day. All right. Just before starting, I just want to ask you to sign this non-disclosure agreement. Essentially, it's saying that everything we do today will be kept confidential and also gives you the right to skip any things that you feel uncomfortable answering as well as leaving the test at anytime. Let's get this started here. Here's a pen for you. All right. Okay. So, I'm just going to go over some of the things that we'll be doing today, kind of give you a rundown of the agenda, and then see if there's any questions, and then we'll just jump right into everything. Okay. So, for the usability test, we'll start off I'll ask you some basic background questions, then we'll complete eight different tasks, and then we'll follow-up by doing a quick debrief, I'll ask you some follow-up questions on the test that you did. All right. For the most part, the session should be as if you're by yourself. So, if you get stuck at any point, I'm not going to try to intervene right away. I want you to troubleshoot it almost like you're at home working the site by yourself. All right. One thing that's really important is to do a think aloud, which basically means I want you to narrate what you're doing. So, whenever you're taking any action, tell me what you're looking at, what you're expecting, what you're hoping to get done, okay? All right. Any questions before we begin? I don't think so, no. Okay. So then maybe just to start off, can you just tell me a little bit about the last trip that you planned? Yes. So, the last trip I planned was a spring break trip to Cancun a few weeks ago. Okay. Tell me a little bit more about how you planned it, who are the people and stuff. So, I was actually meeting some friends down there and I was responsible for my own lodging. I had a debate while planning I was either going to stay at a hotel next to theirs or stay at theirs. So, I also had to book plane tickets too, and then I guess I went on a few different sites to look for planning my trip. Okay. Which sites did you use? So, there was Hotwire, Kayak, Expedia, I checked my credit card site. Then for plane trips, I used Google and then I also, while after using the Expedia, et cetera, if there were hotels I was particularly interested, I then use that specific hotel site in addition. Do you remember in which site that you eventually used? So, after everything was done, I booked the hotel specifically through the Starwood Hotel site and then I booked a flight directly through Delta. Perfect. Okay. Let's jump into the tasks that we have prepared for today. So, the first task we have for you is pretend that your manager asked you to plan a trip for the company. She's heard of a website called Hipmunk and encourages you to use it. It's basically a site that helps you plan trips and aggregate home flights. So, plan a round trip from Detroit to Atlanta for under $250 from January 16th to 19th. Then afterwards, I want you to email the itinerary to [inaudible] @gmail.com. All right. Here you go if you need to review anything. This includes lodging? So, you're just planning your flights for now. Just flights. All right. So, I guess since I'm just doing flights, I would see that flight is already selected, round trips already selected. So, I'm good from Detroit, which I happen to know it's DTW. I don't know Atlanta's code, so I'll just search Atlanta. That appears to be that sounds right. So now, it's from January 16th to January 19th. So, that's 2018. I see them in 2017. I'll just have to click through I guess to get to next year. So, January 16th returning the 19th. So, just one person for me. I see I can either search or then compare with other sites. I guess my first instinct is just to see what happens when I just search this site. I don't particularly want the emails, so it's not there. There. It looks like the cheapest trip is $258 at the moment. So, since it's the cheapest trip is $258, it looks like you can compare and save. That's going to lead me to another site. So, I'm thinking about how I would go back I guess to that last option. Now, I'll try the back button and everything's saved. So, all right. Then sub-sites let you compare all on one site, but this just looks like it's linking me to another site. If we can't find the one flight at $250, then just find the cheapest one. The cheapest. All right. Yeah. That might do. So, I'll go back and search then. So, we've got a few options. It looks like to start, you would select your departure. So, there's a number of ones all the same price. They all seem to be American Airlines are the cheapest one, so this one has the most return options I guess and there's not a time. So now, I've got seven choices of return flights. Since they're all the same price, I don't really know any time constraints. I'll just pick one. So, it looks like you can book or there's an email itinerary button, which is what the goal was, so I'll do that. I'll put my name and I'll send it. So, I think I just did that. Perfect. Whenever you think you're done, just say something like, "I'm done." Yes, so I'm done with the task one. Wonderful. Next task. So, four people from the Chicago office, they want to attend a conference in New York from January 8th to 10th, what's the cheapest total price of the trip? Then note that any arrival, departure time is fine. All right. So, I'll go back, I'll edit the trip. Looks like I can just edit from right here. So, from Chicago, I guess I'll see city and see if it gives me all. I don't know what the difference between these two. Chicago to- What do you expect to happen? So, I would expect that it would give me departures from both O'Hare and Midway, is what I would guess but I would confirm by just doing it I guess. New York's probably going to be the same where there's a number of inbound airports. So, it looks like my dates are saved January 8th to the 10th, and then we've got four people I assume, coach. So, I will search again. When you see this going on, what are you thinking? I just think it looked like it was loading their trips. This kind of looks like an ad and so I'm ignoring the ad and just waiting for the bar to finish, because I assume that it's still looking for different flights until the bar finishes. So, I guess I don't really know what happened if I exit too early but my guess would be that it wouldn't search all the trips. I don't know if that's true. It appears to be $146.40 is the cheapest flight. They're all pretty much the same. So yes, it looks like $146.40 per person is the cheapest flight per person. I think I'm done with task two. Okay. Task three. Okay. So your manager wants to join the Chicago team in New York, your office is in Detroit, but then she wants to go to London for a week and then return to Detroit. She wants to fly business class, what's the cheapest price for her trip? All right. So, I'll go to edit. Sometimes there'll be ways to add multiple trips and I don't see it right now. So, I'll go back to that main page. Because I thought there is a multi-city option on the main page and there is. So, she wants to go Detroit to Chicago. So then, London. I assume London, England not Ontario. There it is. [inaudible]. All right, and then back to Detroit. There is a quick change. She wants to join the Chicago team in New York. All right. I see. So, Detroit to New York, then to London, then to Detroit. I think I filled that out wrong. I was just adding and so it would be from Detroit to New York City, from New York City to London, from London to Detroit. Then get rid of this leg. So, Detroit to New York City. Can I look what the date was? Yes, it's January 8th. So, she wants to get into New York City on January 8th. Scroll back to January, and then she wants to then go to London on the 10th, and then for a week would be seven days. So, it's the 17th, she would come back to Detroit, and she wanted business class into her trip. Business. So, searching. All right. So, for those particular dates, so this one's going to be 79. Select that leg, that leg, still the same, select that leg. $7944.60 would be the cheapest flight that I found and I think I'm done with that task. Okay. Next task. The LA office manager has a meeting in New York on October 16th, 2017. She wants to leave on October 15th after nine a.m. and wants to arrive in the morning before nine a.m. How many flight options do you have? All right. So, I'll just stay at this page. Never mind it stays at multi-trip, so I'll go back to the homepage. So, she wants to leave on October 15th. So, it's from LA to New York. She wants to leave on October 15th, and then return. So, arrive in the morning, that's arriving back home in the morning. She wants to depart LA after the morning and then she basically wants to arrive sometime after the morning as well. Arrive in New York? Yes. Oh, I see. All right. So, this up here appears to just be a one-way trip then, said she's not saying anything about going back. So, LA to, it's already in there, New York. She wants to leave on October 15th. It doesn't say. All right, search. So, I'll wait. So, I'm looking at select departure. This is the whole day. So, actually, it would be more helpful, at least for this purpose, to have the dates, not the weekdays, since I had no idea what. Oh, here it is, all right. Sunday, October 15th, now I see that October 15th is a Sunday, so I didn't know what day that was. She wants to leave after 9:00 AM. Looks like I can cut things off. So, she was to leave after 9:00 AM and then land before 9:00 AM. I guess that. I see that these are the two different time zones. So, it looks like she has 1, 2, 3, 4, 5, 6, 7, 8, 9,10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,35, 36, 37, 38, 39, 40. I don't know what that. Well, I'm not going to count all these. Looks like she has 253 because I see that number there. I also just doing. She's really in such as way of fitting. Is that the morning of the 15th, or the morning of the 16th? As you see on Monday. Okay I just want to confirm. Because that seemed impossible with the time zones, I was like, "This is an adding output." But, yeah. So, my final answer is 253. Wonderful. Next one. So, hope your manager booked a place to stay from October 16th to 18th. You want to find the top-rated hotel that has Wi-Fi, for under 350 a night in New York City. All right. So, I'll go to hotels now. I don't like having a lot of tabs open, so I'll just go and stay in this tab and then go back to hotels. There's 16th to 18th. Search. So, it is going to need to have WiFi. I'll wait for it to load. I'm trying to relate what you're thinking and feeling as well during each step. I'm just waiting for it to load. Seems like it's taking a little long, but I'll deal with it compared to what I'm used to. So, I need Wi-Fi, so I'll check the filters, amenities, free Wi-Fi, uncheck that, and then under 350 a night, so I'll drag this to max out at 350 per night. Well, I can id 349, 353. I'll do 349 then the other. I can't plug in the exact prices. So, I want the top rated hotel, so I should be able to sort somewhere right here by rating stars or I guess it's top rated, so rating. I don't know why that didn't disappear, but it appears to be the citizenM New York Times Square would be my answer. Correct. Task six. So, the more total. Yeah. You want to surprise your family with a visit over Christmas, but money is really tight. You want to set up a fare alert for a trip from Detroit to Seattle from December 22nd to December 26th. All right. So, I need to do a fare alert. So, when it's a trip that's just airfare. Yeah. Just airfare. So, I'll go to flights because I thought I remembered them doing stuff like that. I'm just saying if there's anything on the homepage. Mobile. I just knew that was the mobile site because I saw it pop up in the bottom left that URL was taking me to, but so now it looks like I'm round-trip. Detroit to Seattle on December 22nd to December 26th, that's so we're not going back in time because it says at 16, but would be nice if I could go valentine. One person, money's tight coach. So, this looks like what I was talking about when an email, when the price plummets. I'm hesitant to see what happens when I finish searching. Yeah, it goes away. So, I didn't want to type things in it for it to go away. So, I do see fare alerts right here though because that didn't say specifically fare alerts, so I don't know if that's what I wanted, but it appears to be. You could this one. That one right there? Yeah. I probably don't want that, so I'll subscribe. I'm not sure what it's doing. But, it looks like it just reloaded and I assume that I'm subscribed, I guess. Yeah. I don't exactly know what that means, but I'm subscribed to the fare alerts. Okay. I would say I'm done with the task. Perfect. Some good news, your parents decided to pay for your flight home. Save the flight you just planned on this account here, and so your parents can access it later, then just name it Family Trip. All right. Did I really plan a trip though? I just searched updates, I guess. Well, let's say you want to save this trip, so that they can kind of see these dates and the time. All right. So, I need to save a trip. I see this saying my saved flights and my eyes just went up to the top to see where the saved stuff was and I see mine. So, you ought not to tell me how you can save flights and assign it to a trip, save file by clicking the save flight link after selecting your departure and return. So, I'll just forward to do it. I'll have to select departure, return, and then save flight in the top right again. So, I'll save flight. Now, I need to log in and I again I don't want email letters. Oh, wait. Well, that's the sign-up buttons. So, I'll login. So, I saved it which in. No, thanks, I've saved the flight. All right, so I saved the flight. That's one. Yes, I saved the flight. I guess I forgot to name it. So, I remember if I go back here and here my activities and trips that looks like the saved flight or the saved one. My bookings, I swear I was just on my way I saved my saved flights and now I forgot where. What are you looking for at the moment? I'm trying to get back to my saved flights because I forgot to name it and I don't know, because I went to my bookings and there's nothing in my bookings. Nothing in settings, and then I went to my activity and trips and there's a search under activity which I guess I'll run again, and now I see my saved flights but I don't know why it didn't save. I think we're running a little bit low on time so let's just pretend you named it. All right. So, after you've seen your itinerary, your parents decided they want you to come over at least a week so I want you to basically access your saved flight, change the dates from December 22nd to January 4th. Well, that's what I was doing and apparently it didn't save so then I'll go back and save flight number, so I'm going to save it. There is the name that I forgot [inaudible] I guess I have to add this. I guess that's probably why I didn't save it. So, we move this search. Got it. All right, and so now I have to go back to access it. So, I search again and then my saved flights. So, I need to change the dates from 22nd to the 4th. So, I'll click it and then I will look for someway to change it, edit seems right. So, I'm staying all the way to the 4th and I'll search again. Wait, so here's the cheapest one, I'll select that leg, I'll select that leg and then I'll save this flight I guess. I think that fixed it because it says save to family trip. So, family trip, and I guess if I go back I don't know how else to get there. Yes, if I go back to flight results and then I go to my saved flights. It did not update so I'll go back. I don't know why it didn't update but. All right, so instead of update, I'm just going to delete this and then add it again to the 4th, and then search and then wait again and then there it is. So, it's saved. Add this search to a trip. Family trip. All right it's a family trip so I think I saved it. They're good. If I go back to my saved flights. All right, it's saved. Okay, so let me jump to this post-test questionnaire. This one, you fill this out. Try it. Fill it out or should I talk as I fill it out. You just fill it out and then I'll ask you a couple of questions. Since you're running the demo, I think it's okay to breeze right through it. Thank you. So, general thoughts? I guess reminded me a lot of the sites I've used. In reality, I would probably use a site like this or other ones. I usually use these sites to just look for flights and then go directly to the airline to book them unless they're giving me specific deals that I don't find at the airline. Look, I thought it was user friendly in booking a multi-trip one. It's pretty easy to sort of set time constraints, stuff like that. Okay. The saving the flight and saving the trip stuff was the one thing that was confusing and trip me off and I still don't know if I did that right. Okay, why don't we run through some of the task themselves and I'll ask you a little more specific questions. Well, since we're already talking about the saved flights. Let's say on a scale of 1-10, 10 being very confident might be knockoff now. How confident are you that you actually completed the task? So, well I know I didn't originally and then I went back and fixed it and then after I went back and fix it on the next task. I would say 7.5 out of 10 and then the second task about changing the flight, probably about five or six out of 10 because I mean, I don't even know if I changed it or if I just deleted it and re-added it because I tried to change it wouldn't change and then just to hurry up I just deleted it and redid it because it was pretty quick. So, that one's a lower number. Okay, and if I asked you what would help you validate that you did it correct. So, there was a thing that when you clicked it like the heart lit up and it said flight saved, but then when I went to my saved flights it didn't appear which confused me. So, some sort of notification or if it took me to the, if I saved it and there is some way to then go to the saved flights section to confirm that it's there, is what I would say. Okay, so I'm hearing is you got some sort of feedback when you initially clicked. Yes. But then when you actually double-check there is nothing there. Didn't appear to be accurate. So, if there was just it just showed up there, that's what you were expecting? Yes. Okay. Going back a little bit. On task one, you hovered on the X for awhile while all the thing was loading. Yes. What were you thinking on that point? My instinct because it appeared to be an ad and a pop up was to X out which is just an instinct I have but then I realized that it was searching, and I didn't know if X-ing out would just stop the search and give me what it had like there or fairly continue searching. So, I just had to stayed on the X and decided to wait because I didn't want to risk losing some flights. It sounds like you expect when you click X it'll cut the search? That yes, I don't know if I certainly expect it but I would see that as a possibility, yes. Okay. That would be it. That it could do that, that it could just stop the search. Okay, and if you're at home would you have waited for the entire time or? I would've waited depending on how many, by about the sixth or seventh flight, I may have just said I tried it. If I was just a two or three flights as long I would have waited eventually if I kept using and I probably would have X and see what it did, and then if it was good then I just stopped it I would realize that and go back, but if it didn't I would start X-ing from then on. For task two, so you typed in the city name and then both the city and different airports popped up. Yes. How do you choose the specific one that you chose? So, I saw that all the task ask for was to a city and I guess to my knowledge that all the airports are generally within the city or close to each other enough and so if all I had to do is get to the city, I just wanted the city because in parentheses I noticed that it showed all of the individual airports so my guess would that it would just combine those and then find the cheapest of any of those airports. So, since I just wanted the city and not a specific airport, I just did the city. All I'm hearing is the parentheses with all the airports after the city name gave you some validation of airports I would say that it validated it. Okay, I have one more question for you. So, on task three you mentioned that you filled it out wrong. What were you thinking at that point? I guess for some reason, I guess I wasn't paying complete attention, I was used to Google Maps in which when you add a destination it just goes below it and below it. So, if you want to set the destination you do destination one, two, three, and then the final one. So, I did the four and then after I've done that I realized that it actually had meant flight one to two, two to three, three to four, and so I guess that's why I miss stepped or whatever, just missed it. Okay, I think that's all the questions I have for you. I want to leave a little bit of time to see if you have any questions for me about the website, test or anything like that. Nothing comes to mind. Okay. Then I just want to say thank you very much for participating, and if you have any follow-up questions please let me know and we really appreciate your time. All right. No problem. Thank you
