Mobile Phone Sites Fail to Connect

August 12th, 2009

Despite selling the technology frequently used for social networking, Britain’s biggest mobile phone e-retailers are failing to plug into the phenomenon, according to a report from benchmarking specialist Syntagm (http://www.syntagm.co.uk). The study of 12 leading mobile sites found that the majority offer very limited interactive features, and don’t encourage user-generated content or attempt to foster a sense of community – in stark contrast to e-commerce leaders like Amazon.

Recent research from Ofcom revealed that 30% of British adults now have a social networking profile, with many accessing this virtual world via their mobile phone. And consumers have come to expect the same level of interactivity when shopping online: user reviews, self-service and control over purchasing are commonplace everywhere from ASOS to eBay.

When it comes to buying a mobile, however, customers might as well skip the online experience and stroll into a ‘bricks and mortar’ store if they’re looking for an interactive shopping experience. Whilst some of the sites provided interactive search facilities or 3D phone viewers, only two (Three and T-Mobile) allowed potential customers to chat online to a sales advisor.

None of the sites made personalised recommendations based on previous visits, and most did not even attempt to show users the products they most recently viewed – a serious flaw given the amount of ‘site hopping’ people tend to do when shopping online.

Several sites subscribed to ‘read only’ third-party product reviews, but it wasn’t possible for users to make comments about their own experience of a particular product, or leave questions to be answered by other punters.

William Hudson, Syntagm’s CEO, comments: “Successful e-commerce sites echo the sense of community that has made social networking sites so popular, by enabling users to communicate and share information. In our survey, only outsider Expansys came close, whereas its bigger competitors seem to have got their wires crossed when it comes to social networking opportunities.  There’s a lot more they could be doing.”

Syntagm’s benchmarking report compared user experience between Argos, Carphone Warehouse, Expansys, mobiles.co.uk, O2, Orange, phones4u, Tesco, T-Mobile, Three, Virgin Mobile and Vodafone.

The study concluded that the sites are failing consumers on some of the most basic services needed to buy a phone. And the biggest names were often the worst culprits: relative unknown Expansys was the only site to score above 50% on overall experience for online shoppers, putting its mainstream competitors to shame.

Hudson adds: “At a time of intense competition between the main mobile traders, they need to offer their online shoppers a more engaging experience. Our report clearly shows where their strengths and weaknesses lie – we hope the companies concerned will make use of the information it provides.”

Ends

Notes for editors

About Syntagm:
Syntagm is a small consultancy based in Oxford. Established in 1985, it specialises in design for usability (user-centred design and user experience) and people development. It has worked more than 100 blue chip organisations across Europe and North America.

About the benchmarking report:
Benchmarking took place in late May and early June 2009, comparing the 12 mobile phone sites across 13 categories: content, visual design, navigation, engagement, accessibility, trust, persuasion, shopping basket, search, selection, checkout, account management, and online support.

Additional sites can be benchmarked on request. Syntagm and its staff have no financial interests in any of the organisations benchmarked.

 For further details or to purchase the summary report visit www.uxbench.com

Buying a Mobile Phone Online? Prepare for Confusing Content and Poor Customer Service

August 6th, 2009

A substandard shopping experience and second-rate service are blighting Britain’s biggest mobile phone e-commerce sites, according to a new report from benchmarking specialists Syntagm (http://www.syntagm.co.uk). Findings show that the majority of mobile sites suffer from poor navigation, lack of information and virtually no online support – failing consumers on some of the most basic services needed to buy a phone.

The benchmarking report compares user experience between 12 of the leading vendors – Argos, Carphone Warehouse, Expansys, mobiles.co.uk, O2, Orange, phones4u, Tesco, T-Mobile, Three, Virgin Mobile and Vodafone. Whilst visual design scored highly, with an overall average of 75%, account management scored just 12%, and online support was the weakest area across all sites, trailing the results table with a paltry 2%.

Most sites did not provide login accounts for purchasing – customers had to rely on email updates or ring customer service to track or cancel an order. And features which are mainstream on most e-commerce sites – allowing users to view or modify recent orders – are conspicuously absent if you want to buy a phone online (unless you’re shopping on the Expansys site).

When it comes to dealing with delivery delays and reporting or returning damaged goods none of the sites came up to scratch. Not one site lets customers ask a support question and get an immediate response online, although Expansys, O2 and Virgin offered online forums. When asked ‘How do I return a phone?’ Virgin’s automated Q&A service wonders if you would like to buy a phone. The other nine benchmarked sites would only deal with problems or questions through telephone help desks (often at national-rate charges).

Many sites performed poorly in providing effective product information. The content average was only 57% with O2 scoring the top mark of 83%. Trying to find a handset by feature was impossible on most sites, and only two (Three and T-Mobile) allowed potential customers to chat online to a sales advisor. And Tesco’s search facility returned products from across its entire product range – even though the search was performed from the mobile phone pages (a search for ‘LG’ turned up fridge-freezers and flat screen TVs as well as phones).

For online reviews and control over purchasing and support, mobile e-retailers could learn a lot from facilities customers have come to expect from sites such as Amazon. None of the sites made personalised recommendations based on previous visits, and most did not even attempt to show users the products they most recently viewed – a serious flaw given the amount of ‘site hopping’ people tend to do when shopping online.

Tasks which consumers take for granted in a physical ‘bricks and mortar’ shop, such as checking whether a product is actually in stock, are surprisingly absent in UK mobile phone e-commerce. Expansys was the only site to include clear availability information for every product, and Carphone Warehouse provided a fully functional ‘phone finder’, but seven of the 12 sites surveyed scored zero on this measure. In fact, some even promised next day delivery of a phone that was out of stock.

As a whole, the sites left users to research and understand terminology, features and key purchasing decisions themselves. Other common problems included a lack of clear delivery information and, in some cases, checkout pages that would challenge even the most determined purchaser.

William Hudson, Syntagm’s CEO, comments: “Customer service should be a priority even if you aren’t physically dealing with a person. Based on our study, consumers are, at best, confused and, at worst, badly served by mobile e-retailers. If they want to build their brands as trusted online retailers, in the same league as sites such as Amazon, Dabs and More Computers, we would recommend they look at creating a more rewarding shopping experience and providing proper online support beyond the purchasing process – and especially when things go wrong.”

Ends

Notes for editors

About Syntagm:

Syntagm is a small consultancy based in Oxford. Established in 1985, it specialises in design for usability (user-centred design and user experience) and people development. It has worked more than 100 blue chip organisations across Europe and North America.

About the benchmarking report:

The benchmarking took place in late May and early June 2009, comparing the user experience between 12 of the leading UK vendors: Argos, Carphone Warehouse, Expansys, mobiles.co.uk, O2, Orange, phones4u, Tesco, T-Mobile, Three, Virgin Mobile and Vodafone.

Syntagm compared the sites across 13 categories: content, visual design, navigation, engagement, accessibility, trust, persuasion, shopping basket, search, selection, checkout, account management, and online support.

Each category received a numerical score and written observations. The numerical scores make it possible to see clearly where strengths and weaknesses lie for each of the mobile e-commerce organisations concerned, and improved scores can be used as targets for future development.

Additional sites can be benchmarked on request. Syntagm and its staff have no financial interests in any of the organisations benchmarked.

For further details of the report or to purchase a copy visit www.uxbench.com

Mobile Phone E-Commerce Report Delivers Blow to Big Brands

August 6th, 2009

Only outsider Expansys scores above 50% on overall experience for online shoppers

Britain’s biggest mobile phone brands are lagging behind relative outsider Expansys when it comes to the design and usability of their e-commerce sites, according to a new report from benchmarking specialists Syntagm (http://www.syntagm.co.uk). Findings show that the majority of mobile sites suffer from poor navigation, lack of information and limited online support – failing consumers on some of the most basic services needed to buy a phone.

The benchmarking report compares user experience between 12 of the leading vendors (Argos, Carphone Warehouse, Expansys, mobiles.co.uk, O2, Orange, phones4u, Tesco, T-Mobile, Three, Virgin Mobile and Vodafone), revealing some big surprises in the ratings – and an unprecedented overview of the market as a whole.

Specialist handheld reseller Expansys topped the results tables in eight of the 13 groups (including some tied scores) but its overall score was still only 62%. O2 was ranked in second place overall with a score of 50% and Argos scraped into third position with 47% for overall user experience.

Tesco may be marching further into the mobile market, but online it slows to a crawl: it received the lowest scores for trust and shopping basket design, and suffered from problematic navigation and an indiscriminate search facility. This combination of weak results earned Tesco the bottom position overall, with a user experience score of just 36%.

Many of the mobile providers were found guilty of prioritising style over substance: half of the sites scored 80% or above for visual design but only O2 had a respectable content score (83%), with the majority performing poorly in providing effective product information (three quarters achieved 60% or lower).

Online support was the weakest area across all sites, scoring just 2% overall. Expansys, O2 and Virgin received some marks for offering online support forums, but the other nine benchmarked sites received zero scores for only dealing with problems or questions through telephone help desks (often at national-rate charges).

With current levels of spam, phishing attacks and credit card fraud, consumers have every right to be suspicious of e- commerce sites. Yet very few of the benchmarked sites provided adequate levels of reassurance. Clicking the ‘Internet shopping is safe’ logo on the Orange checkout page produced a certificate error.

A surprisingly poor performance came in the area of persuasion (trying to sell services), which only averaged 11% in its benchmarked group. And, where accessories or upgrades were offered they were often irrelevant to the phone selected. The most commonly offered additional service was handset insurance, despite this already being included in many home contents policies.

“Having spent seven years benchmarking and offering design advice on intranets we’re now turning our attention to e-commerce sites, starting with mobile phones. The overall results were unexpected and disappointing, but we hope that the companies in the survey will find the information useful,” said William Hudson, Syntagm’s CEO.

“At a time of intense competition between the main mobile traders they can’t afford to lose customers due to confusing content, shocking customer service and missed sale opportunities. Expansys may not be a mainstream mobile phone site, but the user experience it offers puts many of its bigger competitors to shame.”

Ends

Notes for editors
About Syntagm:
Syntagm is a small consultancy based in Oxford. Established in 1985, it specialises in design for usability (user-centred design and user experience) and people development. It has worked more than 100 blue chip organisations across Europe and North America.

About the benchmarking report:
The benchmarking took place in late May and early June 2009, comparing the user experience between 12 of the leading UK vendors: Argos, Carphone Warehouse, Expansys, mobiles.co.uk, O2, Orange, phones4u, Tesco, T-Mobile, Three, Virgin Mobile and Vodafone.

Syntagm compared the sites across 13 categories: content, visual design, navigation, engagement, accessibility, trust, persuasion, shopping basket, search, selection, checkout, account management, and online support.

Each category received a numerical score and written observations. The numerical scores make it possible to see clearly where strengths and weaknesses lie for each of the mobile e-commerce organisations concerned, and improved scores can be used as targets for future development.

Additional sites can be benchmarked on request. Syntagm and its staff have no financial interests in any of the organisations benchmarked.

For further details or to purchase the report visit www.uxbench.com

Dates study, user experience benchmarking and more…

July 8th, 2009

Many thanks to the hundreds of people who took part in our dates study. We had almost 1,000 responses in a one week period. I have written a full report of the study and put it in the resources section of our site – www.syntagm.co.uk/design/datesstudy.htm – but read on for a headline summary. Respondents who used month-first order were torn between ‘8/2/09’ and ‘August 2 2009’ with the former slightly ahead. Those who used day-first ordering had a strong preference for ‘2 August 2009’ while a very small number of respondents used the ISO date standard: 2009-08-02. Having said that, the thing I really wanted to know was whether people used leading zeroes without being asked. The answer is pretty much ‘no’. 76% of all responses used no leading zeroes in the day or month. Read the full report to find out why this is interesting (to me, at least).

Having spent 7 years benchmarking intranets, it is now the turn of e-commerce sites. The first in our new series of user experience benchmarking reports addresses the UK mobile phone market and covers 12 of the leading vendors: Argos, Carphone Warehouse, Expansys, mobiles.co.uk, O2, Orange, phones4u, Tesco, T-Mobile, Three, Virgin Mobile and Vodafone. There were some big surprises in the ratings – see our overview at www.uxbench.com. Also, join our UX benchmarking email list to have a chance to win a personal copy of the report. Just send an empty email to subscribe-uxbench@listman.syntagm.co.uk (unsubscribing is easy too).

Finally, a reminder of our upcoming courses. We are running our full-day card sorting and Ajax interaction design courses in London, Las Vegas and Berlin in the Autumn (the later two venues as part of the Nielsen-Norman Group conferences). See www.csadvances.com and www.ajaxusability.com for further details (and to book the London dates).

Card Sorting and Ajax Usability Courses in London, Las Vegas and Berlin

June 3rd, 2009

We have been asked by the Nielsen-Norman Group to present our card sorting and Ajax usability courses at their Usability Week conferences later in the year. (The half-day versions were well-received at CHI 2009 in Boston and will appear at HCI 2009 at Cambridge University in September – see http://www.hci2009.org).

So here is the current Autumn schedule:

ADVANCES IN CARD SORTING (FULL DAY)

  • 5 October 2009, London
  • 11-16 October 2009 (part of NNG Usability Week), Las Vegas

AJAX DESIGN AND USABILITY (FULL DAY)

  • 6 October 2009, London
  • 11-16 October 2009 (part of NNG Usability Week), Las Vegas
  • 15-20 November 2009 (part of NNG Usability Week), Berlin

The dates for the Nielsen-Norman Group presentations have not been fixed yet, but will appear on their site in due course (http://www.nngroup.com). I would be happy to answer any questions about the content of these courses, but all practical arrangements will be through NNG.

For the London courses, you can book directly through our web site. Discounts are available for groups of three bookings or more. A further discount is available (on the one-day price) for places on both days.

Visit:
http://www.csadvances.com for Advances in Card Sorting (5 October, London)
http://www.ajaxusability.com for Ajax Design and Usability (6, October, London)

Both days can be booked from either page. All of our courses (including the above) can be found at http://www.syntagm.co.uk/design/courses.htm

UCD Courses at HCI 2009 and in London

May 13th, 2009

We’re pleased to announce that two of our courses have been accepted for presentation at the annual British HCI Group conference (HCI 2009), this year at the University of Cambridge from 1-5 September. The courses are

  • Innovations in Card Sorting: A Hands-On Approach – this is our half-day workshop on paper-based card sorting using our advanced analysis software, SynCaps V2. (See http://www.syncaps.com for further details.)
  • Ajax Design and Usability – a half-day course on how to use Ajax and similar web technologies to improve the user experience rather than to frustrate it.

Both of these courses were very well received at CHI 2009 in Boston last month. They will be available at the HCI 2009 conference in Cambridge on Tuesday 1 September. Half-day courses at the conference will cost only £40 plus, you do not have to register for the conference to attend the courses.

Full details of conference courses can be found at http://www.cl.cam.ac.uk/conference/hci2009/tutorials.html

We will also be running a more extensive one-day course on both paper and online card sorting in London on 5 October. The cost includes a fully-licensed copy of our SynCaps V2 analysis software. For further information and to book online visit http://www.csadvances.com

News Summary

April 27th, 2009

I’m moving news items from my design home page to a new blog category called News (unimaginative, I know). Here is a summary of news for 2008/2009.

We presented three courses and a short paper at the CHI 2009 conference in Boston. This puts us on a par with Carnegie Mellon (who also had three courses but slightly more papers<g>):

  • Innovations in Card Sorting: A Hands-on Approach
  • Web Design for Usability
  • Ajax – Design and Usability

(These courses can also be run in-house. Details are available at http://www.syntagm.co.uk/design/courses.htm)

We’ve posted the slides and a recording from our half-our card sorting webinar, plus four videos showing how to prepare for and capture data from card sorting sessions.

The paper concerns a study we conducted using Simon Baron-Cohen’s empathising and systemising quotients. It provides a psychological explanation for the difficulties technologists have in seeing problems from a user’s perspective:

William Hudson was the invited keynote speaker for the CADUI 2008 conference in Albacete (near Madrid). He also presented a half-day tutorial on card sorting. An essay based on his keynote address is available here.

Do you like using or teaching Card, Moran and Newell’s Keystroke Level Model but hate the arithmetic? Treat yourself to our free KLM calculator!

Bandwagon Cues – To Jump or Not to Jump?

April 24th, 2009

Continuing on the theme of new research from the CHI 2009 conference, I came across another interesting poster – this time from Penn State University (S. Shyam Sundar, Qian Xu and Anne Oeldorf-Hirsch, “Authority vs. Peer: How Interface Cues Influence Users”). Here the researchers were considering the effect of an authoritative logo and customer comments on purchase intention, participants’ product attitudes and a raft of other factors they referred to as “bandwagon perceptions”.

For the studies they used a typical product page, as shown in Figure 1 (adapted from Amazon). The “authority” seal shown (above the camera) was either for a fictitious organisation (Zig! as displayed here) or for CNET, a well-known technology review site (or it was absent altogether). Both seals were fairly similar in appearance.

Typical product page showing Zip! authority seal 

Figure 1: Typical Product Page

The researchers also made use of “outlying reviews” – that is, one where the reviewer expressed dissatisfaction with a product, even though the average score among all reviewers was high. So, in summary, there were two main effects of interest: faux versus “real” authority logo and presence or absence of “outlying” (dissenting) review.

Some of the reported results are as might be expected, but there was one very interesting interaction between the authority seal and an outlying review:

  • When no authority seal was shown, the presence of an outlying review had no effect on purchase intention
  • When the CNET seal was shown, purchase intention was contingent on the absence of an outlying review (that is, an outlying review reduced the likelihood of purchase)
  • When the Zig! Seal was shown, the situation was reversed (that is, an outlying review increased the likelihood of purchase)

The relatioship between the seals and the presence of outliers is shown in Figure 2. Needless to say, this final point – that purchasing intention increase with an outlying review – is somewhat counter-intuitive. The researchers offer no explanation for it and I have none to offer. It would be interesting to know whether other studies show any similar effect. I’ll write more on this if something turns up!

Chart showing negative effect of outlying review with CNET seal and opposite for Zig! seal

Figure 2: Purchasing Intention by Seal Condition and Presence of Outlying Review 

Facial Avoidance in Page Design

April 20th, 2009

I am a great admirer of Tom Tullis’s work. He’s made important contributions to our understanding of screen design over the years (plus he is even older than me, which gives me great cause for hope!). In a recent poster at CHI 2009, in collaboration with Fidelity Investment colleagues Marisa Siegel and Emily Sun, Tom has done it again. He explained to me their important findings on the use of face images to attract attention on web or intranet pages. The results certainly run contrary to the perceived wisdom in this area (that humans are naturally attracted to facial images).

The Fidelity studies show that not only were users disinclined to look at facial images of the type shown in Figure 1, but that a significant number of users were unable to find the text immediately adjacent when given a task requiring that information.

Box containing facial image next to link text

Figure 1: Link panels with and without facial image (Study II)

In fact, overall performance was negatively affected by the presence of the face: accuracy dropped from 93% to 78%, mean time on task increased from 37 to 54 seconds and perceived task ease dropped from 4.2 to 3.4 (on a 5-point scale).

In a separate study, Tullis and colleagues not only found that facial images reduced performance; they also had the completely unexpected effect of reducing user confidence. Figure 2 shows the quite common approach that was tested (with and without the images). Again accuracy dropped, but this time by a smaller margin, from 94% to 87%. However, trust also dropped, from 6.3 to 6.0 on a 7-point scale. This result may seem small but there is only a 5% probability of it being due only to chance, which is taken by statisticians to mean that it was significant.

 Box show author images next to link text

Figure 2: Authors’ images reduced trust in content (Study III)

So, contrary to what I remember being told in design seminars years ago, and even contrary to advice that I would have given up until I saw these studies (typified by my article on using images as links – http://www.syntagm.co.uk/design/articles/fitts50.htm) it looks like we need to be careful about how we use facial images in particular.

(You can see the full work-in-progress paper in the CHI 2009 proceedings, which will eventually be available from the ACM Digital Library – www.acm.org/dl)

User What? How to Alienate Your Colleagues with Live Meeting.

February 23rd, 2009

I am not a Microsoft basher, or at least I wasn’t until Vista stole a year of my life in its troublesome pre-SP1 version. However, my recent experience with Live Meeting 2007 has me questioning any remaining pro-Microsoft sentiment that I have been nostalgically harbouring.

It was a simple enough problem: set up a half-hour webinar and invite participants without publishing all of their e-mail addresses to everyone attending. With the Live Meeting plug-in, it even looked deceptively simple. The most complicated part appeared to be creating a distribution list to prevent individual e-mails from being shown. But it’s a steep, slippery, downhill slope from there.

The first problem is that Outlook creates a ‘dummy’ message for you to send to attendees. Users are supposed to realise that they can add their own text at the top of the message but the bottom part of the message, where the important details like telephone numbers and Web URLs go, will actually be entirely replaced when the message is sent. So, if you want to correct the phone number (the one supplied had a superfluous zero) or add details for local numbers in other countries, you can just type it all in and then have it discarded without notice. Simple huh? Happily, I discovered that in a test meeting I created beforehand.

For the real meeting, participants were signing up over several days. The initial invitation went out correctly but on opening the calendar entry to add further recipients, I discovered the distribution list was gone, having been replaced by a list of individual e-mail address.  What’s a user supposed to do at that point? Update the distribution list or the calendar entry? I may never know if I guessed wrong, but when I started getting two or three separate acceptance e-mails from participants, I suspected that all was not well.

On checking my Live Meeting account, I discovered to my horror that there were now three separate Live Meetings: the original, plus one for each time I updated the attendee list. They had separate meeting codes, so what would have actually happened on the day is that the lucky attendees who were invited first would get to see and hear about card sorting in its full glory, while those in the other meetings would be in an expensive state of limbo (since the calls and webinar connections were all being charged for through Live Meeting). The only remedy was to send an apology to everyone, cancel all the meetings and start again from scratch.

It may be that some of these problems were caused by the Live Meeting plug-in for Outlook as it’s hard to guess how this functionality is divided. What I do know is that if I try to send invitations from the Live Meeting website, I get precisely the problem I wrote about in my last blog entry: the Microsoft server pretends that it is sending e-mail messages from me and it isn’t authorised by our Sender Policy Framework (SPF) to do that. So any e-mail server doing SPF checking will reject the message as forged. And since the Outlook-based e-mail invitations work so badly, I had to devise my own mail merge and calendar entries to be certain that none of this was going to happen again.

It is almost as if some parts of Microsoft have never heard of user experience. And they’re not the only large organisation with this problem. But they really, really should know better.