|
 |
 |
 |
|
|
Your advert here!!! Technical Editors:
Last modified: |
Putting WAP to the Test
By Chris Goswami and Peter Bird, The NCC
Group WAP
technology is so tempting, but so frustrating for users, because applications
and equipment are not undergoing necessary tests before they are online. It
has been said that, for any WAP application to be useful, it has to do one of
two things: Save time or kill time. Within this broad definition lie thousands
of WAP sites offering the tantalising prospect of transacting business anytime,
anyplace, anywhere. The problem starts when they reach the phone in your pocket;
many sites disappoint in a big way. Many
common problems marring users' experiences on the wireless Internet can be
avoided by the use of test services and tools currently available. By way of
example, here are some failures encountered by the NCC Group when testing
wireless sites and gateways over the past 6-9 months (for actual stats, see the
next page): *
Sites which cannot scale for large numbers of users *
Reliance on card titles and fonts (which some phones do not support) to
convey information *
Getting 'trapped' in a card where the only way out is to call the
browser's main menu or drop the connection *
Inconsistent navigation structures *
Displaying outdated information *
Displaying improper content without warning *
Inaccurate links (mainly external) *
WML deck sizes exceeding phone limits (either statically or dynamically
created) *
WML DTD definitions not matching the document *
Fundamental lack of interoperability between gateways and handsets Users
deserve a better experience when accessing wireless content. Furthermore, WAP is
evolving; new services like 'push' functionality and enhanced security are being
added to the protocol. These will facilitate development of far more
sophisticated applications. Success
will depend on ensuring that the technology can meet stringent performance,
resilience and security demands. And there are tools and services out there to
help developers, suppliers and operators do just that. Independent
testing programs have been established: *
The WAP Forum operates a certification scheme to cover application level
(WML) testing, and WAP protocol compliance testing in its drive to help industry
provide interoperable WAP handsets and applications. They operate a reference
pool for tested devices and offer free test tools for members' use. The WML
application level certification is available now; the WAP protocol certification
will be later this year. *
The Motorola Certification Program - WAP Module (MAGNET scheme) launched
early this year aims to give mobile operators and end users confidence that
their applications will work well with listed handsets, platforms and gateways.
NCC has been selected as the authorised test centre. The scheme requires WAP
developers to submit applications (sites) for certification testing which covers
content, presentation, ease of use, interoperability, performance and
resilience. Successful developers
may access further benefits offered by Motorola's developer support program. *
Other tools and services Like WAP
itself, production of test tools is a new industry and the requirements from
developers, service providers and content providers will emerge. The types of
tools required include interoperability testers, phone or gateway emulators, WML
checkers, load testers and security testers. Sources of these tools are the
phone manufacturers them-selves, the WAP Forum, suppliers of Web-testing
technology and independents (see Table). Just as
with tools, the requirements for testing services are developing. What is clear
is there are urgent requirements for load-testing services and interoperability
services to determine whether an application will operate on a particular
handset on a specific network provider. Some of this testing will be done by
application developers and phone manufacturers themselves, but there is a
growing need for independent testing by specialists. The
lesson from this? Success is guaranteed for those who can deliver, but failure
is very public. Those who succeed will to be the far-sighted ones who understand
that testing is as important as development. Statistics for most common WAP application failures (October
2000 to May 2001) General 1.
Many applications use deck structures that are far too deep, eg one email
application needed to go through over eight cards just to send an email.
Applying the Internet adage of 'you lose 50% of users with any click, and you
wont have many left! 2.
The test results for any applications suggested that they had only been
tested with an emulator, or a single handset (most often the 7110). Most
common core failures; 1.
30% of applications tested have had either a screen with either no/broken
links, or trapped the user in a recursive loop of links, forcing them to drop
the connection 2.
35% of applications failed testing for consistent navigation 3.
18% contained decks which exceeded the phones capacity, and thus did not
display at all 4.
20% relied on WML card titles to convey significant information to the
user (some phones do not display card titles) Style/supplementary
failures 1.
Almost 70% of applications tested relied on labels assigned to the 'prev'
softkey 2.
90% did not make use of the <emptyok='true'> or <emptyok='false'>
tag as appropriate 3.
30% of applications made no provision for text only mobile agents 4.
Almost 50% of all sites certified contained grammatical errors in the
text Other Most
common failures were due to developers lack of knowledge regarding the vast
difference in capabilities and display features of the variety of handsets
available. (But still, only 1/3 of all applications made use of the
HTTP_USER_AGENT). Table
|
|
