This section presents the indicators analysed within the scope of this study, including measurements for Internet access reliability and application performance, i.e. web browsing, file transfer, etc.
Service access failure
Given that the service is available in a certain area (the possibility of accessing the Internet, guaranteed by the operator's existing coverage and the availability of the requisite technology), it is important to know that it is ready to be used at any time.
This indicator provides the percentage of cases where it was not possible to establish a connection to the data network. The reasons for failure can be related to network conditions.
Causes of failed service access
This indicator details that fault that is responsible for the service access failure.
The connectivity of mobile networks to external networks (Internet) is based on the creation and maintenance of ''PDP contexts'' for the relevant mobile and data networks. PDP means Packet Data Protocol. In order to send and receive data the mobile device (mobile phone or card) creates what is called a ''PDP Context''. This PDP context enables the transfer of data between the mobile device and the Internet access (a mechanism which establishes, for the system, the authentication and authorization of the user to benefit from that service).
IP means ''Internet Protocol'' and it specifies how to segment the information by data packets so these can be transferred from one point to another, including the specification of the source and destination addresses from which and to where the packets must be transferred.
In short, and reducing the process to how the analysis is carried out, the mobile device guarantees the connectivity between itself and the operator’s data network through the ''PDP Context'', while the connectivity from the operator’s data network to the Internet is guaranteed through the ''IP Connection''.
Activation/connection time
This indicator represents the time needed to access the service successfully, i.e., to activate the ''PDP Context'' and establish a specific access to the service.
Latency
Latency is the time elapsed from the beginning of a transaction and the first reply to that transaction. It is one of the fundamental properties that affects the efficiency of the HTTP and TCP communication protocols.
In order to guarantee a good performance of the connections, it is best to have a low latency (the lower the latency, the better the performance). The latency determines the speed at which a TCP connection can be established, and in some cases the maximum data transfer speed.
For the end user this indicator signifies that if the latency is very high the desired speed may not be reached, or it may take longer to reach it. This indicator is particularly important when loading pages (web browsing) made up of several objects.
PING is the way of measuring a network’s latency, using the ICMP protocol's echo command. It sends a data pattern to a network point and requests it to send back a reply with the same pattern. This process allows the PING to measure the time that elapses between the request and the reply. The size of the PING packet used in this study was 256 bytes.
NOTE: latency measured with PING is used as a generic indicator. There are many factors that affect the latency of different protocols and applications.
Time to load a page (web browsing)
Page download times while browsing the web are usually more perceptible for the end user since they represent the time spent waiting to see the entire page requested. Port 80 is used (usual for HTTP over TCP).
File transfer speed
The file transfer speed is defined as the volume of data transferred from one network point to another, in time. It is usually measured in Kilobit/s (Kbps) or Mbit/s (Mbps). It is important to notice that quite big files should be used with respect to the access speed in order to correctly assess it. This is because, in each FTP session for example, there is a number of constant delays due to the connection time and to the start-up of the related protocols (TCP), whose impact should be minimized with respect to the real transfer time. In the case of transfer via Bittorrent, it used a file of the same total size as that used in FTP, although that application divides the file to be transmitted into equal portions of 250 kB, or even into 16 kB sub-portions (when the access is slow).
So in order to guarantee a rigorous evaluation of the upload and download transfers, test files should have a recommended size (in kB) of at least twice the maximum size of the connection in kbps. But whenever possible the size implemented was four times greater.
However, since there was some variability in the actual speeds recorded (lower speeds) versus those advertised, with the immediate repercussion of compromising the time frames available for carrying out the battery of tests, a scheme of adaptive file dimensions was implemented. This way, and regarding the test files used for fixed accesses, their size took into account the average download and upload speed obtained for each access tested (considering the rule about file size described above).
In the case of mobile network accesses, randomly created binary files were used, with a size of 4 MB for download and 1.6 MB for upload. The same file was used for all operators since all their advertised speeds should be similar.
NOTE: the file transfer speed (FTP protocol) is quite different from the web browsing speed (HTTP protocol). The speed reached during an HTTP session is much slower that the one achieved during an FTP session. The reason for this is that the HTTP protocol is greatly affected by latency (RTT), mainly on mobile networks. HTTP transfers require multiple TCP connections, as well as DNS lookup. Each TCP connection requires several RTTs to completely open the TCP sending window, and each DNS lookup requires several RTTs before mapping the domain name to the IP. These RTTs and TCP/DNS considerably degrade the HTTP performance. Considering this difference, web browsing speed and file transfer speed are not directly comparable.
Loss of Packets
The loss of packets can be cause by several factors including data corruption, insufficient bandwidth, and the delivery of packets in the wrong order. Any loss of packets affects the quality of service; but its impact varies according to the application in question. In order to evaluate the loss of packets a streaming script is used; this is an application where the loss of packets has an impact on the quality of service.
Jitter
When a datagram is sent, the issuer adds a timestamp to it. When it is received, the receiver adds another timestamp to it. These two timestamps are used to calculate the datagram’s transit time. If the datagrams’ transit time for the same test differs for different sessions, the test experiences jitter. In fact, Jitter effects are similar to those of the loss of packets, since the jitter causes ''buffer starvation'', which is equivalent to the loss of packets.
The volume of jitter in a test depends on the degree of difference between the datagrams’ transit times. If the transit time is similar for all datagrams (regardless of the transit time), the test experiences no jitter.
DNS (Domain Name System) resolution time
The quality of the usage experience during web browsing is usually accredited to the time associated with the location (resolution of the page address) and the loading of the requested page by the web browser.
One of the parameters that most contributes to this application’s good performance is the response time of the page resolution server (DNS – server response time). One of its most common uses is to translate machine names into their IP addresses and vice-versa.
For it to be possible to evaluate the DNS server resolution time, each battery of tests included the access to about one hundred web pages (top 100 pages most visited by Portuguese users).
The purpose of this test was to evaluate the performance perceived by residential users who usually use the DNS servers provided by their ISPs, using for that purpose a DHCP (Dynamic Host Configuration Protocol) configuration in their network interfaces. This way all local cache mechanisms used for DNS resolution (operating system and Internet browser) were eliminated.
