ANACOM - Autoridade Nacional de Comunicações has conducted an evaluation of the quality of SMS and MMS services provided by operators Optimus (Sonaecom), TMN and Vodafone, through an assessment of the technical parameters that reflect the perception of quality from a consumer point of view. The results demonstrate very good performance of these services.
In general, the mobile communication systems studied have messages delivery rates in excess of 98%. The average message delivery times are less than nine seconds for the SMS service, and less than 54 seconds for MMS.
No significant differences in terms of service performance were found over the course of the day, or between working days and the week-end.
With respect to the short message service, the study shows very good levels of performance of the SMS service, whereby no significant differences between operators were observed. The best performances were recorded for TMN and Vodafone, with 99.7% and 99.3% of messages, respectively, successfully delivered to recipients. In addition good average delivery times were recorded. TMN and Vodafone achieved the best level of performance, with average times of seven seconds.
As for the performance of the SMS service over the course of the day, except for Optimus in the period between 21h and 24h, there were no significant variations in the indicators analysed.
The levels of accessibility of the service of TMN and Vodafone are always over 99%, while for Optimus levels above 98% were reported - except between 21h and 24h, when a deterioration is seen for this indicator.
The values reported for message delivery rates are between 98% and 100%. The lowest rates belonged to Optimus, between 21h and 24h, due to poor service accessibility recorded during this period.
In terms of the average message delivery time, the fluctuations over the course of the day were very minor - around a second between the best and worst average time recorded.
Regarding the multimedia message service, the performance of the mobile systems was found to be very good. The message delivery rate indicator recorded levels above 98.5% without significant differences found between operators.
With respect to the average message delivery times, significant differences were found between operators. The best performances was achieved by Vodafone and Optimus, with average delivery times of 35 seconds, rising to 53.4 seconds. in the case of TMN.
No significant variations were found over the course of the day. The MMS transmission rate varies between 98.5% and 100%, except for TMN which between 21h and 22h has a rate of 97.3%.
The MMS delivery rates range from 96% to 100%, with similar results for the three operators.
With respect to the average MMS delivery time, regular results are found for most of the day, with average times for Vodafone and Optimus reported between 30 and 40 seconds, rising to 48 to 58 seconds for TMN. The widest disparity was registered for TMN, with the indicator recording an average of 91 seconds in the period between 15h and 16h.
In work carried out by ANACOM, the main indicators of quality were examined taking into account the point of view of the user. Among the indicators analysed were service accessibility, message delivery rates, message delivery times as well as any disparity between these indicators over the course of the day.
Measurements were made in similar conditions for the three operators, and field measurements were taken between 9 and 15 July. In total 15,117 tests messages were sent, corresponding to 72 hours of measurements.
The sample used means that the overall results for each operator, have maximum margins of error of less than 1% for indicators of service accessibility and messages delivery rates, and less than 1.26 seconds for message delivery time, with a confidence level of 95%.
The test messages used had a size of 120 characters for SMS and 25 KB for MMS.
Consult:
- GSM/UMTS Mobile Communication Systems - Evaluation of the QoS of short messaging (SMS) and multimedia messaging (MMS) services https://www.anacom.pt/render.jsp?contentId=709930
