ANACOM determines alterations to the system used to measure CTT quality indicators

Through Grant Thornton, ANACOM undertook auditing of the quality of service indicators reported by CTT with respect to 2016 and 2017. ANACOM concluded that, in 2016, independence of the measuring system could not be guaranteed, since CTT did not use an independent external entity to measure the indicator between January and September. From October to December 2016 and in 2017, measurement was performed by PwC, an independent external entity contracted by CTT, and auditing led to the conclusion that the measurement procedures implemented run the risk of not properly reflecting the quality of the service offered to consumers.

The audit concluded that calculation of the values of the Quality of Service Indicators was performed according to the methodology defined. Errors were recorded only in the calculation of two Quality of Service Indicator values, both in 2017: non-priority postage not delivered within 15 working days and priority mail not delivered within 10 working days.

According to the audit, the methodology applied in 2017 in the formula for calculating these indicators was incorrect, but the resulting deviations were not significant and did not have an impact on the accomplishment of the established target and minimum values.

Grant Thornton made a set of recommendations aimed at correcting some aspects of the measurement system.

In view of the audit findings, ANACOM notes that the procedures used to measure the Quality of Service Indicators does not guarantee anonymity of the panellists. As a result, the test postage used in the sample to make the measurements might receive specific treatment which is more favourable than the treatment given to actual postage.

Furthermore, in the case of waiting time measurement, since the person contracted to carry out the measurement is easily recognisable, this may lead to CTT staff behaving differently when the measurement is being performed as compared to normal customer service.

ANACOM therefore found that the procedures implemented may not ensure that the values obtained for the quality of service indicators properly correspond to the quality of the service that is actually provided. As such, ANACOM requires a number of changes to be made in order to ensure accuracy and comparability:

• the procedures for sending the test objects to the panellists by the measuring entity must be carried out without CTT's involvement.
• delivery of parcels must take place on different days of the week, not on a single predetermined day;
• the test objects must have an appearance that is similar to actual mail items;
• the use of test objects with transponders is not permitted.

As from six months following notification of the final decision to CTT, panellists and introduction and reception points that are or may have been part of the panel between October 2016 and December 2018 may not form part of the quality of service indicator measurement panel and may not be reincorporated into the panel during the next three years.

ANACOM also determines that no panellist, postage point or reception point may be part of the panel for more than 4 consecutive years.

With regard to Quality of Service Indicators on waiting times in postal establishments, a mystery shopper is to be used. The same mystery shopper may:

• not collect information on more than two occasions in the same postal establishment within a year and;
• not collect information at the same establishment until at least 4 months have elapsed since the last occasion.

The measures to be determined by ANACOM, which are subject to a prior hearing of CTT and public consultation (each having a period of 20 working days¹), will be applicable as from the measurement undertaken with respect to 2019.

The audit of quality of service indicators was organised by ANACOM under article 13, paragraph 5 of Lei Postal (Postal Law). As required by the law, today, ANACOM is also publishing the results of the audit on its website.

ANACOM also audited the indicators on complaints and requests for information, within the scope of the universal postal service, as measured by CTT under Article 41, paragraph 5 of the same law. Following the audit, the results of which are also published on ANACOM's website, ANACOM recommends that CTT adopt measures aimed at reducing subjectivity in classification of the reasons given as motivation for complaints and others aimed at reducing the occurrence of errors in the registration of information in the respective information system.

Consult:

• Levels of quality of service of CTT in 2016 and 2017 (audit) https://www.anacom.pt/render.jsp?contentId=1461558

• Consultation on audits of quality levels of the universal postal service (2016 and 2017) https://www.anacom.pt/render.jsp?contentId=1461561