Category: abt associates

  • The federal government was warned early this year that its $16 million COVIDSafe contact tracing app was ineffective in “very high-risk settings” and was wasting the time of state health authorities in a report it attempted to keep secret.

    Health Minister Greg Hunt tabled a report on the effectiveness of COVIDSafe, which uses a smartphone’s bluetooth technology to log close contacts between users and sends this information to a national database if a user tests positive for COVID-19, last month, well after he was required to under legislation.

    The COVIDSafe app. Credit: Daria Nipot / Shutterstock.com

    But the report tabled by the government differs greatly from the full report provided by independent consultancy Abt Associates, which has now been revealed following a freedom of information battle by The Canberra Times.

    The government had earlier released a version of the full report nearly entirely redacted, before tabling a summary version in Parliament.

    The version put forward by the government is stripped of most of the critical information on the app, including that it was unlikely to provide any benefit in high-risk settings such as restaurants and supermarkets, where significant transmission has been seen this year, and that it is often taking up hours of contact tracers’ precious time with little reward.

    Despite being billed as akin to putting on sunscreen and a key part of the easing of restrictions following the first national lockdown early last year, the app has only unearthed 17 unique close contacts since it was launched, all in New South Wales. It has not found any new close contacts this year, despite a number of outbreaks around the country.

    The report tabled by the government included a foreword by Mr Hunt, saying COVIDSafe is an “important element” of Australia’s contact tracing efforts.

    But the full report provided by Abt Associates for $250,000 paints a very different picture.

    “The app has identified less than 2 per cent of the total pool of close contacts identified in NSW and no new contacts were identified by COVIDSafe in Victoria or Queensland between March and November 2020. COVIDSafe has technology constraints, complications with manufacturer phone settings, and there are limitations to its use in high-risk settings,” the report said.

    “Overall, the state contact tracers interviewed for this report have suggested it has not been a highly effective complementary tool for their state contact tracing systems.”

    The report, which reveals the government has spent about $16 million on the development and promotion of the app, found that due to various constraints, COVIDSafe cannot be effective in high-risk environments such as bars, restaurants and supermarkets.

    These are the type of locations that have seen significant transmissions across the third wave in New South Wales and Victoria, and are a “key priority gap in standard contact tracing that would need the support of digital tracing tools”.

    “However, in its current form, COVIDSafe is not able to be used effectively for these purposes due to the identified constraints discussed in this report,” the report said.

    The “cumbersome” process behind accessing data from the national database and cross-checking this against case interviews, which has to be done manually, means COVIDSafe “adds to [contact tracers’] workload without an optimisation of benefits”, the report found.

    For some contact tracers, using COVIDSafe is adding up to two hours of extra work for each case, and often just providing false close contacts.

    Out of 205 potential individuals flagged by COVIDSafe in NSW, more than 60 per cent were found by contact tracers to be false close contacts, while 30 per cent had already been identified. Only 17 of these individuals were unique new close contacts found by COVIDSafe, accounting for 0.07 per cent of all close contacts in NSW.

    “In the current environment, where public health units must be ready to rapidly surge their capacity and where it is widely acknowledged that experienced contact tracers are at the core of a good system and cannot be easily outsourced or secured through short-term training, inefficient use of contact tracers’ time is a high cost for the system to bear,” the report found.

    “Since experienced contact tracers are currently in high demand and face scarce supply, any additional task required from them that does not deliver substantial benefits carries considerable opportunity costs.

    “Overall, the utilisation of COVIDsafe has resulted in high transaction costs for state contact tracing teams and produced few benefits and efficiencies to the existing contact tracing work-flow.”

    The report found that COVIDSafe often flagged false close contacts, such as neighbours who were close to each other but separated by a wall, or those with other physical separations. The time spent validating these potential close contacts was not worth the benefit the app provided, the consultancy found.

    The report, which was handed to the federal government in March, urged for major changes to be made to the app to address some of the identified issues.

    “We believe it is crucial for all state and Commonwealth stakeholders to explore options for enhancing COVIDSafe. It is particularly important to the seven million Australians that have downloaded the app, to acknowledge the performance barriers that are limiting the effectiveness and efficiency of COVIDSafe,” Abt Associates said in the report.

    No significant changes have been made to COVIDSafe this year, despite this recommendation.

    The version of the report tabled by the government in Parliament did not include most of the critical information in the full report, including on COVIDSafe’s inability to assist in high-risk settings, and the time burden it is placing on contact tracers.

    The development of COVIDSafe was led by the Digital Transformation Agency, which brought in a number of private consultancies and companies including Boston Consulting Group and Delv on contracts worth nearly $10 million. It costs around $60,000 per month to keep COVIDSafe running.

    The post Secret report slams COVIDSafe as ‘high cost, low benefit’ appeared first on InnovationAus.

    This post was originally published on InnovationAus.

  • The federal government has been forced to release a report on the effectiveness of its controversial contact tracing app COVIDSafe, but has removed all parts relating to this and left only basic information and positive comments.

    The report, released this week following a Freedom of Information request, has wholly redacted all of the parts relevant to its effectiveness and comes nearly a year after the government was required to release it.

    COVIDSafe uses a smartphone’s Bluetooth technology to log close contacts between users and then send these to a national database and state health authorities if a user tests positive for COVID-19. It was launched by the federal government with much fanfare in April last year.

    Accompanying legislation enshrining privacy protections around the app in law required the Department of Health to report on the operation and effectiveness of the app and the national database every six months.

    This report must be prepared “as soon as practicable after the end of each six-month period”, and must then be tabled in Parliament within 15 days of completion.

    C
    Credit: Daria Nipot / Shutterstock.com

    Despite it now being nearly 18 months since COVIDSafe launched, the government is still yet to produce any full report on the effectiveness of the app.

    The Department of Health paid global research firm Abt Associates nearly $250,000 last year to complete the first of these six-monthly reports. The report covering April to October last year was handed to the government in March this year, but is still yet to be publicly released.

    Following a Freedom of Information request by The Canberra Times, the government has this week released a highly redacted version of this report, with all analysis of the effectiveness and operation of the act removed.

    The only elements left visible in the report relate to the basics of how COVIDSafe works and its launch, and positive elements, such as the quick timeframe it was developed.

    The FOI release does reveal that the COVIDSafe report was handed to the Department of Health on 30 March this year. Despite being required to table it in Parliament within 15 sitting days of this happening, there have been 17 sitting days since then and the report still hasn’t been released — a contravention of the government’s legislative requirements.

    Across 11 weeks from September last year, Abt Associates and technical partner Bdna conducted interviews with state and territory health authorities, along with staff from the Department of Health and the Digital Transformation Agency.

    “Our evaluation used a mix-methods and multidisciplinary approach blending perspectives from public health evaluation and technology reviews to examine available quantitative and qualitative evidence on the appropriateness, effectiveness and efficiency of the COVIDSafe app and the National COVIDSafe Database,” the report said.

    “A high level technology review was undertaken according to the request for quotation which excluded COVIDSafe technological alternatives and recommendations for future innovations.”

    Abt Associates was paid $233,668 to produce this report.

    All elements of the report relating to the actual evaluation and analysis of the effectiveness of COVIDSafe were redacted by the government, mostly under s47C. This exemption relates to information involving a “deliberative matter”, such as an opinion, advice or recommendation that has been obtained, prepared or recorded.

    This exemption is not meant to apply to reports of “scientific or technical experts…including reports expressing the opinions of such experts on scientific or technical matters”.

    The government did however leave in parts of the report saying it was a “commendable achievement” that COVIDSafe was developed in a short space of time, and another saying that the right model was chosen for the app.

    “As our technology review indicates, based on the parameters of knowledge and capabilities at the time of app launch, it is believed that the COVIDSafe app was the correct tool to employ,” the report said.

    The report said that the key measure of success of the app is its “ability to identify individuals who meet the criteria of close contact”.

    COVIDSafe has not identified any of these people this year, despite significant outbreaks in New South Wales, Victoria and other areas of the country.

    The app is not currently being used at all by NSW health authorities, the only state where it has identified unique close contacts previously.

    COVIDSafe has found 17 new close contacts not identified by manual contact tracers in total, while a further 544 people were identified by health authorities after the app found a new exposure time at a NSW venue.

    The government paid private contractors nearly $10 million for work on COVIDSafe, while it is still costing $60,000 per month to continue running.

    The post Govt releases highly redacted COVIDSafe report appeared first on InnovationAus.

    This post was originally published on InnovationAus.