Data access and use

Recommendation 19

Review the costs and benefits of increasing access to and improving the use of data, taking into account community concerns about appropriate privacy protections.


Government should commission the Productivity Commission (PC) to commence, by the end of 2015, an inquiry into the costs and benefits of increasing access to and improving the use of data, subject to privacy considerations. Increasing access to data could enhance consumer outcomes, better inform decision making, and facilitate greater efficiency and innovation in the financial system and the broader economy but could also involve privacy risks.

The PC should consider potential mechanisms to:

  • Increase private sector, academic and community access to public sector data.
  • Encourage the use of appropriately de-identified public data to inform government, private sector and consumer decision making.
  • Improve individuals’ access to public and private sector data about themselves, such as by defining relevant data, standardising its collection and aggregation in datasets, and formalising access entitlements and arrangements.
  • Increase access to private sector data while maintaining private sector incentives to collect data, such as through data-sharing arrangements, cost-recovery arrangements and user charges.
  • Further standardise the collection and release of public and private sector data and product information, so datasets can be created and combined more effectively.
  • Enhance and maintain individuals’ confidence and trust in the way data is used.

The PC should report to the Treasurer on how better use of data can improve user outcomes, including potential amendments to the Privacy Act 1988 (Privacy Act) and other legislation.


  • Improve the quality of business and consumer decision making, public policy development and implementation, and research into how the financial system and broader economy function.
  • Better enable innovative business models that rely on data, where they improve user outcomes and overall system efficiency.
  • Increase the utility of public institutions that hold data.


Problem the recommendation seeks to address

Data is becoming increasingly integral to how the financial system and broader economy function. By 2020, the amount of data held globally is predicted to be 44 times larger than it was in 2009.82 Ever-expanding computational power and smarter algorithms are enabling this data to be used more effectively. This is helping businesses better understand and meet the needs of consumers, improve product offerings, manage risks and reduce costs.

National governments globally are encouraging these trends through open data policies.83 Some private sector organisations, non-government organisations and academic institutions have also incorporated open data policies or are actively contributing to the amount of publicly available data.84 However, to date, and especially in Australia, there has been very little debate around whether Government policies could increase access to private sector data to boost innovation and competition.

The increasing use of data is not without risk. The 2014 update of the Australian Privacy Principles made significant progress in defining how individuals can access and control their personal data.85 Globally, there is growing debate about the use of data and how societies should balance privacy and efficiency considerations.86

The scenario set out in Box 11: A cameo on the potential benefits of enhanced data usage, based on existing and/or emerging data-driven financial products and services from around the world, highlights the power of data to drive competition and improve user outcomes.

Box 11: A cameo on the potential benefits of enhanced data usage

Alternative text:Sarinda owns a food truck business, Happy Jalapeños, which he has operated for 18 months. He uses payments service provider, SnappyPay, for a mobile point-of-sale solution, which enables him to track his sales, take pre-orders online and accept card payments with an attachment on his smart phone. As part of its service, SnappyPay collects Happy Jalapeños’ payments data.The following example shows how Sarinda’s business operates in the current Australian context, and then how emerging data-driven financial products and services could enhance Sarinda’s business.Example 1: Improving access to financeSarinda wishes to expand his business by buying a second truck.Current situation: Sarinda has limited personal savings and collateral. The traditional banking sector is reluctant to lend to Sarinda without collateral and places limited weight on alternative means of assessing creditworthiness such as cash flow data. Sarinda is also unable to easily compare credit products due to a lack of standardised product data.Potential future situation: Based on an analysis of 18 months of Happy Jalapeños’ cash flow data, SnappyPay identifies Sarinda as an attractive loan candidate and offers him a customised loan to match his needs. Sarinda compares SnappyPay’s offer with competing products by inputting his loan needs and standardised cash flow data into a comparator site.By accessing credit providers’ standardised product data, the site matches Sarinda’s needs with the best available products from various lenders, including banks, peer-to-peer lenders and other providers. The comparator site is also able to identify crowd-sourced equity options. Sarinda is able to select the financing option that best suits his needs.Example 2: Facilitating growth and competitionSarinda wishes to better understand his business and competitive environment.Current situation: While Sarinda knows he is not fully tapping the potential of his business, he is unable to afford the data analytics capabilities his larger competitors possess. He also has limited access to competitor data due to privacy safeguards and a lack of relevant datasets. Instead, he relies on his business intuition and observes trends while operating his food truck.Potential future situation: Through SnappyPay, Sarinda is easily able to review Happy Jalapeños’ sales and cash flow data. He enters a reciprocal data-sharing agreement under which Happy Jalapeños’ data is de-identified and aggregated into industry-wide data he can then access for competitor analysis. He also consents to sharing his truck’s real-time geolocation data in exchange for the geolocation data of other nearby trucks.With analytics support, Sarinda can identify peak and off-peak sales periods throughout each day, and across the year, to better target his promotional offers. He is able to adjust his food and beverage offerings to match trending and popular products. He is also able to determine the best location for his truck throughout each day.

View image enlarged

Access to public sector information

As the PC has previously noted, “... unlike many other countries, Australia makes relatively little use of its public [sector] data resources, even though the initial costs of making data available would be low relative to the future flow of benefits”.87 The National Commission of Audit noted that Australian governments have only released around 3,200 datasets, compared with 10,000 datasets in the United Kingdom and 200,000 datasets in the United States.88 The PC also observed that, “... academics, researchers, data custodian agencies, consumers and some Ministers are eager to harness the evidentiary power of administrative data, but this enthusiasm generally is not matched by policy departments”.89

This reluctance could be due to the costs of making data available and usable, and risks around quality assurance. However, these considerations should be weighed against the fact that decisions not to release data prevent public and private sector decisions from being better informed. The Inquiry is mindful that financial regulators release significant amounts of data, but sees scope to release more, including both aggregated data and de-identified datasets of personal information.

Access to personal information

The Australian Privacy Principles give individuals the right to access personal information about themselves; however, a number of impediments are still preventing consumers from being able to use their data effectively:

  1. Little guidance is available on how personal information should be provided, including delivery method, timelines and standards for representing data.
  2. In most cases, consumers are unable to authorise trusted third parties to access their personal information directly from their service provider. This reduces the ability of competitors to offer consumers better value or tailored services, or develop advice services to better inform consumer decision making.
  3. Confusion exists over what constitutes personal information, which may limit individuals’ access to data.90

Access to private sector data

In many circumstances, private sector organisations have strong incentives to restrict access to the data they hold, as it serves as a competitive advantage. However, this may create inefficiencies where the benefits to the economy of releasing data are greater than the benefits to individual institutions of restricting access. In this sense, publicly accessible data can suffer from the ‘common good’ problem: it is undersupplied because the gains to society are difficult to monetise.

The Inquiry does not suggest that all, or even most, private sector data should be released publicly. In many cases, private returns are necessary to justify investments in developing datasets. The challenge is to maintain commercial incentives for developing datasets, while facilitating the release of data where this improves efficiency.

Standards for datasets and product information

Standards for collecting and representing data can improve the use of data and enhance its network qualities. They can enable datasets to be combined and aid algorithms that mine datasets to find meaning. Although progress has been made in standardising data collection, particularly through programs such as Standard Business Reporting, much work still needs to be done. In some cases, poor coordination prevents standards from being developed or widely used. Another contributing factor is the cost involved in adjusting how data is collected and represented to comply with standards.

Confidence and trust

For the potential of data to be fully realised, individuals must have confidence and trust in how their personal information is stored and used. Without confidence and trust, individuals would be unwilling to volunteer their personal information and may avoid using services that develop datasets based on observations of individuals’ behaviour.91 Some reports suggest that individuals do not understand how their personal information is currently collected and used. When they do find out, they can lose trust and may stop using services that collect their personal information.92


A number of submissions support data sharing within privacy limitations and increasing the use of standards, including those from Choice, the Office of the Australian Information Commissioner, the Association of Superannuation Funds of Australia and the ABA. Others note the benefits to policy makers, regulators and researchers of having access to high-quality data for understanding how the financial system functions and improving policy decision making.93 Although issues regarding accessing and using data are important for the financial system, they also have much broader implications.

The outcomes of the proposed PC inquiry should improve the way Australia’s financial system data ecosystem functions by increasing data sharing and the utility of datasets. As the Privacy Act has only recently been updated after an extensive review, any PC recommendations to amend the Privacy Act could be considered in a broader post-implementation review of the Privacy Act. This would ensure another forum to explore potential trade-offs between efficiency and protecting individuals’ privacy. Both processes would foster much-needed public debate on these complex issues, which Government, business and society will need to grapple with for some time to come.

Option considered

Recommended: The PC should consider the costs and benefits to the financial system and broader economy of mechanisms to: increase access to public sector data; individuals’ access to their personal information; access to private sector data; the use of standards for datasets and product information; and confidence and trust in the use of data. The PC should recommend where amendments to the Privacy Act and other legislation could enable better use of data and improve public welfare.

Option costs and benefits

The costs and benefits of potential options the PC inquiry could consider are discussed below. The cost of the actual PC inquiry is likely to be minimal, considering the potential benefits of improving the use of data.

Access to public sector data

Broadening access to public sector data has significant benefits. One report suggests this could add around $16 billion per annum to the Australian economy, as a conservative estimate.94 The Inquiry strongly supports the National Commission of Audit’s recommendations to extend and accelerate publishing de-identified administrative data and stocktake suitable datasets for public release.95

However, the main challenge will be overcoming the disincentives to release data. The PC may wish to consider price signals to address this, such as a program that provides additional funding to agencies for the first year they release new datasets or charging agencies for each year they do not release data.

Access to personal information

Some submissions note that individuals would benefit from being able to access their personal information more readily and in more standardised formats, such as in machine-readable format, and from being able to share their information with trusted third parties more seamlessly.96 Consumers could better understand their circumstances and improve their decision making, as well as identify better-value offerings from other service providers.

Consumers are increasingly using online resources to inform their financial decision making — up from 25 per cent in 2005 to 40 per cent in 2011,97 and the Inquiry believes this could grow significantly. When considering options in the Australian context, the PC may wish to consider regulatory models such as ‘midata’ in the United Kingdom98 and Smart Disclosure in the United States.99 It may also be possible to create standard protocols to enable consumers to allow trusted third parties to access some of their personal information. For example, the PC could consider whether introducing such standards could facilitate opportunities for ‘data banks’ to store personal data that individuals volunteer. Individuals could then choose to provide access to parts of this information to trusted third parties.

Access to private sector data

Increasing access to private sector data would have large efficiency benefits across the economy. It would support innovation and competition, as new entrants and smaller businesses with smaller datasets could better compete with larger incumbents. Online comparator sites and other advice services could better serve their clients, and consumers could make more informed choices. Industry and government should continue to progress initiatives for sharing data with consent, and initiatives allowing greater access to commercial data where this would improve user outcomes. In particular with greater flexibility of disclosure mechanisms, issuers and firms should make available prominently and publicly at no cost, mandated disclosure in a central place. For example, all PDSs of an issuer could be made available on their website.

The PC should consider the conditions under which the release or sharing of private data would create net benefits to the economy, and not reduce incentives for businesses to collect the data in the future. In many cases, potential disincentives could be addressed by compensating businesses that share their data. It may be efficient to charge users of data to fund this compensation, such as through an access regime. In other cases, businesses could be compensated through greater access to others’ data.100 Alternatively, if the data is viewed as a ‘pure’ public good, Government may be justified in compensating providers and releasing the data for free.

Standards for datasets and product information

Standardising datasets can improve data functionality by improving ease of use and allowing datasets to be combined. This can assist the process of turning data into meaningful information. The Standard Business Reporting initiative has invested considerable resources to standardise financial datasets and currently operates on a voluntary basis.101 The PC may wish to consider whether additional actions are necessary to unlock the potential of data through standardisation — both for public and private sector datasets.

Standards could also cover how financial product information is reported, so third parties could use automated processes to create market-wide datasets of available products.102 The Inquiry believes new advice and comparison services would be developed if product information was better standardised, supporting consumers in making more informed choices and enhancing competition.

Although the Inquiry’s first preference would be for private sector models to develop, some stakeholders are concerned that comparison services can face conflicted incentives, which may lead them to provide poor advice or misleading comparisons.103 The PC may wish to consider these issues, particularly in circumstances where Government provision is the most effective option.104 Government provision can avoid conflicted incentives, but it can come at a cost to taxpayers and involve moral hazard. The Inquiry encourages the PC to consider these issues further.

Confidence and trust

Increasing confidence and trust in the way data is collected and used can increase the amount of data institutions are able to collect and use. As a result, some restrictions on the use and collection of data can actually increase the amount of data available. The Privacy Act is a vital piece of infrastructure in Australia’s data ecosystem.

The PC may wish to consider other means of enhancing confidence and trust and reduce the risk of it being eroded. These outcomes can be achieved by ensuring individuals benefit from sharing their personal information and have visibility and control over how their information is used. Specific options could include standardising processes for correcting data errors and limiting how widely data can be shared. Increasing transparency through data breach reporting and greater disclosure of how data is used and collected could also assist, particularly in building sustainable levels of confidence and trust over time. The PC may also wish to consider how to best balance efficiency and security in relation to controls on international data transfers.


The PC is best placed to consider the costs and benefits to consumers, the financial system and the broader economy of increased access to and improved use of data, and to explore options the Inquiry has not considered.

82 Gantz, J and Reinsel, D 2010, The digital universe decade — are you ready? International Data Corporation iView, Framingham, Massachusetts, page 2.

83 Davies, T 2014, Open data policies and practice: an international comparison, Paper for European Consortium for Political Research Panel P356 – The Impacts of Open Data, page 1.

84 Herzberg, B 2014, The Next Frontier for Open Data: An Open Private Sector, World Bank, viewed 22 October 2014.

85 Privacy Amendment (Enhancing Privacy Protection) Act 2012, which amends the Privacy Act 1988.

86 For example, see Acquisti, A 2010, The Economics of Personal Data and the Economics of Privacy, Organisation for Economic Co-operation and Development, Paris.

87 Productivity Commission 2013, Annual Report 2012–13, Chapter 1: Using administrative data to achieve better policy outcomes, Commonwealth of Australia, Canberra, page 1.

88 National Commission of Audit 2014, Towards Responsible Government: The Report of the National Commission of Audit, Phase One, Chapter 10.5: Data, Commonwealth of Australia, Canberra, page 235.

89 Productivity Commission 2013, Annual Report 2012–13, Chapter 1: Using administrative data to achieve better policy outcomes, Commonwealth of Australia, Canberra, page 1.

90 For example, see Grubb, B 2014, ‘Spies can access my metadata, so why can’t I? My 15-month legal battle with Telstra’, Sydney Morning Herald, 10 October, viewed 23 October 2014.

91 For example, payment services that collect data on individuals’ purchasing decisions.

92 For example, see World Economic Forum and Bain and Company 2011, Personal Data: The Emergence of a New Asset Class, World Economic Forum, page 6.

93 For example, see Centre for International Finance and Regulation 2014, Second round submission to the Financial System Inquiry, pages 20–21.

94 Lateral Economics 2014, Open for Business: How open data can help achieve the G20 growth target, Lateral Economics, Melbourne, page 10.

95 National Commission of Audit 2014, Towards Responsible Government: The Report of the National Commission of Audit, Phase One, Chapter 10.5: Data, Recommendation 61, Commonwealth of Australia, Canberra, page 236.

96 Refer, for example, to Choice 2014, Second round submission to the Financial System Inquiry, pages 26–31.

97 ANZ 2011, Adult Financial Literacy in Australia: Full report of the results from the 2011 ANZ Survey, ANZ, Melbourne, page 102.

98 United Kingdom Department for Business, Innovation and Skills 2011, The midata vision of consumer empowerment, media release, 3 November, London, viewed 18 November 2014.

99 United States Government 2014, An introduction to smart disclosure,, Washington DC, viewed 11 November 2014.

100 For example, financial institutions share consumer credit data through credit bureaus.

101 Australian Business Register 2014, Second round submission to the Financial System Inquiry, page 2.

102 For example, a 2010 Choice survey found that credit card providers use at least 10 different billing methods, making it difficult for consumers to compare information. Choice 2014, Second round submission to the Financial System Inquiry, page 27.

103 Choice 2014, Second round submission to the Financial System Inquiry, supplementary document, page 3.

104 There are already examples of state governments providing comparator services for electricity and gas (see and, and the Federal Government will establish a comparator service for insurance in North Queensland by March 2015. Cormann, M (Minister for Finance and Acting Assistant Treasurer) 2014, Initiatives to help address insurance affordability for North Queensland, media release, 23 October, Canberra, viewed 23 October 2014.