Building trust through audit and certification

As part of the ODI’s R&D programme we continued our research into building trust and how organisations can demonstrate trust and trustworthiness when accessing, using or sharing data.

Over recent months we used this research as the basis for building prototype tools, guides, workshops and certified courses to help organisations demonstrate, build, and increase trust and trustworthiness, and train employees in data ethics.

Over the last months, and throughout our R&D project on Building trust through audit and certification, we’ve been exploring what types of mechanisms are likely to have the most impact in improving trust between organisations around data. This project was built on previous work on how to design trustworthy data institutions and sustainable data institutions, and had the aim of helping the data ecosystem operate more effectively while reducing the risk of causing harm.

Based on findings from the research and alpha phases, we've developed tools and resources to build, assess and demonstrate trust and trustworthiness between organisations when sharing, accessing and using data.

The problem we set out to address

It’s very important for data stewards to be considered trustworthy and to be able to trust each other when it comes to sharing or accessing data. Both the Open Data Institute (ODI) commissioned Frontier Economic report and YouGov survey show how important it is to have trustworthy data ecosystems, but it can be difficult for organisations to assess and demonstrate their own trustworthiness and understand how to define the specific mechanisms to improve it.

From the beginning of the project we wondered whether the role of trusted third parties, auditors and certifying bodies could be applied to data ecosystems to improve trust, as it does in other domains. We’ve been exploring how certification and auditing could tackle the problem of assessing trustworthiness, but also asking: what other types of tools and resources could the ODI develop to support creating trustworthy data ecosystems?

How we chose to try to address that problem

During the research phase of the project we’ve been trying to understand how trust relationships work in practice, what type of assessments support them, and how certifications are perceived to actively establish trustworthiness. Activities such as desk research, expert interviews and a survey brought us insights into how a person (or organisation) goes about demonstrating and assessing trustworthiness within their data ecosystem. The findings from the different research streams published in our interim report showed us that trust and trustworthiness are highly context dependent – as well as the appropriate methods to show and assess it – and that third-party assessments are useful, but only to a degree, and within certain contexts.

Part of the alpha phase of the project involved looking at all the evidence we gathered during the research phase and defining in more depth the main user needs and the problems they faced when building trust. The key user needs were:

Guidance and support when assessing – internally and externally – building and demonstrating trustworthiness.

Upskilling for professionals and certification or qualifications for staff in data ethics.

Following these insights, during the beta phase of the project we’ve developed different prototype tools aimed at addressing the different user needs and journeys identified within this project. These tools are the Trustworthy Data Stewardship Guidebook – which includes a set of prototypes of tools – and a training prototype.

The Trustworthy Data Stewardship Guidebook

The Trustworthy Data Stewardship Guidebook is aimed at people and organisations looking to be more trustworthy when accessing, using and sharing data. The main goals of this guidebook are:

  • Helping organisations identify their trust-related challenges and goals
  • Providing step-by-step guidance on how to assess, build and demonstrate trust and trustworthiness, in order to achieve those goals; and
  • Pointing to relevant tools and resources along the way.

Part of the beta phase of the project included testing and iterating the guidebook. Some of the early feedback we received on the guidebook was very positive in terms of its purpose. A Senior Policy Adviser from a large organisation working with different businesses pointed out the coverage of the three main steps to trustworthiness – ie assess, build and demonstrate – meets the needs of various audiences. The feedback we gathered from people that showed interest in this project also helped us discover the potential uses of the guidebook – people could work through the guidebook on their own, in cooperation with other organisations, or use it to educate or train others.

Tools in the Trustworthy Data Stewardship Guidebook

The variety of tools and resources also received positive reviews. People who tested them noted how useful they were in helping organisations or people get the right information relevant to whatever stage they were in, and to the level of maturity of their data ecosystem. For the beta phase of the project we’ve prioritised the development of some of the tools based on time-effort criteria in order to be able to further test and iterate them.

Some of the tools and prototypes included within the guidebook were developed within this project, while some are existing tools and resources from the ODI and other external organisations. The tools and resources developed for this project include:

Finally, we have developed prototypes of two workshops aimed at organisations interested in working through the activities and tools within the guidebook in a facilitated workshop setting. One workshop helps organisations perform an internal assessment of their trustworthiness in their role as a data steward. Another enables two or more organisations engaged in a shared venture to assess each other’s trustworthiness and agree next steps to become more trustworthy within their partnership or collaboration. We’ll continue working on the content of the workshops, addressing the user feedback we’ve received in terms of the amount of themes the workshop might be able to cover.

Some of the tools and resources developed for this guidebook, including the workshops, are still in the beta phase. We would like to see them tested further to make them better. Please contact us if you are interested in helping us do this.

Training prototype: Certifying Data Ethics Facilitators

Data ethics is of paramount importance to ensure the trusted collection, use and sharing of data. It is especially relevant when data activities have the potential to impact people and society, directly or indirectly. The ODI recognises the importance of data ethics and has developed tools such as The Data Ethics Canvas and Consequence Scanning Agile Kit to provide tangible and replicable ways to ask important questions about tech products and data, and suggest courses of action to ensure ethical data practices.

As part of this programme of work, the ODI is prototyping and trialing a certified Data Ethics Facilitators Programme. This assessed programme offers organisations the opportunity to certify staff in data ethics, equipping them with the knowledge and skills to help organisations minimise potential harms that come from data collection, use and sharing.

This programme builds upon our extensive knowledge of operationalising data ethics and looks to grow a strong network of ODI certified individuals who can help others be more ethical with data.

The prototype course will be run first with a commercial client who has agreed to cover the cost to deliver the programme before we make the course available more widely. An initial register interest page shared only with our training mailing list has already generated over 250 expressions of interest in this programme, and we continue to engage with the community to seek feedback on the programme.

What we didn’t pursue (aka the ‘no-go-types’)

  • An intake questionnaire that organisations can use at the start of their journey to help focus on their specific needs, challenges and goals, spotlight important elements of trustworthy data stewardship for them to focus on, chart a path through the playbook and then prioritise next steps.
  • An internal assessment questionnaire that organisations can send to different people/departments within their organisation to understand the steps each department is taking to be trustworthy and how each department perceives ‘trust and trustworthiness’.
  • An external assessment questionnaire that organisations can send out to their external stakeholders to understand how their trustworthiness is perceived externally. If the process identifies areas of concern or in need of improvement, the guide could recommend consulting the ‘build stage’ for suggested next steps.
  • A trustworthiness assessment guide to help organisations assess the trustworthiness of potential partners – either indirectly via materials published by that organisation, or directly via surveys or interviews.
  • An interview guide/template with suggested questions and discussion points that organisations can use when interviewing their stakeholders about trust.
  • ‘Yelp for data stewards’ a potential (albeit risky) prototype that we uncovered during our research and that the ODI or other organisations could pick up if so desired. When deciding whether to work with an organisation, many people we spoke to said they often relied on the reputation of that organisation and recommendations from someone else they know who had worked with that organisation. In a ‘Yelp for data stewards’, organisations could rate those they’ve interacted with in the past by giving them a general rating on different elements of trustworthy data stewardship. Other organisations who are contemplating interacting with that organisation could then look up the organisations rating to see whether they are generally trustworthy/trusted.
  • Templates and guides for an agreed pact/bond which captures the steps each organisation has agreed to take in order to improve their trustworthiness and lists the checks or audits that need to be put in place.
  • A communication template to help organisations gather the materials and insights produced during the process of working through the guidebook, in order to help them communicate their trustworthiness to external parties.
  • A certification or kitemark that organisations can receive after submitting evidence of having worked through some or all of the playbook. The certification could be of practice or process.
  • A ‘follow up’ questionnaire to send around internally to staff in different departments, or externally to different stakeholders and partners, to identify whether the steps you are currently taking to be trustworthy are still fit for purpose and assess whether any trust relationships have changed, agreements need to be updated or stages of the playbook need to be repeated.

What’s next for the guidebook and these prototypes

This work and the frameworks defined – eg The 10 elements of trustworthy data stewardship prototype – will be available externally to anyone interested in assessing, building or demonstrating trust or trustworthiness of their data or data practices. The ODI will build on this research as part of its work exploring the role of data assurance products and services to improve data governance and trust across a data ecosystem. If you are interested in working with us to explore this topic, or test prototype tools, please get in touch.