We’re continuing our research on the Building trust through audit and certification project within the ODI’s R&D programme.
Since our last worknote, we have been carrying out expert interviews and desk research. We are now putting together a survey on how organisations demonstrate their trustworthiness and assess the trustworthiness of others, which we will send out to a range of organisations in the coming weeks.
This worknote lays out:
- Why we felt a survey on trust and trustworthiness was the right methodology for this stage in the project
- What the survey includes and the research that went into crafting the questions
- How the survey is going to help us push the research forward
If you would like to take our survey, you can find the link here. If there is a particular aspect of trust, data and certification that you believe we should explore further, please get in touch.
Why we’re putting together a survey
In speaking to organisations that use and share data, we found that ‘trust’ means very different things to different people. Different sectors have different definitions – eg trust in the health sector versus trust in engineering. Even people within different departments of an organisation will have different views of trust – eg the legal team compared to the software, sales or communications teams. And it can also differ depending on whether the organisation is a data contributor, data intermediary or user of data.
We therefore decided to survey a range of people and organisations in order to help us better understand how trust works across different sectors and contexts. We believe the results of the survey will help us identify where we can have the most impact in trialing tools, resources and standards aimed at improving trust between organisations around data.
What the survey includes
The survey has three main sections and is set up to gather information on three important areas:
- The context and background of the person taking the survey
- How that person (or their organisation) goes about demonstrating and assessing trustworthiness
- And to what extent they believe standards, certifications and audits help them demonstrate or assess trustworthiness.
The survey works in tandem with our expert interviews and desk research, as it gathers a more quantitative picture of trust and certification.
Demonstrating and assessing trustworthiness
One of the main sections of the survey centres on how organisations demonstrate their trustworthiness to other organisations when accessing, using or sharing data.
We’ve learned during this project that organisations are multifaceted, so demonstrating trustworthiness is multifaceted as well. In some cases or contexts it might be more important to demonstrate that your software systems are safe, whereas in others it might be more important to demonstrate that your commercial practices or ethical review procedures are sound.
We’ve put together a list of 15 different aspects of an organisation that they might feel the need to show are trustworthy. We ask respondents to rank the list in terms of priority for their organisation. The list is a work in progress and has been put together by drawing on our expert interviews, desk research and other research at the ODI. It currently includes things like:
- legal structure and compliance with relevant laws or regulations
- software and technical infrastructure
- commercial practices and how ‘value’ is created or shared
- ethical review procedures and efforts to minimise bias or harms
- datasets supplied by the organisation (eg their quality, safety, value)
- financial sustainability
From our research we know that it is equally important for organisations to be able to assess the trustworthiness of others. We therefore present the same list to respondents and ask them to rank the aspects of an organisation that they would need to assess when gauging whether that other organisation is trustworthy.
Third-party assessments and certifications
The other main section of the survey aims to gauge the perceived value of things like third-party assessments, audits, certifications or kitemarks in helping organisations demonstrate and assess trustworthiness.
In essence, the previous section dealt with ‘what’ about an organisation needs to be trustworthy. This section deals with ‘how’ an organisation might go about demonstrating its trustworthiness or assessing the trustworthiness of others.
To start, we ask people whether there are existing certifications in their sector, whether they are mandatory and whether they find them useful. This should help us identify where current certification schemes are adequate and where there are gaps in current offerings. However, we also know that organisations use a range of methods, approaches and mechanisms beyond certifications to assess trustworthiness, so we also ask people whether they use other methods to demonstrate and assess trustworthiness. This will help us add to a list we have been collecting of different methods for demonstrating and assessing trust.
In addition to certifications, audits and kitemarks, our list of nearly 20 methods includes:
- legal commitments – eg contracts, terms and conditions or licenses
- technical standards
- tests or exams – on individuals, organisations or products
- penalties or redress mechanisms
- performance monitoring or oversight
How the survey will help us progress our research
This survey will be useful in a number of ways.
It will help target our research in trialling new methods for improving trust between organisations. It will make it easier for us to find partners with specific problems related to trust, with whom we can create or improve trust mechanisms. It will also help us identify where some organisations or sectors may be using methods that could be trialled elsewhere. Also, it will help us catalogue the different aspects of an organisation’s trustworthiness (the ‘what’) as well as the methods that an organisation might use to demonstrate and assess trustworthiness (the ‘how’). This type of catalogue could then guide others in choosing the methods that will help increase trust in their sectors given their particular circumstances.
Next steps
In the coming phase, we intend to experiment with different methods for improving trust. These will be led by the needs of the organisations we talk to. It could include crafting new standards, creating training courses or developing canvases for self-led work around data and trust.