The Role of Evidence in Establishing Trust in Repositories

The Role of Evidence in Establishing Trust in Repositories

The types of evidence likely to be of value to an auditor will be drawn from a range of
sources: information services, finance, human resources, and many others. The methods for
selecting and evaluating the evidence need to be regularized. For example, presence or
absence of a particular class of evidence is not necessarily a sufficient metric. Here we only
seek to highlight the relevant questions and concerns, offer a series of common sense
solutions, and prompt further exploration; we do not aspire to examine the issue as
comprehensively as it needs to be. So we have not suggested methods for evaluating the
documentary evidence. The most immediate barrier is establishing an understanding of the
kinds of documentary and testimonial evidence that an auditor would seek to accumulate in
considering a repository's case for certification. From this, a series of sub-questions follow:

• In what circumstances can quantitative metrics be established for assessing whether
individual criteria have been adequately satisfied?

• How might the qualitative merits of evidence be assessed consistently by different
auditors?

• What organisations and types of skilled professionals should be responsible for
gathering information and conducting a dialogue with repository representatives?

• What document procurement powers should be conferred to auditors?

• What assurances must be given to institutions concerned about disclosing sensitive
information? (For example, will non-disclosure agreements be necessary?)

• What external benchmarking evidence might be available to auditors that could
contribute to their forming a view as to whether an institution is compliant and
certifiable?

The initial starting point though is the evidence itself; we accept the checklist format that has
become de rigueur, but propose that at all times evidence requirements ought to be detailed
inline alongside each certification criterion. Needless to say, the means of their satisfaction
will be determined in part by the character and services of the particular repository
undergoing audit. We favour a simple system of classification of the evidence, with
conformance information categorised as documentary evidence, observation of practice
evidence, or testimonial evidence. Here we would propose that observation can be much
more than a passive activity, it might include such proactive steps as sampling, scenario
sequencing, tests, and simulations.

5. Documentary Evidence

Some repository characteristics can be objectively assessed through the provision by the
repository of documentary evidence and its analysis by the auditors. Insights into technical
infrastructure, financial management, resource allocation, and user relationships can all be
gained from the existence and analysis of a range of documentary evidence. Numerous types
of documentation of value to the audit and certification process exist within repositories; for
some, their presence alone will be encouraging, and in other cases their content will require
scrutiny if its role in fostering organisational compliance is to be assessed. To promote an
improved understanding of the kinds of documentation that might be used to support audit
and certification we suggest that the following be considered as an initial list:

• Repository Mission Statement: This is the statement of the repository's mission and,
if the repository is part of a larger organisation, its 'spacing' within the parent
organisation. This provides auditors with evidence of institutional commitment to the
long-term retention and management of digital information on behalf of depositors.

• Example Deposit Agreements: Examination of these agreements would enable
auditors to assess the relationship between depositor and repository, the responsibilities
of both parties, the level of service expected, and the legal rights and obligations of the

<<   <   2   3   4   5   6   7   8   >   >>