The Role of Evidence in Establishing Trust in Repositories
http://www.dlib.org/dlib/july06/ross/07ross.html
undertaken within the system, including the application of security and functionality
upgrades and the repair or replacement of corrupt or lost data objects. This enables the
auditors to evaluate the effectiveness of the team and to make an assessment of the
quality of service that the repository can provide.
• Results of Other Audits: The results of other audits are of value, such as Information
Security, Health and Safety, and even evidence of areas in which the repository was
acknowledged by other independent auditing regimes and particular audits to have
been successfully conducting its business.
• Other Documentation Records: During the process of managing digital materials,
repositories will produce other documentary 'fingerprints'. These will need to be
identified on a repository by repository basis.
Even the processes by which these documents are managed will provide auditors with
valuable insights into the running of the repository. It will be useful to know how decisions
are taken to draft them, how their change is reviewed, how new versions are approved, and
how staff are made aware of changes to procedures and policies.
Associated with many forms of physical documentary evidence will be fears over
confidentiality and privacy - every organisation has documentation that it regards as
sensitive, whether for financial or business planning reasons, or those more attributable to
relationships with depositors. Auditing teams need to put appropriate non-disclosure
agreements in place to secure the confidence of repository representatives that privacy and
confidentiality agreements will not be breached by the audit process. The processes and
practices of conducting audits are, moreover, governed by a range of professional practices.
These types of documentation must be subjected to consistent and unbiased evaluation. The
methods for evaluating documentation and reporting on their evaluation require further
consideration. Moreover the evaluation of the submitted documentation may result in
subsequent requests for additional written documentation, or the collection of evidence
through observation of practice or by means of interviews.
6. Observation of Practice Evidence
Witness accounts describing existing processes within the repository represent a key means
of determining whether certification criteria are being met. In most cases this represents the
most straightforward way in which repository workflow and good practice can be evidenced.
Within the context of an institutional audit, auditors themselves will expect to be exposed to
the processes of ingest, archiving, and dissemination. This might be most fruitfully achieved
by following the passage of a single digital object throughout the full process, or through the
selection of different evidence points related to the passage of different objects through the
process while observing what happens at each of these points. As well as processes, technical
characteristics of the repository can be assessed in this way. While observation may appear
less objectively quantifiable than documentary evidence, it nonetheless represents an
important part of the organisational assessment and is used in other types of audit. In such
areas as procedures and workflow models, auditors can test how well the repository
understands what it does and does what it says it does, its relationship with its users, and how
it has planned to handle disasters. These observations might include walkthroughs or testing
and measurement of the presence of essential characteristics of digital objects against
anticipated or projected characteristic survival after preservation action (e.g., after
regularisation, migration, emulation) has been carried out. Here again our community needs
to define what practices it should adopt to document these audits; there are practices we can
adopt from other communities.
7. Testimonial Evidence
7 of 13
01/08/2006 17:25