The Role of Evidence in Establishing Trust in Repositories
http://www.dlib.org/dlib/july06/ross/07ross.html
Interviews with stakeholders and repository staff will allow the auditor to assess internal
mechanisms and organisational processes. Inevitably, documentary evidence offers
incomplete insights. In most organisations there is a degree of knowledge that is locked away
'in the heads' of experienced repository staff. This in itself is a concern in terms of
certification, and every repository should have mechanisms in place to mitigate risks posed
by this situation. Interviews are an effective means of highlighting the omissions that exist
within formal documentation and to validate whether the aspirations of the documents are
achieved in reality.
A key step is to identify the particular individuals to be included in the interview process.
Initial considerations suggest that interviews with staff fulfilling a representative sample of
roles within the repository are sensible. This could stretch from Director to janitors
(cleaners). For instance, a casual chat with a janitor might reveal that, although the
repository's documentation states that to avoid data leakage all media from CDs to tapes are
shredded or crushed before disposal, this does not happen in practice. Of course, in many
smaller organisations different activities that might in larger organisations be handled by
different individuals might be handled by a single person. The discussion should certainly be
structured in terms of roles, and not people. A series of example roles are described below:
• Repository's Administrators: Those in charge of the repository's operation provide
an obvious starting point, and they ought to be capable of offering an institutional
overview, as well as expert insight into the repository's mission, staff appointment
criteria, financial policy, and risk management strategies. While perhaps insufficiently
hands-on to offer specific technical insights, they will have the greatest sense of the
overall workflow that is undertaken from an object's ingest through to its subsequent
retrieval. This broad view ought to be exploited.
• Hardware and Software Administrators: Interviews with the individual(s)
responsible for the design, implementation and maintenance of repository hardware
and software will assist in assessment of the appropriateness of technologies, and
security, disaster recovery or integrity measures.
• Ingest, Archive and Preservation, and Access Officers: Those responsible for key
functional units within the repository will be able to offer more in-depth insights into
day-to-day operations than senior administrators. Interviews with these classes of staff
should facilitate the understanding of the way procedures and policies are developed
and implemented.
• Depositors: While there may be practical concerns associated with securing depositor
testimonies, their views will contribute to establishing the repository's success in
achieving the targets set within its depositor agreements. For instance, depositors will
be able to verify whether they are adequately informed when processes are completed
and consulted about changes to repository procedures and services. The significance of
their role will be determined in many cases by the nature of the repository and its
relationship with depositors.
• Information Seekers: Similarly, interviews with information seekers may be difficult
to set up, but they will support assessment of whether users are satisfied with the
working of the repository. This would complement or provide validation of user-based
evaluation work. While it is true that most validation of user evaluation can be done
through assessment of protocols, processes, and outputs, there are circumstances where
auditors will find it appropriate to gather their own evidence from the community.
As vital as determining the list of potential interviewees is the identification of the core set of
questions that will direct the dialogue and effectively marry it with the checklist being used.
Current DCC research is committed to the development of a semi-structured interview
template to facilitate the process of interview and personal engagement. This will be
designed in a way that makes it extensible so that when issues arise as part of the pre-audit
8 of 13
01/08/2006 17:25