Research and development

Over the years the Archive has been extremely successful in securing a number of awards in addition to those for running the core services in order to conduct, what is broadly termed research and development. Income has come from a number of sources, including the ESRC, the European Commission and the JISC.

The Archive works in collaboration with organisations worldwide, including partner archives, data libraries, research centres and national statistical institutes, to achieve the goals and objectives of each award.

The research and development activities have always aimed to enhance the Archive's services, consolidate its position as an internationally respected organisation and spearhead new developments in data preservation and dissemination. In particular, the main goal is to develop tools or products that can then be rolled-out or implemented in the main services.

In recent years this has resulted in the development of software for web browsing, data discovery and delivery, as well as enhanced data products.

Acquiring the Family Expenditure Surveys (1975-76)

In 1975 the SSRC agreed to meet the cost of transfer to the Archive of the Family Expenditure Surveys (FES) from 1961 onwards.

This involved conversion of very early tapes to a more up-to-date format, as well as some data standardisation, since methods of coding the survey data were radically different for the period 1961-67. It was noted at the time that:

"the cost of punching or copying alone for the period 1957-73 would be some £62,500. In addition there would be very considerable expense, at present uncosted, in moving and controlling the original documents in preparing explanatory materials and in liaison with the Archive." (Office of Population Censuses and Surveys, April 1975)

Thus, unfortunately, due to what were seen as prohibitive costs, the earlier pre-1961 data were not rescued, in retrospect, perhaps a rather short-sighted decision.

The Domesday Project (1985-86)

At the beginning of October 1985 the Archive reached agreement with the BBC to be a major contractor for the provision of data for a new Domesday Survey which was undertaken from 1985 to 1986.

The BBC launched the publication of the discs on 26 November 1986 to coincide with the 900th anniversary of the Domesday Book, using then up-to-date interactive video disc technology. The discs contained a vast amount of information including 24,000 maps, 250,000 pages of textual information, 60,000 photographs, 1 hour of moving film and sound and 200 megabytes of data.

The Archive supplied over 3,200 datasets for inclusion on the discs which were intended for use in teaching and research, including schools. The Queen was shown the Domesday Project by the then director Howard Newby, during her visit to the University of Essex in 1985.

NESSTAR and FASTER (1996-2002)

In 1996 funding from the European Commission, under the Information Engineering part of the Telematics Programme, supported the project known as Networked Social Science Tools and Resources (NESSTAR).

The aims of this project were to further internet developments of value to the European data archives and their users, with key components being an integrated yet distributed catalogue of data holdings, with further modules devoted to data browsing, data visualisation, simple analysis, data subsetting and downloading.

The Archive's main partners in this enterprise were the Norwegian Social Science Data Service and the Danish Data Archive, with the Archive in a co-ordinating role.

In September 1999 the Archive began contract negotiations with the European Commission for a new research proposal called Flexible Access to Statistics, Tables and Electronic Resources (FASTER) - to take forward some of the developments within NESSTAR, aiming to develop a robust architecture for the dissemination and use of statistics.

As a result of these projects, in late 2000 the Archive launched its first Nesstar product - 'Nesstar Light' for online data discovery, browsing, tabulation and download. Subsequently, additional versions of the Nesstar software have been released, with the latest version (3.5) allowing tabulation of aggregate (cube) and micro data, regression and correlation, recoding and grouping of variables.

MADIERA (2002-06)

The European Union (EU)-funded Multilingual Access to Data Infrastructures of the European Research Area (MADIERA) project aimed to develop and employ a multilingual thesaurus to break down language barriers in the discovery of key social science data resources.

As a result the multilingual thesaurus European Language Social Science Thesaurus (ELSST) was developed as a follow-up to the Language Independent Metadata Browsing of European Resources (LIMBER) project. This has subsequently been used within the CESSDA data portal to provide a seamless interface to datasets from social science data archives across Europe.

The distributed catalogue is currently searchable in nine languages (German, Danish, Greek, English, Spanish, Finnish, French, Norwegian and Swedish).

CHCC (2000-03)

The Archive contributed in three areas to the Collection of Historical and Contemporary Census Data (CHCC) project. The project's main objective was the development of learning and teaching resources achieved by improving accessibility to the primary data resources and by developing an integrated set of learning and teaching materials.

The History Data Service improved access to the Historical Census Collection by making data from census enumerators' books and data from the published census reports available online. This work led indirectly to the historical census download service, now managed by AHDS History and currently accessed by more than 650 users annually.

The Archive's role was to develop a stand-alone Census Resource Discovery Service (CRDS) which is still available and employs metadata schemas in order to index its dataset and learning resource records. The storage and resource discovery of the metadata are centralised but the entry system is distributed, with each CHCC partner entering their own metadata.

Finally the Archive was responsible for the evaluation of the project outputs. This survey-based evaluation work included testing by students and a tutor-led case study of the service.

TRAMSS (1998- 2000)

The Teaching Resources and Materials for Social Scientists (TRAMSS) project aimed to place exemplar data analyses in a substantive context by introducing data sources and methods via research questions. Geared towards research students in quantitative social science research, the resource guided users through finding, extracting and applying analysis complex models to data. It is still in use by teachers and has subsequently been updated.

JISC X4L (2002-03)

The JISC Exchange for Learning Project (JISC X4L) project built, piloted and documented the evaluation of a set of data-based resources for use in teaching in political science courses within higher and further education institutions.

The resources were based on existing Archive holdings that were re-purposed to suit the pedagogical needs identified by the tutors authoring and piloting the teaching resource for their existing courses. This included obtaining permission from the Home Office to create an abbreviated customised version of the British Crime Survey for online analysis, which has subsequently been used as a major resource in its own right.

ARCHIVE DOCUMENTATION DIGITISATION PROJECT (1996-98)

An ambitious project to scan the entire paper documentation holdings of the Archive (around 600,000 pages) aimed to provide machine-readable documentation for all datasets within 18 months.

Tagged Image File Format (TIFF) images were produced, which were then converted into PDF files and combined to make comprehensive user guides to accompany the data.

Hiring teams of temporary staff who worked in shifts 24 hours a day, the project was successfully completed within a year, leaving time to digitise the administrative and deposit- process material associated with studies.

As a result, documentation for all studies within the Archive can be delivered to researchers in machine-readable form (indeed, all data documentation is openly accessible, browsable and downloadable via the Archive web site). In addition, internally, staff can view processing and background information about any study onscreen, without the need to handle original materials.

Digitisation is now a seamless part of data and documentation processing, although these days most material is received by the Archive in a computerised format.

OHPR (2004-07)

The JISC-funded Online Historical Population Reports (OHPR) project undertook a major digitisation exercise to capture and mount on the web a complete collection of British historical population reports from 1801 to 1937.

With almost 200,000 page of images, from all British censuses from 1801 to 1931, and a complete run of Annual Reports of the Registrar General until 1920, the web site, called histpop, is a comprehensive resource for students of population history. Additionally, the histpop site contains a vast quantity of contextual material, including essays and working documents from the General Register Office.

QUADS (2005-06)

The ESRC Qualitative Archiving and Data Sharing Scheme (QUADS) aimed to develop and promote innovative methodological approaches to the archiving, sharing, re-use and secondary analysis of qualitative research and data, in all of their disparate shape and forms.

A range of new models for increasing access to qualitative data resources, and for extending the reach and impact of qualitative studies was explored. The scheme also disseminated good practice in qualitative data sharing and research archiving.