ACROSS THE DECADES - 40 years of data archiving

The social science environment - changing times, changing names

1960s - The Data Bank

Although the 1946 Clapham Committee recommended that public funding of social science research in universities expand, by the early 1960s total research funding for the social sciences from all sources stood at only 5 million and there were still as many historians teaching in Oxford as there were sociologists teaching in universities in Britain as a whole.

Studying the data bank in operation are Mr T.E. Roughley and Prof A.M. Potter

The expansion of universities following the Robbins Report of 1963 and the establishment of the Social Science Research Council (SSRC) in 1965 led to the expansion of social science research within universities and it was within this context that the SSRC Data Bank was set up at the University of Essex in 1967.

1970s - The Survey Archive

The 1970s was a period of growth in empirical social science research. By the mid-1970s approximately 50 million per annum was being spent on social research in Britain, half in universities and the rest in 'in-house' government research and independent research units.

In its early years the Data Bank originally experienced difficulties in populating its collection due to the high standards it required from deposits, an immature culture of data sharing, and the restrictions on use attached to certain studies, particularly government surveys. The turning-point came in the early 1970s when the Government Statistical Service enabled government surveys to pass to the Survey Archive, as the Data Bank had been renamed in 1972.

1980s - The SSRC/ESRC Data Archive

SSRC Data Archive logo

The Survey Archive was renamed the SSRC Data Archive in 1982 in order to reflect the fact that a broader range of data resources, more than just surveys, was now being collected and stored.

In the early 1980s government spending on universities in general, and social science research in particular, was cut. As part of this process the Rothschild enquiry was set up to review the 'scale and nature' of the SSRC's work. Despite government antipathy towards the SSRC, the resulting report was supportive of the Council whilst recommending a stronger focus on empirical research and research considered to be of 'public concern'. Whilst the Government accepted Rothschild's main recommendation - that the SSRC continue - a consequence was that the SSRC was renamed, becoming the Economic and Social Research Council (ESRC) in January 1984. This resulted in a second name change to the Archive in two years - ESRC Data Archive!

UKDA 40 logo The Archive was referred to as 'the Rutherford Laboratory of the social sciences world' in a House of Lords debate on the Research Councils on 21 November 1984.

Whilst the 1980s could be seen to be a low point for the social sciences, in retrospect, pressures on funding had both negative and positive impacts on the Archive - less was spent on primary data collection, yet this in turn encouraged increased secondary use of research data and a greater acceptance of data sharing. The 1980s also saw the Archive branch out through its involvement in a number of large co-operative data-orientated projects - key of which were the Domesday Project and the Rural Areas Database. This set a trend which has continued up to the present.

1990s - The Data Archive

The Data Archive logo

The 1993 White Paper on Science and Technology led to an emphasis on wealth creation and the need to establish closer and deeper partnership between the academic community and users of its research. In line with this, the 1990s witnessed an extension of activity by the Archive. In 1992 the History Data Unit was formed as a specialist unit within the Archive, becoming part of the Arts and Humanities Data Service (AHDS) three years later (and as a result renamed the History Data Service).

In 1996 direct funding from the Joint Information Systems Committee (JISC) was received in recognition of the support provided by the archive for teaching and learning. This led to a further name change with the dropping the Council preface to become The Data Archive, together with a new colour scheme and logo being selected to co-ordinate with the new image of the University of Essex.

2000s - UK Data Archive

This latest period started with yet another name change. In order to reflect both its UK-wide remit (post devolution) and the importance of its role within the international data network, it became the UK Data Archive (UKDA).

Qualidata, a specialist unit initially set up in the Department of Sociology at Essex in 1996 to acquire and support the secondary use of qualitative materials, merged with UKDA in 2001, both streamlining procedures and facilitating greater cross-over between qualitative and quantitative approaches.

The UKDA's portfolio was also further expanded by the creation of the Census Registration Service (CRS) in 2001. Although it had provided census data to researchers for 20 years prior to this, the creation of CRS marked the UKDA becoming a member of the ERSC/JISC-funded Census Programme to a range of UK census outputs.

The new UKDA building

The changing nature of the Archive's role within the social science community and the requirements of its users were reflected in the ESRC's green paper on 'Data Archiving and Data Resources' (2000) and resulted in the creation of a major new initiative in the form of the Economic and Social Data Service (ESDS) which came into operation in 2003. This formulated changes already underway focusing on a more user-driven service with specialist support and data enhancement for key data resources through a distributed service, based on collaboration between four centres of expertise: the UKDA and Institute for Social and Economic Research (ISER) at Essex, and the Cathie Marsh Centre for Census and Survey Research (CCSR) and Manchester Information and Associated Services (MIMAS) both located at Manchester.

In recognition of its position in disseminating and preserving an increasingly diverse collection of government data, from 1 January 2005 the UKDA became a designated Place of Deposit for public records for The National Archives (TNA), thus making the deposit of materials a legal requirement for the first time, and thereby ensuring the supply of key social surveys for future research.

Most recently, in the 40th anniversary year, together with ISER, the UKDA moved into a new purpose-built social science research centre costing 6.5 million.