• Trust : Enrichment : Openness

    Archives Unlocked, the vision for Archives in England 

    Archives Unlocked was launched by the National Archives yesterday, 29th March. This is a compact, but important document: “IN SHORT: ARCHIVES MATTER. Our collections need to be used to be useful.”  This is not a new philosophy, but it has new implications, driven by three changes in the context of archives which have become more apparent over the last decade or so, and the last months. The technological and social context is characterised by the concern for digital and accessibility in the UK Digital Strategy section on heritage. This is joined by a concern for confidence in information in an era of false news, and the removal of old obfuscations and lies through examination of archival material.

    “TECHNOLOGICAL CHANGE Digital technology has fundamentally changed what it means to be an archive. Archivists can help the IT and knowledge management communities by bringing professional archival practice to this digital world”,

    “USER EXPECTATIONS Society is changing, opening up new uses for data and records, and posing new questions about what is collected now and in the future, in both paper-based documents and digital formats”. 

    The third change is “CONFIDENCE IN DATA AND INFORMATION People need to have confidence in the integrity of institutions. Organisations need to be open and transparent, and high profile enquiries into the history and culture of public, corporate and charitable bodies have highlighted the evedential value of records.” 

    The Vision document changed significantly in response to the changes we experienced particularly through the second half of 2016: the importance of access, particularly digital access and access to born-digital information highlighted when the importance of this data for confidence in institutions became clearer: it is not enough for the data to be preserved, or for it to be reliably transmitted, but also for it to be open and transparent.

    This context leads to three high-level visions, for Trust, Enrichment and Openness, with case studies and think pieces for those who would like to delve further, and action plans for those who are involved with delivering the vision, in whatever capacity.

    How Free UK Genealogy helps to achieve that vision (using the language of Archives Unlocked).

    Trust

    People and institutions trust in the quality of our type-what-you-see transcriptions as an authentic representation of archived records, supported by our openness about the limitations of a transcription, and the need for researchers to verify information. 

    • Democracy and society are strengthened by enabling free, comprehensive, remote scrutiny of the archival record, holding institutions and individuals to account.
    • Users have confidence in the integrity and authenticity of our transcriptions, and in the charity and its volunteers who support their research.
    • We embrace the opportunities of technological change, ensuring confidence in both born-digital and transcribed records.

    Enrichment

    Our work enhances and enriches our society intellectually, economically and culturally.

    • Our culture of knowledge and learning and our commitment to open data expands through new ways to discover and use archive material.
    • Open data means value in businesses(1) can grow through the use of archive material to support change, innovation and efficiency.
    • People’s lives are enhanced through their engagement with archive collections.

    Openness

    Free UK Genealogy cultivate an open approach to knowledge, makes archive records accessible to all.

    • We aim to deliver an excellent user experience, enabling people to find, access and interpret archive records
    • The rich diversity of society is increasingly reflected in our archives’ collections, users and workers (including volunteers).
    • We are networked globally to maintain excellent practice and open new possibilities for institutions and users.

    In some of these areas, we have almost 20 years’ experience as an institution, and huge experience as individuals.  In others, we have just started on our paths towards truth, enrichment and openness. The work plan will help us in that, and we in turn can help others in the wider archive world.

    The plan focuses on three themes:

    DIGITAL CAPACITY. Develop the digital capacity of the archives sector, to preserve digital records, and increase discoverability of the paper and digital archive. 

    RESILIENCE. Build the sectors resilience to ensure more archives can meet and sustain the Archive Service Accreditation standard, open the sector to new skills and a more diverse workforce, increase income generation capacities, and support innovative service models. 

    IMPACT. Demonstrate the impact of archives by developing and expanding audiences, piloting approaches to using data and evidence, and influencing thinking in the IT, commercial and knowledge sectors.

    http://www.nationalarchives.gov.uk/documents/archives/Action-Plan-Accessibility-Version.pdf

    The plan will be delivered over the next three years, each a separate phase:

    PHASE 1 - BUILDING THE PLATFORM. Scope and design the infrastructure that will give archives the capacity, knowledge and development tools for delivering the three themes of the action plan. 

    PHASE 2 - DEVELOPING CAPACITY. Design and test new models of delivering world-class archive services, working with partners on research and guidance in order to enable the development of new archive practice. 

    PHASE 3 - SHAPING THE FUTURE. Enable services to influence new delivery streams in emerging technologies, policies and strategies, within and beyond the archives sector.

    (1) One change between the consultation version of Archives Unlocked and the published version which we argued for was a fundamental shift from seeing ‘commercial’ relationships in terms of behind-paywall datasets: a wider vision of the contribution of archives to economic sustainability (as opposed to the contribution of business to the budgets of archives) is both more representative of the wider archive community, and fit much better with a vision for archives that has truth, enrichment and openness as its aims.  This is not to say that there is no role in this world for commercial partners who limit access: if they are providing enrichment that cannot be made by the archive or not-for-profit partners, they still have an important role, and will still be contributing to economic sustainability.

    Quotations and adaptations from Archives Unlocked are © Crown copyright 2017.

    This publication is licensed under the terms of the Open Government

  • Guest Post: The GRO Searchable Database and PDF Pilot

    We are happy to welcome Anne from Leaves Family History Research Service as our first Guest Poster. Here, she presents her musings on the new GRO pilot scheme.

        Updated 31st January 2018

    A Brief Background:

    There have been calls to improve access to civil registration records for many years going back at least 25 years.  Various Government papers looked at the issues, including a 1990 White Paper on ‘Registration: Proposals for Change’, but little if anything was ever agreed.

    In 2002 the 'Civil Registration: Delivering Vital Change', report mentioned electronic access to ‘historic’ records could be provided by a ‘not-for-profit’ organisation.  The report may have been referring to FreeBMD, which had started to transcribe a few years before.  Between 2005 and 2012 there were several attempts to digitise and index the General Register Office (GRO). records, primarily the DoVE (Digitisation of Vital Events) and MAGPIE (Multi-Access to GRO Public Index of Events) projects, but none were completed.  It was not until the Deregulation Act 2015 that different ways of accessing historic civil registration records were discussed again.  This Act allows the relevant Government Minister to make regulations dealing with searching and supplying information from civil registration records held in the GRO. (1)

    This month (November 2016) the GRO began trialling the first of 3 pilot schemes, allowing the purchase and emailing of PDF copies including birth records dated 1837-1934 and death records dated 1837-1957.  The purchase of marriage records are not included in the trial.  These copies can only be used for research, not for official identification purposes, as they are not certified. Phase 2 will pilot the delivery of the PDF records within 3 hours, and phase 3 the delivery of PDF copies of civil registration entries that are not held by GRO in a digital format.

    The Searchable Index

    To assist in the ordering process a free online searchable database was also introduced.  To access this you must register and login into the GRO website.  Unlike the original GRO indexes, which many family history researchers are familiar with, these indexes include the mother’s maiden name for most birth registrations prior to 1911, and ages of death prior to 1860.  Both of these will be a huge boost for researchers.  Sadly, the birth index only goes up to 1915, although the death index continues to 1957.  This means that in order to purchase a PDF copy of a post 1915 birth record, the reference details must be found on the FreeBMD website or other partner databases.  There is currently no searchable GRO index for marriages.

    To search either index is easy but also surprisingly restrictive, as can be seen from the image below, and can be accessed via: https://www.gro.gov.uk/gro/content/certificates/indexes_search.asp.


    The search for names can be exact spellings, phonetic or similar sounding.  The names are also broken down into three parts, surname (which is a requirement), followed by first and second forenames.   Although this can be a useful feature there are issues if the person was not known by their 1st forename.  It is possible however to search without inputting any forenames, but a surname must always be included.

    As the mother’s maiden name can also be added this can making the search for popular surnames easier.

    The main issues with this search is that you must choose the gender (male or female, but not both), and a year, but you can only search for up to 2 years on either side.

    An interesting omission is that you cannot search the indexes by county.  Currently you can either search by registration district, which can be restrictive if the family moved around, or by the whole of England and Wales.  

    The search page for death registration is similar but includes the age at death (+/- up to 10 years) instead of the mother’s maiden name.

    The Search Results

    To try this new system, I decided to look for the births and deaths of some of the people in my family tree, and in each case I found all of them, despite some reports of missing entries.  In fact, because of the mother’s maiden name search, I found a couple of births that I had not previously found as they had been born and died between census years.   
    In most cases when I searched for an exact spelling of a surname with no forenames given the results were displayed very quickly, although you have to scroll below the search box to see them.  When I requested a phonetic or similar sounding search, it could take up to a minute for the results to be listed, and several seconds to change to the next page.  Whether this was because of a long search or because the site was busy I do not know.
    My main concern with the results in general, was that the quarters were listed by initial letter. M = March, J=June, S=September and D=December.  For experienced researchers this is not too much of a problem, but for new researchers it can be confusing, especially as J could be taken to mean January.  There has been some online discussion on various forums about the naming of quarters with some preferring 1st Qtr and 2nd Qtr etc., but my students usually find the JFM, AMJ formats easier to remember.

    Search Results – Births.

    Another issue with the results is the lack of county.  I appreciate that counties moved their boundaries, but I needed to do an internet search to find that the ‘Lexden and Winstree Union’ was in Essex.  

    An interesting omission in the results shown above is the mother’s maiden name for birth in the Blofield Union.  As this child is in my family tree I know he was illegitimate.  I searched for other known illegitimate births, where the father is not recorded and in each case the mother’s maiden name column is blank.  So this is a good indication of an illegitimate birth.

    Early reports of the use of this database suggested that the deaths of infants contained errors relating to their ages.  Using known infant deaths from my own family tree I looked up several and only one gave the age as 0 years.  In other cases 15 years was shown instead of 15 months, and 1 year instead of 1 day.

    Birth and death in the same Qtr but showing age at death as 1 year

    The GRO have included a system to correct any incorrect or missing entries, as shown below.  The form opens in a new browser window and you are required to complete all of the details yourself.  There is no link between the record and the report, unlike the system on the FreeBMD website.  Whereas the FreeBMD website entries are linked to the corresponding index page, the GRO entries are not, so possible transcription errors cannot be checked.  

    Reporting Issue

    Ordering PDF Copies

    Ordering PDF copies or the actual certificates is now easy. Once the record has been found in the index search, you simply click on the relevant option, which takes you to the order page where all the information has already been completed - you just need to make the payment.

    Conclusion

    It seems clear that the new GRO searchable index is simply to help researchers to purchase the correct record, rather than a general research tool. The addition of the mother’s maiden name is very useful, but tempered by the restrictive search of +/- 2 years and the male/female requirement, meaning that several searches for family members must be made rather than one inclusive search. The popular FreeBMD website will, in my view, continue to be a vital resource for the majority of general searches, especially as their double entry system can help to weed out transcription errors.

    ____________________________________________________________________________

    References: 

    [1] Fairbairn, Catherine. (2015)  Briefing Paper. Researching ancestry: access to civil registration records. Number 02722, 9 July 2015. Accessed online : http://researchbriefings.files.parliament.uk/documents/SN02722/SN02722.pdf

    ____________________________________________________________________________

    Update - January 2018

    Phase one was clearly a success as in October 2017 a new pilot was started to run for a minimum of 3 months. Within that time over 79,600 PDF applications had been processed. The pilot was then extended for a minimum of a further 6 months until at least the 12th July 2018.

    In addition the end date for birth records has been extend by a year to 1916. Each PDF cost £6 compared to £9.25 per Certificate.

    Anne Sherman of Leaves Family History is a qualified and experienced Genealogist and Tutor.  She can research your family history, help you with your own research or teach you how to start to get started with her online course, using free websites, including the FreeUK Genealogy sites. Anne was a transcriber for FreeBMD and now transcribes for FreeREG.

  • Who really owns population and census data: and should commercial use be allowed?

    Over the last ten years, the power and use of data for transparency, accountability and research has become a central feature of the way that government and academia are doing their work. Both have come to recognise that data and publications - including research - have the most benefit when it is shared with everyone, and that they have no moral case for restricting what other people do with factual information.
    The central idea is that since government pays for this information through our taxes, people should be able to do whatever they like with it without having to ask permission first. As Dave Mayall explains, this approach results in people doing interesting, clever, and unexpected things.

    FreeUKGen is dealing with data that was collected and created at great expense, through the will of the Crown and Parliament. Going back to Henry VIII, laws have demanded the creation of population records by the Church. From 1801, the state collected additional information through the census, and in 1837 started collating birth, marriage and death records through the GRO.
    If there were simple justice in the world and we were not going through economically hard times, the government would have a duty to digitise and release historic data such as this for free and without restriction. Birth, marriage and death data which they currently hold digitally will, if the UK government acts consistently, be released freely once it ceases to relate to living individuals.

    As a project we have a simple dilemma. We know this data should exist, and everyone should have free access to it. That is why the project started and it is our mission. Some of us are, however, uncomfortable with our efforts potentially being used by commercial outlets. To those who feel this way I would ask four questions:

    (1) The vast majority of the expense in creating this data was borne by the public purse, at the time that governments decreed that it be collected and preserved. Why should we claim that our efforts, valiant as they are, should then deny the public full and free access to the dataset when everybody’s ancestors paid for it in the first place?

    (2) If the government digitised and released this data today we would surely wish them to release it without restriction, so that everyone can benefit and use the data as they like. There would be no good reason for the state to choose who can use the data and who cannot. If we would expect the government to do this, why would we apply a different standard to ourselves?

    (3) As a charity, we must seek to ensure the best and fullest use of our charitable work for the public good. From this perspective, who are we to say whose use of data is valid or invalid? Why should we choose who can do what?

    (4) In the long run, from a national or global perspective, this data like any other publicly created dataset should be open and free to ensure all the benefits can be made of it. If we try to keep it closed, then someone else will feel the need to re-digitise it. The first full, free and open version will be what is used, in the long run. Is it really the best way forward to risk that our work in digitising the data, by being partially closed (to commercial use for instance), should be supplanted and discarded as the result of a second effort in the future?

    This change of perspective may well also change the way that commercial players work. Today, they compete by charging for access to closed datasets. They have no incentive to encourage sharing of transcriptions and data. If we start the sharing, they may (where our shared resources are best) find it is pragmatic to help us with these key resources. Of course, they won’t open up all their data, but where we do it best (as we do!) it will be pointless, expensive and bad for their business to try to duplicate what we do.

    This is not fantasy: this is what happens in the software world today. Free and open source software, like LinuxApache, -and a whole range of other software- is built by IBM, Apple, Google, Facebook and others, including many, many volunteers. They all work with open projects for a variety of reasons but the underlying point is that it makes business sense, for instance for reasons of quality and efficiency.
    The same applies in other fields, including Wikipedia, which dominates the world of public domain knowledge. Even freely licensed photography helps the world illustrate their blogs and websites. 
    There are other freely licensed historical records, too, such as historic weather, digitised by volunteers. As Open Data, it is making huge contributions to climate science. Perhaps we could use it for family histories in some way!

    As trustees, we know we are asking for some faith that opening our data is worth doing. We believe that we will become more relevant in the future by going down the road of Open Data. We can become the lodestone of accurate and rich genealogical data, and bring more people into our endeavours on more genealogical projects. But this only works if we see our mission as being that of public benefit for all, rather than restricting that mission to individual researchers or projects we pre-approve.

    Ben Laurie, one of our trustees, went through a similar process with the Apache Software Foundation some years ago. Apache is software that runs most websites you visit today. There were voices of opposition to ‘commercial’ repurposing of the software, with some people worried that the software would simply be taken away by companies, who would then seek to create their own web server monopolies.
    However, this hasn’t happened. Today, Apache is a huge success story. This could only happen because they released it as an open project, and made no judgement about who might use it. This isn’t to say that Open Data for our projects is without risk - but it does show that for public charitable endeavours in today’s digital world, being open and allowing any kind of use of your creations can be a winning strategy.

    Jim Killock

    Trustee, Free UK Genealogy