• How FreeCEN is changing...

    ... and how this affects researchers and volunteers

    The changes we're making at the moment will take FreeCEN into the future. We are no longer a lookup service, which is how we started. We are becoming a dynamic source of data for both the amateur and the professional. 

    In this post we will explain the issues driving the changes, and how they will affect you. It's broken down into the following sections:

    Changes for Researchers

    Changes for Volunteers

    • Transcribers
    • Proofreaders(formerly 'Checkers')
    • Validators
    • Coordinators

    Future Developments (for Researchers)

    Changes for Researchers

    A growing number of users are accessing the web through phones and tablets, rather than laptops and larger computers - they expect an interface that looks good and works well on a small screen. We think we are ‘almost there’ on this.

    We also want to make the website more accessible. This work is just beginning.

    We want to tackle some issues in searching, including:

    • Place: The boundaries used by the census are not easy to visualise, often cross historic boundaries (e.g. including parts of two counties) and change name and shape - so even if you know the village or part of a town your ancestor might have resided in, choosing which census district to filter by is not easy, particularly for those unfamiliar with the geography of the 19th century.
    • Names: The names used by people to record their place of birth are faithfully transcribed. But this also makes searching difficult, plus the person entering the data might know that their home village was in one county when they were born, and another at the time of the census, and chosen either. So we are looking into more choice of how you search, including a map-based search.
    • Too much choice: It can be difficult to know which of the many fields you should fill in, in order to be able to identify the person you are seeking, but not rule them out due to something not being recorded quite as you might have expected (we are looking into Artificial Intelligence to solve this one for you).

    We also want to improve the quality of the data we transcribe - in the past, we have had to use abbreviations and compress two fields into one, in order to keep within the spreadsheet restrictions of 20 years ago. We're moving to a flexible spreadsheet (CSV) system which will mean transcribers can type what they see everywhere.

    Changes for Volunteers

    Volunteers may think "I am happy with what I am doing now." That's fine, you can keep doing what you are doing and we will continue to appreciate your work. It is very valuable. Alternatively, you can come along for the ride. You will find it is going to be a very exciting one!

    Transcribers

    You will have an option of either transcribing onto a spreadsheet as you have been doing, or transcribing online. The spreadsheet will look and feel very similar to the one you are using now, but there will be some changes.

    Columns in the spreadsheet become fields in the FreeCEN database. As the fields in the database have become more flexible we can be more flexible in what you enter on your spreadsheet. For instance, transcribers put a lot of effort into making the Occupation fit the Occupation column. That will no longer be the case. You will be able to enter what you see without worrying about the length and some columns can be amended. Column H in the existing spreadsheet, for instance, contains details of unoccupied houses, people visiting away, buildings in progress etc. but it is also used for the query flag. We are looking at moving the query flag into its own column. 

    In the online version, you will be entering data directly into a database. You will, therefore, see field boxes on your screen instead of columns. Instead of your coordinator sending you images, via Dropbox, for instance, the image will be displayed on your screen. When you have transcribed all the rows on an image you then move on to the next image. Each field in the online database will exactly correspond to a column in the spreadsheet.

    The online version will also capture the number attached to the image. This will allow FreeCEN the ability to link the image to the transcription and display the images on-screen for the researcher, much the same as commercial sites do now. We still need to work out how we are going to add this information to the almost 36 million records already online (as of November 2018) so the displaying of images is something for the future. Nevertheless, we can start to collect that information with the online version.

    One difference between the spreadsheet and the online database is that a transcriber can stop and start wherever they wish. The online version is displayed one page at a time so transcription has to stop at the end of each page, and not in the middle of a page. Another difference is that transcribers can share a census piece. The next available page is displayed to each transcriber (not the same page). Once a page is transcribed it will not be displayed to anybody else.

    So, whether you choose to keep transcribing on a spreadsheet or to transcribe online, the changes should make transcription easier.

    When will all this happen?

    The first test piece was transcribed in August 2018 and has been proofread and validated. We are creating a detailed audit trail for this piece so that the developers can refine the system.

    There is a test piece available online if you wish to have a go. This is an 1871 Somerset piece that has already been published online. It has been made available for multiple people to transcribe, a page at a time. Once it has been completed we will perform a quality check to compare the shared transcription with the published version. We would like to know whether our quality drops using this method. You do not need to be a Somerset volunteer to work on this; anybody can give it a go. If you do give it a go, please give us feedback. That way we can continue to improve. 

    You can access it at https://csindexing.com/projectinformation.php?p=358 (link opens in a new tab). This version is a prototype version and therefore is not very pretty. Nevertheless, we do need to know of any problems transcribers may experience with it.

    Proofreaders

    You will be using new software for the proofreading process. (No, we no longer have checkers - the transcribers' work is being proofread. Transcribers are not being checked on!). 

    Some proofreaders will remember the old WinCC software, used before we moved to proofreading spreadsheets. WinCC became incompatible with modern operating systems as it is badly in need of an upgrade. It certainly is not compatible with the new database format. 

    This upgrade will happen in conjunction with the introduction of the online transcription system. Once the transcription trial is finalised we will know the design of the proofreading software. We will also be able to make the decision whether this will be an online or offline process, or whether the proofreader will have the choice. 

    The trial transcription has been converted to a spreadsheet and the Proofreader is using a spreadsheet to proofread.

    Validators

    You will definitely be using new software. Neither FCTools nor ValdRev is compatible with the new database format. The validation process will not vary a great deal. The biggest change is that a map reference will be automatically attached to every Place of Birth. This will affect the way that the Validator manages the Validator’s Choice Place of Birth where the original entry cannot be found. The development of the map location system is in progress (as of November 2018). Once again we are not sure whether Validation will be online or offline, or whether there will be a choice.

    Coordinators

    Coordinators have already seen changes in registration of new volunteers. This will continue but you will have two methods of issuing transcription work: by sending images as you do at the moment, and by giving access to online images. The Coordinator will upload the online images to be transcribed from their master copies. The online software will record who has worked on these images, so the Coordinator will only need to keep track of who is working on the spreadsheet images. 

    Coordinators will also be able to upload their own completed data to the database. You used to be able to do this but during the changeover to FreeCEN2 this function was centralised to give more control. Uploading will be devolved back to the Coordinators in the foreseeable future.

    Future developments (for Researchers)





    We're about to give each record (household) a permanent URL. To do this, we recently made the “FreeCEN2 website” available at www.freecen.og.uk, and the existing website freecen1.freecen.org.uk. The old FreeCEN will be phased out in the future. We intend to develop new ways to search the data. 

    The addition of map references is one way of doing this, for example. Instead of having to know the county border in the year being researched, (or in the birth year of the person being looked for) a place will be able to be chosen. This place has a map reference. The search can then be done around that map reference. This would be transparent to the end user. 

    An example would be a researcher in the future looking for me. I was born in Bristol, census county GLS. However, I was brought up 7 miles away in Somerset (Census county SOM). Whenever I fill in a form I show my Place of Birth as Bristol (that was where the hospital was!). A future researcher would be frustrated searching the Bristol 1951 Census. I am not there. I am in a different county. However, a search of 10 miles around the map reference would discover my entry. A much better result for the researcher and for FreeCEN.

    We tend to think of a researcher as somebody building their family tree. However, there are other types of researchers out there. A university student doing a PhD may need to identify the demographics of a specific geographical area in a specific timeframe. Because FreeCEN is an Open Data platform we can provide a data dump of that geographic area for the student to use. Once again this is a win-win for both the student and FreeCEN.

  • Censuring Censuses!

    If you're a part of any family history forum or group, it's not unusual to find a genealogist bemoaning the spelling and handwriting, or casting aspersions on the intentions of a particular Census Enumerator. We recently shared this image on our Facebook page and our followers certainly appeared to see the veracity behind the humour:

    However, we then came across this poem which seems to lay at least some of the blame at the feet of our ancestors! 
    (Please note that this was published in 1881, and we haven't been able to see the original)

    The Enumerators Complaint

    The census may be good and right and useful to the human natur'
    But I can swear there's no delight
    In being an enumerator;
    For up and down six blessed streets,
    I've tramped it morning after morning,
    And the reception that one meets
    Should I serve as a most wholesome morning.

    This house, their writing isn't plain;
    That house, their language is exotic;
    And some describe themselves as sane,
    Who seem to me quite idiotic.
    Towns such as countess never knew
    Are given as the natal places;
    While you're supposed to find what's true,
    And to correct in faulty cases.

    Then ladies of a certain age
    Decline to make it clear by telling;
    And others fly into a rage,
    And oh, such awful slips and spelling!
    And some deduce - in humour bold -Their line from non-existent nations,
    And state they've grown uncommon old
    In most unheard of occupations.

    Here, you perceive that you intrude;
    And here, the party's an objector;
    And here, they are positively rude -They fancy you're the tax collector.
    So what with humbug and rebuff,
    And cutting many fruitless capers,
    I have already had enough,
    And cry - Confound these census papers!

    We complain about the enumerators; they pass the buck on to the public. It seems that in the taking of the census, the old adage needs updating:

    There can be three sides to a story!

  • Opening Death Data for Genealogists and Other Historians

    Open Data image with logos

    Open Data Day is an annual celebration of open data all over the world. On Saturday 3rd March groups from around the world will create local events on the day where they will use open data in their communities. It is an opportunity to show the benefits of open data and encourage the adoption of open data policies in government, business and civil society.

    All outputs are open for everyone to use and re-use.  Research Data is one of four themes for this year's Open Data Day.

    All three of our current projects contain information which is invaluable to family historians and other researchers. The indices to the registrations of death in England and Wales are, of course, freely available on www.freebmd.org.uk. Civil registration only started in 1837, so to find deaths which occurred earlier, you can look on www.freereg.org.uk, to see Church of England and other burials.  Later burials are there too, from the Church of England Registers and a growing range of religious organisations and secular bodies.  Most recently, we have received images of burial registers from Lancashire that are awaiting transcription - sign up here to help get them on line sooner!

    Image of a desk with genealogy paraphernalia


    Surprisingly, perhaps, the census records we transcribe and share on www.freecen.org.uk also have information about death. On https://freecen1.freecen.org.uk you can search by occupation, and this includes those who worked in various aspects of the businesses surrounding death.  Restricting the search to Cornwall, in 1841 there was just one (funeral) "undertaker" recorded (in St Austell) In 1851, four undertakers are recorded:

    Image showing details of four undertakers


    In 1861, just one again is recorded, and in 1871 five including Jabez Parkyn.  A decade later, the Parkyn name becomes even more visible, as the children of the family (shown below in the 1871 census) continued the family trade, all three describing themselves as "Builder & Undertaker":

    1871 Census, Parkyn Family


    But in 1891, although the number had grown to 11, none of them was a Parkyn.  Jabez senior and Jabez junior (now spelled Parkin) are recorded purely as Builders, Jabez William A had become a painter.

    Parkin 1881 census


    This brief look raises many questions - many undertakers had more than one occupation (carpenter or mason being common).  Were others who were recorded only as masons or carpenters also arranging funerals? We have not yet enabled a search-by-occupation feature on FreeCEN2 - we'd love to know if you would use this feature, and how you would like the search of occupations to work there.

    I restricted the data to Cornwall, as we now have permission to share this dataset as Open Data - please contact us to request access to this dataset. Sharing this data as Open means that the history of undertaking in Victorian Cornwall can be undertaken (excuse the pun!) much more easily than for other counties.

    Please join us in exploring our records on 3rd March, commenting here or on our Facebook event.  We'd love to know anything you are doing with the data of death - for example if you are researching the Undertaking Parkyns of Cornwall, exploring longevity, or if you would like us to transcribe the records of your church or share the transcriptions from a graveyard survey.

  • Nothing from Northern Ireland?

    Some reseachers have - rightly - questioned why Free UK Genealogy has 'UK' in its name as we have, until recently, not transcribed records created in Northern Ireland.

    The UK in our name reflects ambition rather than the actuality of the three projects which are currently providing data (FreeBMD - England and Wales, FreeCEN - England and Wales, Scotland and Crown Dependencies, and FreeREG - England and Wales, Scotland, and Crown Dependencies). 
    This is not to say that we do not have data relating to Northern Ireland.  It is most easily accessible on the original FreeCEN website, where you can search by place of birth.   For example, Harriett Smith, a widow born in Argmagh, was working as a servant in Nottingham in 1891:

    Screenshot of transcription of a household from FreeCEN website
    Household in Nottingham, including Hariett Smith, born Armagh
    Grand brick-built Victorian villa
    Cliff House, Hermitage Walk, Nottingham (c) Savilles (1)

    You can currently search on FreeCEN by Northern Irish county, and Eire counties will be added shortly. You cannot, currently, search by place of birth within any county - we would love to hear from you about how we can best help you to search for Irish ancestors, so please fill in the survey there. Sadly, the early censuses of Ireland have rarely survived. We would like to work with the National Archives of Ireland, who hold those fragments which remain, to enable the early censuses to be transcribed, and their data for the 1901 and 1911 to be shared on FreeCEN. 

    On FreeREG, we have recently been given transcriptions of registers in Co. Fermanagh, and these are the start of our coverage for Northern Ireland.  Country Coordinator Sandra Adams-West is contacting record-holders and recruiting new transcribers (head over to https://www.freereg.org.uk/cms/opportunities-to-volunteer-with-freereg if you would like to help out).

    Civil registration of births, marriages and deaths started in Ireland in 1864.  Records are searchable at https://geni.nidirect.gov.uk/, but in a very limited form.  We would like to work with the government of Northern Ireland, to enable searching across the UK (in FreeBMD), which would make their data more discoverable while enabling them to retain their income from pay-per-view to the certificates.  We are hoping to transcribe registration of Birth, Marriage and Death from military contexts and these include instances of events in Ireland.  For example one folio of the Registry of Marriages, Births and Baptisms, C Brigade Royal Horse Artillery. http://discovery.nationalarchives.gov.uk/details/r/C11665645 begins with a marriage at Charlemont, Armagh, in 1889, and ends with the birth of their daughter in Aldershot the following year.  This is a good example of a record which is currently inaccessible, as it has not been digitised

    So while we do not yet have much from Northern Ireland we have a little, and plans to do a great deal more. we would be delighted to hear from any organisation with such records that they would like us to transcribe.

    (1) This image shows the house where Harriett was a General Servant in a household with 10 young children  The house was offered for sale a few years ago, and the details are still available at http://www.rightmove.co.uk/property-for-sale/property-22783515.html

    *********************************************************************************************

    Look out for our series of guest posts starting next week, where professional Irish genealogist Nicola Morris of timeline.ie will be exploring Irish records in more depth.