I'll show you my research data if you show me yours...

My research data

A few months ago I was having a clear out at home and came across a bunch of floppy disks in the drawer of my bedside table.

This is my research data...

Actually, that is not strictly true. I did a taught masters course and my research consisted of just a short dissertation at the end of the course. Most of these disks contain files from the taught element of my course and the subsequent dissemination of results. 

I published a paper at the end of the masters on the findings of my dissertation. 

If you are interested in the placement of
Iron Age hillforts in the landscape then
this is the book to look for.

No-one has since approached me and asked if they can see the data that underlies this publication

...but this was the 1990's! 

Times are different now. We expect our researchers to be able to produce the data and share it (where appropriate) so that others can build on their research. 

I'm now involved in teaching researchers here at York about Research Data Management (RDM) and how they should look after their data for future re-use.

When I created and stored this data I was not a digital archivist. I had no idea I would become a digital archivist. I like to think I would have managed my data differently if I had known more. 

Let's start with documentation. Much of the documentation for this data is what is actually written on the disk labels. I gave myself a little pat on the back for having recorded what was on the floppies so well on *most* of the disks. This of course was particularly useful in those days. File names were restricted to 8.3 characters so very little detail about the files could be incorporated into the name. Documenting things on disk labels helps add a bit of context. All well and good until you notice the disk on the far right with no label at all. This one remains a mystery!

So what are the issues here. First and most obviously, as a student in the 90's I was using cutting edge storage technology - the floppy disk! Can we read these today? Yes and no. Floppy disks fall firmly into the category of 'obsolete media' which is a topic that we digital archivists like to talk about. I found I could read about a quarter of these using the USB floppy reader that is attached to my PC. For the others I saw a lot of error messages like this:

The answer is "No"!

Fortunately I had more success using an old PC I keep in my office for the very purpose of reading old floppy disks - all but two of the floppies could be read and copied using this PC. On one disk I could view the list of files on it but couldn't copy all of them off the disk so I considered this to be a partial success. The one disk which I couldn't access at all was interestingly the one with no label. Perhaps this mystery disk was in fact never formatted or put into active use. 

Not too bad a result so far?

So what about the contents of the disks?

The contents of one of the floppy disks. Windows Explorer identifies the DOC files as
Microsoft Word 97-2003 but they are likely to be an earlier version of Word than this

As mentioned above the file and folder naming is noticeably brief (as is the way with media from this period). Today we talk to our researchers about the importance of naming files in such a way that you know what it is before you double click on it. This was near on impossible when faced with only 8 characters. I created this data but have no idea what I might expect to find in a directory called 'DISTEX' (though the label on the disk does help give a clue).

Note too the lack of organisation of the contents. At the end of my masters degree whilst finishing off my papers and publications I was also clearly focusing on what my next steps would be. Personal data (my CV for job applications) is stored alongside data relating to my research*. This again is something we discourage when we talk to researchers about data management. It is much easier when working with filestore to organise and categorise data more effectively, keeping personal data separate from research data. We have come a long way since the days when we were squashing any files that would fit on to a floppy disk regardless of content or context.

Here is some data on another of the disks (viewed in Windows Explorer as tiles). I have no idea what possessed me to store scanned photographs as GIF images. They look terrible! Did they always look this bad? Choosing the right file format is something we also cover in our RDM training and though file size is still a consideration for today's research students, at least they don't have to try and fit numerous images for one presentation on a single floppy disk.

More coded file names - this was a necessity when you had so few characters available.
I still remember what these mean but very much doubt anyone else would.

Some are my files are fairly easy to read, others less so (more detective work is required to find the right software). The Word documents are OK but come up in 'Protected View' (which means I'm not allowed to edit them). The default settings here are to treat a Word 6 or 95 document with suspicion but this can be easily resolved by editing these settings.

These old MS Word docs are still readable (and editable if I change the policy settings)

So, digging out my old research data has been an interesting diversion. I now use this as an example at the beginning of RDM teaching sessions and ask the students to imagine how their research data might look 20 years from now. 

Another added bonus from this exercise is that I now have even more files that I play with as I test Archivematica and file identification tools.

*Interesting to note that a first (unsuccessful) attempt to get a job in York occurred in 1998. I got here 5 years later!

Jenny Mitcham, Digital Archivist

A short detective story involving 5 ¼ inch floppy disks

Earlier this year my colleague encountered two small boxes of 5 ¼ inch floppy disks buried within the Marks and Gran archive in the strongrooms of the Borthwick Institute. He had been performing an audit of audio visual material in our care and came across these in an unlisted archive.

This was exciting to me as I had not worked with this media before. As a digital archivist I had often encountered 3 ½ inch floppies but not their larger (and floppier) precursors. The story and detective work that follows, took us firmly into the realm of ‘digital archaeology’.

Digital archaeology: 

“The process of reclaiming digital information that has been damaged or become unusable due to technological obsolescence of formats and/or media” (definition from Glossaurus)

Marks and Gran were a writing duo who wrote the scripts of many TV sitcoms from the late 1970's on-wards. ‘Birds of a Feather’ was the one that I remember watching myself in the 80’s and 90's but their credits include many others such as ‘Goodnight Sweetheart’ and ‘The New Statesman’. Their archive had been entrusted to us and was awaiting cataloguing.

Clues on the labels

There were some clues on the physical disks themselves about what they might contain. All the disks were labelled and many of the labels referred to the TV shows they had written, sometimes with other information such as an episode or series number. Some disks had dates written on them (1986 and 1987). One disk was intriguingly labelled 'IDEAS'. WordStar was also mentioned on several labels 'WordStar 2000' and 'Copy of Master WordStar disk'. WordStar was a popular and quite pioneering word processing package of the early 80’s.

However, clues on labels must always be taken with a pinch of salt. I remember being guilty of not keeping the labels on floppy disks up to date, of replacing or deleting files and not recording the fact. The information on these labels will be stored in some form in the digital archive but the files may have a different story to tell.

Reading the disks

The first challenge was to see if the data could be rescued from the obsolete media they were on. A fortuitous set of circumstances led me to a nice chap in IT who is somewhat of an enthusiast in this area. I was really pleased to learn that he had a working 5 ¼ inch drive on one of his old PCs at home. He very kindly agreed to have a go at copying the data off the disks for me and did so with very few problems. Data from 18 of the 19 disks was recovered. The only one of the disks that was no longer readable appeared from the label to be a backup disk of some accounting software - this is a level of loss I am happy to accept.

Looking at file names

Looking at the files that were recovered is like stepping back in time. Many of us remember the days when file names looked like this - capital letters, very short, rather cryptic, missing file extensions. WordStar like many of the software packages from this era, was not a stickler for enforcing use of file extensions! File extensions were also not always used correctly to define the file type but sometimes were used to hold additional information about the file.

Looking back at files from 30 years ago really does present a challenge. Modern operating systems allow long and descriptive file names to be created. When used well, file names often provide an invaluable source of metadata about the file. 30 years ago computer users had only 8 characters at their disposal. Creating file names that were both unique and descriptive was difficult. The file names in this collection do not always give many clues as to what is contained within them.

Missing clues

For a digital archivist, the file extension of a file is a really valuable piece of information. It gives us an idea of what software the file might have been created in and from this we can start to look for a suitable tool that we could use to work with these files. In a similar way a lack of file extension confuses modern operating systems. Double click on a file and Windows 7 has no idea what it is and what software to fire up to open it. File characterisation tools such as Droid used on a day to day basis by digital archivists also rely heavily on file extensions to help identify the file type. Running Droid on this collection (not surprisingly) produced lots of blank rows and inaccurate results*.

Another observation on initial inspection of this set of files is that the creation dates associated with them are very misleading. It is really useful to know the creation date of a file and this is the sort of information that digital archivists put some effort into recording as accurately as they can. The creation dates on this set of files were rather strange. The vast majority of files appeared to have been created on 1^st January 1980 but there were a handful of files with creation dates between 1984 and 1987. It does seem unlikely that Marks and Gran produced the main body of their work on a bank holiday in 1980, so it would seem that this date is not very accurate. My contact in IT pointed out that on old DOS computers it was up to the user to enter the correct date each time they used the PC. If no date was entered the PC defaulted to 1/1/1980. Not a great system clearly and we should be thankful that technology has moved on in this regard!

So, we are missing important metadata that will help us understand the files, but all is not lost, the next step is to see whether we can read and make sense of them with our modern software. 

Reading the files

I have previously blogged about one of my favourite little software programmes, Quick View Plus – a useful addition to any digital archivist’s toolkit. True to form, Quick View Plus quickly identified the majority of these files as WordStar version 4.0 and was able to display them as well-formatted text documents. The vast majority of files appear to be sections of scripts and cast lists for various sitcoms from the 1980’s but there are other documents of a more administrative nature such as PHONE which looks to be a list of speed dial shortcuts to individuals and organisations that Marks and Gran were working with (including a number of famous names).

Unanswered questions

I have not finished investigating this collection yet and still have many questions in my head:

• How do these digital files relate to what we hold in our physical archive? The majority of the Marks and Gran archive is in paper form. Do we also have this same data as print outs? 
• Do the digital versions of these files give us useful information that the physical do not (or vice versa)? 
• Many of the scripts are split into a number of separate files, some are just small snippets of dialogue. How do all of these relate to each other? 
• What can these files tell us about the creative process and about early word processing practices?

I am hoping that when I have a chance to investigate further I will come up with some answers.

* I will be providing some sample files to the Droid techies at The National Archives to see if they can tackle this issue.

Jenny Mitcham, Digital Archivist