Overview The data longevity promise (and a challenge) The Model Editions Partnership A report card Lessons for the future

The data longevity promise

Coombs, Renear, DeRose (1987):

... an author who is not using descriptive markup may have to modify the markup of document files ... whenever the author or the installation changes the computing environment. When [systems are replaced], authors either [spend] hours converting their files to the new format ... or [accept] the possibility of losing their data. ... In fact, almost any change in the computing environment poses a threat to one’s files if they contain procedural or presentational markup.

But in contrast

As we point out [elsewhere], descriptive markup eliminates maintenance concerns.

Twenty years later, ...

One disadvantage of clear statements: they are falsifiable.

The Model Editions Partnership

Multi-stage, multi-year project.

funded by U.S. National Historical Publications and Records Commission (NHPRC) based at University of South Carolina led by David R. Chesnutt

MEP I

Original project (1995-1999): (1996) Adaptation of TEI P3 for historical documentary editions (TEI Tag Set Description) Web site (http://mep.cla.sc.edu/) with samples (150-200 pages each) from seven partner projects: Documentary History of the First Federal Congress Documentary History of the Ratification of the Constitution and the Bill of Rights Papers of General Nathanael Greene Papers of Henry Laurens Lincoln Legal Papers Margaret Sanger Papers Papers of Elizabeth Cady Stanton and Susan B. Anthony

MEP II etc.

Later work: tweaks to the DTD XML version of the DTD more samples The Frederick Douglass Papers The Papers of Dwight David Eisenhower The Marcus Garvey and UNIA Papers The Papers of Joseph Henry The Papers of George Catlett Marshall Eleanor Roosevelt Papers stand-alone deliverable on CD-ROM

Time spares no software

2009: the server at USC dies.

2010-2011: attempts to re-host the material. Currently in the cloud (http://mep.blackmesatech.com/mep/).

So ... how'd we do?

Did descriptive markup buffer us from changes in the environment?

Did it eliminate maintenance concerns?

What worked? What didn't work?

If we had it to do again, ...?

Keepers

Things that worked / if you were doing it again today, what would you do in the same way?

Use descriptive markup

Yes, it works.

No, it's not flawless.*

Use TEI

Because the analysis works fairly well.

Not for reuse of stylesheets etc.

Modify TEI

Because modifications allow a better fit.

Multiple vocabulary flavors

MEP's design distinguishes gold-standard archival vocabulary (for long-term storage, convenient delivery) data-capture vocabulary (lighter-weight structure and metadata)

Deliver early and often

MEP had some skeptical partners (and funders).

So there was pressure to have something nice to show people.

Big books and little books

MEP's practice* distinguishes big-book encoding little-book encoding * The practice, but not the theory ... (why not?)

Assign identifiers

MEP's practice assigns identifiers to each singly deliverable item (document, biographical note, major section of introductory essay).

We may have done this for the wrong reasons.

But it's the right thing to do.

Lessons for the future

What would you do differently if you were doing it today?

Use current standards and software XML, not SGML XSLT XQuery XProc XForms

This is different in detail but not substance.

Serve the XML directly

More feasible today (but Panorama).

Don't fake small caps

In 1999, browsers didn't do small caps at all (let alone well).

But this is not the answer: In committee of the whole on the State of the Union, Mr. TRUMBULL in the chair.

]]>

Make the archival form

MEP's design distinguishes gold-standard archival vocabulary (for long-term storage, convenient delivery) data-capture vocabulary (lighter-weight structure and metadata)

But we never actually got around to making the archival form.

Delivery system was built around data-capture format.

Hire a metadata Nazi

The documents do have change logs, but they are almost entirely useless. cm 23 Jan. 1997; QA and validation by LG/ 24 Jan. 9 2/24/97; QA CM- 2/16/97 9 April 97, Level 3; QA 4/11/97 rg 97-05-13; QA 97-05-20, cm 97-08-28 rg 97-09-26 ml 98-02-05 ah 98-02-10 rg 98-05-27 mm 98-07-06 ml 98-07-15 rg 99-04-05 ah FC10720 , 20 May 1789 ... ]]>

Have a toolmaker around

Even today, projects will have special needs hard to meet with off-the-shelf packages.

Document processing and styles better

The more effort you put into styling, the less you want to lose it.

If you can't use standard languages, at least document what you did in prose.

Multiple-use

Work harder to provide delivery using two different systems.

Also revise your first version. (Don't stand still.)

Memento mori. Routine backups (even if nothing has changed) Document behaviors and appearance (?) Have a successor lined up.

(All easier said than done.)

Conclusions

What does this mean?