Where does the raw TEI XML live?

[jeffreycwitt]

Hi there, I'm just trying to understand the technology stack of this project/cms a little better

I was wondering if it is possible to tell me where the encoded TEI XML text files "live" and how they are accessed by the application? I don't see the TEI XML within this repository itself, so I wonder whether they live in another github repository to which the web application points and then retrieves when it builds?

Or are the TEI XML files created directly in the application itself and stored directly in the running database?

Is it possible to view the XML files through the webapp? I don't see any links/info on the website point to the accessible XML text (https://cotr.ac.uk/). The only thing I can find is the json text api: https://cotr.ac.uk/api/texts/36/transcription/whole/whole/ which is obviously some kind of derivative export.

I would be extremely grateful for any thoughts, explanations, or clarification.

[geoffroy-noel-ddh]

Hi, the texts were encoded using a variant of XHTML which preserves some semantic attributes compatible with TEI.

We are indeed planning to export those texts as TEI in the coming months, although the markup will be very light. I can post an update here when that's available.

I believe the encoded texts are already very standardised (spelling, expansions, ...) as the focus of the project was on some specific types of differences in the wordings (the unsettled areas). There's more documentation about that on the website itself.

Hope this helps.

Geoffroy

[geoffroy-noel-ddh]

@jeffreycwitt I see from your github profile that you worked on a TEI web editor. You might be interested to know that we've customised TinyMCE web editor to let the research team collaboratively input and mark up their texts, which is saved automatically into the dabatase. This editor (which also works as a viewer, so they can preview their work any time) has been used on multiple projects. This web editor + relational storage approach has been a successful alternative to Oxygen + TEI files exchange.

[jeffreycwitt]

@geoffroy-noel-ddh Many thanks for your reply and helpful notes.

Sounds like the approach that has worked best for you guys is to use the web editor to allow people to save xhtml directly in a relational database.

In this case, any TEI files then would be, for you, "derivative exports".

The "source of truth" so to speak is always the xhtml fragments in the running database. (Do you worry at all about whether this means the long term storage of the "source of truth" depends on the running database rather than static files?)

Has the project departed then a little bit from the idea expressed in the grant to produce first and foremost a "Genetic TEI" edition:

We shall also pioneer new methods and models for working with medieval texts, whose content and
structure change over the long term. The concept of the 'genetic TEI edition' was introduced above. Working
with the resources and scholarly expertise of the Digital Lab of King's College London, this would be the
first project to extend the concept of a 'digital genetic edition' to a text, like Regiam, whose working
processes took place over centuries, involving multiple anonymous individuals and scribes

Have you found that the Oxygen + TEI file exchange has too steep a learning curve for scholars? Has that motivated the use of the web editor + relational storage? (i.e. is the latter method preferred by those working on the texts?).

Please feel no obligation to answer my questions, I'm just thinking out loud. But I'm definitely interested in your perspective.

[geoffroy-noel-ddh]

Hi Jeffrey,

That's correct, the TEI is derivative in this case. This choice is made on a case by case basis; in other projects we might favour the traditional Oxygen + TEI file as primary source; mainly when the schema is more dense or advanced. Both approaches have pros and cons obviously.

Indeed, the project departed slightly from the grant description. It was mainly a conceptual shift as we gradually learned more about the nature of the edition we wanted while analysing the texts and their differences. That type of evolution is typical of a research project. Hence the distinction made later by the researchers between a genetic edition and dynamic edition. That said our XML schema is based TEI constructs and semantic.

This project finishes this year and, by the end, the edition should be stable and will be stored as XHTML and derivative TEI on a long term archival solution (e.g. Github or Figshare). So to answer your question, the ground truth will move from the DB records to a file format for sustainability and public sharing purposes.

Regarding your last question, as I said in the beginning, it is a case by case decision that depends on the ability and preferences of the research team, the level of collaboration, the density and complexity of the markup, ... But the web editor approach, when feasible and adequate to their needs, is usually much preferred by scholars. The key advantages I have observed are: less prone to errors (in markup, and file exchange); much easier to learn; better for close collaboration; and usually better integrated with the rest of the editorial and publishing environment (e.g. immediate preview the edition exactly as it will appear to the public on the site). Another key advantage is that it is excellent for linking up relational records with the text (e.g. looking up a person or place record from editor and adding a reference in the XML); or storing extensive manuscript metadata in relational tables rather than in XML.

There are also many drawbacks to that approach: not convenient for rich or nuanced markup, need to maintain a semantic-preserving conversion from TEI to XHTML (and risks of irreconcilable divergences), editorial features are nowhere near what Oxygen offers, ...

I hope that answers your questions!

Geoffroy

[geoffroy-noel-ddh]

Another point I'd like to add is that, strangely enough, using a limited editor can be beneficial for larger projects with high number of texts or many participants. Although we always design a schema from the start based on the research needs I have noticed that scholars tend to be seduced by the vast array of tags and constructs offered by TEI. With Oxygen we can constrain those possibilities by imposing a schema. With a Web Editor, the tag is simply unavailable unless there is a button for it. So new TEI tags or attribute can't be used unless the team decide to add it.

This is a burden. But it is also a great way to enforce consistency across a corpus and avoid unnecessary digressions (which would cause issues in the processing down the line if one editor forget to communicate their use of new elements to the rest of the team). I believe scholars do enjoy that sense of focus.

[jeffreycwitt]

That's so much. It is very generous of you to offer me such a detailed response.

I definitely agree about the "seduction by the vast array of tags" that come with TEI. I have found that severely constraining the set of available tags has been a key part of finding success at scale.

Many thanks again for your help. :)