By Eric Kansa, Technical Director, Open Context and Emily Cunningham, Professional Fellow, Arches Team at the Getty Conservation Institute
The initial development of Arches in 2012 was informed by a great deal of prior experience and learning. Arches is an open-source platform that has to date been deployed by over 120 organizations and projects worldwide.
Arches itself was informed by data management work initiated by a prior collaboration between the Jordanian Department of Antiquities (DOA), the Getty Conservation Institute (GCI), and World Monuments Fund (WMF) to develop Jordan’s Middle Eastern Geodatabase for Antiquities (MEGA-Jordan). The DOA has utilized MEGA-Jordan to maintain a comprehensive inventory of culturally and historically significant archaeological sites and monuments across the country.
The landing page and legacy interface of MEGA-Jordan’s website, megajordan.org
MEGA-Jordan now records over 15,000 sites and over 54,000 site components. It tracks condition monitoring activities and helps inform conservation decision-making for Jordan’s unique, deep, and incredibly rich archaeological heritage. However, MEGA-Jordan is now over 14 years old and it needs an update. This sets the stage for the histories of MEGA-Jordan and Arches to become intertwined yet again!
The Arches team at the GCI recently began work through a partnership between the GCI, the Jordanian Department of Antiquities and the University of Oxford to help support a new effort to migrate data from MEGA-Jordan into a current version of Arches.
The migration process is providing the Arches team the opportunity to review and enhance the MEGA-Jordan data. As a part of this effort OpenRefine is being used to reconcile values against established name authorities, such as the Getty Thesaurus of Geographic Names (TGN), as well as to deduplicate person and organization records. A record of all previously used spellings is being maintained so that the values can be stored as alternative terms, which will be indexed to enhance search capabilities. Since MEGA-Jordan was designed to support both English and Arabic, special attention is being given to the handling of Arabic names for individuals and organizations.
Mapping MEGA-Jordan data to Arches resource models and graphs is an ongoing task. The team is working to transform the data from its original relational database structure into a linked data environment, ensuring it aligns with Arches semantic framework and capabilities.
Tips for Migration
Migrating data from a legacy database like MEGA-Jordan is complicated because data must be transformed and enriched to take advantage of the semantic capabilities of Arches. Here are some lessons and strategies that have been helpful during the MEGA-Jordan migration process:
Going from “strings” to “things”
It is helpful to think of the migration process as a way to turn “strings” into “things.” Extracting named entities from free-text values (“strings”) in the legacy data and modeling them as distinct, meaningful entities (“things”) allows for more detailed semantic modeling, which enhances querying capabilities, visualizations, and integration with other datasets.
For example, if a survey was carried out by “John Smith, Department of Antiquities,” we would separate that value into two distinct entities: the person, John Smith, and the organization, Department of Antiquities. Then, we can describe the entities and the relationship between them in a way that is both machine- and human-readable. This method supports long-term use and reuse of the data across platforms and projects, making it more sustainable and interoperable.
This can be time consuming, because a given “thing” can be named with different capitalizations, acronyms, spelling variations, etc. It can often require domain knowledge and attention to detail to organize “strings” into “things”!
Assign UUIDs before importing to Arches
Generating unique identifiers (UUIDs) for resources before importing them into Arches (as opposed to letting Arches generate UUIDs during the load process) helps with troubleshooting and makes it easier to reference specific records during and after migration. The pre-generated UUIDs are preserved during the load process, ensuring consistency throughout.
Tracking alternative names
Arches supports attributes with multiple values, so it is easy to store and retrieve alternative names (such as nicknames or common misspellings). This is especially useful for handling transliterations of Arabic names into Latin characters, which can vary greatly, and improves searchability.
Expect a labor-intensive process
Successful migration requires deep familiarity with the source data and the Arches resource models. The process of massaging the data into a new structure requires time and expertise, but it will lead to better-structured, more expressive, and ultimately more powerful data. This enhanced data is easier for both humans and machines to query. The added context offered by semantic data can even enable users who leverage AI tools to ask more precise questions and receive more accurate answers.
We will continue to share updates as the work progresses, and we hope these early lessons are helpful to others navigating similar efforts. If you’ve tackled a migration of your own, we’d love to hear more about your process over in the Arches Community Forum or through community contributed blog posts.
