From Person Mentions to Person Reconstructions: A Transparent, Provenance-Centred Entity Resolution Pipeline for the Gouda Time Machine
Bob Coret, ORCID 0000-0002-7328-1312
Draft v0.4
Abstract
Local time machine projects aim to reconstruct the historical development of a city from heterogeneous archival sources. For the person dimension of such a reconstruction, the central obstacle is that historical individuals appear as scattered, unlinked mentions: a line in a census, a name in a civil certificate, an abbreviated entry in an address book. This article presents the design, implementation, and evaluation of the person reconstruction pipeline of the Gouda Time Machine, which resolves 1,636,040 person observations from two structurally different linked data sources - the archival collections of Streekarchief Midden-Holland and a harvest of mentions of Gouda-born individuals in other Dutch archives - into 650,176 person reconstructions conforming to the Persons in Context (PiCo1) model. The pipeline combines multi-pass blocking, a deliberately transparent hand-weighted scoring model with hard rejection rules, constrained union-find clustering with cluster-level consistency guards, and a post-clustering cohort-splitting step that separates same-name individuals from different generations. Every reconstruction carries full PROV-O provenance2, including an empirically calibrated confidence score per constituent observation, and reconstruction URIs remain stable across reruns through a persistent identifier register. Calibration with stratified samples judged by independent large language model annotators, validated internally against pairs known to be distinct, indicates a volume-weighted precision of approximately 95% at the operational threshold, with 73.6% of all mentions absorbed into multi-member reconstructions. Version 2 of the pipeline additionally exploits 2,578 user-contributed family trees from the Genealogie Online platform as a strictly bounded auxiliary signal: tree persons never become reconstruction members, but deduplicated tree profiles that anchor two archival observations to the same tree person contribute a capped bridge feature to the pairwise score. A controlled A/B experiment shows that this signal promotes 47,537 additional pairs above the acceptance threshold while raising volume-weighted precision from 95.4% to 96.1%. We discuss why a transparent weighted-sum model was preferred over probabilistic and learned alternatives, report a negative result for unsupervised Fellegi-Sunter estimation in a blocked candidate space, and argue that calibrated, filterable confidence combined with human-in-the-loop review offers a workable template for person reconstruction in other local time machine projects.
Keywords: record linkage, entity resolution, person reconstruction, linked open data, PiCo, provenance, time machine, historical demography, digital heritage
1. Introduction
The time machine paradigm in digital heritage aspires to reconstruct the past of a city as a navigable information space, connecting buildings, parcels, streets, and people across centuries of archival documentation (Kaplan, 2015). The Gouda Time Machine, a collaboration between the regional archive Streekarchief Midden-Holland (SAMH) and the historical society Die Goude, publishes the sources of the city of Gouda as linked open data: censuses, civil registration, population registers, address books, tax registers, burial records, and more, enriched and transcribed largely by volunteers.
For buildings and parcels, the connecting tissue of such a reconstruction is location. For people, no such natural key exists. The same historical individual surfaces in dozens of sources under varying name forms - "Dirk Gijsbertus van Vreumingen" in an 1879 marriage certificate, "D. G. van Vreumingen" in the address book of 1902 - and nothing in the data states that these mentions belong together. Without resolving this, a time machine can show records but not lives: no life courses, no family networks, no connection between a census line and a burial entry.
This article describes the pipeline that performs this resolution for the Gouda Time Machine. Its input is 1,636,040 person observations from two source families: the linked data dump of the Gouda Time Machine itself (1,053,381 observations, SAMH sources) and a harvest from the aggregation platform Open Archieven of Gouda-born individuals appearing in archives other than SAMH (582,659 observations). Its output is a set of 650,176 person reconstructions - one node per presumed historical person - published as RDF conforming to the Persons in Context model (PiCo; Woltjer et al., 2024), in which 1,204,244 mentions (73.6%) are grouped into 218,565 multi-member reconstructions and the remainder are retained as single-member reconstructions with stable identifiers.
The contribution of this article is threefold. First, we document an end-to-end, reproducible architecture for municipal-scale person reconstruction over heterogeneous linked data sources, running on a single machine in about thirty minutes, and we motivate each design decision against explicit functional and quality requirements. Second, we describe several techniques that, to our knowledge, have not been reported in combination in the historical record linkage literature: cluster-level consistency guards inside a union-find merge procedure, a post-clustering cohort-splitting step against generation conflation, a widow-aware scoring mode for sources that record widows under their husband's name, and a persistent identifier register that keeps reconstruction URIs stable across pipeline reruns with documented tombstones. Third, we present a pragmatic evaluation methodology in which stratified samples are labelled by independent large language model (LLM) annotators and cross-checked against an annotator-free internal validation set of pairs that are guaranteed to be distinct persons, yielding calibrated confidence scores that downstream consumers can filter on.
Throughout, we emphasise a design stance that we believe is essential in a heritage context: the pipeline never modifies the source data, never invents persons, exposes the reasoning behind every match as a feature-level breakdown, and records its own uncertainty and its own contradictions (rejected merges, disputed parent relations) as first-class output. Section 2 situates the work in the literature; Section 3 describes the source data; Section 4 presents the requirements and the conceptual model; Sections 5 to 7 walk through the pipeline; Section 8 documents the output model; Section 9 presents the evaluation; Section 10 describes the human-in-the-loop review component; Section 11 discusses limitations and lessons; Section 12 concludes.
2. Related Work
2.1 Record linkage and entity resolution
The formal foundation of record linkage is the probabilistic framework of Fellegi and Sunter (1969), which models the decision to link two records as a likelihood ratio over field-level agreement patterns, with match (m) and non-match (u) probabilities estimated from data. Subsequent decades produced a rich toolbox of approximate string comparators (Jaro, 1989; Winkler, 1990), phonetic encodings such as Double Metaphone (Philips, 2000), blocking and indexing techniques to tame the quadratic comparison space (Christen, 2012a), including the sorted neighbourhood method of Hernandez and Stolfo (1998), and clustering formulations of entity resolution in which pairwise decisions are reconciled into a consistent partition, optionally under must-link and cannot-link constraints (Wagstaff et al., 2001). Christen (2012b) provides a comprehensive overview. Recent work in economic history has systematically compared automated linking methods on historical census data and highlighted the trade-off between match rates and false positive rates (Abramitzky et al., 2021).
The pipeline described here is best understood as a hand-calibrated, transparent variant of the Fellegi-Sunter tradition: a weighted sum of nameable features with explicit thresholds, augmented with hard rejection rules and cluster-level constraints. We return in Section 11 to an empirical comparison with a proper Fellegi-Sunter estimation on our data.
2.2 Historical person linkage in the Netherlands
The Dutch research infrastructure has a long tradition of large-scale historical person linkage. The LINKS system (Mandemakers et al., 2023) links the indexed civil certificates of the Netherlands (1812 onwards) into family reconstructions, using rule-based matching over name variants and family roles, and underpins the Historical Sample of the Netherlands. Its designated successor burgerLinker (Raad, 2021), developed within the CLARIAH infrastructure, reimplements candidate generation as a fast Levenshtein-automaton search over civil certificates modelled in a purpose-built Civil Registries schema, with recent versions producing output conformant to PiCo. The Golden Agents project developed Lenticular Lens, an interactive tool for linking persons across heterogeneous sources with user-defined matching logic and recorded provenance (Idrissou et al., 2022). The edited volume of Bloothooft et al. (2015) collects methodological work on population reconstruction from this community, including multi-source entity resolution for genealogical data.
Our work differs from LINKS and burgerLinker in scope and shape. Those systems link one structurally homogeneous source type - civil certificates with fixed role structures - at national scale, optimising candidate recall and leaving precision decisions partly to the user. The Gouda pipeline instead links a deliberately heterogeneous, municipal-scale corpus (certificates, population registers, censuses, address books with initials only, tax registers with institutional entries, burial records, and externally harvested mentions), which forces the matcher to reason over attestation years, register periods, address evidence, and abbreviated name forms rather than over certificate structure alone. It also differs in its output commitments: calibrated per-member confidence scores, stable reconstruction URIs across reruns, and machine-readable tombstones - requirements that arise when reconstructions are published as citable linked data rather than consumed as a research dataset.
2.3 Linked data models for historical persons
The Persons in Context model (Woltjer et al., 2024) standardises the crucial distinction, long implicit in genealogical practice, between a person observation - one attestation of a person in one source, immutable and source-faithful - and a person reconstruction - the falsifiable claim that a set of observations refers to the same historical individual. PiCo reuses existing vocabularies: the Person Name Vocabulary (PNV3) for name constituents, schema.org for factual person properties, and PROV-O2 for derivation and activity provenance. Records in Contexts (RiC-O4) describes the archival context of record sets. The Gouda Time Machine publishes its person data in PiCo, and the pipeline described here consumes and produces PiCo without introducing a new data model, adding only a small project namespace (gtm:) for confidence scores, run parameters, and dispute flags. To our knowledge, this pipeline is among the first production systems to exercise the full PiCo observation-to-reconstruction cycle at scale, including qualified derivations carrying calibrated confidence.
3. Source Data
3.1 The SAMH linked data dump
The primary source is the N-Triples dump5 that the Gouda Time Machine publishes from its Omeka S collection management system (goudatijdmachine.nt, 5.4 GB, approximately 35 million triples), containing 1,053,381 resources typed pico:PersonObservation. These observations originate from indexed civil registration (births, marriages, deaths), population registers, censuses of 1830 and 1840, address books (1867-1906), the 1806 verponding property tax register, church marriage registers (1771-1795), burial records, the cadastral OAT of 1832, and a long tail of smaller sources (office holders, biographical dictionaries, war graves, and others). Name constituents are modelled in PNV, roles via the Thesaurus of Historical Person Data6, family relations as schema:spouse, schema:parent, and schema:children between observation URIs (identified by Archival Resource Keys), and each observation points via prov:hadPrimarySource to its source scan and, for civil registration, to the certificate resource.
The heterogeneity of this corpus is the central challenge. Civil certificates carry day-precise dates and explicit family roles; censuses and population registers carry ages and household context but often only year ranges; address books list heads of household by initials and surname only; the verponding register mixes persons with institutional entries ("leprooshuis"); widows appear under their husband's name with an optional maiden name ("wed. L. C. Kint geb. van Dam"). Only 6,137 observations carry a recorded schema:gender.
3.2 The Open Archieven harvest
The second source, non-samh.nt.gz, contains 582,659 pico:PersonObservation resources harvested from the aggregation platform Open Archieven: mentions of Gouda-born individuals in the collections of archives other than SAMH (for example, marriage certificates of children of Gouda families registered in Rotterdam or Haarlem). These records are structurally different: schema.org fields sit directly on the observation, there are no collection item sets, blank node labelling conventions differ, and prov:hadPrimarySource points to the source record rather than to a certificate or scan. Birthplaces are literal place names rather than place URIs. The harvest comprises population register entries (412,866), civil registration (137,902), and smaller categories including militia registers, judicial records, memorial cards, and pre-1812 baptism, marriage, and burial registers.
The purpose of the second source is cross-archival closure: a person's Gouda records and their records elsewhere should meet in one reconstruction. As Section 9 shows, 47,183 reconstructions combine mentions from both source families.
3.3 The Genealogie Online family trees (version 2)
The third input is not an archival source but a derived one: the family trees that amateur and professional genealogists publish on Genealogie Online, available as PiCo RDF through a QLever SPARQL endpoint (~1 billion quads; 49M pico:PersonObservation resources in 6,294 trees, each tree a named graph with author, title, and URL). Phase 13 harvests the Gouda subset - every person whose birth, death, or life-event place is Gouda (GeoNames 2755420): 107,760 mentions in 2,578 trees - plus their one-hop relatives (spouse/parent/children/sibling, another ~202k persons harvested for family context). Because corpus-wide joins inside named graphs exceed the endpoint's memory, the harvest uses only single predicate-path queries and VALUES batches of five thousand URIs, with a per-query TSV cache. Dates are partly GEDCOM strings ("ABT 1828", "BEF 1850"); qualified dates deliberately yield estimated-year intervals rather than registered years, so that sloppy tree dates can never trigger a hard date rejection downstream.
Family trees copy from each other on a large scale - of the Gouda mentions with a name and day-precise birth date, roughly 30% is an exact copy of a mention in another tree - and a person appearing in twenty trees is not thereby better attested. Phase 14 therefore collapses the harvested mentions into tree profiles (107,760 mentions → 82,075 profiles) through two conservative union-find passes guarded by the same compatibility checks as clustering, and derives tree families: publications sharing ten or more deduplicated persons (or ≥ 25% of the smaller publication), or sharing an author, form one family - the 2,578 trees collapse to 848 families. A profile's independence is the number of tree families contributing to it, not the number of trees: the largest profiles occur in 26-27 trees that constitute a single copying family, and count as one independent attestation.
Crucially, tree persons never become members of a reconstruction (Section 7.1). Phase 35 links each profile, as a pseudo-observation, to archival observations using the full scoring model including every hard rejection rule; of 5.8M candidate links, 579k survive the linking threshold, anchoring 437k archival observations (27%) to at least one tree profile.
4. Requirements and Conceptual Model
4.1 Requirements
The design of the pipeline was steered by fourteen explicit requirements, functional and qualitative, summarised in Table 1. We list them in full because, in our experience, the qualitative requirements (transparency, precision over recall, calibrated confidence, stable identifiers, human reviewability) shaped the architecture at least as much as the functional ones, and because they may serve as a checklist for comparable projects.
Table 1. Requirements for the person reconstruction pipeline.
| # | Requirement | Summary |
|---|---|---|
| R1 | Group mentions | Person observations from heterogeneous sources are automatically grouped into person reconstructions: one node per presumed historical person. |
| R2 | Sources untouched | Observations remain source-faithful and immutable; the algorithm adds an interpretation layer and never modifies the sources. |
| R3 | No invented persons | A reconstruction arises only from actual observations. Singletons receive a one-member reconstruction so that every mention has a stable reconstruction URI; fully nameless mentions and institutional tax entries are excluded. |
| R4 | Full traceability | Every reconstruction points via provenance to its observations and to the run (version and threshold) that produced it. |
| R5 | Confidence per claim | Every observation-in-reconstruction carries an explicit confidence score and a weakest-link score, so that consumers can filter more strictly than the pipeline did. |
| R6 | Lifted relations | Family relations (partner, parent, child) are lifted from observation to reconstruction level, with contradictions flagged (disputed parents). |
| R7 | Transparent, no black box | The match decision is explainable: a weighted sum of nameable features with a per-pair feature breakdown; weights are hand-tunable. |
| R8 | Precision over recall | Rather a reviewable grey zone than an erroneous merge; hard rejections and merge guards prevent impossible and transitively overreaching merges. |
| R9 | Calibrated confidence | The confidence score is empirically calibrated, so that a threshold is a statement about precision rather than an arbitrary number. |
| R10 | Reuse standards | The pipeline consumes existing vocabularies (PiCo, PNV, schema.org, PROV-O, RiC-O) and delivers RDF (N-Triples); it invents no data model of its own. |
| R11 | Stable identifiers | Reconstruction URIs remain stable across reruns (persistent identifier register), with traceable tombstones on merges and splits. |
| R12 | Reproducible and modular | The outcome is deterministic; every phase is idempotent and independently rerunnable. |
| R13 | Humanly reviewable | Grey-zone pairs, disputed relations, and weakly connected clusters can be reviewed by humans through a web application; verdicts survive reruns and feed the next calibration. |
| R14 | Feasible on one machine | A full run fits in the memory of a single machine and completes in tens of minutes; thresholds and weights are configurable. |
4.2 Observations and reconstructions
The pipeline adopts the PiCo distinction between two entity types. A person observation (pico:PersonObservation) is one attestation of a person in one source: "D. G. van Vreumingen" in the 1902 address book is a different observation from "Dirk Gijsbertus van Vreumingen" in an 1879 marriage certificate, even if the same individual is meant. Observations are source-faithful and immutable (R2). A person reconstruction (pico:PersonReconstruction) is an interpretation: the claim that a set of observations refers to the same historical person. Reconstructions are traceable (every claim points back to its observations), dated (every run is a prov:Activity), and replaceable (a better matcher yields better reconstructions without touching the source data).
Three design principles operationalise the requirements throughout the pipeline. First, transparency over black boxes (R7): matching uses a weighted sum of nameable features rather than a learned model, and every candidate pair retains a feature breakdown showing which characteristics contributed how much, displayed integrally in the review application. Second, rather a grey zone than a wrong merge (R8): pairs with insufficient evidence are not merged but parked in a reviewable grey zone, and an observation that links to nothing becomes an honest single-member reconstruction rather than a forced match. Third, provenance everywhere (R4, R5): every reconstruction carries a per-member confidence score, every run a version and threshold statement, so that downstream consumers can apply stricter filters than the pipeline itself did.
5. Architecture and Data Preparation
5.1 Architecture
The pipeline is a sequence of independent Python scripts around a single DuckDB database, modelled after the "Aanknopingspunten v3" matcher7 of the genealogical platform Genealogie Online, which processes 76 million persons in roughly ninety minutes. At 1.64 million observations the entire problem fits comfortably in the memory of one machine; a full run takes about thirty minutes. Distributed infrastructure would add complexity without benefit at this scale (R14). DuckDB performs the heavy set-oriented work (joins, sampling, aggregation, Parquet I/O); per-pair scoring is a tight Python loop over C-backed libraries (rapidfuzz for Jaro-Winkler similarity, metaphone for Double Metaphone encodings). Bulk data transfer uses Apache Arrow tables registered directly into DuckDB, which proved roughly two orders of magnitude faster than row-wise insertion.
Figure 1. Pipeline phases, from the two source dumps to the published N-Triples and the review application. Counts and timings are those of the reported run.
Each phase is an idempotent script (extraction, normalisation, blocking, scoring, clustering, serialisation, reporting, calibration), orchestrated by a driver that can restart at any phase (R12). Intermediate products live in Parquet files or DuckDB tables; expensive external steps (a 1.2 GB gender lexicon dump, a twelve-minute lexicon aggregation) are cached. All configuration - paths, the acceptance and grey-zone thresholds, a blocking cap, and every scoring weight - resides in a single module and can be overridden through environment variables, so that, for example, a maximum-certainty run requires no code change.
5.2 Extraction
The SAMH dump is subject-sorted, so a single streaming pass suffices: lines are buffered per subject block and flushed on subject change, with the rdf:type triple deciding whether the block is a person observation. Two blank node conventions require care: PNV name-part nodes are joined to their observation through the collection system identifier (blank node labels are not stable across dumps), and auxiliary nodes carrying occupations, roles, and places are resolved in the normalisation phase. The parser extracts eight Parquet files - observation properties, name parts, family relations, resource titles, address edges, part-of links, and register period dates - in about forty seconds.
The Open Archieven harvest is handled by a separate adapter that streams the gzipped N-Triples file and normalises each person directly into the common observation schema, so that everything downstream of the merge point is source-agnostic. Three aspects deserve mention. First, the pipeline keys on an integer observation identifier, and identifier continuity (Section 8) depends on stable membership; since harvested records carry no such identifier, the adapter assigns synthetic identifiers from a persistent register in a reserved high range, so that the same mention keeps its identifier across re-harvests. Second, the harvested record URI is registered as the observation's certificate URI, so that the existing same-certificate rejection (two persons named in one record are by definition distinct) applies unchanged. Third, the register period of the source (for example "Bevolkingsregister 1874-1893") supplies an attestation year range, marks the source as a living-persons register, and, combined with a recorded age, yields a birth-year interval.
5.3 Lexicons
Three auxiliary tables are derived from the database of Genealogie Online and thus ultimately from genealogist-contributed family trees: a gender-by-given-name lexicon (73,967 given names with an F/M distribution, filtered at confidence above 0.75 and at least ten bearers, derived from 46 million gendered persons), roughly 8,000 undirected given-name synonym pairs (aagjeagatha, teunistheunis), and 458,036 symmetrised surname synonym pairs after normalisation ('s Gravemade~Schrama). A deliberate choice, learned from the Genealogie Online matching project, is to use synonyms only as direct pairs, without transitive closure: transitively closed name groups fuse into unusably large megaclusters.
5.4 Normalisation and enrichment
The normalisation phase builds the central observation table: one row per observation with every matchable characteristic in normalised form. The most consequential operations are the following.
Name parts. PNV fields take precedence; failing those, the family name is split with a Dutch prefix list ("van der Berg" yields prefix "van der" and core "berg"). All name material is lowercased and stripped of diacritics, and Double Metaphone codes are computed for the first given name and the surname core as phonetic blocking keys.
Initials. Records such as "D. G. van Vreumingen" are flagged as initials-only; full names receive derived initials ("Dirk Gijsbertus" yields "dg") so that both forms fall into the same blocking pass. Where the given-name field contains a stray surname token ("A. C. Visser" next to family name "Silvius", a double surname split inconsistently by the source), the token is moved to the surname; otherwise the derived initials would be wrong and the mention would co-block with nothing.
Dates with a precision flag. Distinguishing day-, month-, and year-precision (xsd:date versus xsd:gYear, and patterns such as "1809-10-00") is crucial for the conflict rules: a year-precise "1580" must never hard-conflict with "1580-05-01".
Attestation year chain. For sources without dates of their own, a registration year is derived along a priority chain: a year in the collection title ("Volkstelling Gouda 1830"); a publication date on the observation (address books); a period statement in the title of the source scan (civil registration, 645,000 observations); or the register period reached through the scan's part-of link to a RiC-O record set (population registers, 345,000 observations, ranges such as 1880-1900). This chain leaves only two of the 990,156 SAMH records without an attestation year or range.
Source classification. SAMH observations are classified from collection titles; harvested observations receive a source label from the PiCo source-type vocabulary of their record, with keyword rules over free-text labels for smaller categories, in a separate label namespace so that statistics remain legible.
Gender inference. For observations without a recorded gender and without initials-only names, the gender lexicon is consulted; above 75% lexicon confidence the observation receives an inferred gender, explicitly marked as inferred rather than recorded. Across both sources the large majority of the 1,636,040 observations obtain a gender (SAMH approximately 98%). The recorded/inferred distinction determines how severely a gender conflict is weighted in scoring: two conflicting recorded genders are a hard rejection, while conflicts involving inferred genders incur a penalty only.
Family context. The 263,000 partner and 379,000 parent-child observation pairs are denormalised into per-observation lists of partner given names, father and mother given names, and child given names, so that scoring can compare family context without per-pair joins.
Widow notation. Address books and registers record widows as "wed. L. C. Kint", sometimes with a maiden name: "wed. L. C. Kint geb. van Dam". The widow token is recognised (setting gender to female with a dedicated provenance flag), the initials are understood to be the husband's, and the maiden name is stored separately. Without this step, the notation variants of one widow acquire mutually incompatible name profiles and can never cluster.
Building location. Where the observation, its administrative designation, or its source scan is linked to a building geometry, the building's location point is attached to the observation, connecting mentions of the same building across sources.
6. Blocking
Comparing all pairs is infeasible (order 10^12 at this scale). Blocking restricts comparison to candidates sharing at least one key. Seven passes each target a different error model in the sources: (P1) phonetic surname plus phonetic first given name, for spelling variants; (P2) exact normalised surname plus first given name; (P3) surname plus birth year within one, for given-name variants; (P4) surname plus patronym, for early sources without dates; (P5) phonetic surname plus first initial, bridging initials to full names; (P6) surname-synonym pairs plus first given name, for surname synonyms that differ phonetically, emitted only when at least one side of the synonym pair is rare, since frequent pairs are already covered phonetically; and (P7) phonetic code per surname token plus first initial, for inconsistently split double surnames ("visser silvius", "sylvius visser", "silvius").
Blocks exceeding a cap of 300 members are hierarchically re-keyed through a salt chain (birth decade, then second given name, then patronym or street, then source; for initials, the second initial). Whatever remains oversized after the full chain - almost exclusively undated namesakes of high-frequency names such as "Johanna de Jong" - is not skipped but handled with a sorted neighbourhood window (Hernandez and Stolfo, 1998): sorted within the block on second given name, patronym, and street, only pairs within twenty positions are emitted, linear rather than quadratic in block size. Structural non-coverage is thereby zero. Alongside the passes, a small maiden-name bridge directly pairs widows carrying a "geb. X" maiden name with bearers of surname X: such pairs share no name key at all (the widow is recorded under her husband's surname), and the volume is small enough for a direct join. Over both sources, blocking yields 107.8 million unique candidate pairs (66 million for the earlier SAMH-only corpus).
7. Scoring, Clustering, and Cohort Splitting
7.1 Transparent pairwise scoring
Each candidate pair receives a score between 0 and 100 from a transparent weighted sum over nameable features (R7). Table 2 reproduces the full weight table; weights were set by hand and calibrated by sampling (Section 9). Jaro-Winkler similarity is the fuzzy string measure for names; estimated birth years are compared as intervals, with the gap defined as the smallest distance between intervals, so that a twenty-year register range does not generate false conflicts.
Table 2. Feature weights of the scoring model (excerpt of the configuration; positive weights reward agreement, negative weights penalise conflict).
| Feature | Points | Feature | Points |
|---|---|---|---|
| Given name exact | +25 | Birth year conflict (> 2 y) | -40 |
| Given name synonym | +22 | Death date day-exact / year | +20 / +8 |
| Given name JW >= 0.92 / >= 0.85 | +18 / +10 | Death year conflict | -30 |
| Given name hard conflict | -25 | Birthplace URI equal / unequal | +10 / -12 |
| Second given name conflict | -10 | Birthplace name string equal | +6 |
| Initials subset of given names | +15 | Gender conflict (incl. inferred) | -40 |
| Initials both, equal / prefix | +12 / +6 | Age-derived year fits (+-2) | +6 |
| Initials conflict | -15 | Age-derived conflict (> 5) | -10 |
| Surname exact | +25 | Partner given-name overlap | +20 |
| Surname synonym | +20 | Father / mother given name equal | +15 / +15 |
| Surname JW >= 0.92 / >= 0.85 | +18 / +8 | Parent name overlap (gender unknown) | +15 |
| Surname tokens all matched / subset | +18 / +18 | Child name overlap (Jaccard >= 0.5) | +10 |
| Surname conflict (JW < 0.75) | -30 | Same street URI + house number / street only | +12 / +6 |
| Patronym equal / conflict | +12 / -8 | Occupation equal | +7 |
| Birth date day-exact | +30 | Second-order family bonus | +15 |
| Birth year equal / +-1 | +12 / +6 | Widows of the same man (name + initials) | +20 |
| Same building location point | +10 | Widow maiden name equal / conflict | +15 / -20 |
| Widow maiden name = other's surname | +30 |
Occupations are compared case-insensitively after splitting on separators, with the many notational variants of "no occupation" collapsed into one canonical token, while genuine occupations that merely contain such words ("Vragtschipper zonder vast verblijf") are left intact.
Hard rejections precede the weighted sum; these pairs receive no score but a rejection reason. Table 3 lists the rules with their volumes in the reported run. Two rules merit comment. The rule that both birth dates are day-precise and incompatible tolerates likely transcription errors - same year with month or day off by one, or day and month transposed - which are scored neutrally so that name and birth year evidence can still carry the match. The two timeline cross-checks (nobody is attested in any source before their own estimated birth year; nobody dies before they are born, checked across the pair as death of A against birth of B) catch a failure mode that field-against-field comparison misses: name reuse after child mortality, where a deceased child's name is given to a later sibling.
Table 3. Hard rejection rules with pair volumes (run of 5 July 2026).
| Rule | Pairs rejected | Rationale |
|---|---|---|
| Both birth dates day-precise and incompatible | 14.1 M | Day precision is the strongest source of truth; probable transcription errors are scored neutrally instead |
| Both death dates day-precise and incompatible | 429 k | As above |
| Period impossibility | 2.85 M | Attestation in a living-persons source (address book, census, population register) outside the candidate's lifespan |
| Attested before birth | 11.3 M | Universal timeline cross-check, margin 1 year (recorded) / 5 years (age-estimated) |
| Died before born (cross-check) | 10.8 k | Death of A precedes birth of B: catches name reuse after child mortality |
| Generation gap | 41.5 k | Family-context overlap plus at least 18 years between birth-year intervals: the father-son naming pattern |
| Explicitly related | 25.2 k | The two observations are each other's parent, child, or partner according to the source |
| Gender conflict (both recorded) | 1,674 | Hard only for two recorded genders |
| Same certificate or record | 450 | Two persons in one certificate (bride and groom) are never the same |
Two source-specific scoring modes address systematic notation patterns. For initials, prefix-compatible initials ("dg" within "dirk gijsbertus") score +15, which together with an exact surname (+25) yields 40 - grey zone, no automatic merge. But if the attestation year of the initials-only mention falls within the recorded lifespan of a full-name candidate (address books and voter lists record living adults), a period-fit bonus (+18) lifts the pair to 58, above threshold. This is precisely the founding use case: "D. G. van Vreumingen" (address books 1875-1903) links to "Dirk Gijsbertus van Vreumingen" (born 1842, died 1907). As protection against frequent names, if more than three distinct full given-name variants compete for the same initials-surname combination, all involved pairs are demoted to the grey zone as ambiguous.
For widows, scoring is widow-aware. Two widow mentions of the same man (identical husband name and initials, +20) reach 57 with surname (+25) and equal initials (+12), which is how notation variants of one widow cluster; an agreeing maiden name confirms (+15), while a conflicting maiden name - successive wives of the same man - penalises (-20). When exactly one side is a widow with initials, given-name and initials evidence is skipped entirely (the initials are the husband's and say nothing about the woman herself), and her maiden name may match the other observation's surname (+30), replacing the regular surname comparison that would wrongly count "Kint" against "van Dam" as a conflict. That +30 alone remains below the grey zone; with building or address evidence it reaches the review queue.
After the first scoring round, a second-order round adds +15 to a pair (A, B) whenever the partners (or parents, or children) of A and B themselves form an accepted pair. Families thus reinforce each other - in our experience the strongest signal in genealogical data, consistent with the design of certificate-based linkers such as LINKS and burgerLinker.
The namesake guard and its blind spot. Family context is weak evidence on its own. Dutch naming conventions name sons for fathers and draw given names from a small pool, so a father and son whose wives share a first name agree on name and on family context alike. The scoring model therefore rejects a pair outright when it shows family-context agreement and a generation-sized gap between the two birth-year intervals. That rule requires a birth year on both sides, and is silently inoperative when either side lacks one - which is exactly the population it exists to catch: identical name plus shared spouse given name and no date at all scores 70 on the weights above, filling the 60-80 range at 54% precision (Section 9). Where birth years are unavailable we therefore fall back to the source year as the only remaining time axis. Within that population the median source-year distance is 13 years for pairs an annotator judged identical and 58 years for pairs judged distinct; beyond 40 years apart only 23% are correct. A gap of 30 years or more incurs a penalty of 20 points rather than a rejection, which moves the dominant combination from 70 to 50 - into the reviewable grey zone rather than out of the data, since roughly a quarter of these pairs are genuine. The penalty is applied after second-order evidence has been resolved, so a pair whose spouses themselves match strongly is exempt, as is any pair carrying a date. In the reported run it fired on 212,284 pairs.
Two thresholds partition the outcome: pairs scoring at least 55 become cluster edges; 40 to 55 is the grey zone, reviewable in the web application but not counted in reconstructions; below 40 the pair lapses. The operational threshold was lowered from 60 to 55 after the second calibration round, when the 55-60 band measured 98% precision following the introduction of the timeline cross-checks; it measures 99% in the current calibration. The reported run yields 5.92 million accepted edges and 16.9 million grey-zone pairs.
The tree signal (version 2). After the second-order round, the tree-bridge round adds the family-tree evidence of Section 3.3. If a tree profile links archival observations A and B to the same tree person, the pair receives exactly one of three capped features - never stacked, and deliberately small because user-contributed trees are less reliable than archival sources: tree_bridge (+8, profile from a single tree family), tree_bridge_multi (+12, profile from two or more independent tree families - copies do not count), or tree_bridge_day (+15, additionally requiring both profile links to be day-anchored on an exact birth or death date). A fourth feature, tree_family_context (+6), fires when the tree's family context fills a gap in the archival family context of the other side. Four safeguards keep the signal subordinate to archival evidence: (i) a tree-only clamp - a pair that would cross the acceptance threshold through tree features alone, without any date feature or second-order relation, is clamped into the grey zone for human review (1.40M pairs in the current run); (ii) candidate injection of profile bridges that blocking never generated (a baptism and a burial seventy years apart share no blocking key) is restricted to bridges whose both links are day-anchored, and injected pairs pass every hard rejection plus the source-year generation guard - without the day-anchor requirement, dateless name+spouse injections were observed to chain same-name individuals into homonym clusters; (iii) contradictory tree data never rejects an archival pair - the tree can confirm but not deny; and (iv) profiles linking more than 80 archival observations are considered ambiguous and do not bridge.
7.2 Constrained clustering with merge guards
Naive connected components over accepted edges would produce transitively linked chains: A matches B and B matches C, therefore A equals C - even where A and C exclude each other. The clustering phase therefore runs a union-find in which edges are processed in descending score order and every prospective merge is checked against the composite cluster profile. Table 4 lists the guards with the number of merges each blocked in the reported run.
Table 4. Merge guards with blocked merge counts (run of 9 August 2026, both sources).
| Guard | Merges blocked |
|---|---|
| Cluster size would exceed 80 | 15,479 |
| Incompatible day-precise birth dates in the composite (transcription errors tolerated) | 362,206 |
| Incompatible day-precise death dates | 71,658 |
| Recorded birth years more than 11 years apart (namesake conflation) | 446 |
| A member recorded as born after another member's death (margin 1 year) | 4,856 |
| Cohesion: subclusters of more than 3 members merge only on an edge >= 75, or on >= 2 independent cross-edges - since version 2 counted on the tree-free score: copy-based tree evidence is not independent corroboration and can never carry a large-cluster merge on its own | 66,481 |
| Two members from the same certificate or harvested record (censuses exempt) | 33,343 |
| Two distinct birthplace URIs | 16,448 |
| Gender conflict | 128 |
The two birth-year sanity guards operate at cluster level rather than pairwise and are therefore robust against chains through undated mentions: one person cannot have recorded birth years decades apart, and nobody in a cluster is born after that cluster's death. These guards catch the namesake conflations - grandfather and grandson of the same name connected through undated mentions - that became visible after the cluster size cap was raised from 40 to 80 (the raise itself being motivated by hundreds of well-documented individuals, attested in address books across four decades, that the lower cap truncated).
7.3 Cohort splitting
A subtler conflation escapes the merge guards. Two same-named individuals from different generations - for instance Johannes Wiezer, a dock worker born around 1807, and Johannes Hendricus Wiezer, a pipe maker born in 1871 - become connected through undated "Johannes Wiezer" bridge mentions. At pair level this is inseparable from legitimate matches: the abbreviated given name scores exact, the missing second name is neutral, and the family-context bonus lifts the bridge over the threshold; tens of thousands of correct abbreviated-name links follow the same pattern. The distinction is visible only at cluster level, as two widely separated birth cohorts.
A post-clustering pass therefore splits every cluster whose effective birth years (recorded, else age-estimated) span more than 12 years and decompose into cohorts (gaps larger than 8 years) each containing at least two dated members. Undated bridge mentions are assigned by label propagation to the cohort they are most strongly connected to. The 8-year gap is safe because the age-estimated birth years of one person lie within 5 years of each other in 95% of cases; the requirement of two dated members per cohort prevents a single noisy outlier from triggering a split. The Wiezer reconstruction was thereby cleanly separated (dock worker cohort 1805-1815 versus pipe maker cohort 1868-1872, with the correct occupations on each side), as were nearer-generation conflations such as a "Johannes Petrus Spruijt" reconstruction that proved to contain three individuals - a bargeman's mate (around 1798), a clay treader (1818), and a coppersmith (1829) - held together by undated "Jan/Johannes Spruijt" mentions and betraying itself through an impossible canonical profile (an occupation attested in 1825 against a birth year of 1818). The reported run split 1,613 such generation conflations; splitting requires positive evidence of multiple cohorts, so abbreviated-name links between mentions of one person remain intact.
7.4 Finalisation
Observations that join no cluster become one-member reconstructions with their own stable URI (R3), excepting fully nameless mentions and institutional tax-register entries; a singleton has no internal pair scores and consequently no confidence statement. For multi-member reconstructions, each member's confidence is the mean pairwise score from that observation to its co-members, and a weakest-link score records the member's weakest connection - a cluster of five observations in which one hangs by a single 62-point edge makes that fact explicit (R5). Canonicalisation selects the most precise date, takes name constituents from the member with the fullest given-name form (initials never become canonical when a written-out name is available), and unions occupations. Family relations are lifted from observation to reconstruction level with support counts; a consistency rule permits at most one father and one mother reconstruction per child, with supernumerary candidates flagged as disputed parents (10,879 cases), a quality signal with its own review queue (R6). Self-references, where a cluster is its own parent, mark the cluster as suspect.
The reported run (11 July 2026, threshold 55, both sources) produced 650,176 reconstructions: 218,565 multi-member reconstructions absorbing 1,204,244 observations (73.6% of all 1,636,040 mentions) and 431,611 single-member reconstructions, with only 185 mentions receiving no reconstruction (86 nameless, 99 institutional). Multi-member reconstructions average 5.5 observations. The cross-source effect - the purpose of the second source - is substantial: 47,183 reconstructions contain both a SAMH and a harvested mention (148,881 harvested observations thereby attach to a SAMH person), against 102,430 SAMH-only and 68,952 harvest-only multi-member reconstructions.
8. Output Model
The reconstructions are published exclusively as N-Triples (17.1 million triples, syntax-validated), conforming to PiCo with PROV-O provenance in the same style as the source data (R10). Listing 1 shows a representative reconstruction.
Figure 2. One reconstruction of Dirk Gijsbertus van Vreumingen, with a qualified derivation per constituent observation. The address-book mention scores 45 and is held in the grey zone rather than merged.
Listing 1. A person reconstruction with qualified provenance (abridged).
<https://www.goudatijdmachine.nl/id/reconstructie-7c3a9f21-...>
a pico:PersonReconstruction, schema:Person ;
schema:name "Dirk Gijsbertus van Vreumingen"@nl ;
pnv:hasName <...#name> ; # givenName / baseSurname / surnamePrefix
schema:birthDate "1867-05-02"^^xsd:date ; # most precise among the members
schema:birthPlace <https://www.gemeentegeschiedenis.nl/gemeentenaam/Gouda> ;
schema:gender schema:Male ;
schema:hasOccupation "Sigarenfabrikant" ; # union over the members
prov:wasDerivedFrom <https://n2t.net/ark:/60537/b...> ; # one per member
prov:qualifiedDerivation <...#d-427> ;
schema:spouse <.../another-reconstruction> ;
prov:wasGeneratedBy <.../id/reconstructie/run/2026-07-05> .
<...#d-427> a prov:Derivation ;
prov:entity <https://n2t.net/ark:/60537/b...> ;
gtm:confidence "0.85"^^xsd:decimal ; # mean edge score / 100
gtm:minEdgeScore "0.80"^^xsd:decimal . # weakest link
Three design decisions structure this output. First, confidence as qualified derivation: the score sits on the connection between reconstruction and observation - modelled as a prov:Derivation with hash URIs rather than blank nodes - not on the observation itself, since the same observation may sit in a different reconstruction with a different confidence after the next run. Second, run provenance: each run is one prov:Activity carrying the software version and the operational threshold, so consumers know under which regime the reconstructions arose (R4). Third, stable URIs through a persistent register (R11): every reconstruction receives a UUID-based URI, and a register that survives reruns stores the membership per UUID. Each run matches its clusters against the register on mutual best membership overlap with deterministic tie-breaking. On a split, the largest fragment retains the UUID; on a fusion, the largest contributor wins and the losers receive a tombstone (dcterms:isReplacedBy in a dedicated retirement file); vanished reconstructions are invalidated with prov:invalidatedAtTime. An identical rerun demonstrably inherits 100% of its UUIDs.
Two by-products of the pipeline are independently useful enrichments of the source data at observation level: a file with 645,249 exact registration years (schema:datePublished) and 344,905 register period ranges (schema:temporalCoverage) for SAMH mentions, derived through the attestation-year chain of Section 5.4; and a file with 1,029,403 inferred genders (schema:gender) from the given-name lexicon at over 75% confidence.
9. Evaluation
9.1 Calibration methodology
The confidence score is calibrated against a stratified sample: one hundred random pairs per score band, drawn by reservoir sampling with a fixed seed, each band judged independently by a separate LLM annotator under a strict instruction to answer "same" only on genuine evidence and "uncertain" in doubt. Using one annotator per band prevents cross-band leakage; using eight independent annotators prevents a single model conversation from drifting. All eight annotators were instances of the same model, Claude Opus 5 (claude-opus-5), each in a fresh context receiving only the verbatim instruction reproduced in Appendix A together with the two observations as JSON; the pipeline's own score, band, and feature breakdown were withheld, without which the precision measurement would be circular. Sampling is deterministic - DuckDB reservoir sampling with seed 42 - so the evaluated pairs are exactly reproducible, but the annotation is not: no temperature or seed was fixed, and indeed neither is available, since this model generation removed the sampling parameters from the API and the API exposes no seed. Re-running the annotation therefore reproduces the sample but not the individual verdicts, and per-band figures should be read with that in mind alongside the sampling error of roughly ±4 to 9 points at n = 100.
LLM annotation of record pairs is itself an evaluation choice that requires validation. We therefore added an internal validation that requires no annotator at all: pairs of persons co-occurring in the same certificate (bride and groom, deceased and informant) are guaranteed distinct individuals. The check is only sound where deed_uri genuinely identifies a certificate listing different people, which holds for harvested civil registration (97.1% of those pairs carry different names) but not for the censuses, where deed_uri links the same person across the 1830 and 1840 enumerations (66.7% carry identical names); the censuses are therefore excluded from the metric. On harvested civil registration, 0.05% of these guaranteed-distinct pairs would still score 55 or higher in the absence of the same-certificate rule - a stable, annotator-free upper bound on the false-positive pressure of the scoring model, and a sanity check that any revision of the model must pass. Applying the metric to the undifferentiated population of certificate pairs instead yields 0.15%, but that figure is diluted: 93% of those pairs come from population registers, where nearly every pair is already rejected on a day-precise birth date.
9.2 Precision per score band
Table 5. Precision per score band (calibration of 9 August 2026, round 3, judged blind, after the source-year generation guard; n = 100 per band, sampling margin roughly +-3 to 5 percentage points).
| Band | Same | Different | Uncertain | Precision (a) | Strict (b) | Edge volume in run |
|---|---|---|---|---|---|---|
| 40-45 | 2 | 83 | 15 | 2% | 2% | (grey zone) |
| 45-50 | 4 | 63 | 33 | 6% | 4% | (grey zone) |
| 50-55 | 21 | 34 | 45 | 38% | 21% | (grey zone) |
| 55-60 | 77 | 1 | 22 | 99% | 77% | 1 091 k |
| 60-70 | 75 | 10 | 15 | 88% | 75% | 505 k |
| 70-80 | 77 | 7 | 16 | 92% | 77% | 306 k |
| 80-90 | 85 | 6 | 9 | 93% | 85% | 3 213 k |
| 90+ | 93 | 1 | 6 | 99% | 93% | 808 k |
(a) same / (same + different). (b) uncertain counted as error.
Figure 3. The score distribution is bimodal, and precision is weakest where the distribution is thinnest. Bar height is the number of candidate pairs per band; the label above each bar is the measured precision of Table 5.
The distribution is strongly bimodal: below 55 almost exclusively non-matches (the grey zone does its work), above 80 almost exclusively true matches. The error analysis of this round directly produced two of the hard rejections of Section 7.1 - the universal attested-before-birth check and the cross-pair died-before-born check - together with the second-given-name conflict penalty. Rescored on the same 800 labelled pairs, these rules reject 91 "different" pairs without losing a single "same" pair, lifting the 60-70 band to approximately 92% and 70-80 to approximately 80%. Notably, 55-60 (98%) is then the best band below 80: every newly added form of evidence (synonyms, register periods, addresses) lifts true matches upward out of the band, while 60-80 partly fills with pairs riding stacks of weaker bonuses - a caution against reading mid-range scores as mid-range certainty.
The calibration was re-run on the reported run (9 August 2026), with each score band judged blind by an independent annotator that saw only the two observations - never the pipeline's own score, band, or features. Volume-weighted over accepted edges, precision at threshold 55 is 94.6% over 5.92 million edges; at threshold 60 it is 93.7%, and at threshold 80 it is 94.5%. An earlier round exposed a pronounced non-monotonicity: the 55-60 band measured 94% while 60-70 fell to 83% and 70-80 to 77%, recovering only above 80. Diagnosis traced this to a single evidence pattern - identical name plus a shared spouse given name, with no date evidence whatsoever - which measured 54% precision and made up half the 70-80 band. Dutch naming conventions make a father and son with same-named wives indistinguishable on that evidence, and the existing parent/child guard was inoperative there because it requires a birth year on both sides. Adding a source-year fallback (Section 7) lifted 70-80 to 92% and 60-70 to 88%. That guard was derived from the preceding round's annotations, so the present figures are not an independent validation of it: the pre-registered simulation predicted 95.3% at threshold 55 and the measurement returned 94.6%, the gap one expects from a rule fitted on adjacent data. The weakest accepted band is now 60-70 at 88%. The choice of 55 as the operational threshold remains deliberate: the 55-60 band contributes 1.09 million edges at 99% precision, and predominantly cross-source edges. At threshold 80, nearly all cross-source links vanish, because censuses, address books, and burial records rarely offer day-precise date agreement; yet precisely those links are the point of a time machine. Consumers requiring maximum certainty filter on gtm:confidence >= 0.80: the information is in the output (R5).
9.3 Coverage
Table 6. Coverage per source (reported run): observations, observations absorbed into multi-member reconstructions, and percentage.
| Source | Observations | In multi-member reconstruction | % |
|---|---|---|---|
| SAMH (civil registration + population register + genealogy) | 990,156 | 846,038 | 85.4 |
| Harvest: population registers (non-SAMH) | 412,866 | 197,230 | 47.8 |
| Harvest: civil registration (non-SAMH) | 137,902 | 107,626 | 78.0 |
| Harvest: other registers (non-SAMH) | 20,216 | 12,589 | 62.3 |
| Harvest: militia / judicial / memorial cards / church registers | 11,675 | 5,572 | 47.7 |
| Burial records | 14,568 | 6,934 | 47.6 |
| Census 1840 | 14,469 | 10,799 | 74.6 |
| Census 1830 | 12,577 | 8,309 | 66.1 |
| Church marriages 1771-1795 | 6,137 | 2,310 | 37.6 |
| Cadastral OAT 1832 | 4,758 | 3,268 | 68.7 |
| Verponding tax register 1806 | 3,970 | 1,935 | 48.7 |
| Office holders / biographical / address books / other | 6,746 | 1,634 | 24.2 |
| Total | 1,636,040 | 1,204,244 | 73.6 |
The harvested population registers link at a markedly lower rate (47.8%) than harvested civil registration (78.3%): they are often loose household or address lines that can link only through name, birth year, and family context, and their birthplaces are strings rather than URIs, which restricts them to the weaker place-name feature. The initials mechanism of Section 7.1 raised address-book coverage from 80.6% to 90.3%, with 270 of 298 initials-only mentions linked automatically; only genuinely ambiguous cases (multiple full given-name variants on the same initials) remain in the review queue.
9.4 Quality signals per run
Every run reports its own contradictions: 561,238 guard-refused merges broken down by reason (Table 4), 10,879 disputed parent relations, the count of ambiguous initials pairs, and the blocking non-coverage percentage (structurally zero since the salt chain and sorted neighbourhood window). These signals constitute the agenda of the next improvement iteration; the register periods, the gender inference, and the generation-gap rejection all originated as responses to them. We regard this self-reporting of contradiction as good practice for any heritage linkage system: the errors a system refuses to make are as informative as the links it accepts.
9.5 The tree signal: a controlled A/B experiment (version 2)
Because version 2 with the tree signal disabled is construction-identical to version 1, the effect of the family trees can be measured as a controlled experiment on the same codebase and archival data: run A (baseline) versus run B (tree signal enabled). Run B accepts 47,537 additional pairs - every one carrying a tree feature; no pair was promoted without tree evidence and no baseline acceptance was lost. The promotions are dominated by spelling-variant pairs just below the threshold (surname synonym +20, given name exact +25, estimated year compatible +6 = 51, lifted to 59 by +8 tree evidence) and by cross-source links (14.7k promotions connect SAMH with Open Archieven or census mentions). The 10,052 injected day-anchored life-course bridges connect baptism, marriage, and death mentions that blocking could never pair. The consistency guards moved in the right direction: date-conflict guard refusals decreased relative to baseline (birth-day conflicts −3,379, born-after-death −656), while the cohesion guard fired 9,811 times more often - the tree-free-score rule doing its work.
Calibration (800 pairs, eight blind annotators, stratified by band × tree/non-tree) shows tree-corroborated pairs matching or exceeding non-tree pairs in every acceptance band: 98% vs 86% (55-60), 98% vs 87% (60-70), 96% vs 91% (70-80). Volume-weighted over all accepted pairs within the same annotation round, precision rises from 95.4% (baseline) to 96.1% (tree signal): the signal adds pairs and raises average precision, because an independent tree confirmation lands predominantly on pairs that were already sound. The same-certificate sanity check counted zero guaranteed-distinct pairs that tree features alone would push over the threshold. Two failure modes were caught by the A/B measurement itself during development - dateless injections chaining homonyms, and tree promotions counting as independent cohesion corroboration - and both were closed structurally rather than by weight tuning, which we regard as the main methodological lesson: every new evidence source must pass through all existing guards, including the ones that run before it in the pipeline.
10. Human-in-the-Loop Review
A lightweight web application reads the pipeline database read-only and offers four entry points: a reconstruction browser sortable by lowest confidence, showing per member the confidence and per pair the full feature breakdown; the grey-zone queue; the list of disputed parent relations; and an overview of verdicts. The grey-zone queue presents each doubtful pair (score 40-55) as a field-by-field comparison of both mentions - name, gender, dates, places, address, occupation, full family context, and attestation year - colour-coded on the same thresholds as the scoring model, so that the reviewer sees exactly what the machine saw (R13). Names link to the source pages. Verdicts land in a separate store that survives pipeline reruns. Integration of this review interface into the existing crowdsourcing platform of the Gouda Time Machine ("Vele Panden") is under consideration, which would extend review capacity from the project team to the volunteer community.
Verdicts are not a passive archive: a dedicated module feeds them back into the next run as hard constraints, following the must-link/cannot-link paradigm of constrained clustering (Wagstaff et al., 2001). A "same" verdict becomes a must-link: the scoring phase forces an edge at score 100 even where blocking generated no candidate, and clustering waives the soft merge guards (cluster size, cohesion) for it. Hard data contradictions with a third cluster member - incompatible day-precise birth dates, a recorded gender conflict, co-occurrence in one certificate - continue to apply and are logged, so that a human mistake cannot silently corrupt a cluster. A "different" verdict becomes a cannot-link: the pair is rejected, and clustering keeps the two observations apart transitively through a per-cluster forbidden set. On direct contradiction, cannot-link defeats must-link. Reconstructions containing a confirmed pair are marked gtm:humanVerified in the output, and verdicts serve as high-grade labels in the supervised weight estimation of Section 11, overriding LLM labels where they overlap. Each hour of human review thus improves both the immediate reconstruction and, over time, the model.
11. Discussion
11.1 Hand-set weights versus estimated weights
An obvious objection to a hand-weighted scoring model is that Fellegi-Sunter weights can be estimated from data. We tested this (a dedicated pipeline phase implements the estimation and can be swapped in at runtime), with two instructive outcomes. Unsupervised estimation by expectation maximisation collapses in a blocked candidate space: the latent class that EM discovers is "shares a blocking key", not "is the same person", because blocking has already removed the mass of trivial non-matches that the two-class mixture relies on. This failure mode is worth reporting because blocking-then-EM remains a common default. The supervised variant - m-probabilities from the LLM labels, u-probabilities from random pairs - produces plausible weights and narrowly wins on the labelled pairs, but loses on the same-certificate sanity check of Section 9.1, because rare-but-decisive outcomes (day-precise death agreement, address evidence) are undercovered by the labels. The hand model therefore remains the default until targeted labelling closes those gaps. We take from this a broader point: in heritage linkage, where the reviewing public must be able to understand why two mentions were joined, the burden of proof lies on the learned model to beat a transparent one, and on our data it does not yet do so.
11.2 LLM annotation
Using LLM annotators for calibration is economical and reproducible, but has a structural limitation: the annotators see the same fields as the scorer, so systematic errors residing in the data itself - a mistranscribed date, a consistently misread name - are invisible to both. The bands are therefore honestly comparable with one another, but not an absolute ground truth. The annotator-free same-certificate validation set partially compensates, and human verdicts from the review application progressively replace LLM labels where available. A planned comparison of LLM verdicts against accumulated human verdicts on the same pairs will quantify annotator quality directly.
11.3 Other limitations
Three further limitations should be named. First, within oversized blocks of high-frequency undated namesakes, the sorted neighbourhood window compares only near neighbours; undated namesakes of top names thus remain harder to match exhaustively than dated ones. Second, the URI continuity rules are heuristic: under drastic reconfiguration (a very different threshold) a UUID can shift to a substantially different membership; the register and the tombstones make such shifts traceable, but not impossible. Third, recall is deliberately subordinated to precision (R8): the 26.3% of mentions that remain singletons include genuine members of existing reconstructions whose evidence simply does not reach the threshold, and the grey zone of 16.6 million pairs is the price of refusing to guess. We consider this the correct trade-off for published heritage data, where an erroneous merge propagates into every downstream presentation, but it should be understood when using the data for, for example, demographic rates.
11.4 Generalisability
Little in the pipeline is specific to Gouda. The requirements of Section 4, the observation/reconstruction discipline of PiCo, the guard-based clustering, the cohort splitting, the identifier register, and the calibration methodology transfer directly to any municipality publishing person observations as linked data; the Dutch-specific components (prefix lists, widow notation, synonym lexicons) are data, not architecture. Within the growing family of local time machine initiatives, we would argue that person reconstruction of this kind - transparent, calibrated, provenance-complete, and reviewable - is the natural person-dimension counterpart to the georeferencing and parcel filiation work that such projects already undertake for the built environment.
12. Conclusion
We have described a person reconstruction pipeline that resolves 1.64 million heterogeneous person mentions of one Dutch city into 650,176 PiCo-conformant person reconstructions, at a volume-weighted precision of approximately 95% at the operational threshold, on a single machine, in half an hour. The design consistently privileges transparency, precision, and provenance over coverage and modelling sophistication, and treats its own uncertainty - confidence scores, grey zones, refused merges, disputed parents - as publishable output rather than internal state. Future work includes closing the label gaps that currently hold back supervised weight estimation, opening the review interface to the volunteer community, deeper use of address and household evidence for the weakly linking population registers, and alignment of reconstructions with external person hubs. The reconstructions, the enrichment by-products, and this documentation are available through the data platform of the Gouda Time Machine.
Data and Software Availability
The person reconstructions are published as an openly licensed dataset, Person Reconstructions for the Gouda Time Machine, identified by the persistent ARK identifier https://n2t.net/ark:/60537/b03ndm7. The dataset is distributed as a single compressed N-Quad file (personreconstructions.nq.gz) under a Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) licence, and is described in machine-readable form in the data catalogue of the Gouda Time Machine (https://www.goudatijdmachine.nl/.well-known/datacatalog). The description is registered in the Dataset Register of the Dutch Digital Heritage Network (Netwerk Digitaal Erfgoed), where it can be inspected, together with its distributions and licence, at datasetregister.netwerkdigitaalerfgoed.nl.
Besides the file download, the reconstructions can be queried directly. The Gouda Time Machine operates a public SPARQL endpoint at https://sparql.goudatijdmachine.nl/, which offers a YASGUI query editor in the browser and accepts programmatic requests at the path /sparql (GET or POST with a query parameter, content negotiated through the Accept header for SPARQL-results JSON, XML, or CSV). The reconstructions are loaded there as the named graph <https://www.goudatijdmachine.nl/id/person-reconstructions> - the same graph name that the N-Quads distribution carries, so that a local triple store reproduces the endpoint's structure exactly - holding the 17.1 million triples described in Section 8. Queries should scope themselves to that graph with an explicit GRAPH clause. Listing 2 illustrates the pattern that makes the calibrated confidence of Section 9 usable in practice: retrieving reconstructions together with their constituent observations, filtered on a stricter confidence threshold than the pipeline itself applied (R5).
Listing 2. Retrieving reconstruction members above a confidence threshold from the SPARQL endpoint.
PREFIX pico: <https://personsincontext.org/model#>
PREFIX prov: <http://www.w3.org/ns/prov#>
PREFIX schema: <https://schema.org/>
PREFIX gtm: <https://www.goudatijdmachine.nl/def#>
SELECT ?reconstruction ?name ?observation ?confidence
WHERE {
GRAPH <https://www.goudatijdmachine.nl/id/person-reconstructions> {
?reconstruction a pico:PersonReconstruction ;
schema:name ?name ;
prov:qualifiedDerivation ?derivation .
?derivation prov:entity ?observation ;
gtm:confidence ?confidence .
FILTER (?confidence >= 0.80)
}
}
LIMIT 100
Because the observation URIs returned by such a query are the ARK identifiers of the source data (or, for harvested mentions, Open Archieven record URIs), the result set is directly joinable with the observation graphs of the Gouda Time Machine and with the source archives themselves. Federated queries from other endpoints can address the graph in the same way through SERVICE <https://sparql.goudatijdmachine.nl/sparql>.
The source code of the pipeline is publicly available at https://github.com/gouda-tijdmachine/person-reconstructions, released under the Eclipse Public License 2.0 (EPL-2.0). The repository contains the phase scripts described in Section 5 (extraction, normalisation, blocking, scoring, clustering, serialisation, reporting, calibration), the shell orchestration that drives them, and the configuration module that holds the thresholds and scoring weights of Section 6, so that the run reported here can be reproduced or re-parameterised - for example at the maximum-certainty setting - without modifying code. The data and the code are thus separately licensed: the reconstructions under CC BY-SA 4.0, the pipeline under EPL-2.0.
References
Abramitzky, R., Boustan, L., Eriksson, K., Feigenbaum, J., & Perez, S. (2021). Automated linking of historical data. Journal of Economic Literature, 59 (3): 865–918. https://doi.org/10.1257/jel.20201599
Bloothooft, G., Christen, P., Mandemakers, K., & Schraagen, M. (Eds.). (2015). Population reconstruction. Springer International Publishing AG. https://doi.org/10.1007/978-3-319-19884-2
Christen, P. (2012a). A survey of indexing techniques for scalable record linkage and deduplication. IEEE Transactions on Knowledge and Data Engineering, 24(9), 1537-1555. https://doi.org/10.1109/TKDE.2011.127
Christen, P. (2012b). Data matching: Concepts and techniques for record linkage, entity resolution, and duplicate detection. Springer. https://doi.org/10.1007/978-3-642-31164-2
Fellegi, I. P., & Sunter, A. B. (1969). A Theory for Record Linkage. Journal of the American Statistical Association, 64(328), 1183-1210. https://doi.org/10.1080/01621459.1969.10501049
Hernandez, M. A., & Stolfo, S. J. (1998). Real-world Data is Dirty: Data Cleansing and The Merge/Purge Problem. Data Mining and Knowledge Discovery, 2(1), 9-37. https://doi.org/10.1023/A:1009761603038
Idrissou, A., Wissen, L. van, & Zamborlini, V. (2022). The Lenticular Lens: Addressing Various Aspects of Entity Disambiguation in the Semantic Web. Graphs and Networks in the Humanities 2022, 3-4 February. Amsterdam, The Netherlands. https://graphentechnologien.hypotheses.org/files/2022/01/The_Lenticular_Lens_large_Addressing_Various_Aspects_of_etc-Idrissou_Wissen_Zamborlini.pdf
Jaro, M. A. (1989). Advances in record-linkage methodology as applied to matching the 1985 census of Tampa, Florida. Journal of the American Statistical Association, 84(406), 414-420. https://doi.org/10.2307/2289924
Kaplan, F. (2015). The Venice Time Machine. Proceedings of the 2015 ACM Symposium on Document Engineering, 73. https://doi.org/10.1145/2682571.2797071
Mandemakers, K., Bloothooft, G., Laan, F., Raad, J., Mourits, R. J., & Zijdeman, R. L. (2023). LINKS. A System for Historical Family Reconstruction in the Netherlands. Historical Life Course Studies, 13, 148-185. https://doi.org/10.51964/hlcs14685
Philips, L. (2000). The Double Metaphone search algorithm. C/C++ Users Journal, 18(6), 38-43.
Raad, J. (2021). burgerLinker: Command line tool for linking civil registries [Computer software]. CLARIAH. https://github.com/CLARIAH/burgerLinker (TODO: unsure about citation, was Joe the only author, reference year is from the now legacy burgerLinker)
Wagstaff, K., Cardie, C., Rogers, S., & Schroedl, S. (2001). Constrained k-means clustering with background knowledge. Proceedings of the Eighteenth International Conference on Machine Learning (ICML), 577-584. https://doi.org/10.5555/645530.655669
Winkler, W. E. (1990). String comparator metrics and enhanced decision rules in the Fellegi-Sunter model of record linkage. Proceedings of the Section on Survey Research Methods, American Statistical Association, 354-359.
Woltjer, P., Zandhuis, I., Coret, B., Lindeman, M., Balkenende, J., Zijdeman, R., & Mourits, R. (2024). Persons in Context. A Model to Represent Observations and Reconstructions of Historical Persons in Linked Data. Historical Life Course Studies, 14, 105-125. https://doi.org/10.51964/hlcs19312
Appendix A. Annotation instruction
The following instruction was given verbatim to each annotator, in Dutch, together with one band's pairs. Each
pair was supplied as two JSON objects (a and b) containing only the observation fields - name parts, gender,
dates, places, age, occupation, source and source-year range, and family context (spouse, parent and child given
names). No score, band, or feature breakdown was included. The Dutch text is authoritative: it is what the
annotators received. The English below is a translation for readers and was never sent to the model.
A.1 Base instruction (Dutch, as sent)
Je beoordeelt of twee historische persoonsvermeldingen uit Gouda (18e-20e eeuw) dezelfde persoon betreffen. Wees streng: alleen 'same' als de evidentie dat echt ondersteunt; 'different' bij tegenstrijdige evidentie (afwijkende datums, onverenigbare gezinscontext); 'uncertain' als het niet te beslissen is. Let op: naamvarianten en patroniemen komen voor; initialen (bv. "D.G.") kunnen bij een volledige naam horen; leeftijden zijn vaak geschat. Twee vermeldingen uit dezelfde bron in hetzelfde jaar zijn zelden dezelfde persoon.
A.2 Base instruction (English translation)
You are judging whether two historical person mentions from Gouda (18th-20th century) concern the same person. Be strict: answer 'same' only where the evidence genuinely supports it; 'different' where the evidence is contradictory (diverging dates, incompatible family context); 'uncertain' where it cannot be decided. Note: name variants and patronyms occur; initials (e.g. "D.G.") may belong to a full name; ages are often estimated. Two mentions from the same source in the same year are rarely the same person.
An earlier round revealed that annotators resolved recurring borderline cases differently - whether a duplicate register entry counts as the same person, how to treat a wide gap bridged only by family context - so the following four conventions were added to harmonise them. They state how to decide, not what to decide.
A.3 Harmonising conventions (Dutch, as sent)
- Twee vermeldingen met IDENTIEKE dag-precieze geboortedatum EN identieke naam: 'same', ook als ze uit dezelfde bron/periode komen (dubbele registerinschrijving).
- Alleen naam-overeenkomst zonder enig gedateerd of contextueel aanknopingspunt: 'uncertain', nooit 'same'.
- Gezinscontext zonder datums (partner-/kindvoornamen) met een gat > 25 jaar tussen de vermeldingen: 'uncertain', tenzij de naamcombinatie zeldzaam is.
- Onverenigbare leeftijd/datum (bv. > 100 jaar oud, of overlijden vóór een latere vermelding): 'different'.
A.4 Harmonising conventions (English translation)
- Two mentions with an IDENTICAL day-precise birth date AND an identical name: 'same', even when they come from the same source or period (a duplicate register entry).
- Name agreement alone, without any dated or contextual anchor: 'uncertain', never 'same'.
- Family context without dates (spouse or child given names) with a gap of more than 25 years between the mentions: 'uncertain', unless the name combination is rare.
- An impossible age or date (e.g. over 100 years old, or a death preceding a later mention): 'different'.
Each annotator returned one JSON object per pair - the pair identifier, the verdict (same / different /
uncertain), and a short justification - and was asked to verify that its output covered every input pair
exactly once. Those identifier sets were checked against the inputs before aggregation; a further check
confirmed no annotator's output contained pairs belonging to another band.
The repository additionally ships a batch implementation of the same instruction against the Messages Batches
API (75_calibrate.py), which pins the model through the CAL_MODEL environment variable and constrains the
verdict with a JSON schema. It was not used for the figures reported here.
Footnotes
Persons in Context (PiCo) model specification. https://personsincontext.org/model/ ↩
PROV-O: The PROV Ontology. W3C Recommendation. https://www.w3.org/TR/prov-o/ ↩ ↩
Person Name Vocabulary (PNV). https://w3id.org/pnv ↩
Records in Contexts - Ontology (RiC-O), version 1.1. International Council on Archives. https://www.ica.org/standards/RiC/ontology ↩
The dump is produced by the linked data pipeline of the Gouda Time Machine, which harvests the Omeka S collection management system, transforms the records to RDF, and publishes the N-Triples dump that is indexed for the SPARQL endpoint. Source code: https://github.com/gouda-tijdmachine/lod-pijplijn (EPL-2.0). ↩
Thesaurus of Historical Person Data. https://terms.personsincontext.org/ ↩
Aanknopingspunten v3 - how it works, methods and evaluation. https://www.genealogieonline.nl/en/aanknopingspunten/uitleg.php ↩