Rob Ekl 621d581d87 Fix large GEDCOM files failing to load or freezing the browser (#296)
* fix(upload): bypass history.pushState size limit with in-memory GEDCOM store

Firefox caps history.pushState state at 640KB; Safari at ~512KB. The upload
handler was passing the raw GEDCOM string through navigate(..., {state}),
which calls history.pushState. A 10MB GEDCOM caused an immediate SecurityError
in Firefox and Safari, preventing the chart from ever loading.

Add a bounded in-memory store (gedcom_store.ts) capped to 1 entry — uploading
a new file evicts the previous one, preventing unbounded memory growth during
a session. navigate() now carries only a file fingerprint hash in the URL;
loadGedcom() retrieves the GEDCOM from the store by hash.

Update embedded.ts to store its injected GEDCOM under the key 'embedded'
before calling loadGedcom(), keeping the data path consistent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(fingerprint): replace 500ms full-file md5 with fast metadata + 4KB sample hash

The upload handler called md5() over the full GEDCOM string before navigating
— ~240ms for a 10MB file, blocking the main thread before the user saw
any feedback.

Replace with a hash over file.name|file.size|file.lastModified plus the first
and last 4KB of content. Runtime drops to ~1ms. The 4KB sample captures the
GEDCOM header (software name, export date, submitter info) which differs
between any two real exports, providing strong collision resistance for a local
session cache without a cryptographic guarantee.

Extract fileFingerprint to src/util/file_fingerprint.ts so it can be unit
tested independently of the React component.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(parse): async GEDCOM conversion with event-loop yields and onProgress callback

convertGedcom was synchronous, blocking the main thread for the full parse
duration (~1.2s parseGedcom + ~700ms gedcomEntriesToJson in Safari). The
browser spinner could not update and GC could not run between steps.

Make convertGedcom async and insert await setTimeout(0) yields between the
four major steps (parse, convert, normalize, index). This breaks the work
into chunks, allows incremental GC, and keeps the spinner responsive.

Add onProgress?(status: string) => void to convertGedcom, prepareData,
loadGedcom, loadFromUrl, loadAndPrepareFile, and the DataSource interface.
All data source implementations (UploadedDataSource, GedcomUrlDataSource,
GoogleDriveDataSource) forward the callback down the load pipeline.
WikiTree and Embedded accept the parameter but do not use it.

Also skip the sessionStorage JSON.stringify for GEDCOM files over 512KB.
The serialized TopolaData (chartData 7.85MB + raw parse-gedcom tree 25MB
= 33MB) always exceeded the 5MB sessionStorage limit, wasting 10-20s in
Safari on a write that was guaranteed to fail.

Simplify loadGedcom to (hash, onProgress?) now that the GEDCOM string is
always retrieved from the in-memory store rather than passed as a parameter.
Remove console.log timing instrumentation and window.__topolaDebug globals
that were left in from debugging.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(ui): skip redundant chart re-renders; persist progress overlay through D3 layout

Two related issues caused the chart to freeze or appear unresponsive:

1. The Chart useEffect has no dependency array, so it ran after every React
   render. Each call re-created a JsonDataProvider from all 23K individuals,
   re-ran the ancestor/descendant layout, and called getComputedTextLength for
   every visible text element (~500ms per call in Chrome, longer in Safari).
   For unrelated state changes this froze the browser 3-5 extra times per load.
   Fix: return early in renderChart() when neither initialRender nor
   resetPosition is set — the SVG is already correct.

2. The "Loading..." progress pill was mounted only in AppState.LOADING. When
   data finished parsing, React transitioned to SHOWING_CHART and unmounted the
   pill before the D3 layout (the slow part) had run. Users saw a blank screen
   with no feedback for several seconds.
   Fix: replace the state-gated pill with a persistent fixed overlay driven by
   a loadingStatus string, visible through both LOADING and the initial chart
   render. Add onFirstRender() callback to Chart; App clears loadingStatus when
   the first D3 layout completes.

Wire the onProgress callback from the load pipeline into setLoadingStatus so
users see step-by-step progress ("Step 1/4: parsing GEDCOM..." through
"Rendering chart (23,909 people)...").

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(search): pre-build Lunr index during browser idle time

Building the full-text search index over 23K individuals takes ~600ms
synchronously. This was deferred to the first keystroke, freezing the input
field for up to 3 seconds in Safari with no user feedback.

Schedule the build via requestIdleCallback (5s timeout) after data loads,
with a 200ms setTimeout fallback for browsers without requestIdleCallback.
The index is typically warm before the user types. getOrBuildIndex() still
builds synchronously on the first keystroke if idle scheduling has not yet
fired (e.g. user types immediately after load).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: add coverage for gedcom_store, loadGedcom store path, and fileFingerprint

gedcom_store.spec.ts
  - Returns undefined for unknown hash
  - Stores and retrieves GEDCOM by hash
  - Evicts the previous entry when a new file is stored (bounded-to-1 behavior)
  - Returns undefined for a hash that was evicted

load_data_store.spec.ts
  - loadGedcom throws ERROR_LOADING_UPLOADED_FILE when hash is absent from store
  - loadGedcom successfully parses GEDCOM retrieved from the in-memory store
  - onProgress callback receives all four step messages in the correct order

upload_menu.spec.ts (via src/util/file_fingerprint.ts)
  - Returns a valid 32-character hex hash
  - Produces identical hashes for identical inputs
  - Differs for different filename, file size, content beyond 4KB boundary,
    and image file list

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(ui): clear loading pill for all chart types including Donatso

The progress pill persisted indefinitely on the Donatso chart because
DonatsoChart did not call onFirstRender — only the D3-based Chart did.

Add onFirstRender? to DonatsoChartProps and call it from the existing
useEffect on the first render, consistent with how Chart handles it.
Pass onFirstRender={() => setLoadingStatus('')} from renderChart in app.tsx.

Update visual regression snapshots for macOS (darwin) to reflect the new
progress pill and its correct dismissal across all chart types. The 12
non-Donatso failures were pre-existing snapshot staleness; chart-donatso
was the one newly introduced by this PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix thread onProgress through EmbeddedDataSource.onMessage to loadGedcom

The _onProgress parameter accepted by loadData was never forwarded to the
onMessage event handler, silently dropping all step-progress callbacks for
embedded GEDCOM loads. The progress pill stayed at 'Loading…' for the full
parse duration instead of showing the four-step breakdown.

Thread onProgress through the event-listener closure by adding it as a
parameter to onMessage and passing it to loadGedcom.

* Make gedcom_store tests independent of execution order

Jest caches module instances, so the module-level let variables persisted
across tests. Running with --randomize or in parallel could cause test 1
('returns undefined for unknown hash') to see a hash left by a previous test.

Export clearStoredGedcom() for testing and call it in beforeEach so each test
starts from a clean slate regardless of execution order.

* Document intentional pre-animation timing of onFirstRender in Chart

onFirstRender fires when the chart SVG is in the DOM, before the D3
fade-in animation completes (~400ms). A reviewer noted this as a potential
issue, but moving it into animationPromise.then() would keep the progress
pill visible for 400ms after the chart is already visible and interactive,
creating a confusing overlap. The current timing is correct.

Add a comment explaining the design intent.

* Pass onProgress to WikiTree and Embedded data sources at call site

Both call sites in the loadData callback omitted the onProgress parameter
that all other sources (UPLOADED, GEDCOM_URL, GOOGLE_DRIVE) already forward.
WikiTree's implementation ignores it today (_onProgress), but wiring it here
ensures parity and makes future progress reporting possible without touching
app.tsx again. Embedded now receives it and threads it to loadGedcom.

* Use const-scoped handles in search useEffect to eliminate type casts

The shared 'let handle' variable was typed as a union of setTimeout and
requestIdleCallback return types, requiring unsafe casts (handle as number,
handle as ReturnType<typeof setTimeout>) in the cleanup closures. Each
cleanup closure captured the shared variable by reference rather than value.

Use a const declaration in each branch so TypeScript infers the correct
type automatically and no casts are needed. The early-return form also
makes the two branches structurally symmetric.

* Remove window event listener in EmbeddedDataSource after GEDCOM received

The listener registered in loadData was never removed, causing it to
accumulate on every loadData call. In a multi-file workflow the second
load would have two listeners firing on the same message, the first
with a stale resolve/reject pair.

Wrap resolve and reject so the listener removes itself on first
settlement, making the registration effectively once-per-load.

* Minor code fixes. Moved test to a different file

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Przemek Więch <pwiech@gmail.com>
2026-06-05 00:14:58 +02:00
2026-05-12 00:29:35 +02:00
2026-05-29 00:04:13 +02:00
2026-04-25 21:07:18 +02:00
2019-01-24 00:04:24 +01:00
2026-05-09 11:07:55 +02:00
2026-05-18 23:14:16 +02:00

Topola Genealogy Viewer

Node.js CI

View your genealogy data using an interactive chart.

Website: https://pewu.github.io/topola-viewer

screenshot

If you find this project useful, consider buying me a coffee.

buy me a coffee

Features

  • Hourglass chart
  • All relatives chart
  • Click on a person to focus
  • Open standard GEDCOM files you can export from any genealogy application
  • Load from URL (just point to a GEDCOM file on the Web)
  • Privacy your files do not leave your computer
  • Print the whole genealogy tree
  • Export to PDF, PNG, SVG
  • Side panel with details
  • Configuration options
  • Permalinks when loading from URL
  • Cool transition animations

Changelog

Examples

Here is an example from the Web:

If you have data in a genealogy database, you can export your data in GEDCOM format and load it using the "Load from file" menu.

Integrations

Topola Genealogy Viewer is being integrated into more and more Web and desktop applications. Here are the current integrations:

Gramps

To view your Gramps data in Topola Genealogy Viewer, install Interactive Family Tree plugin from the Gramps plugin manager. The plugin will add a Tools->Analysis and Exploration->Interactive Family Tree menu item to Gramps.

Source code: https://github.com/gramps-project/addons-source/tree/master/Topola

Webtrees

Embed Topola Genealogy Viewer in your Webtrees installation with the Topola interactive tree addon.

Source code: https://github.com/PeWu/topola-webtrees

WikiTree

You can browse the WikiTree genealogy tree using Topola Genealogy Viewer. On a WikiTree profile page go to the Family Tree & Tools tab and click the Dynamic Tree by Topola link.

Example: Stephen Hawking

Topola Genealogy Viewer is hosted on apps.wikitree.com to benefit from the ability of being logged in to the WikiTree API.

Running locally

npm install
npm start

Self-hosting

You can host Topola Genealogy Viewer on your own server. There are no specific requirements for the hosting server. There is no code that is executed on the server side. The server only hosts the application files and whole application runs in the browser.

You can build Topola Genealogy Viewer from source code or take a ready-to-deploy package.

Bulid your own

Here are the commands to build the application:

git clone https://github.com/PeWu/topola-viewer.git
cd topola-viewer
npm install
npm run build

Now, take the contents of the dist/ folder and host it on your own server.

Use an existing package

Download the following file, unpack it and upload the contents to your server: https://github.com/PeWu/topola-viewer/archive/refs/heads/gh-pages.zip

These are the exact files that are hosted on GitHub pages.

Build for your own data only

You can run Topola Viewer in a "single tree mode" that displays only the GEDCOM you specify. Specify the URL to a GEDCOM file in the VITE_STATIC_URL environment variable when building and running the application.

Run locally with the specified data URL:

VITE_STATIC_URL=https://example.org/sample.ged npm start
For Windows CMD:
set VITE_STATIC_URL=https://example.org/sample.ged && npm run build

Build with the specified data URL:

VITE_STATIC_URL=https://example.org/sample.ged npm run build
For Windows CMD:
set VITE_STATIC_URL=https://example.org/sample.ged && npm run build

The dist/ folder will contain files that can be hosted on a Web server.

Build without Google Analytics

Set VITE_GOOGLE_ANALYTICS=false to exclude Google Analytics from the build output. This will remove the external JavaScript dependency.

VITE_GOOGLE_ANALYTICS=false npm run build
For Windows CMD:
set VITE_GOOGLE_ANALYTICS=false && npm run build

This may be combined with the other build environment variables described above.

Alternative build

The topola-webpack tool can build a Topola Genealogy Viewer package bundled together with a GEDCOM file.

Docker Container Deployment

Topola Viewer can be run locally or deployed to standard cloud environments using Docker.

Running Topola Viewer

To pull and run Topola Viewer:

docker run -d -p 8080:8080 ghcr.io/pewu/topola-viewer:latest

Open your web browser and go to http://localhost:8080 to upload your family tree files locally.

Running with Your Own Data (Zero-Build Run)

You can serve a standalone, pre-loaded family tree with zero compilation by mounting your family tree data (a .ged file or a zipped .gdz archive containing photos) directly into the running container:

docker run -d -p 8080:8080 \
  -e STATIC_URL=my_family.gdz \
  -v ./my_family.gdz:/app/public/my_family.gdz \
  ghcr.io/pewu/topola-viewer:latest

Building the Base Image Locally

To build the base image from source:

docker build -t topola-viewer -f docker/Dockerfile .

Ready-To-Use Standalone Templates

For creating completely self-contained Docker images that bundle your genealogy data and serve it instantly, see these pre-configured examples:

  1. Simple Standalone Tree: Demonstrates how to package and pre-load a .ged file directly inside a custom image.
  2. Standalone Tree with Photos: Packages your family tree and a photos/ folder into a valid .gdz archive on-the-fly.

Additional options

handleCors

Add &handleCors=false to the URL to avoid using the CORS proxy

embedded

Add &embedded=true to the URL. This option removes the options to open a different file. It is an option that was intended to be used when Topola Genealogy Viewer is in an iframe.

Description
Topola Genealogy Viewer – interactive genealogy visualization
Readme Apache-2.0 138 MiB
Languages
TypeScript 97.2%
CSS 1.8%
Dockerfile 0.4%
HTML 0.3%
JavaScript 0.2%