Line Detection
Identify text regions, line structure, and page features in scanned historical documents.
AI: Historical handwriting transcription and genealogy
Ancient-Script.org is a research and engineering platform operated by Ancient Script Org LLC. The project explores AI-assisted tools for transcribing historical handwriting, organizing document evidence, training machine-learning models for document analysis, and supporting genealogy research workflows.
Access is currently invitation-based so uploads, collaborative review, and model-training workflows can be managed carefully.
Identify text regions, line structure, and page features in scanned historical documents.
Refine word and fragment crops so document evidence can be inspected, compared, and reviewed.
Compare AI suggestions with human corrections, votes, and source-document context.
Use reviewed document evidence to develop and evaluate machine-learning workflows for document analysis.
The engineering beneath the tools: the machine-learning, distributed-systems, and cryptographic work that supports document analysis at scale, together with research directions explored along the way.
A framework instantiates coordinated hierarchies of language-model agents, each bounded by its own tool set, context window, and cost budget, in order to transform a page image into a structured record. Tool invocations are streamed, and context is compacted automatically as a conversation approaches its limit.
Historical handwriting is transcribed by a sequence model that couples a gated-convolutional visual encoder with a Transformer and a connectionist temporal classification decoder. Training minimizes character error rate and incorporates a diffusion-style denoising objective to improve robustness on degraded scans.
Lines, words, and page regions are recovered by combining learned and classical vision: instance segmentation isolates word crops, a U-Net segments text lines, and Hough transforms, morphological skeletonization, and ridge tracing reconstruct page geometry.
A research line investigated an active-perception policy for transcription: a Transformer critic approximating the action-value function, with a dueling value/advantage decomposition and reward shaping over glimpse actions, learning when the accumulated evidence is sufficient to stop.
A diffusion-style noise schedule is embedded within the recognizer as a denoising objective, and a separate generative experiment examined the reconstruction of whole document pages from noise using a U-Net backbone.
Records are held in a horizontally distributed Citus and PostgreSQL cluster, sharded by surname hash and document identity, with reference tables for frequently joined lookups and an explicit strategy for cross-shard fuzzy search. Services coordinate over a NATS JetStream message bus.
Each model trains and serves as a containerized microservice on Kubernetes, with multi-GPU distributed-data-parallel training over CUDA, persistent volumes for model checkpoints, self-healing coordinator and worker pods, and automated certificate management.
Canonical individuals are reconstructed from many dispersed record sightings through surname-hash blocking and fuzzy matching, and competing machine and human judgments are reconciled within a corroboration-voting knowledge graph.
Reviewers work concurrently over Socket.IO, incremental language-model output is delivered by server-sent events, and a tiered vote cache keeps shared state responsive under simultaneous editing.
Credentials are stored as bcrypt hashes, and per-user API keys are protected with AES-256-GCM under keys derived per record via HKDF-SHA256 with user-scoped authenticated data. Access is mediated by stateless, audience-separated JSON Web Tokens and an OAuth 2.1 authorization server with PKCE and dynamic client registration, over managed TLS.
AI-generated transcriptions, document labels, and genealogy suggestions require human review. They should not be treated as authoritative records or professional advice.
Participants should upload only materials they have the right to process and should avoid uploading sensitive modern personal information.