A Multi-Stage Framework for Kuzushiji Character Recognition in Japanese Historical Documents

Kuzushiji was a widely used cursive writing system in pre-modern Japan. Due to simplification and glyph variation, most modern Japanese readers cannot read Kuzushiji characters. Consequently, recent studies have developed optical character recognition (OCR) systems for Kuzushiji. Despite recent progress, Kuzushiji character recognition (KCR) in Japanese historical documents remains challenging because of seal-character overlap and complex layouts, which interfere with character recognition and hinder accurate reconstruction of the reading order. To address these challenges, we propose a multi-stage KCR framework comprising character detection, cropping, classification, ordering, and large language model (LLM)-based post-OCR correction. Specifically, we employ a synthetic data augmentation strategy to improve character detection robustness against seal interference and introduce an adaptive column clustering algorithm to reconstruct the reading order. Finally, we leverage the contextual capabilities of the LLM to correct OCR errors. In addition, we correct annotation omissions, reconstruct the benchmark dataset, and introduce a synthetic test set with simulated seal interference and an out-of-domain (OOD) test set for evaluation. Compared with the conventional character-level OCR baseline, our framework achieves relative CER reductions of 43.48%, 46.02%, and 39.11% on the real, synthetic, and OOD test sets, respectively.

Conventional pipeline (blue flow) and the proposed pipeline (red flow) for seal-interfered Japanese historical document images. Dashed arrows indicate additional processes performed in parallel with character detection without affecting the detection results.

Among the 1,000 document images, 267 were found to contain missing annotations. To improve annotation quality, the missing character bounding boxes were manually added with the assistance of a Kuzushiji expert. Red bounding boxes indicate the annotations newly added in this work, while green bounding boxes correspond to the original annotations.

We construct a synthetic test set to simulate severe seal interference. The seals shown in the top row originate from real historical documents, while those in the bottom row are synthetically overlaid.

Visual examples of detection results produced by the YOLO11-L model trained with the proposed Synthetic Data Augmentation (SDA) strategy. The top row shows low-confidence bounding boxes produced by the detection model. Stains in Japanese historical documents may cause false positives, resulting in background noise being mistakenly detected as Kuzushiji characters with confidence scores as low as 0.001. The bottom row shows the detection results with a confidence threshold of 0.1, which effectively removes most false positives.

Visual examples of document restoration results obtained using the proposed color-based thresholding algorithm with τ_r = 90 and (τ_rg, τ_rb) = (1.3, 1.3).

Visual examples of the input document images (top row) and the corresponding recognition results projected onto the original document images (bottom row). Despite the complex document layouts, the proposed visualization facilitates intuitive interpretation of historical documents by preserving the recognized characters within their original spatial context.