pdf-jsonl-smith

pdf-jsonl-smith converts PDF-derived text into compact BigQuery-ready JSONL.

The local app is a thin wrapper around PyMuPDF for born-digital PDFs. The docai app runs Google Document AI Form Parser, but still emits compact purpose-built records instead of full Document AI payloads.

Contents

• Usage
  • CLI
  • Python API
  • Output Fields
• Document AI
  • Beginner Setup
  • Run
  • Inspect Output With jq
  • Why Unit Type Matters
  • Policy
• API Reference
  • Public API
  • Modules
• Release Process
  • Versioning
  • Publishing
  • Read the Docs

Supported

• Born-digital PDFs.

• PyMuPDF block-based text extraction.

• Conservative same-page wrapped-line merging.

• Google Document AI Form Parser compact output.

• UTF-8 JSONL output.

• BigQuery-friendly flat records.

• One unified JSONL stream with unit_type filters.

Not Supported

For the local extractor:

• Scanned PDFs / OCR.

• Forms.

• Image text.

For all extractors:

• Full Document AI payload mirroring.

• Perfect reading order.

• Cross-page paragraph reconstruction.

• Semantic heading detection.

• Lossless table reconstruction.