Usage
=====

CLI
---

Install the GCP extra before using the ``docai`` app:

.. code-block:: bash

   uv pip install -e ".[dev,gcp]"

.. code-block:: bash

   pdf-jsonl-smith local input.pdf -o text_units.jsonl

Optional metadata:

.. code-block:: bash

   pdf-jsonl-smith local input.pdf \
     --output text_units.jsonl \
     --source-uri gs://bucket/input.pdf \
     --schema-version 0.1

By default, the CLI prints a short completion summary to stdout:

.. code-block:: text

   Wrote 42 records to text_units.jsonl from input.pdf.

Use ``--quiet`` to suppress the summary.

The older ``pdf2jsonl`` command remains available as a compatibility alias for
the local PyMuPDF extractor:

.. code-block:: bash

   pdf2jsonl input.pdf -o text_units.jsonl

The ``docai`` app uploads a local PDF to Cloud Storage, runs Google Document AI,
and writes compact JSONL records:

.. code-block:: bash

   pdf-jsonl-smith docai input.pdf \
     --output compact.jsonl \
     --project my-project \
     --location us \
     --processor-id PROCESSOR_ID \
     --staging-uri gs://bucket/pdf-jsonl-smith/input/

By default, ``docai`` emits compact ``text``, ``key-values``, ``tables``, and
``lists`` records. Use ``--include`` to narrow the output:

.. code-block:: bash

   pdf-jsonl-smith docai input.pdf \
     --output compact.jsonl \
     --project my-project \
     --location us \
     --processor-id PROCESSOR_ID \
     --staging-uri gs://bucket/pdf-jsonl-smith/input/ \
     --include text,key-values

Use ``--jsonl-gcs-uri`` to upload the compact JSONL output after writing it
locally.

See :doc:`docai` for a friendlier walkthrough and useful ``jq`` inspection
commands.

The ``bq`` app is reserved for the BigQuery workflow:

.. code-block:: bash

   pdf-jsonl-smith bq

The ``docai`` app policy is to extract from rich Google Document AI payloads,
but emit only compact, purpose-built records. Full Document AI payloads are not
part of the final JSONL output.

Python API
----------

.. code-block:: python

   from pdf_jsonl_smith import extract_pdf_to_jsonl

   count = extract_pdf_to_jsonl(
       input_path="input.pdf",
       output_path="text_units.jsonl",
       source_uri="gs://bucket/input.pdf",
   )

The function returns the number of JSONL records written.

Output Fields
-------------

Each line is one standalone JSON object with flat fields:

.. code-block:: text

   schema_version STRING
   doc_id STRING
   source_uri STRING
   filename STRING
   page_number INT64
   unit_type STRING
   unit_index INT64
   text STRING
   bbox_x0 FLOAT64
   bbox_y0 FLOAT64
   bbox_x1 FLOAT64
   bbox_y1 FLOAT64
   char_count INT64
   word_count INT64
   extractor STRING
   extracted_at TIMESTAMP
   warning STRING