# Extracting text from scanned RFP PDFs with an OCR fallback

A scanned Request for Proposals (RFP) — a Notice of Funding Opportunity that a program officer printed, signed, and re-scanned, or a National Institutes of Health (NIH) appendix delivered as image-only pages — carries no text layer at all, so pdfplumber returns an empty string and every downstream parser silently sees a blank document. That silent blank is the dangerous failure: a compliance matrix built from nothing looks structurally valid, clears shallow checks, and ships a package that omits half its requirements. This page builds the detect-and-fall-back path that turns those image-only pages into coordinate-tagged text, as one hardened workflow inside the broader PDF text extraction with pdfplumber stage. The approach: measure the text layer before trusting it, render the offending page with PyMuPDF, run Tesseract optical character recognition (OCR) through pytesseract, keep every word’s bounding box, and gate the result on a confidence score so a bad scan routes to a human instead of poisoning the batch.

# Phase 1 — Detect image-only pages and install the OCR toolchain

The cheapest mistake is running OCR on every page. It is an order of magnitude slower than reading a native text layer, and rasterizing a page that already has clean, positioned glyphs throws away the exact coordinates that the table-extraction workflow depends on. OCR is a fallback, invoked only when the native extraction is empty or unusable, so the detector runs first and decides per page.

A page needs OCR when either condition holds: the extracted character list is empty, or the text is present but garbage — a decorative or broken embedded font can produce characters that render but decode to noise. Both are detectable without rendering anything.

Implementation steps:

1. Measure the character count. Read page.chars. An empty list means no text operators exist on the page — a pure image. This is the unambiguous case.
2. Screen for garbage text. When characters exist, compute the ratio of alphanumeric-or-space characters to the total. A page of real prose sits well above 0.85; a decode-broken font collapses toward zero. Below a floor, treat the layer as unusable.
3. Confirm the page is actually inked. A genuinely blank page (a separator sheet) also has no characters but should not be sent to OCR. Check page.images or the rasterized pixel variance so an empty page is skipped, not “recognized.”
4. Record the decision, never guess. Tag every page with native, ocr, or blank and carry that tag downstream, so an auditor can see exactly which text came from a machine-read layer.

import pdfplumber

def needs_ocr(page: "pdfplumber.page.Page", min_clean_ratio: float = 0.6) -> bool:
    """Decide whether a page's native text layer is trustworthy.

    Returns True when the page is image-only (no chars) or when the
    embedded text decodes to mostly non-textual noise.
    """
    chars = page.chars
    if not chars:                      # no text operators at all -> image page
        return bool(page.images)       # ...but only OCR it if something is inked
    text = "".join(c["text"] for c in chars)
    if not text.strip():
        return False                   # whitespace-only: a blank separator sheet
    clean = sum(ch.isalnum() or ch.isspace() for ch in text)
    return (clean / len(text)) < min_clean_ratio   # garbage embedded font

Pin the toolchain. pytesseract is a thin wrapper over the Tesseract binary, which must be installed at the operating-system level; PyMuPDF (imported as fitz) does the rendering:

python -m venv .venv
source .venv/bin/activate
pip install "pdfplumber==0.11.4" "pymupdf==1.24.10" "pytesseract==0.3.13" "Pillow>=10.2"
# Debian/Ubuntu: the OCR engine itself is a system package
sudo apt-get install -y tesseract-ocr

# Phase 2 — Render, recognize, and preserve coordinates

Once a page is flagged, the fallback renders it to a bitmap at a resolution high enough for the recognizer, runs OCR in a mode that returns per-word data — text, confidence, and bounding box together — and re-projects those pixel boxes back into PDF points so the output is drop-in compatible with the native page.extract_words() shape. Preserving coordinates is what lets the same section-boundary detection and table logic run identically over recognized and native pages.

The rendering resolution matters more than any other single parameter. Tesseract wants roughly 300 dots per inch (DPI) of input; below about 200 DPI accuracy falls off a cliff, and above 400 DPI you pay rendering cost for no accuracy gain. PyMuPDF renders at 72 DPI by default, so scale by dpi / 72. The same scale factor, inverted, maps recognized pixel boxes back to points.

import fitz  # PyMuPDF
import pytesseract
from pytesseract import Output
from dataclasses import dataclass

@dataclass(slots=True)
class OcrWord:
    text: str
    x0: float          # PDF points, top-left origin like pdfplumber
    top: float
    x1: float
    bottom: float
    confidence: float  # 0-100, per word, from Tesseract

def ocr_page(doc: "fitz.Document", page_index: int, dpi: int = 300) -> list[OcrWord]:
    """Render one page and return recognized words in PDF-point coordinates."""
    scale = dpi / 72.0                          # PyMuPDF renders at 72 DPI by default
    pix = doc[page_index].get_pixmap(matrix=fitz.Matrix(scale, scale))
    img = pix.tobytes("png")
    data = pytesseract.image_to_data(
        _png_to_image(img), output_type=Output.DICT, config="--oem 1 --psm 6"
    )  # oem 1 = LSTM engine; psm 6 = assume a single uniform block of text
    words: list[OcrWord] = []
    for i, raw in enumerate(data["text"]):
        token = raw.strip()
        conf = float(data["conf"][i])
        if not token or conf < 0:               # Tesseract emits -1 for non-text boxes
            continue
        words.append(OcrWord(
            text=token,
            x0=data["left"][i] / scale,         # pixels back to points
            top=data["top"][i] / scale,
            x1=(data["left"][i] + data["width"][i]) / scale,
            bottom=(data["top"][i] + data["height"][i]) / scale,
            confidence=conf,
        ))
    return words

The decision flow below shows how a single page moves from native extraction, through the image-only test, into the render-and-recognize fallback, and finally through a confidence gate that decides whether the recognized text is trusted or a human is called in.

# Phase 3 — Edge cases and agency scanned-appendix realities

A recognizer that works on a clean flatbed scan still fails on the documents grant offices actually receive. Four failure modes dominate, and each has a specific mitigation.

Mixed-mode documents. A single PDF routinely mixes native pages (the agency’s typeset boilerplate) with scanned inserts (a signed letter of support, a re-scanned budget appendix). Never decide OCR once for the whole file — the needs_ocr test runs per page, and the output stitches native and recognized words back into one ordered stream tagged by source.

Rotation and skew. Scanned pages arrive rotated 90 or 180 degrees or skewed a few degrees off-axis, and Tesseract’s accuracy is rotation-sensitive. Enable orientation detection so the recognizer corrects the page before reading it, and prefer the page-segmentation mode that auto-detects orientation on suspect scans.

Low DPI and thresholds. A fax-quality 150-DPI scan cannot be rescued by rendering it at 300 DPI — you cannot add detail that was never captured. Render from the source resolution, and when the mean confidence still falls short, that is the signal to route the page, not to lower the bar.

Agency-specific handling. Where scanned material appears differs by funder, which changes where the fallback fires. Keep the confidence floor data-driven per agency, checked against the Pydantic schema-validation layer so a recognized field that fails its type contract is caught even when OCR reported high confidence.

# Phase 4 — Validation with committed fixtures

OCR is nondeterministic across Tesseract versions, so tests assert on structure and thresholds, not on exact recognized strings. Commit small fixture PDFs — one native page, one clean scan, one deliberately poor scan — and pin the Tesseract version in the environment so a run reproduces. The goal is to prove the router chooses the right path and the confidence gate fires, not to snapshot brittle text.

import pytest

def mean_conf(words: list[OcrWord]) -> float:
    return sum(w.confidence for w in words) / len(words) if words else 0.0

def test_native_page_is_not_sent_to_ocr() -> None:
    with pdfplumber.open("tests/fixtures/native_text.pdf") as pdf:
        assert needs_ocr(pdf.pages[0]) is False   # has a real text layer

def test_scanned_page_is_flagged_and_recognized() -> None:
    with pdfplumber.open("tests/fixtures/clean_scan.pdf") as pdf:
        assert needs_ocr(pdf.pages[0]) is True
    doc = fitz.open("tests/fixtures/clean_scan.pdf")
    words = ocr_page(doc, 0)
    assert words and mean_conf(words) >= 70          # clears the trust floor
    assert all(w.x1 > w.x0 and w.bottom > w.top for w in words)  # coords intact

def test_poor_scan_falls_below_floor_and_routes() -> None:
    doc = fitz.open("tests/fixtures/fax_150dpi.pdf")
    words = ocr_page(doc, 0)
    assert mean_conf(words) < 70                      # must NOT be silently trusted

Close the loop with a manual acceptance checklist that fixtures cannot cover: confirm every page carries a native/ocr/blank source tag, that recognized coordinates land inside the page’s media box, that a rotated fixture is corrected before reading, and that any page below its agency confidence floor produces a review task with the page image attached. Only after that gate does recognized text flow to the downstream parser and, eventually, the audit record.

# Frequently asked questions

# Related

• Extracting tables from NIH FOA PDFs using pdfplumber — the coordinate-aware table workflow that runs identically over recognized pages.
• NLP section boundary detection — the stage that consumes the stitched native-plus-OCR word stream.
• Schema validation with Pydantic — catches a recognized field that fails its type contract even when OCR reported high confidence.
• PDF text extraction with pdfplumber — the parent stage this fallback plugs into.

Up one level: PDF Text Extraction with pdfplumber