# Document Assembly & Audit Logging

Document assembly is the stage where a grant proposal stops being structured data and becomes the single submittable artifact a funding portal will accept or reject. Everything upstream — the modeled requirements, the parsed solicitation, the compliance verdicts — exists to make this one moment safe: composing narrative, boilerplate, agency forms, and attachments into a package that matches an agency’s exact byte-level expectations, then recording precisely how that package was built. For the National Institutes of Health (NIH), the National Science Foundation (NSF), and the Department of Defense (DoD), assembly failures are as fatal as scientific weakness. A form field left blank on the SF424 Research & Related (R&R) form set, a PDF that is encrypted rather than flattened, a technical volume that exceeds a Broad Agency Announcement (BAA) cap, or an unsigned Authorized Organization Representative (AOR) certification will halt a submission at the portal gate — often minutes before a deadline, with no path to correction.

This section is the output stage of the pipeline. It sits downstream of requirement modeling, downstream of ingestion and parsing, and downstream of the compliance validation rule engine whose verdicts it consumes. Where those layers answer what must be true, assembly answers produce the thing, and prove how you produced it. That second clause — provenance — is why assembly and audit logging belong together. A funding cycle can span a year from draft to award to closeout, and an institution must be able to reconstruct, months later, exactly which template version, which data snapshot, and which compliance verdict produced the bytes that were transmitted. This page maps how narrative rendering, form population, PDF generation, approval routing, and audit logging compose into one deterministic, defensible output layer.

# How document assembly fits together

Assembly is not a single transformation; it is an ordered pipeline where each stage consumes validated inputs and emits an artifact plus a provenance record. The organizing principle is that composition is deterministic and every decision is logged. Given the same validated data, the same template versions, and the same agency profile, the assembler must produce a byte-identical package and an identical manifest — because a package you cannot reproduce is a package you cannot defend when a sponsored programs office contests it. Five concerns make up the layer, each with its own dedicated workflow, and a sixth cross-cutting concern that records all of them.

The first concern is turning structured data into prose. Narrative Template Rendering assembles the Specific Aims, Project Description, and boilerplate sections from validated requirement data using a templating engine such as Jinja2 or docxtpl, so that a facilities statement or a data-management narrative is generated from a single source of truth rather than copy-pasted between proposals. Rendering is where placeholder substitution, conditional boilerplate, and per-opportunity variation are resolved into finished text.

The second concern is machine-readable forms. Agency Form Population fills the agency’s fillable forms — the SF424 (R&R), the PHS 398 supplemental forms, the R&R Budget — programmatically, mapping validated fields onto the exact form field names the portal expects. Forms are unforgiving: a mismatched enumeration value or a truncated organizational identifier fails silently at render time and loudly at submission time.

The third concern is the final rendered artifact. PDF Package Generation composes the rendered narrative and populated forms into the submittable PDF package, adding bookmarks and navigation, stamping page numbers and identifiers, and — where an agency or archive requires it — enforcing PDF/A archival conformance with libraries such as reportlab, WeasyPrint, and pikepdf. This is where font embedding, flattening, and encryption removal are enforced, because a portal will reject a package whose fonts are not embedded or whose security settings block extraction.

The fourth concern is human authority. No package leaves an institution on the strength of an automated pipeline alone; an AOR must certify it. Institutional Approval Routing models the internal Sponsored Programs Office sign-off as an explicit state machine — draft, routed, department-approved, AOR-certified, released — so that the transition from “assembled” to “authorized to submit” is auditable and cannot be skipped.

The fifth concern is memory. Audit Logging & Provenance records every assembly decision as an immutable, tamper-evident entry: which template rendered which section, which data snapshot fed it, which compliance verdict cleared it, and what the resulting bytes hashed to. It is the layer that makes version diffs across drafts and provenance across funding cycles possible, and it is deliberately drawn as a cross-cutting ledger below, because every other stage writes to it.

Once assembled and certified, the package leaves this section and enters transmission — the submission portal sync layer that pushes it to Grants.gov or Research.gov and polls for acknowledgment. Assembly ends at the moment of release; what happens on the wire belongs to that adjacent section.

# Agency-specific constraint matrix

Assembly cannot be a single hardcoded routine because NIH, NSF, and DoD expect fundamentally different output artifacts, encoded differently, transmitted through different portals, and retained under different rules. The assembler must be parameterized by an agency profile so that switching the active agency switches the form set, the render format, the navigation expectations, and the archival requirements all at once. The matrix below captures the load-bearing differences for the output concern specifically — not the upstream formatting rules, but what the finished package has to be.

The agency identifiers in this table are the same profiles the upstream layers produce. NIH form semantics originate in the NIH FOA schema mapping, NSF structured fields in the NSF proposal guide taxonomy, and DoD volume structure in DoD BAA requirement extraction. Budget-specific output — the R&R Budget form and its justification narrative — draws its cross-agency layout rules from the budget justification format standards section, because the same justification content has to be rendered into an NIH modular grid, an NSF line-item table, and a DoD cost volume from one structured source. Keeping the assembler agnostic to which column it is rendering is what lets one institution ship all three agencies through a single, testable pipeline.

# Conditional logic and branching rules

The single most consequential decision in assembly is what artifact to produce, and that branches on the target agency and its submission portal. A proposal bound for NIH must become a flattened PDF form package; the same scientific content bound for NSF becomes structured web-form data with a handful of PDF uploads; bound for a DoD BAA it becomes a set of separately-capped technical volumes. These are not cosmetic differences — they select different renderers, different validators, and different release paths. The assembler resolves this branch first, before any section is rendered, so that every downstream stage operates in the correct output mode.

Three kinds of branching dominate the output stage. Format branching is the top-level fork above: PDF form package versus structured entry versus multi-volume submission, keyed on the agency profile and the portal it targets. Scope-activated components add artifacts to the package when the proposal touches a particular scope — an NIH human-subjects study injects the PHS Human Subjects and Clinical Trials form; an NSF proposal with postdocs requires a Mentoring Plan upload; a DoD proposal with foreign collaborators activates export-control markings. Version-activated templates select the template revision in force on the deadline, so that re-rendering an archived proposal reproduces the layout it was actually submitted with, not today’s template. This mirrors the version-aware logic in the compliance validation rule engine, and for good reason: the verdict and the artifact must agree on which policy version they were built against.

The diagram below shows how the top-level format branch fans a validated, compliant package into three agency-specific output paths, each converging on release once its portal-specific artifact is produced.

# Production pipeline implementation

A production assembler makes composition a pure function of its inputs. It sorts components by an explicit order, hashes every rendered artifact, binds the package to the compliance verdict that cleared it, and emits a manifest that can regenerate — or at least fully account for — the transmitted bytes. The Pydantic v2 models below define an assembled-package manifest, and the PackageAssembler composes components deterministically. The manifest digest is a function of the ordered component hashes and the policy version, so two runs on different machines produce the same digest, which is precisely what the audit layer needs to prove reproducibility. This shares its typed-model discipline with the Pydantic validation layer that hardens data on the way in.

from __future__ import annotations

import hashlib
from datetime import datetime, timezone
from enum import Enum

from pydantic import BaseModel, Field, field_validator


class Agency(str, Enum):
    NIH = "NIH"
    NSF = "NSF"
    DOD = "DoD"


class RenderFormat(str, Enum):
    PDF_PACKAGE = "pdf_package"       # NIH flattened SF424 (R&R) set
    STRUCTURED_ENTRY = "structured"   # NSF Research.gov web entry + uploads
    VOLUME_SET = "volume_set"         # DoD BAA technical/cost volumes


class ComponentKind(str, Enum):
    NARRATIVE = "narrative"           # rendered from a Jinja2/docxtpl template
    FORM = "form"                     # populated agency fillable form
    ATTACHMENT = "attachment"         # supporting PDF (biosketch, letter, plan)


_HEX64 = 64  # length of a hex-encoded SHA-256 digest


class AssembledComponent(BaseModel):
    """One rendered artifact plus the provenance of what produced it."""

    component_id: str
    kind: ComponentKind
    order: int                        # explicit position in the package
    source_hash: str                  # digest of the validated input snapshot
    render_sha256: str                # digest of the rendered bytes
    page_count: int = 0

    @field_validator("component_id")
    @classmethod
    def _normalize_id(cls, value: str) -> str:
        token = value.strip().lower()
        if not token:
            raise ValueError("component_id must be a non-empty token")
        return token

    @field_validator("source_hash", "render_sha256")
    @classmethod
    def _require_sha256(cls, value: str) -> str:
        token = value.strip().lower()
        if len(token) != _HEX64 or any(c not in "0123456789abcdef" for c in token):
            raise ValueError("hash must be a 64-character hex SHA-256 digest")
        return token

    @field_validator("order")
    @classmethod
    def _non_negative(cls, value: int) -> int:
        if value < 0:
            raise ValueError("order must be zero or positive")
        return value


class PackageManifest(BaseModel):
    """The auditable record of one assembled submission package."""

    package_id: str
    agency: Agency
    opportunity_id: str
    policy_version: str               # template/policy revision in force on deadline
    render_format: RenderFormat
    compliance_verdict_hash: str      # binds the package to the verdict that cleared it
    components: list[AssembledComponent]
    assembled_at: datetime = Field(
        default_factory=lambda: datetime.now(timezone.utc)
    )

    @field_validator("components")
    @classmethod
    def _unique_ordered(cls, value: list[AssembledComponent]) -> list[AssembledComponent]:
        orders = [c.order for c in value]
        if len(set(orders)) != len(orders):
            raise ValueError("component order values must be unique within a package")
        ids = [c.component_id for c in value]
        if len(set(ids)) != len(ids):
            raise ValueError("component_id values must be unique within a package")
        return value

    @property
    def manifest_digest(self) -> str:
        """Deterministic digest over ordered component hashes + policy version.

        Independent of insertion order: components are sorted by `order` first,
        so the same package always yields the same digest.
        """
        hasher = hashlib.sha256()
        hasher.update(self.policy_version.encode("utf-8"))
        hasher.update(self.render_format.value.encode("utf-8"))
        for component in sorted(self.components, key=lambda c: c.order):
            hasher.update(component.component_id.encode("utf-8"))
            hasher.update(component.render_sha256.encode("utf-8"))
        return hasher.hexdigest()


class PackageAssembler:
    """Compose validated, rendered components into a deterministic manifest."""

    # Per-agency default output format; a full system loads this from a profile.
    _FORMATS: dict[Agency, RenderFormat] = {
        Agency.NIH: RenderFormat.PDF_PACKAGE,
        Agency.NSF: RenderFormat.STRUCTURED_ENTRY,
        Agency.DOD: RenderFormat.VOLUME_SET,
    }

    def __init__(self, agency: Agency, policy_version: str) -> None:
        self.agency = agency
        self.policy_version = policy_version

    def assemble(
        self,
        package_id: str,
        opportunity_id: str,
        components: list[AssembledComponent],
        compliance_verdict_hash: str,
    ) -> PackageManifest:
        # Sort by explicit order so composition never depends on call order.
        ordered = sorted(components, key=lambda c: c.order)
        manifest = PackageManifest(
            package_id=package_id,
            agency=self.agency,
            opportunity_id=opportunity_id,
            policy_version=self.policy_version,
            render_format=self._FORMATS[self.agency],
            compliance_verdict_hash=compliance_verdict_hash,
            components=ordered,
        )
        return manifest


if __name__ == "__main__":
    zero = "0" * 64
    parts = [
        AssembledComponent(component_id="specific_aims", kind=ComponentKind.NARRATIVE,
                           order=0, source_hash=zero, render_sha256=zero, page_count=1),
        AssembledComponent(component_id="sf424_rr", kind=ComponentKind.FORM,
                           order=1, source_hash=zero, render_sha256=zero, page_count=3),
        AssembledComponent(component_id="research_strategy", kind=ComponentKind.NARRATIVE,
                           order=2, source_hash=zero, render_sha256=zero, page_count=12),
    ]
    assembler = PackageAssembler(Agency.NIH, policy_version="nih-2026-fy1")
    manifest = assembler.assemble(
        package_id="pkg-r01-0417",
        opportunity_id="PA-26-123",
        components=parts,
        compliance_verdict_hash=zero,
    )
    print(manifest.render_format.value, manifest.manifest_digest)

Three design choices carry the guarantees. Validating that every hash is a well-formed SHA-256 digest at construction means a manifest can never quietly reference a missing or malformed artifact. Enforcing unique order values means the package layout is unambiguous — there is exactly one correct sequence of components, not a set that happens to iterate in insertion order. And computing manifest_digest over sorted component hashes plus the policy version means the digest identifies the content and layout of the package independent of how the assembler was called, which is the anchor the audit layer signs.

# Institutional scale and failure modes

An assembler that produces one clean package on a developer’s laptop behaves very differently against a research portfolio pushing hundreds of proposals toward a shared deadline. Several failure modes appear only at institutional scale.

Portfolio contention on shared deadlines. NIH standard due dates and NSF program deadlines concentrate assembly load into narrow windows. Rendering, form population, and especially PDF generation are CPU-bound and can starve a single-threaded pipeline the night before a deadline. Because each package is independent, assembly parallelizes cleanly across workers — the same fan-out discipline used by the async batch processor for ingestion applies here, with the added constraint that each worker must still emit a deterministic manifest so two runs never disagree about what was built.

Multi-PI and multi-volume packages. A large center grant or a multi-institution BAA response assembles sections authored by many principal investigators, each with their own boilerplate and figure conventions. Without a shared canonical data model feeding the templates, the same nominal facilities statement arrives in three subtly different forms, and the assembler produces a package that is internally inconsistent. Normalizing to one structured source before rendering — the same discipline the narrative template rendering layer enforces — is what keeps a fifteen-component package coherent.

Template drift across cycles. Templates evolve: an office edits a facilities boilerplate, an agency revises a form, a budget table gains a column. When a template changes mid-cycle, a portfolio can briefly contain packages built from two template revisions at once. Selecting the template revision per package by effective policy version — never by a global “current” pointer — is what keeps an in-flight proposal from silently adopting a layout it was never reviewed against.

Partial-assembly and release-gate failures. At scale, some fraction of packages will fail late: a form field maps to an enumeration the portal rejects, a PDF fails PDF/A validation, an AOR is unavailable to certify before the cutoff. The pipeline must fail closed — a package that cannot be fully assembled and certified must be blocked from release and routed to a human queue with diagnostic metadata, never transmitted in a partial state. An irreversible submission of a broken package is worse than a missed internal target, because only the former burns the funding cycle.

# Audit and version control

Because a contested submission is defensible only if you can show exactly what was assembled, from what, and who authorized it, the audit layer treats every assembly run as a durable, versioned record rather than a transient build artifact. Three practices make that record hold up over the year-long life of an award.

Bind the package to its inputs and its verdict. Each manifest records the policy version, the compliance verdict hash, and a per-component source and render hash. Reproducing an archived submission with the same templates and data must regenerate the same manifest_digest; if it does not, either the data, the templates, or the policy version changed, and the differing hash tells you which. This extends across funding cycles the same reproducibility guarantee the compliance validation rule engine provides for verdicts, so the artifact and the verdict that cleared it can always be shown to agree.

Diff packages across drafts. As a proposal iterates toward its deadline, the useful signal is not the current package but the change from the last one — which components were re-rendered, which forms changed, whether a scope-activated form appeared or disappeared. Storing structured manifests rather than opaque PDFs turns a package diff into a straightforward comparison of component hashes, and that diff is exactly what the audit logging and provenance layer captures as tamper-evident entries, so a reviewer can see the assembly history without re-opening every draft.

Make approval and rollback first-class. The transition from assembled to authorized is an audited state change owned by the institutional approval routing state machine, and its signatures are written into the same ledger as the render hashes. When a template revision turns out to be defective, teams need to pin an affected portfolio back to the prior revision while a fix is validated; keeping templates and manifests immutable and addressable by version makes that rollback a pointer change rather than a reconstruction. Once released, the package is handed to the submission portal sync layer, and the acknowledgment it returns is appended to the same provenance chain — closing the loop from validated data to a transmitted, receipted, fully accounted-for submission.

# Related

• Narrative Template Rendering — assemble narrative and boilerplate from structured data with Jinja2 and docxtpl templates.
• Agency Form Population — fill SF424 (R&R), PHS 398, and R&R Budget forms programmatically from validated fields.
• PDF Package Generation — render the final PDF package with bookmarks, page stamping, and PDF/A conformance.
• Institutional Approval Routing — model the Sponsored Programs Office and AOR sign-off as an auditable state machine.
• Audit Logging & Provenance — immutable, tamper-evident logs, version diffs, and provenance across funding cycles.

Up: grant-automation.org home