# Core Architecture & RFP Taxonomy

Federal grant proposal automation fails at the seams where agency rules disagree. A submission built for the National Institutes of Health (NIH) will be administratively rejected by the National Science Foundation (NSF) for the wrong biosketch format; an NSF package will collapse against a Department of Defense (DoD) Broad Agency Announcement (BAA) that demands export-control attestations NSF never asks for. Without a rigorously defined core architecture paired with a precise Request for Proposal (RFP) taxonomy, these divergences surface at intake — inside Grants.gov, eRA Commons, or a contracting officer’s compliance review — where they are expensive, deadline-fatal, and impossible to remediate. This page defines the architecture that prevents that: a system that treats every funding opportunity as a structured data contract, where parsing, validation, and rendering are governed by agency-specific schemas rather than by generic templating or manual oversight.

Treating the solicitation as a contract turns proposal development from a fragmented, error-prone effort into a deterministic pipeline that scales across an institutional portfolio while preserving regulatory fidelity. The sections below map how the taxonomy layers fit together, tabulate the concrete NIH/NSF/DoD differences the architecture must reconcile, formalize the conditional logic that fires when agency rules override one another, and show the production Pydantic validation code, portfolio-scale failure modes, and audit trail that keep the system trustworthy across funding cycles.

# System Overview: How the Taxonomy Layers Fit Together

The foundation of any compliant automation system is a hierarchical taxonomy that maps unstructured funding announcements into machine-readable requirement sets. Federal solicitations embed explicit constraints inside narrative text, appendices, and cross-referenced policy documents. A robust taxonomy begins by isolating the solicitation type, then extracting its governing compliance matrix and routing it to the correct agency schema. Upstream of this architecture, raw documents are acquired and decomposed by the RFP ingestion and parsing workflows, which convert PDFs into structured text before the taxonomy assigns meaning to that text.

For biomedical and clinical research opportunities, the NIH FOA Schema Mapping process establishes the baseline for translating narrative constraints into validation rules. A Funding Opportunity Announcement (FOA) dictates precise project-narrative lengths, mandatory section ordering (Specific Aims, then Research Strategy), and strict typography — Arial, Helvetica, Palatino Linotype, or Georgia at an 11-point minimum with 0.5-inch margins. Parsing these constraints programmatically is what prevents an administrative rejection during intake, long before a study section evaluates scientific merit.

For foundational science proposals, automation operates under a highly standardized but rigidly enforced framework. Implementing the NSF Proposal Guide Taxonomy lets Python parsers dynamically adjust document-assembly parameters to the specific program solicitation. NSF compliance hinges on exact page limits, a biographical sketch limited to two pages per senior person in the NSF-approved format, and precise placement of the broader impacts, data management, and postdoctoral mentoring plans. Static templates fail here because NSF revises its Proposal & Award Policies & Procedures Guide (PAPPG) on a recurring cadence, so automated systems must ingest versioned policy deltas and propagate them to downstream validation engines.

Defense-related solicitations add layers of complexity through Broad Agency Announcements and topic-specific calls that mandate security classifications, proprietary-data handling, and cost-reasonableness justifications. The DoD BAA Requirement Extraction methodology shows how natural-language processing and rule-based parsers isolate mandatory deliverables, International Traffic in Arms Regulations (ITAR) and Export Administration Regulations (EAR) compliance triggers, and subcontracting limitations from dense narrative text.

Finally, budget compliance cuts across all three agencies and is one of the highest-risk failure points in automated generation. The Budget Justification Format Standards taxonomy isolates agency-specific financial schemas, mapping line-item categories to allowable-cost definitions under the federal Uniform Guidance (2 CFR Part 200). Once the taxonomy has classified a requirement, the structured output is handed to the compliance validation rule engines for enforcement — the taxonomy decides what a rule means, and the rule engine decides whether a given draft satisfies it. This separation of concerns is the load-bearing idea of the whole architecture: classification is stable and reusable, enforcement is where agency-specific verdicts are rendered.

# Agency Constraint Matrix: NIH vs NSF vs DoD

The taxonomy exists to reconcile concrete, divergent constraints. The matrix below captures the core dimensions the architecture must model for each agency. Every row is a field the schema must carry so that a single intermediate representation can render compliant output for any of the three funders.

Two properties of this matrix drive the code. First, most fields are agency-conditional, not universal — the schema must model absence (an NSF proposal has no modular-budget path) as cleanly as presence. Second, several fields are version-scoped: a page limit or font rule is only valid relative to a specific FOA reissue or PAPPG edition, which is why the taxonomy stores a policy version alongside every constraint and why the threshold tuning for compliance workflow keeps numeric limits externally configurable rather than hard-coded.

# Conditional Logic and Branching Rules

Where agencies diverge, the pipeline needs explicit branching rather than a lowest-common-denominator template. Defense automation is the sharpest example: the taxonomy must support boolean logic gates, because requirements activate only when a project crosses a funding threshold, involves foreign collaborators, or touches controlled technology. If a proposal exceeds a specified dollar threshold or involves foreign persons, the pipeline automatically injects the required cost-reasonableness narrative and export-compliance matrices. This conditional routing is what prevents the late-stage failures that otherwise surface during a contracting officer’s review.

The same branching discipline governs cross-agency overrides that are less obvious than defense triggers. An NIH modular budget suppresses the detailed line-item justification that NSF requires, so a portfolio that reuses one budget object across agencies must select the rendering path from the agency field rather than assume a single output shape. Financial normalization makes this tractable: a cross-agency normalization layer abstracts disparate inputs into a unified intermediate representation before rendering agency-specific outputs. By decoupling ingestion from presentation, administrators keep one source of truth for personnel effort, equipment depreciation, and fringe calculations, then dynamically generate compliant justifications for NIH modular budgets, NSF detailed budgets, or DoD cost-reimbursement structures. Every branch in this logic ultimately resolves to a rule the compliance validation rule engines can evaluate deterministically.

# Production Pipeline Implementation

A production-ready pipeline enforces schema validation before document generation, so non-compliant data fails fast and never reaches a renderer. The implementation below is a Pydantic v2 validation layer that enforces taxonomy-driven constraints; it is the same Pydantic validation layer pattern used across the ingestion pipeline, specialized here to agency taxonomy objects.

from pydantic import BaseModel, field_validator, ValidationInfo, ValidationError
from typing import Literal

class ProposalSection(BaseModel):
    section_id: str
    title: str
    max_pages: int
    # Font family is agency-specific: NIH allows Arial/Helvetica/Palatino Linotype/Georgia;
    # NSF allows Arial/Courier New/Palatino Linotype and similar. This field stores the
    # primary font declared at document-assembly time for audit purposes.
    font_family: str
    font_size: int
    content: str

    @field_validator("content")
    @classmethod
    def enforce_length(cls, v: str, info: ValidationInfo) -> str:
        # Rough page estimate: ~500 words/page at standard settings.
        word_count = len(v.split())
        max_words = info.data.get("max_pages", 1) * 500
        if word_count > max_words:
            title = info.data.get("title")
            raise ValueError(f"Section '{title}' exceeds {max_words}-word estimate.")
        return v

class AgencyTaxonomy(BaseModel):
    agency: Literal["NIH", "NSF", "DoD"]
    policy_version: str  # e.g. "PAPPG 24-1" or the FOA reissue id — pins constraints to a source
    sections: list[ProposalSection]
    requires_data_management_plan: bool = False
    requires_budget_justification: bool = True

    def validate_compliance(self) -> dict:
        """Return compliance status and any flagged violations."""
        violations: list[dict] = []
        for sec in self.sections:
            try:
                ProposalSection.model_validate(sec.model_dump())
            except ValidationError as exc:
                violations.append({"section": sec.title, "errors": exc.errors()})
        return {
            "agency": self.agency,
            "policy_version": self.policy_version,
            "compliant": len(violations) == 0,
            "violations": violations,
        }

def process_proposal(taxonomy_data: dict) -> dict:
    try:
        schema = AgencyTaxonomy(**taxonomy_data)
        return schema.validate_compliance()
    except ValidationError as exc:
        return {"status": "schema_invalid", "details": exc.errors()}

This layer integrates directly with document-generation engines such as python-docx or lxml, guaranteeing that rendered output matches the structural and typographic requirements declared in the taxonomy. Because the same objects that validate also carry a policy_version, institutions can run these checks as continuous-integration gates against draft proposals, flagging deviations well before a submission deadline. The full taxonomy-driven pipeline proceeds end to end as follows.

# Institutional Scale and Failure Modes

An architecture that validates one proposal cleanly can still break at portfolio scale, where the failure modes are structural rather than per-document. Three dominate.

Versioned policy drift. The most common portfolio failure is a silent PAPPG or FOA update that changes a page limit or an allowable-cost category mid-cycle while in-flight proposals still validate against the prior edition. Because every AgencyTaxonomy object pins a policy_version, a nightly reconciliation job can diff each active proposal’s pinned version against the currently published edition and quarantine any that reference a superseded rule set — turning a class of surprise rejections into a routine queue. Keeping the numeric limits themselves in an external, version-tagged store rather than in code is what makes the threshold tuning for compliance workflow safe to update without redeploying.

Multi-PI and multi-project fan-out. A single research administrator may shepherd dozens of concurrent submissions across all three agencies, each with several senior personnel who share biosketches and current-and-pending records. The failure mode is combinatorial: a biosketch valid as an NSF two-page document is invalid when reused in an NIH package, and effort commitments that sum cleanly per project can exceed 100% when a person appears on many. The intermediate representation must therefore model people and effort as first-class, cross-proposal entities, validated globally, not copied per document. Processing that volume within a submission window is exactly what the asynchronous batch processing for large RFPs pattern exists to handle.

Heterogeneous source quality. At scale the inputs are never uniform: scanned FOAs, malformed tables, and inconsistent section headers all degrade extraction accuracy. When the upstream PDF text extraction with pdfplumber stage returns low-confidence structure, the taxonomy must fail loudly — routing the document to human review rather than assembling a plausible-looking but non-compliant package. A pipeline that silently degrades is more dangerous at portfolio scale than one that halts, because a single bad template can propagate across every reuse.

The unifying principle is that scale converts individual mistakes into systemic ones. The architecture defends against this by validating shared entities once and globally, pinning every constraint to a source version, and preferring a hard halt over a soft, undetected compliance drift.

# Audit and Version Control

Deterministic assembly is only trustworthy if its decisions are reconstructable after the fact. Across funding cycles, reviewers, program officers, and internal auditors will ask why a given proposal was assembled a particular way — which rule set applied, which version, and what changed since the last submission. The architecture answers this by treating every compliance verdict as an append-only, diffable record rather than transient runtime state.

Each validation run emits an immutable record keyed by proposal id, agency, and policy_version, capturing the full violation set and a content hash of the assembled package. Storing these as structured events makes two operations cheap: diffing two cycles to see exactly which constraints changed, and rolling a proposal back to the last known-compliant state when a mid-cycle policy update breaks it.

from dataclasses import dataclass, field, asdict
from datetime import datetime, timezone
import hashlib
import json

@dataclass(frozen=True)
class ComplianceRecord:
    proposal_id: str
    agency: str
    policy_version: str
    compliant: bool
    violations: list[dict]
    content_hash: str
    recorded_at: str = field(
        default_factory=lambda: datetime.now(timezone.utc).isoformat()
    )

def snapshot(proposal_id: str, result: dict, rendered_bytes: bytes) -> ComplianceRecord:
    """Build an immutable audit record for a single validation run."""
    return ComplianceRecord(
        proposal_id=proposal_id,
        agency=result["agency"],
        policy_version=result["policy_version"],
        compliant=result["compliant"],
        violations=result["violations"],
        content_hash=hashlib.sha256(rendered_bytes).hexdigest(),
    )

def diff_records(prev: ComplianceRecord, curr: ComplianceRecord) -> dict:
    """Report what changed between two compliance snapshots of one proposal."""
    prev_v = {json.dumps(v, sort_keys=True) for v in prev.violations}
    curr_v = {json.dumps(v, sort_keys=True) for v in curr.violations}
    return {
        "proposal_id": curr.proposal_id,
        "policy_changed": prev.policy_version != curr.policy_version,
        "content_changed": prev.content_hash != curr.content_hash,
        "resolved": [json.loads(v) for v in prev_v - curr_v],
        "introduced": [json.loads(v) for v in curr_v - prev_v],
    }

With records shaped this way, diff_records isolates exactly which violations a revision resolved or introduced, and the content_hash proves whether the rendered artifact itself changed even when the verdict did not. The same audit stream feeds the broader automated checklist generation reporting, so program staff see a human-readable trail rather than raw events. Retaining these snapshots across cycles gives an institution genuine version control over compliance state — the ability to log every decision, diff any two funding cycles, and roll back to the last compliant assembly when policy shifts underneath an active proposal.

# Related

• NIH FOA Schema Mapping — translate narrative FOA constraints into validation rules
• NSF Proposal Guide Taxonomy — model versioned PAPPG requirements programmatically
• DoD BAA Requirement Extraction — isolate conditional defense obligations and export triggers
• Budget Justification Format Standards — normalize agency financial schemas to one representation
• Compliance Validation & Rule Engines — enforce the taxonomy’s rules against draft proposals
• RFP Ingestion & Parsing Workflows — acquire and decompose solicitations upstream of the taxonomy

Up: Grant automation home