Submitting to Grants.gov with the System-to-System API
Grants.gov exposes a System-to-System (S2S) web-service interface that lets an institution transmit an application package directly from its own software instead of a human uploading a Workspace form by hand — and for high-volume research offices submitting to the National Institutes of Health (NIH), that interface is the difference between a controlled, logged, repeatable submission and a frantic browser session minutes before a deadline. This guide walks the submit half of that exchange end to end: assembling the application XML and its attachments, wrapping them in the S2S envelope, transmitting once, and capturing the Grants.gov tracking number that becomes your proof of receipt. It is the concrete transmit step behind the broader submission portal sync discipline, and the exact failure it prevents is the silent non-submission — a package that never actually reached the portal but was assumed to have.
Phase 1 — Establish authorization and build the package
The S2S submit call will reject you long before any XML is parsed if the institutional authorization is not in place, so the first phase is entirely about prerequisites and package construction — none of which the client code can shortcut.
Implementation steps:
- Confirm SAM.gov registration is active. The submitting organization must hold a current registration in the System for Award Management (SAM.gov) with a valid Unique Entity Identifier (UEI). A registration that lapses even a day before the deadline blocks submission across every federal portal, so make an active-registration check a hard pre-flight gate that runs before assembly, not a runtime discovery.
- Resolve the Authorized Organization Representative (AOR). Only a delegated AOR may submit on the organization’s behalf. The S2S client authenticates with an institutional certificate bound to that AOR authority — never a personal login — and the pipeline should verify the AOR delegation is current before transmitting.
- Assemble the application XML. The Grants.gov application is a single XML document that carries the cover components and every form as structured data. Do not hand-build it here: it is the serialized output of the agency form population stage, and the SF-424 (R&R) population walkthrough shows how the cover form and its fields are produced. The submit client consumes that XML as an opaque, validated artifact.
- Collect the attachments. The narrative PDFs, biosketches, and budget justification come from the PDF package generation stage. Each attachment is referenced by the application XML and transmitted alongside it.
- Capture the approval token. Transmission is permitted only for a package that institutional approval routing has certified, so the signing official’s sign-off travels with the envelope as evidence the AOR was authorized to submit this exact package.
Pin the environment so a submission attempted today reproduces exactly if it must be retried after a portal outage:
python -m venv .venv
source .venv/bin/activate
pip install "zeep>=4.2" "requests>=2.31" "pydantic>=2.6" "tenacity>=8.2"
Phase 2 — Construct and submit the S2S request
The Grants.gov S2S submit operation historically accepts a SOAP request whose body carries the application XML, with each attachment transmitted as a binary Message Transmission Optimization Mechanism (MTOM) part rather than base64-inlined — MTOM keeps a multi-hundred-megabyte package from ballooning by a third on the wire. The operation returns a Grants.gov tracking number, and that number is the single most important value the whole exchange produces: it is your timestamped receipt and the key you will later hand to the status service. The sequence below shows the two-call rhythm — submit, then a first status read — that every S2S submission follows.
The client wraps zeep for the SOAP transport but exposes a narrow typed surface, so the rest of the pipeline sees a submit method that takes a package and returns a receipt. The service endpoint is injected through configuration and never appears as a literal:
from datetime import datetime, timezone
from pydantic import BaseModel, Field
from zeep import Client
from zeep.transports import Transport
class GrantsGovPackage(BaseModel):
"""A ready-to-transmit application: XML plus attachment file paths."""
opportunity_id: str = Field(pattern=r"^[A-Z0-9-]+$")
application_xml: str # from agency form population
attachment_paths: list[str] # from PDF package generation
package_sha256: str = Field(min_length=64, max_length=64)
class GrantsGovReceipt(BaseModel):
grants_gov_tracking_number: str # the timestamped proof of receipt
received_at: datetime
class GrantsGovS2SClient:
"""Thin typed wrapper over the SOAP submit/status operations."""
def __init__(self, wsdl_url: str, transport: Transport) -> None:
# wsdl_url is injected config, not a literal — swap it for a mock in tests.
self._client = Client(wsdl=wsdl_url, transport=transport)
def submit(self, pkg: GrantsGovPackage) -> GrantsGovReceipt:
# zeep streams each attachment as an MTOM part; large PDFs are not
# base64-inlined, so the wire size stays close to the file size.
response = self._client.service.SubmitApplication(
GrantsGovApplicationXML=pkg.application_xml,
Attachments=[open(p, "rb") for p in pkg.attachment_paths],
)
return GrantsGovReceipt(
grants_gov_tracking_number=response.GrantsGovTrackingNumber,
received_at=datetime.now(timezone.utc),
)
The submit is driven through an idempotency guard so an ambiguous timeout never produces two tracking numbers. The guard consults a durable ledger keyed by the package hash before every transmit; only a package it has never seen is actually sent:
from tenacity import retry, stop_after_attempt, wait_exponential
class GrantsGovSubmitter:
def __init__(self, client: GrantsGovS2SClient, ledger: dict[str, GrantsGovReceipt]) -> None:
self._client = client
self._ledger = ledger # durable in production: package_sha256 -> receipt
def submit_once(self, pkg: GrantsGovPackage) -> GrantsGovReceipt:
prior = self._ledger.get(pkg.package_sha256)
if prior is not None:
return prior # duplicate guard: already transmitted
receipt = self._transmit(pkg)
self._ledger[pkg.package_sha256] = receipt # record BEFORE returning
return receipt
@retry(stop=stop_after_attempt(4),
wait=wait_exponential(multiplier=2, min=2, max=30))
def _transmit(self, pkg: GrantsGovPackage) -> GrantsGovReceipt:
# Safe to retry at the transport level: the ledger check above makes
# logical submission at-most-once per distinct package.
return self._client.submit(pkg)
Once the tracking number is captured, the submit job is complete — but the submission is not yet safe. The number now feeds the status side of the exchange, covered in polling eRA Commons for submission status updates, because Grants.gov acceptance is only the first of two validation layers an NIH application must clear.
Phase 3 — Edge cases and agency-specific overrides
Four situations break a submit client that only handles the happy path, and each has a specific remedy.
Large attachments. A package with high-resolution figures or a lengthy appendix can exceed the portal’s per-message size expectations. MTOM transmission keeps the encoding overhead low, but the client must still stream attachments from disk rather than loading them all into memory, and it should surface a clear size-limit error rather than timing out opaquely.
XML schema version drift. The Grants.gov application schema is versioned, and a package built against a superseded schema version is rejected at validation even when every field is correct. Read the required schema version from the opportunity’s form set and stamp it into the application XML at assembly time; never assume the version that worked last cycle still applies.
Resubmission and replacement. If a submitted package is found to be defective before the deadline, the correct action is a replacement submission that supersedes the prior one, not a blind resend. A replacement carries the original tracking context and produces a new tracking number; the idempotency ledger must key on the corrected package’s hash so the fix is recognized as a distinct, intentional transmission rather than suppressed as a duplicate.
Deadline-window contention. In the final hour before a common deadline, the portal slows under load. Distinguish a transient server error (retry with backoff, keep trying until the cutoff) from a validation rejection (stop and correct), and record every attempt with its timestamp so the institution has evidence if it must request a late-submission accommodation. The retry and backoff discipline here is the same one documented for high-volume runs in retry and rate-limit strategies for overnight RFP batches.
| Edge case | Trigger | Correct handling |
|---|---|---|
| Large attachment | Package exceeds message size expectation | Stream MTOM parts from disk; surface an explicit size error |
| Schema version drift | Application XML built to a superseded schema | Read required version from the opportunity form set; stamp at assembly |
| Resubmission | Defect found before deadline | Replacement submission, new hash, new tracking number |
| Deadline contention | Portal slow/503 near cutoff |
Backoff-retry transients, escalate rejections, log every attempt |
Phase 4 — Validation against a mocked S2S endpoint
Never test a submit client against the live portal — a real transmission is a legal event, and near a deadline it is a hazard. Drive the client with a mock that returns a scripted tracking number, then assert the two properties that make the client safe: submission is idempotent, and the tracking number is captured and persisted.
import pytest
from datetime import datetime, timezone
class MockS2SClient:
"""Deterministic stand-in that counts transmissions."""
def __init__(self) -> None:
self.calls = 0
def submit(self, pkg: "GrantsGovPackage") -> "GrantsGovReceipt":
self.calls += 1
return GrantsGovReceipt(grants_gov_tracking_number="GRANT12345678",
received_at=datetime.now(timezone.utc))
def _pkg() -> GrantsGovPackage:
return GrantsGovPackage(opportunity_id="PA-24-001",
application_xml="<Grants><App/></Grants>",
attachment_paths=[],
package_sha256="b" * 64)
def test_submit_captures_tracking_number() -> None:
submitter = GrantsGovSubmitter(MockS2SClient(), ledger={})
receipt = submitter.submit_once(_pkg())
assert receipt.grants_gov_tracking_number == "GRANT12345678"
def test_resubmit_of_same_package_is_idempotent() -> None:
client, ledger = MockS2SClient(), {}
submitter = GrantsGovSubmitter(client, ledger)
first = submitter.submit_once(_pkg())
second = submitter.submit_once(_pkg()) # identical bytes
assert first.grants_gov_tracking_number == second.grants_gov_tracking_number
assert client.calls == 1 # transmitted exactly once
def test_corrected_package_transmits_again() -> None:
client, ledger = MockS2SClient(), {}
submitter = GrantsGovSubmitter(client, ledger)
submitter.submit_once(_pkg())
corrected = _pkg().model_copy(update={"package_sha256": "c" * 64})
submitter.submit_once(corrected) # a genuine replacement
assert client.calls == 2
A manual release checklist covers what the mock cannot: confirm the SAM.gov and AOR pre-flight gates run before assembly, that the application XML carries the correct schema version, that every tracking number is written to audit logging and provenance the instant it returns, and that a forced timeout does not produce a second submission. Only with those green should the client be aimed at the live S2S endpoint in the submission window.
Submitting through the Grants.gov S2S API turns the most consequential moment of a funding cycle into a deterministic, logged, idempotent transmission — a package leaves the institution exactly once, a tracking number returns as proof, and the status machinery takes over from there.
Frequently asked questions
Is the Grants.gov tracking number the same as the eRA Commons accession?
No. Grants.gov issues its tracking number the moment it accepts the transmission — that is your first proof of receipt. For NIH applications, eRA Commons then retrieves the package and issues its own accession number, and runs a second layer of validation. You capture the Grants.gov number here and use it to query status, but the eRA Commons errors and warnings are what ultimately decide whether the application reaches peer review.
Why encode attachments as MTOM parts instead of base64 inside the XML?
Base64-inlining a binary attachment inflates it by roughly a third and forces the whole package through the XML parser as one giant string. MTOM transmits each attachment as a separate binary part referenced from the XML, so a large figure-heavy PDF stays close to its on-disk size and streams rather than being buffered whole. On a multi-hundred-megabyte package near a deadline, that difference is the margin between a clean submit and a timeout.
What happens if I submit the same package twice?
Without a guard, you get two Grants.gov tracking numbers for one proposal — a duplicate that a program officer must reconcile by hand. The idempotency ledger keyed on the package content hash prevents it: a retry of identical bytes returns the stored receipt instead of transmitting again. A genuine correction has a different hash, so it is correctly recognized as a new, intentional replacement.
How do I know which application XML schema version to build against?
Read it from the opportunity’s form set rather than assuming last cycle’s version. Grants.gov versions the application schema, and a package built to a superseded version is rejected at validation even if every field is correct. Stamp the required version into the XML at assembly time, cross-checked against the populated SF-424 (R&R) form set.
Can I test this against the real Grants.gov endpoint?
No — treat a live submit as a legal event, not a test. Drive the client with a mock that returns a scripted tracking number and assert idempotency and tracking-number capture. Reserve the live endpoint for the actual submission, and only after the pre-flight authorization gates pass.
Related
- Submission portal sync — the full submit, track, and reconcile discipline this transmit step belongs to.
- Polling eRA Commons for submission status updates — what the tracking number feeds once submission succeeds.
- Populating SF-424 (R&R) forms programmatically — the source of the application XML this client transmits.
- PDF package generation — the attachments the submit call carries.
- Retry and rate-limit strategies for overnight RFP batches — the backoff discipline reused for deadline-window contention.
Up one level: Submission Portal Sync