# Polling eRA Commons for submission status updates

A National Institutes of Health (NIH) application that Grants.gov accepted is not yet a submitted application — it still has to pass through eRA Commons, where a second validation layer runs and where the principal investigator gets a short, unforgiving window to view the assembled image and reject it if something is wrong. Missing that window is how a technically-received proposal fails to reach peer review: the errors sat unread until the correction period expired. This guide builds a resilient poller that tracks an NIH submission through its Grants.gov and eRA Commons status transitions, deduplicates the events it sees, and raises an alert the moment an error appears — the tracking half of the submission portal sync loop, picking up exactly where submitting to Grants.gov with the System-to-System API leaves off with a tracking number in hand.

# Phase 1 — Map the status lifecycle and set up

Before writing a poller, model the states it can observe, because polling without a state model produces code that cannot tell “still working” from “silently failed.” An NIH submission moves through a small, well-defined lifecycle, and every transition means something operationally distinct.

The lifecycle to model:

1. Received. Grants.gov has accepted the transmission and issued its tracking number. Nothing about compliance is known yet — this only proves the bytes arrived.
2. Validated. Grants.gov’s own checks passed and the package is being handed to the agency. A package that fails here never reaches eRA Commons.
3. Agency Retrieved. eRA Commons has pulled the package and assigned it an accession, and now runs the NIH-specific validation that produces errors and warnings.
4. Errors present. eRA Commons found a blocking defect. The submission will not advance until the package is corrected and resubmitted — this is a terminal-until-fixed state.
5. Ready for viewing / no errors. Validation produced warnings at most, the assembled application image is available, and the two-business-day viewing window opens for the signing official to accept or reject the image.

Distinguishing an error from a warning is the entire point of the exercise: errors block, warnings do not, and a poller that treats them alike either cries wolf on every submission or misses a real rejection. Pin the environment so a polling run reproduces if it must be restarted:

python -m venv .venv
source .venv/bin/activate
pip install "requests>=2.31" "pydantic>=2.6" "tenacity>=8.2"

# Phase 2 — Build a resilient, deduplicating poller

A production poller reads the current status, normalizes it into a typed model, and records only genuinely new status events — the status service will report the same state many times before it changes, and writing every repeated read to the audit log buries the one transition that matters. The state machine below shows the transitions the poller drives between, and the polling loop that re-reads the transient states with backoff until it reaches a terminal one.

The typed status model normalizes whatever wording the status service returns into a stable enum, and separates errors from warnings so downstream logic can branch on severity:

from datetime import datetime
from enum import Enum
from pydantic import BaseModel, Field, field_validator

class SubmissionState(str, Enum):
    RECEIVED = "received"
    VALIDATED = "validated"
    RETRIEVED = "retrieved"      # eRA Commons has the package
    REJECTED = "rejected"        # blocking errors present
    READY = "ready_for_review"   # warnings at most; image viewable

TERMINAL = {SubmissionState.REJECTED, SubmissionState.READY}

class StatusEvent(BaseModel):
    """One normalized observation of a submission's status."""
    tracking_id: str
    state: SubmissionState
    observed_at: datetime
    errors: list[str] = Field(default_factory=list)
    warnings: list[str] = Field(default_factory=list)

    @field_validator("state")
    @classmethod
    def _ready_has_no_errors(cls, v: SubmissionState, info) -> SubmissionState:
        # A "ready for review" image must be clean — refuse a READY state that
        # still carries blocking errors so the two cannot be confused downstream.
        if v is SubmissionState.READY and info.data.get("errors"):
            raise ValueError("READY state cannot carry blocking errors")
        return v

    @property
    def is_terminal(self) -> bool:
        return self.state in TERMINAL

    def dedup_key(self) -> tuple[str, SubmissionState]:
        # Two reads of the same state for the same submission are the SAME event.
        return (self.tracking_id, self.state)

The poller drives the read on an exponential backoff, records only unseen dedup_key events, and stops the instant it reaches a terminal state or raises an alert on a rejection:

from tenacity import retry, stop_after_attempt, wait_exponential

class StatusPoller:
    def __init__(self, client, sink, alerter) -> None:
        self._client = client          # returns a raw status; maps to StatusEvent
        self._sink = sink              # append-only audit sink
        self._alerter = alerter        # notifies humans on a rejection
        self._seen: set[tuple[str, SubmissionState]] = set()

    def poll_until_terminal(self, tracking_id: str, max_reads: int = 20) -> StatusEvent:
        last: StatusEvent | None = None
        for _ in range(max_reads):
            event = self._read(tracking_id)
            if event.dedup_key() not in self._seen:
                self._seen.add(event.dedup_key())
                self._sink.append(event)             # only NEW transitions logged
                if event.state is SubmissionState.REJECTED:
                    self._alerter.rejection(event)   # surface errors immediately
            last = event
            if event.is_terminal:
                return event                          # stop on terminal state
        return last                                   # exhausted reads: still pending

    @retry(stop=stop_after_attempt(5),
           wait=wait_exponential(multiplier=2, min=4, max=60))
    def _read(self, tracking_id: str) -> StatusEvent:
        # Transient network/rate-limit failures are retried; a real status is
        # returned to the caller for dedup and terminal-state handling.
        return self._client.get_status(tracking_id)

Deduplication on (tracking_id, state) is what keeps the audit trail readable: the poller may read “retrieved” a dozen times while eRA Commons runs its checks, but only the first observation of each distinct state is recorded, so the log shows a clean progression rather than a wall of repeats. Every recorded event flows into audit logging and provenance so the institution can later prove exactly when each transition was observed.

# Phase 3 — Edge cases and agency-specific overrides

Four realities complicate a poller that only walks the happy path.

Errors versus warnings. eRA Commons emits both, and only errors block. A warning — a page that is close to a limit, a non-fatal formatting note — is informational and the application still advances. The poller must carry the two lists separately and alert only on errors; alerting on warnings trains the grant office to ignore the alerts entirely.

The two-business-day viewing window. Once eRA Commons assembles the application image without errors, the signing official has roughly two business days to view it and, if necessary, reject it and submit a corrected package — but only up to the deadline. The poller should compute the window from the retrieval timestamp and escalate as it nears expiry, because an image that is never viewed is accepted by default, defects and all.

Transient failures and rate limits. The status service will occasionally return a 5xx or throttle a client that polls too aggressively. Exponential backoff with a ceiling handles both: it retries transients without hammering the service, and it spaces reads far enough apart that a portfolio of submissions does not trip a rate limit. This is the same backoff-and-jitter discipline documented for high-volume runs in retry and rate-limit strategies for overnight RFP batches.

Polling cadence near the deadline. A submission read once an hour is fine days out but dangerously slow in the final hours, when a rejection leaves almost no time to correct. Tighten the interval as the deadline approaches so a late-breaking error is caught while there is still time to act.

# Phase 4 — Validation of the poller

Test the poller against a scripted status client, never a live submission. The three properties that make it trustworthy are: it deduplicates repeated states, it stops at a terminal state, and it alerts on a rejection.

import pytest
from datetime import datetime, timezone

def _event(tracking: str, state: SubmissionState, errors=None) -> StatusEvent:
    return StatusEvent(tracking_id=tracking, state=state,
                       observed_at=datetime.now(timezone.utc),
                       errors=errors or [])

class ScriptedClient:
    """Yields a fixed sequence of states, repeating the last one."""
    def __init__(self, states: list[SubmissionState]) -> None:
        self._states, self._i = states, 0
    def get_status(self, tracking_id: str) -> StatusEvent:
        state = self._states[min(self._i, len(self._states) - 1)]
        self._i += 1
        errs = ["E01: missing biosketch"] if state is SubmissionState.REJECTED else []
        return _event(tracking_id, state, errs)

class ListSink:
    def __init__(self) -> None: self.events: list[StatusEvent] = []
    def append(self, e: StatusEvent) -> None: self.events.append(e)

class RecordingAlerter:
    def __init__(self) -> None: self.alerts: list[StatusEvent] = []
    def rejection(self, e: StatusEvent) -> None: self.alerts.append(e)

def test_poller_dedups_repeated_states() -> None:
    client = ScriptedClient([SubmissionState.RECEIVED, SubmissionState.RECEIVED,
                             SubmissionState.VALIDATED, SubmissionState.READY])
    sink, alerter = ListSink(), RecordingAlerter()
    StatusPoller(client, sink, alerter).poll_until_terminal("T-1")
    logged = [e.state for e in sink.events]
    assert logged == [SubmissionState.RECEIVED, SubmissionState.VALIDATED,
                      SubmissionState.READY]          # no duplicate RECEIVED

def test_poller_stops_at_terminal_state() -> None:
    client = ScriptedClient([SubmissionState.READY, SubmissionState.RECEIVED])
    sink = ListSink()
    result = StatusPoller(client, sink, RecordingAlerter()).poll_until_terminal("T-2")
    assert result.state is SubmissionState.READY       # never read past terminal

def test_poller_alerts_on_rejection() -> None:
    client = ScriptedClient([SubmissionState.RETRIEVED, SubmissionState.REJECTED])
    alerter = RecordingAlerter()
    StatusPoller(client, ListSink(), alerter).poll_until_terminal("T-3")
    assert len(alerter.alerts) == 1
    assert alerter.alerts[0].errors                    # errors carried to the alert

A manual checklist covers the rest: confirm the viewing-window clock starts at the retrieval timestamp, that warnings never trigger an alert, that the polling interval tightens near the deadline, and that a throttled read backs off rather than retrying immediately. Only a poller that passes all four should be trusted to watch a real submission unattended.

Polling eRA Commons well converts the anxious wait after a submit into an observable, alerting process: every transition is captured once, a blocking error reaches a human the moment it appears, and the two-business-day window is never lost to an unread status. That is what turns a received package into a submission that actually reaches peer review.

# Frequently asked questions

# Related

• Submission portal sync — the full submit, track, and reconcile loop this poller completes.
• Submitting to Grants.gov with the System-to-System API — the submit step that produces the tracking number this poller reads.
• Retry and rate-limit strategies for overnight RFP batches — the backoff and jitter discipline reused for status polling.
• Audit logging and provenance — where every deduplicated status event is recorded for later proof.

Up one level: Submission Portal Sync