# Building tamper-evident audit logs for grant submissions

An audit log becomes evidence only when a reader can prove that no record was altered after it was written, and a plain database table cannot make that promise: anyone with write access can edit a row and reset its updated_at, leaving no trace. This guide builds the alternative — a hash-chained, append-only audit log for grant submissions in which each record commits to the one before it, so that changing any historical entry is not prevented but is made mathematically detectable. It is the concrete implementation behind the audit logging & provenance topic, walked end to end: an event schema and a threat model, the hash chain itself, the edge cases that corrupt honest chains, and a test suite that proves a mutated record is caught. Every code block runs on Python 3.10+ with only Pydantic v2 added to the standard library.

# Phase 1 — Define the event schema and threat model

Before writing a single hash, decide precisely what “tamper-evident” is going to mean, because the term is often oversold. A hash chain does not prevent tampering — a determined administrator can still overwrite the whole store. What it guarantees is detection: any edit to a record that is not accompanied by recomputing every subsequent hash will fail verification, and recomputing every subsequent hash is impossible for anyone who does not control the published head value. That is the property a sponsored programs office actually needs when a submission to the National Institutes of Health (NIH), the National Science Foundation (NSF), or a Department of Defense (DoD) contracting officer is contested.

Make the threat model explicit so the design targets it directly.

The last row is the honest limitation: a chain proves internal consistency, but if an attacker can rewrite every record and the stored head, they can forge a self-consistent history. The countermeasure is to periodically anchor the head hash somewhere the attacker does not control — a countersignature, a second system, a printed record in the grant file. With that boundary understood, define the event schema. Each record carries a monotonically increasing seq, the event kind, the actor, a timezone-aware timestamp, an arbitrary payload, the prev_hash linking it to its predecessor, and its own record_hash.

Setup and steps:

1. Pin the environment so a verification run years later behaves identically to today’s.
2. Fix the genesis value — a constant all-zero hash that the first record chains from, so even record zero has a defined predecessor.
3. Freeze the record model so that once a record is constructed it cannot be mutated through the object API, catching accidental edits before they ever reach the store.

python -m venv .venv
source .venv/bin/activate
pip install "pydantic>=2.6"   # hashlib, hmac, json, datetime are standard library

from __future__ import annotations

from datetime import datetime, timezone
from enum import Enum
from typing import Any

from pydantic import BaseModel, ConfigDict, Field, field_validator

GENESIS = "0" * 64  # the defined predecessor of the very first record


class EventKind(str, Enum):
    ASSEMBLY_START = "assembly.start"
    VERDICT = "compliance.verdict"
    APPROVAL = "approval.signoff"
    SUBMISSION = "submission.sent"


class AuditRecord(BaseModel):
    model_config = ConfigDict(frozen=True)  # immutable once constructed

    seq: int = Field(ge=0)
    kind: EventKind
    actor: str
    at: datetime
    payload: dict[str, Any]
    prev_hash: str
    record_hash: str

    @field_validator("at")
    @classmethod
    def _aware(cls, value: datetime) -> datetime:
        if value.tzinfo is None:  # naive stamps cannot be ordered across hosts
            raise ValueError("timestamp 'at' must be timezone-aware")
        return value.astimezone(timezone.utc)

# Phase 2 — Implement the hash chain

The chain is built from two primitives: a canonical serializer that turns a payload into stable bytes, and a hashing step that folds the predecessor’s hash into the current record. The reason canonicalization comes first is that sha256 is exquisitely sensitive to bytes — reorder two dictionary keys and the digest changes completely, so a verifier that serializes differently from the writer will see tampering where there is none. Sorting keys and stripping incidental whitespace makes serialization a function of the data alone.

With canonical bytes in hand, each record’s hash is sha256(canonical(core_fields) + prev_hash). Folding in prev_hash is what creates the chain: because record n’s hash depends on record n−1’s hash, which depends on n−2, the head hash is a fingerprint of the entire history. The append function reads the current head, computes the new hash, and returns a frozen record; verify_chain walks the list and returns the sequence number of the first record whose recomputed hash disagrees with what is stored — or None when the chain is intact.

import hashlib
import json


def canonical(payload: dict[str, Any]) -> bytes:
    """Deterministic bytes for a payload: sorted keys, no incidental whitespace."""
    return json.dumps(
        payload, sort_keys=True, separators=(",", ":"), ensure_ascii=False
    ).encode("utf-8")


def hash_record(seq: int, kind: EventKind, actor: str, at: datetime,
                payload: dict[str, Any], prev_hash: str) -> str:
    """Fold the predecessor's hash into this record's canonical digest."""
    core = {
        "seq": seq,
        "kind": kind.value,
        "actor": actor,
        "at": at.astimezone(timezone.utc).isoformat(),  # normalize before hashing
        "payload": payload,
        "prev_hash": prev_hash,
    }
    return hashlib.sha256(canonical(core)).hexdigest()


def append(chain: list[AuditRecord], kind: EventKind, actor: str,
           payload: dict[str, Any]) -> AuditRecord:
    """Append one record, linking it to the current head of the chain."""
    prev = chain[-1].record_hash if chain else GENESIS
    seq = len(chain)
    at = datetime.now(timezone.utc)
    digest = hash_record(seq, kind, actor, at, payload, prev)
    record = AuditRecord(seq=seq, kind=kind, actor=actor, at=at,
                         payload=payload, prev_hash=prev, record_hash=digest)
    chain.append(record)  # append-only: never rewrite an existing record
    return record


def verify_chain(chain: list[AuditRecord]) -> int | None:
    """Return the seq of the first broken record, or None if the chain is intact."""
    prev = GENESIS
    for record in chain:
        if record.prev_hash != prev:  # link to predecessor was severed
            return record.seq
        recomputed = hash_record(record.seq, record.kind, record.actor,
                                 record.at, record.payload, record.prev_hash)
        if recomputed != record.record_hash:  # this record's bytes changed
            return record.seq
        prev = record.record_hash
    return None

The diagram below shows what verification catches. An attacker edits the payload of the middle record but cannot recompute the hashes that depend on it; when verify_chain recomputes that record’s digest from its altered payload, the result no longer matches the prev_hash the next record stored, and the break is pinpointed to the exact sequence number.

# Phase 3 — Edge cases and agency-specific overrides

A correct hash chain still fails in production for reasons that have nothing to do with attackers. Four edge cases account for most of them, plus one agency-specific override that reverses the usual advice.

Concurrent writers. A chain has exactly one head, so two processes appending at once will both read the same prev_hash and produce a fork — two records claiming the same predecessor. There is no clever hash trick that fixes this; the append operation must be serialized. In practice that means a single writer process, or a database transaction that reads the current head and inserts the next record atomically under a unique constraint on seq, so the second concurrent insert fails and retries against the new head. The approvals feeding this log arrive from asynchronous human steps modeled in modeling Sponsored Programs approval workflows, so a queue that funnels those events through one serialization point is usually the cleanest fit.

Canonical serialization pitfalls. The serializer is where honest chains most often break. Floating-point numbers are the classic trap — 1.0 and 1 are equal in Python but serialize differently, and a round-trip through a database can change which one you get, so store money and measurements as integers (cents, points) or fixed-precision strings rather than floats. Non-ASCII characters must be handled consistently, which is why ensure_ascii=False is fixed at both write and verify time. And any non-JSON-native type — a datetime, a Decimal, a set — must be converted to a canonical string form before hashing, never left to a default serializer whose output can change between library versions.

Key rotation when signing. Anchoring the head externally often means signing it, and signing introduces keys that must rotate. If every record simply carries an HMAC under the current key, rotating the key breaks verification of all older records. The fix is to record the key identifier alongside each signature so a verifier can select the right historical key, and to keep retired keys available for verification even after they stop signing new records. Rotate the signing key, never the hash chain: the chain stays valid across rotations because it depends on hashes, not keys.

import hmac

# Retired keys stay available for verification; only the active key signs new records.
KEYRING: dict[str, bytes] = {"2025-key": b"...", "2026-key": b"..."}
ACTIVE_KEY_ID = "2026-key"


def sign_head(head_hash: str, key_id: str = ACTIVE_KEY_ID) -> dict[str, str]:
    """Sign the current head so a wholesale rewrite is detectable, not just an edit."""
    mac = hmac.new(KEYRING[key_id], head_hash.encode("ascii"), "sha256").hexdigest()
    return {"head": head_hash, "key_id": key_id, "sig": mac}  # key_id enables rotation


def verify_head(anchor: dict[str, str]) -> bool:
    key = KEYRING.get(anchor["key_id"])  # select the key that actually signed it
    if key is None:
        return False
    expected = hmac.new(key, anchor["head"].encode("ascii"), "sha256").hexdigest()
    return hmac.compare_digest(expected, anchor["sig"])  # constant-time compare

PII and export-controlled data. Because the log is append-only, a Social Security number or International Traffic in Arms Regulations (ITAR) or Export Administration Regulations (EAR) controlled string written into a payload cannot be deleted later without breaking every downstream hash. Redact before hashing: replace the sensitive value with a hash reference, keep the real value in an access-controlled store, and validate every payload against an allowlist of loggable keys at append time. For a DoD Broad Agency Announcement (BAA) whose terms are extracted through DoD BAA requirement extraction, the override is to log less: record a hash proving the controlled artifact was handled, and let the chain reference it without ever containing it.

# Phase 4 — Validation and verification

The chain is only trustworthy if its detection guarantee is itself tested. The suite below proves the three properties that matter: an untouched chain verifies clean, a mutated middle record is caught at its exact sequence number, and the same inputs reproduce the same digests run after run. Because AuditRecord is frozen, the tamper test constructs a mutated copy rather than editing in place — which is precisely the maneuver a real attacker with store access would perform.

import pytest


def sample_chain() -> list[AuditRecord]:
    chain: list[AuditRecord] = []
    append(chain, EventKind.ASSEMBLY_START, "pipeline", {"proposal": "R01-0001"})
    append(chain, EventKind.VERDICT, "engine", {"passed": True, "score": 0})
    append(chain, EventKind.APPROVAL, "sp-office", {"approver": "jdoe"})
    append(chain, EventKind.SUBMISSION, "portal", {"pkg_sha256": "abc123"})
    return chain


def test_intact_chain_verifies() -> None:
    assert verify_chain(sample_chain()) is None


def test_mutated_middle_record_is_located() -> None:
    chain = sample_chain()
    tampered = chain[1].model_copy(update={"payload": {"passed": False, "score": 0}})
    chain[1] = tampered  # swap in an edited record, leaving its stale record_hash
    assert verify_chain(chain) == 1  # caught at the exact seq, not just "somewhere"


def test_digests_are_reproducible() -> None:
    at = datetime(2026, 7, 17, 9, 0, tzinfo=timezone.utc)
    args = (0, EventKind.VERDICT, "engine", at, {"passed": True}, GENESIS)
    assert hash_record(*args) == hash_record(*args)  # pure function of inputs


def test_key_rotation_preserves_verification() -> None:
    anchor = sign_head("9f3a1c", key_id="2025-key")  # signed under a retired key
    assert verify_head(anchor) is True  # still verifies after rotation to 2026-key

Round out the automated tests with a release checklist: confirm no payload carries raw PII or ITAR/EAR-controlled data, that numeric fields are stored as integers or fixed strings rather than floats, that appends are serialized through a single writer, that the head hash is anchored externally on a schedule, and that a detected break reports the first offending seq rather than failing blindly. The pkg_sha256 recorded in the submission event should match the hash of the exact bytes generated by PDF package generation; reconciling the two is what lets you prove which package was submitted. Where numeric tolerances enter a logged verdict, keep them owned by threshold tuning for compliance so the audited value and the rule that produced it never drift apart.

# Frequently asked questions

# Related

• Audit logging & provenance — the topic overview this build implements, including the provenance graph and retention policy.
• Modeling Sponsored Programs approval workflows in Python — produces the approval events this log serializes and records.
• Threshold tuning for compliance — owns the numeric tolerances behind the verdicts the chain captures.
• PDF package generation — the source of the package hash a submission record commits to.
• DoD BAA requirement extraction — where ITAR/EAR terms that drive controlled-data redaction originate.

Up one level: Audit Logging & Provenance