# Incoming webhooks ## What this builds The receiving end for webhooks that other services send an app: a payment provider, a code host, an email service. Each request is checked against its signature, stored, answered at once, and processed afterwards exactly once, with retries when processing fails. Read `spekkr.inputs.json` first. A signed time may be `tolerance_seconds` seconds off the clock. `dashboard` decides whether R8 exists. ## Out of scope - Sending webhooks. `spekkr/outgoing-webhooks` builds that, and the first signature scheme here is the one it sends. - The handlers for any one service's events. The spec builds one example source and one example handler. ## Requirements Each requirement has an id. Tests name the id they cover. ### R1. Sources - A source is a service that sends webhooks. Sources are a list in code: a name, a signature scheme from `design.md`, the header the signature comes in, the environment variable that holds the secret, and where in the body the event's id and type are. - Each source has its own address, `POST /webhooks/NAME`. A name that is not in the list gets 404. - The example source is called `example` and uses the scheme `timed`. Accept when: a request to an unknown name gets 404 and stores nothing. ### R2. Check the signature - The signature is checked against the raw bytes of the body, before the body is parsed. - Signatures are compared in constant time. - For a scheme that signs a time, a request whose time is more than `tolerance_seconds` seconds from the clock is refused, even with a good signature. - A source can have two secrets, the second in a variable with `_OLD` added, so a secret can be changed without losing events. A signature made with either passes. - A request that fails gets status 400 and is not stored. Why it failed is logged without the secret or the signature. Accept when: both test vectors in `design.md` pass, a body changed by one byte fails, a good signature with a time 301 seconds old fails, and a signature made with the old secret passes. ### R3. Store and answer - A request that passes is stored with its source, the event's id and type, its headers and its raw body, and answered with status 200 and an empty body. - Nothing else happens before the answer. Handlers run afterwards. - A body over 1 MB (1,000,000 bytes) gets status 413. Accept when: a handler that takes 10 seconds does not delay the answer. ### R4. Once - An event is known by its source and its id. A second request with the same two is answered with 200 and neither stored nor processed again. - A source whose events carry no id uses the SHA-256 of the body. Accept when: the same event sent 5 times at once is stored once and its handler runs once. ### R5. Handlers - A handler is registered for a source and an event type. An event with no handler is stored and marked ignored. - A handler gets the parsed body and the time the event says it happened. It must not assume events arrive in the order they happened. - The example handler counts the events it has seen. Accept when: an event of a type with no handler is marked ignored and nothing fails. ### R6. Retry - A handler that raises an error is tried again after 1 minute, 5 minutes, 30 minutes, 2 hours and 8 hours. After the sixth failure the event is marked failed. - Each failure stores the first 2,000 characters of the error. - An event being processed when the process dies is processed again. That counts as an attempt. Accept when: a handler that fails twice and then succeeds is run three times with waits of 60 and 300 seconds, and one that always fails is run six times. ### R7. Keeping and forgetting - Events are kept for 30 days and then deleted, bodies and all. Accept when: a clean-up run deletes exactly the events that are old enough. ### R8. Dashboard Only if `dashboard` is on. - A list of events: source, type, when, state and attempts, newest first, with a filter for each source and each state. - One event's page shows its headers, its body and each failure. - "Process again" runs the handler once more, whatever the state. - Only administrators reach it. Bodies can hold personal data. Accept when: "Process again" on a failed event runs its handler, and a person who is not an administrator gets 403. ## Definition of done - Every acceptance check above has an automated test, and all tests pass. Tests set the clock and use no outside network. - No log line holds a secret or a whole signature. - A README explains how to add a source and a handler, how to change a secret without losing events, and how to run the tests.