Skip to content

[BUG] - Payments taking a long time to clear, or don't clear #342

Closed

Description

@nQuiz

Hi guys - thanks for making your tool available to the Nostr community!

We'd like to report an issue we have observed with the 'paid relay' feature, which we have deployed to relay.nquiz.io

When we do a fresh deploy, everything works as it should.

But, after some time of usage, generating multiple invoices, of which some are paid, some not, some expired, some not - the relay stops processing payments. So, even though the user has paid the lightning invoice, the relay can't figure it out and the user doesn't get admitted (/invoices page never says "payment successful", QR code remains on screen)

After we notice that this is happening, we perform the following hack:

  1. Launch the docker nostream-db container bash: docker exec -it nostream-db bash
  2. Log in to the DB: psql -U nostr_ts_relay
  3. Remove all of the invoices: delete from invoices;

After these steps, the relay begins to admit users again.

Is this something you are aware of? If so, is there a fix planned, or are there any workarounds we can use?

System:

  • OS: [Ubuntu]
  • Platform: [docker]

Activity

  1. added
    bugSomething isn't working
    on Nov 6, 2023
  2. cameri commented on Jan 12, 2024

    @cameri
    Owner

    @nQuiz We are going to need more information to understand what is causing the issue.
    Which payment processor are you using?
    Can you provide us with a count of the number of invoices by status? e.g. SELECT status, COUNT(*) FROM invoices GROUP BY status
    Please enable debug logging when this is happening and provide the logs while attempting to pay an invoice. You can enable debug logging by setting the DEBUG environment variable to worker:*. Make sure to disable debug logging once you are done testing.

  3. Anshumancanrock commented on Aug 28, 2026

    @Anshumancanrock
    Contributor

    I dug into this one a bit. Pretty sure the issue isn’t the payments themselves, it’s the poller window.

    getPendingInvoices just grabs a fixed window of ten, oldest first. The maintenance worker hits it every 60s with no paging at all, so it keeps staring at the exact same ten rows every single tick. Once you’ve got ten that never resolve, nothing newer ever gets looked at.
    I ran the worker against a local DB with twelve stale invoices and three fresh ones, processor just reporting everything still unpaid:

    tick 1: old-00 old-01 old-02 ... old-09
    tick 5: old-00 old-01 old-02 ... old-09
    invoice states after 5 ticks: pending=15
    

    The three new ones never even get fetched.
    As for why they stick around forever: the only place the worker actually retires an invoice is in the catch block, and it needs both a 404 from the processor and isExpiredInvoice coming back true. That second check only passes when the invoice has a real expiry date. expires_at is nullable though, and the Zebedee mapping leaves it null whenever the response doesn’t carry an expiry. Same worker, same setup, but with the processor 404ing on two of them:

    null-expiry  expires_at null:true   -> status=pending
    past-expiry  expires_at null:false  -> status=expired
    

    So a 404 only cleans up invoices that actually have an expiry date. The null ones just sit there forever. Get ten of those and the poller is stuck, which matches exactly what @nQuiz described: works fine right after a fresh deploy, then dies later.

    @cameri I’m guessing @nQuiz’s payment processor is LNURL or Zebedee, since that null expiry comes straight out of its mapping and Zebedee is the default anyway. Status count should show ten or more stuck in pending. Debug logs would just keep saying “found 10 pending invoices” every tick with the same ten ids, never the one that’s actually getting paid.

    select status, count(*) from invoices group by status;
    select count(*) from invoices where status = 'pending' and expires_at is null;

    There are two fixes for it:

    1. The starvation stops if the offset advances on each tick and wraps around, so every
      pending invoice gets a turn. Worth keeping the window itself narrow, since it's
      doubling as a rate limiter and plain paging would hammer the processor API.

    2. The pile-up stops if invoices get retired off their own expiry date instead of only
      when the processor 404s, which also clears out whatever is already stuck on relays
      running today.

    Either one by itself should get payments clearing again. Happy to op3n a PR for whichever approach you’d rather go with.

  4. cameri commented on Sep 7, 2026

    @cameri
    Owner

    Thank you @Anshumancanrock !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions