We had a problem with inbound mail taking up to 3 minutes to enter our inbox, even though we saw TMG logs getting the email within 5 seconds of it being sent.
If you have the edge Anti-spam configured and enabled, specifically “IP Block List Providers” & “Sender ID” you might get this latency.
We fixed it by creating an access rule that says:
From: Localhost
To: Internet
Protocol: DNS [not dns server]
Action: Allow
Emails are flowing in with no issues after this change.