Inbound mail not delivered from some senders

[Nexus6]

I'm using the docker image of the server. Today, after a shutdown of the server, a manual backup of the stalwart dir, pulling stalwart:latest (0.13.3), and a restart, some external senders can't send mail inbound to my server. For example, a message from the Outlook web app (actually live.com in my case) to an address (not an alias) on my Stalwart server looks like this:

2025-09-10T06:32:27Z INFO SMTP EHLO command (smtp.ehlo) listenerId = "smtp", localPort = 25, remoteIp = 40.92.21.53, remotePort = 65473, domain = "NAM12-BN8-obe.outbound.protection.outlook.com"
2025-09-10T06:32:27Z INFO SPF EHLO check passed (smtp.spf-ehlo-pass) listenerId = "smtp", localPort = 25, remoteIp = 40.92.21.53, remotePort = 65473, domain= "NAM12-BN8-obe.outbound.protection.outlook.com", result = SPF check passed (spf.pass), elapsed = 5ms
2025-09-10T06:32:27Z INFO TLS handshake (tls.handshake) listenerId = "smtp", localPort = 25, remoteIp = 40.92.21.53, remotePort = 65473, listenerId = "smtp", version = "TLSv1_3", details = "TLS13_AES_256_GCM_SHA384"

It appears that the MAIL FROM exchange is not even happening. This is occurring with the senders Protonmail, Posteo, Addy.io, and Riseup, so far anyway (those are all I've tested) and the logs always look just like the above. Strangely, mail from Gmail is received normally, as is mail from Yahoo. And mail from those two senders to alias addresses works as well.

My config is pretty nearly the default. I did update my letsencrypt TLS certificate many hours before the server update/restart, and everything seemed to be working normally after the new cert was processed. There's nothing in the logs that indicates TLS errors. At this point the failing senders are still retrying so eventually they'll get tired and I might wake up to a bunch of bounce messages with more clues. I updated the Web server and that seemed to go fine and I haven't noticed any problems with the latest one.

Apart from the inbound delivery problem Stalwart seems to be functioning normally, I can fetch messages with IMAPS as usual and can send mail outbound successfully.

Setting the logs to TRACE and restarting, I see, among many other things:
025-09-10T19:09:33Z INFO Processing ACME certificate (acme.process-cert) id = "letsencrypt", hostname = ["mail.stillwarm.net"], validFrom = 2025-09-09T22:15:59Z, validTo = 2025-12-08T22:15:58Z, due = 2025-11-08T22:15:58Z

The cert also appears in the management UI.

When an inbound port 25 connection from a sender happens, in this case addy.io, I see:

2025-09-10T19:11:47Z TRACE Raw SMTP output sent (smtp.raw-output) listenerId = "smtp", localPort = 25, remoteIp = 213.108.105.57, remotePort = 60032, size =211, contents = "250-mail.stillwarm.net you had me at EHLO\r\n250-STARTTLS\r\n250-SMTPUTF8\r\n250-SIZE 104857600\r\n250-REQUIRETLS\r\n250-PIPELINING\r\n250-NO-SOLICITING\r\n250-ENHANCEDSTATUSCODES\r\n250-CHUNKING\r\n250-BINARYMIME\r\n250 8BITMIME\r\n"
2025-09-10T19:11:47Z TRACE Raw SMTP input received (smtp.raw-input) listenerId = "smtp", localPort = 25, remoteIp = 213.108.105.57, remotePort = 60032, size= 10, contents = "STARTTLS\r\n"
2025-09-10T19:11:47Z DEBUG SMTP STARTTLS command (smtp.start-tls) listenerId = "smtp", localPort = 25, remoteIp = 213.108.105.57, remotePort = 60032
2025-09-10T19:11:47Z TRACE Raw SMTP output sent (smtp.raw-output) listenerId = "smtp", localPort = 25, remoteIp = 213.108.105.57, remotePort = 60032, size =31, contents = "220 2.0.0 Ready to start TLS.\r\n"
2025-09-10T19:11:47Z INFO TLS handshake (tls.handshake) listenerId = "smtp", localPort = 25, remoteIp = 213.108.105.57, remotePort = 60032, listenerId = "smtp", version = "TLSv1_3", details = "TLS13_AES_256_GCM_SHA384"
2025-09-10T19:11:47Z TRACE Raw SMTP input received (smtp.raw-input) listenerId = "smtp", localPort = 25, remoteIp = 213.108.105.57, remotePort = 60032, size= 6, contents = "QUIT\r\n"
2025-09-10T19:11:47Z DEBUG SMTP QUIT command (smtp.quit) listenerId = "smtp", localPort = 25, remoteIp = 213.108.105.57, remotePort = 60032
2025-09-10T19:11:47Z TRACE Raw SMTP output sent (smtp.raw-output) listenerId = "smtp", localPort = 25, remoteIp = 213.108.105.57, remotePort = 60032, size =16, contents = "221 2.0.0 Bye.\r\n"
2025-09-10T19:11:47Z DEBUG SMTP connection ended (smtp.connection-end) listenerId = "smtp", localPort = 25, remoteIp = 213.108.105.57, remotePort = 60032, elapsed = 94ms

I did have some bounce messages show up in the mailboxes at the external senders. One, from Protonmail, contains:
Diagnostic-Code: X-Postfix; Server certificate not trusted

Any idea what's happening here and how to fix?

[frenzybiscuit]

Yikes, I’ll hold off on updating until this gets addressed. Thank you for the info!

[max-baz]

I've just tried to send email from a @protonmail.com to my stalwart server running v0.13.3, and it has arrived without any issue 🤔

[Nexus6]

I don't think the problem that I'm seeing has to do with Proton specifically. Addy.io, Posteo, Outlook, and Riseup can no longer send inbound messages to my server either. My best speculation at this point is that either something went wrong in updating my TLS cert from Letsencrypt (even though there were no log errors and the web admin shows a cert), or something went wrong in the auto-migration when 0.13.3 was first run, or something in the cert-related part of the db (Rocks) is now corrupted, or there's a bug in the SMTP protocol handler involving TLS negotiation. I've no idea how to test any of this (trace logs don't show any errors during connection attempts, so ....?) or how any of these problems might be fixed on an otherwise functioning install that I don't want to blow away if I can avoid it.

[Nexus6]

In order to cast doubt on my TLS error theories, I tried:

$ openssl s_client -connect mail.stillwarm.net:25 -starttls smtp
CONNECTED(00000003)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = E8
verify return:1
depth=0 CN = mail.stillwarm.net
verify return:1
---
Certificate chain
0 s:CN = mail.stillwarm.net
i:C = US, O = Let's Encrypt, CN = E8
a:PKEY: id-ecPublicKey, 256 (bit); sigalg: ecdsa-with-SHA384
v:NotBefore: Sep  9 22:15:59 2025 GMT; NotAfter: Dec  8 22:15:58 2025 GMT
1 s:C = US, O = Let's Encrypt, CN = E8
i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
a:PKEY: id-ecPublicKey, 384 (bit); sigalg: RSA-SHA256
v:NotBefore: Mar 13 00:00:00 2024 GMT; NotAfter: Mar 12 23:59:59 2027 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
<snip/>
-----END CERTIFICATE-----
subject=CN = mail.stillwarm.net
issuer=C = US, O = Let's Encrypt, CN = E8
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2646 bytes and written 433 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 256 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
250 8BITMIME
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
Protocol  : TLSv1.3
Cipher    : TLS_AES_256_GCM_SHA384
Session-ID: E7D8E9FA735A18EEAEC8D064736D24FC88201C368C221662A8BEA61E81D662EB
Session-ID-ctx:
Resumption PSK: 66C0F230266095E759772359CA0CF1390A05C9A05D338CEE1562982E2729C4F49E9901FAB1065B9B7C716898FD115D1E
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 86400 (seconds)
TLS session ticket:
0000 - ad 37 f4 bf a9 b9 23 59-88 2c c1 d6 43 06 32 da   .7....#Y.,..C.2.
0010 - 92 fa 0f c7 3a 72 99 fc-42 59 15 4c 29 87 b8 64   ....:r..BY.L)..d

Start Time: 1757642296
Timeout   : 7200 (sec)
Verify return code: 0 (ok)
Extended master secret: no
Max Early Data: 0
---
read R BLOCK
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
Protocol  : TLSv1.3
Cipher    : TLS_AES_256_GCM_SHA384
Session-ID: 6E3CB05F8398D9446F83030EDC9CDE8E44755096A55217D4C32DFDE7F60BBD18
Session-ID-ctx:
Resumption PSK: 340DBDD061710A774457A18F70879EEB2A993D9EC1F1199FC68E92F666F235A562C2492C6D0095907306A0DEE6A71344
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 86400 (seconds)
TLS session ticket:
0000 - ec ae 3d ff 87 e7 f7 5f-dc 16 34 f5 63 96 e3 51   ..=...._..4.c..Q
0010 - 8e 08 2b 27 42 09 e7 eb-ba e1 c5 77 e0 b0 45 c5   ..+'B......w..E.

Start Time: 1757642296
Timeout   : 7200 (sec)
Verify return code: 0 (ok)
Extended master secret: no
Max Early Data: 0

This isn't at all in my wheelhouse, but, that said, it looks like the openssl app fetched the cert and was able to create a new session to the Stalwart instance. The cert appears to be the new one, just issued a few days ago.

If the problem I'm having doesn't have to do with establishing an inbound TLS session on port 25, what would be the next suspect?

[frenzybiscuit]

I updated and everything appears to be working fine on my end.

If you have issues with outlook mail, register for an outlook account and try to troubleshoot.

[Nexus6]

The problem turned out to be old/invalid TLSA records in my DNS config. After getting a new key (fetched via Stalwart), I did not know that I needed to update them. Deleting them fixed the problem, and adding back new TLSA records (associated w/the new key) did not resurface any problems. Apparently only some senders will refuse to send to a server with an invalid TLSA config while others will ignore the issue.

This problem was not related to the 0.13.3 release at all. It would be nice if Stalwart could reinforce (to the admin) the fact that these records need updating whenever what's in DNS doesn't match a new key. A notification of some sort to the admin. It's a foot-gun than many new admins might not have dealt with before.