Mail provider adds a few Received From lines with 127.0.0.1 making MailScanner into thinking the Sending IP is 127.0.0.1

[RobinR1]

I've set up MailScanner and MailWatch (with fetchmail - postfix - dovecot). And from docs on MailWatch I had to add 127.0.0.1 to the Whitelist to be able to release quarantined mails. I have several mail addresses from different providers, but I noticed that now all mails directed to one specific provider are always whitelisted because MailScanner thinks those are always sent from 127.0.0.1.

An example header of a mail from the problematic provider (real email addresses obfuscated):

Received: from hachiman.sicho.home (localhost)
     (no client certificate requested)
     by hachiman (MailScanner Milter) with SMTP id B4BF345C2
     for <[email protected]>; Sat, 15 Jan 2022 22:16:38 +0100 (CET)
Delivered-To: *******@disroot.org
Received: from disroot.org [178.21.23.139]
     by hachiman.sicho.home with IMAP (fetchmail-6.4.22)
     for <[email protected]> (single-drop); Sat, 15 Jan 2022 22:16:38 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
     by disroot.org (Postfix) with ESMTP id F1BB2535C0
     for <******@disroot.org>; Sat, 15 Jan 2022 22:16:36 +0100 (CET)
X-Virus-Scanned: SPAM Filter at disroot.org
Received: from knopi.disroot.org ([127.0.0.1])
     by localhost (disroot.org [127.0.0.1]) (amavisd-new, port 10024)
     with ESMTP id kLCyggn_9Grg for <******@disroot.org>;
     Sat, 15 Jan 2022 22:16:35 +0100 (CET)
X-Greylist: delayed 26808 seconds by postgrey-1.36 at mail01; Sat, 15 Jan 2022 22:16:34 CET
Authentication-Results: disroot.org;
     dkim=pass (1024-bit key; unprotected) header.d=statuspage.io [email protected] header.b="NTrkw0ST";
     dkim-atps=neutral
Received: from o5.notifications-sendgrid.statuspage.io (o5.notifications-sendgrid.statuspage.io [167.89.65.100])
     by disroot.org (Postfix) with ESMTPS id 6923F877B
     for <******@disroot.org>; Sat, 15 Jan 2022 22:16:34 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=statuspage.io;
     h=content-type:from:mime-version:subject:in-reply-to:references:to;
     s=smtpapi; bh=H1rFM4/FgPXmg9f3Vqthiw0yOklwwwIVZU4QvIjSNYM=;
     b=NTrkw0STsAHv34DJuC9boP9BCTTXaPr6OFxssKv9jXwNXvNj64UfK/nuIPfIVWcwduT7
     iDJb7oyu/Mf9ql9apNt/pD+tbBIYvSHSjOXHbckfaSMBzz98abmu+9w7ck0azjgSglki4n
     6SPcqiFFfTZiC2+eKIwyrYutSB40vucAg=
Received: by filterdrecv-69c466f987-gg9ph with SMTP id filterdrecv-69c466f987-gg9ph-1-61E339B1-15
2022-01-15 21:16:33.278851929 +0000 UTC m=+11746577.056673094
Received: from MTgzNDM5Mw (unknown)
     by geopod-ismtpd-3-0 (SG) with HTTP
     id NOv1WmuURVSfwSDwbYDLsg
     Sat, 15 Jan 2022 21:16:33.172 +0000 (UTC)

An example header from another provider without this problem:

Received: from hachiman.sicho.home (localhost)
     (no client certificate requested)
     by hachiman (MailScanner Milter) with SMTP id 26B184EBC
     for <[email protected]>; Mon, 17 Jan 2022 19:00:25 +0100 (CET)
Received: from imap.telenet.be [195.130.132.15]
     by hachiman.sicho.home with IMAP (fetchmail-6.4.22)
     for <[email protected]> (single-drop); Mon, 17 Jan 2022 19:00:25 +0100 (CET)
Received: from jules.telenet-ops.be (LHLO jules.telenet-ops.be)
(2a02:1800:120:4:0:0:f00:c) by zcsnocm102.telenet-ops.be with LMTP; Mon, 17
Jan 2022 19:00:24 +0100 (CET)
Received: from mailc-gc.linkedin.com ([108.174.0.163])
     by jules.telenet-ops.be with bizsmtp
     id ju0N2604Y3X0nTg01u0PVZ; Mon, 17 Jan 2022 19:00:24 +0100
X-Spamcause: gggruggvucftvghtrhhoucdtuddrgedvvddruddugddutdekucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuthgvlhgvnhgvthdrsggvnecuuegrihhlohhuthemuceftddtnecunecujfgurhephffkufggtgfvfffjsegrtdgssgdttdejnecuhfhrohhmpefnihhnkhgvugfknhcuoehnohhtihhfihgtrghtihhonhhsqdhnohhrvghplhihsehlihhnkhgvughinhdrtghomheqnecuggftrfgrthhtvghrnhepveefveeujeefgeduvdeuffetgfehteevkeeuieduiefgleffvdejfedttdejtefhnecuffhomhgrihhnpehlihhnkhgvughinhdrtghomhenucfkphepuddtkedrudejgedrtddrudeifeenucevlhhushhtvghrufhiiigvpeeggeenucfrrghrrghmpehinhgvthepuddtkedrudejgedrtddrudeifedphhgvlhhopehmrghilhgtqdhgtgdrlhhinhhkvgguihhnrdgtohhmpdhmrghilhhfrhhomhepshdqvdhfgehfvghnudhkhhhjphhkuddushhfjhhuiedukehvlhekphhjughpghhilheiphhgfhhkleifhiefjheklehftdhmghgvgeigiiejphiiqhhqsegsohhunhgtvgdrlhhinhhkvgguihhnrdgtohhmpdhsphhfpehprghsshdpughmrghrtgepphgrshhspdhnsggprhgtphhtthhopedupdhrtghpthhtoheprhhosghinhdrrhhovghvvghnshdusehtvghlvghnvghtrdgsvg
Delivered-To: *******@telenet.be

In MailScanner.conf Read IP Address From Received Header is still set to no since it didn't seem to be a problem using fetchmail in combination with the other mail providers. I assume MailScanner detects that Fetchmail is used and first tries to read the IP from the second Received: header despite that setting is set to no; And this works correctly for most mails from the other providers. So I can't use Read IP Address From Received Header = 4 as that does work for this specific provider but won't work for the others anymore as those, most of the time, only contain 2 Received: headers with IP addresses in, making MailScanner to use 127.0.0.1 again (I assume as fallback it takes the actual IP of the SMTP client which is fetchmail here).

I now worked around this by using postfix header_checks to strip out the localhost entries from that provider, so that the 2nd Received: header also on this provider would be the actual sending IP. Except when the provider itself sends me mail; since then there is only 1 Received header and MailScanner uses 127.0.0.1 again. But that is not that much of a problem as I trust that provider.

I assume this is not really a "bug" as such and more of an "enhancement request" as in a Fetchmail system it is probably always a bit of guessing which IP is the actual sending IP. But I think this could be solved if the logic when fetchmail is detected would be:

• Try to use the 2nd Received header, as it does now,
• if that IP is 127.0.0.1/localhost try the next,
• and the next...until a non-localhost IP is found.
• If no more Received headers are found: use the IP from the first Received header (the one Fetchmail added) which contains the IP of the mail provider itself;
• at last resort use 127.0.0.1.

Or would that introduce other problems ?

Server :

• OS: openSuse Leap 15.3
• Mandatory Access Control Enforcement: Apparmor
• MailScanner Version: 5.4.3
• OS Version: 15.3 - kernel 5.3.18-59.37-default
• Installation method: Package
• Installation: New
• Containerized: No

[github-actions]

Thank you for submitting your first issue to MailScanner! We will respond to you soon!

[shawniverson]

In MSMail.pm:

 $getipfromheader = 1 if $getipfromheader<1;
    $message->{clientip} = $rcvdiplist[$getipfromheader-1];
    $IPFound = 1;

Which basically says that if Read IP Address From Received Header is no (0), then bump to 1 (milter isn't passing SMTP client IP currently, and this is the same behavior with direct Postfix)

The regex to match the IP in the received header is:

/^Received: .+?\(.*?\[(?:IPv6:)?([0-9a-f.:]+)\]/i

It turns out the Fetchmail doesn't get matched because the IP in brackets is not in parenthesis, so the subsequent Received header is used. So, this is just a consequence of Fetchmail not formatting the Received header properly with comments in parenthesis.

According to https://datatracker.ietf.org/doc/html/rfc2822:

received        =       "Received:" name-val-list ";" date-time CRLF

name-val-list   =       [CFWS] [name-val-pair *(CFWS name-val-pair)]

name-val-pair   =       item-name CFWS item-value

item-name       =       ALPHA *(["-"] (ALPHA / DIGIT))

item-value      =       1*angle-addr / addr-spec /
                         atom / domain / msg-id

[Corrected]

...and...

comment         =       "(" *([FWS] ccontent) [FWS] ")"

CFWS            =       *([FWS] comment) (([FWS] comment) / FWS)

[shawniverson]

@RobinR1 The enhancement you suggest would require new logic to detect fetchmail. I'll need to ponder this some more.

[RobinR1]

Ok, so it is actually a coincidence that it works most af the time when fetchmail is used due to the fetchmail formatting 😅. I assumed it was already actively detected as it is (according the the docs) with BarricadeMX.

But yes, the logic I proposed needs fetchmail detection.
Maybe this can be avoided by adjusting the way the Read IP Address From Received Header setting works? It should be set to 2 by the user if fetchmail is in use.. (as it already would have been if the fetchmail header would have matched the regex). This way Mailscanner can start at the second, as specified by that setting, skip to the next if it is 127.0.0.1 until the last and as failover then check the first despite that setting being set at 2. ?