Having recently built my home email server and
wanting to be a good MTA I decided to look a number of anti-spam mechanisms.
Whilst host-based anti-virus solutions and the like offer anti-spam engines,
there also exist a number of other technologies to help MTAs determine if email
is legit and which can have unintended consequences. There is also a brief look
at understanding the internals of an org without access it (recon).
Here I'll briefly cover four of them: SPF,
DKIM and DMARC.
The fourth is a a common practice that if you see email originating from you
(as defined by the From:
address) which did not originate from your server, you just discard the email.
A good number of organisations do this.
Finally, we will have a look at a couple of
examples.
The first three listed aren't always enforced.
However, that doesn't infer your email is going to get through. Those anti-spam
engines can and do use the results to rate the email, so a misconfigured setup
can result in mail been rated as spam or discarded, not by spf/dkim/dmarc
directly, but by the anti-spam mechanism.
Further, as you will see, to have a good
all-round solution, you will need DNSSec fully implemented.
There is a wealth of information on this
subject, including dmarc.org and Google itself. Just search
for it.
SPF
At a basic level, Sender Policy Framework (SPF)
works by a domain publishing a DNS TXT record in it's domain to describe who
sends mail on the behalf of the domain. Typically this ends with an all action which describes how
the receiving MTA should handle unmatched senders. For example:
m0noc.net text =
"v=spf1 +mx -all"
silentcircle.com text =
"v=spf1 include:spf.protection.outlook.com -all"
google.com text =
"v=spf1 include:_spf.google.com ~all"
yahoo.com text =
"v=spf1 ptr:yahoo.com ptr:yahoo.net ?all" # NB: the main rec is a
redirect
A minus preceding the all infers that you should
perform a a 'hard fail', a tilde infers a 'soft fail' and a question mark
infers treat it as if the SPF record didn't exist (i.e. is not a positive
validation; it is 'neutral').
Oftentimes, the receiving MTA does not discard
a mail, even if it is a hard fail. However, an anti-spam solution may make a determination or adversely
rank it. It is up to the policy that the receiver chooses to implement as to
how to proceed. My server rejects hard fails but currently allows soft fails
(which may get rated as spam by either my mail server or mail client).
As a result of this, and organisations
rejecting email that states it came from them, but from an unknown host,
mailing lists will often rewrite the From:
address so you see the on
behalf of statements in your email.
DKIM
At a basic level DKIM works by adding a mail
header to outgoing messages. This is an assertion to protect certain header
fields from being altered and to provide an assertion that it was signed by an
authorized host; for example protecting the From:
and Subject:
fields.
The linkage to the authorized host is via a selector. This is a hook to a DNS TXT
record. For example, a yahoo.com originated email may have the selector s2048,
so we prepend this to domain and the subdomain _domainkey to get published bit including the public key.
The following is truncated for brevity.
s2048._domainkey.yahoo.com text
= "k=rsa\; p=MIIBIjANBgkqhkiG.....”
With DKIM there is a lot of debate about what
mailing lists and forwarders do with this. As previously mentioned, due to a
number of reasons the From: field will get re-written. However, this will then
break the DKIM signature. So, do you keep it and risk a receiving MTA or
end-user anti-spam solution treating it as SPAM, or not.
DMARC
Domain-based Message Authentication, Reporting
& Conformance (DMARC) sits on top of SPF and DKIM. It allows a domain to
publish a policy on how to handle messages that fail SPF and DKIM tests, and
also publish a reporting function.
Like SPF and DKIM the policy is published as a
DNS TXT record. For example:
_dmarc.m0noc.net text =
"v=DMARC1\; p=reject\; pct=100\; rua=mailto:dmarc-feedback@m0noc.net\;
aspf=r\; adkim=r\;"
_dmarc.gmail.com text =
"v=DMARC1\; p=none\; rua=mailto:mailauth-reports@google.com"
_dmarc.google.com text =
"v=DMARC1\; p=reject\; rua=mailto:mailauth-reports@google.com"
_dmarc.yahoo.com text =
"v=DMARC1\; p=reject\; pct=100\; rua=mailto:dmarc_y_rua@yahoo.com\;"
The p=
part is the policy: none
infers to just report on it, quarantine
infers mark failed messages as spam, and reject
infers reject the message at the SMTP layer.
Again, it is up to the receiving MTA to
determine if they honour the policy or not.
Example:
Sending to (some) BT.com addresses
A recent email I sent to someone at bt.com had
a delivery receipt set. The receipt will contain an attachment which is the
header of the original message that
was sent, and revealed a few interesting things about the path it (most likely)
took. It also reveal a bit of information about how the network is set up and
what's running on them. Useful information you can publicly get hold of;
remember that the organisation voluntarily gave this information as part of a
standard email.
Here is an edited version containing the Received: headers.
Received: from HE1PR06CA0108.eurprd06.prod.outlook.com
(10.169.114.34) by
DB5PR06MB1495.eurprd06.prod.outlook.com
(10.164.40.141) with Microsoft SMTP
Server id 15.1.557.21
Received: from VE1EUR02FT011.eop-EUR02.prod.protection.outlook.com
(2a01:111:f400:7e06::204) by
HE1PR06CA0108.outlook.office365.com
(2a01:111:e400:7a1b::34)
with Microsoft SMTP Server id 15.1.587.13 via Frontend Transport
Authentication-Results: spf=fail (sender
IP is 62.239.224.235)
smtp.mailfrom=m0noc.net;
btgroupcloud.mail.onmicrosoft.com; dkim=fail
(signature did not verify)
header.d=m0noc.net;btgroupcloud.mail.onmicrosoft.com;
dmarc=fail
action=oreject
header.from=m0noc.net;bt.com; dkim=fail (signature did not
verify) header.d=m0noc.net;
Received-SPF: Fail (protection.outlook.com: domain of m0noc.net does not
designate
62.239.224.235 as permitted sender)
receiver=protection.outlook.com;
client-ip=62.239.224.235;
helo=RDW083A006ED62.bt.com;
Received: from RDW083A006ED62.bt.com (62.239.224.235) by
VE1EUR02FT011.mail.protection.outlook.com
(10.152.12.150) with Microsoft SMTP
Server id 15.1.587.6 via
Frontend Transport
Received: from EVMHT66-UKRD.domain1.systemhost.net (10.36.3.103) by
RDW083A006ED62.bt.com
(10.187.98.11) with Microsoft SMTP Server id
14.3.181.6
Received: from smtpe1.intersmtp.com (10.187.98.11) by
EVMHT66-UKRD.domain1.systemhost.net
(10.36.3.103) with Microsoft SMTP Server id 8.3.342.0
Received: from tbhext01.m0noc.net (82.70.154.45) by
smtpe1.intersmtp.com
(62.239.224.235) with
Microsoft SMTP Server id 14.3.181.6
We have to read these bottom up to track the
flow.
First we note from http://www.oxfordsbsguy.com/2014/01/27/exchange-server-and-update-rollups-build-numbers-2/
(or other sources) that a Microsoft SMTP server id of 15.1.x.x is probably
Exchange Server 2016; it seems that Microsoft are running a really new version
of Microsoft Exchange ;-)
We also note that 14.3.181.6 is Update Rollup 5 for Exchange Server 2010 SP3
and 8.3.342.0 is probably Update Rollup
12 for Exchange Server 2007 Service Pack 3.
Now for the interconnect.
82.70.154.45 is my mail server; the origin of
the email. This connected to smtpe1.intersmtp.com (ip address 62.239.224.235).
The next line up then states the message was received
from smtpe1.intersmtp.com (10.187.98.11) by a different host. This tells us
that the public interface for smtpe1.intersmtp.com has an IP address of
62.239.224.235 and an internal interface of 10.187.98.11 (which is also an
RFC1918 address; not valid on the Internet; hence internal – or there is a bit
of NAT). We can repeat this up the chain to reveal the likely path.
We can also see that smtpe1.intersmtp.com is
also known internally as RDW083A006ED62.bt.com (the reverse path with the same
IP's and the receiving path).
Finally, the outgoing path goes from a public
address to an rfc1918 address. This suggests some DMZ with a B2B link to
Microsoft (there are other possibilities).
In some cases, viewing whois records can also
offer some intelligence.
I will leave it as an exercise for the reader
to produce a diagram of the interconnect.
What is of particular interest is that the
email goes into BT then back out again to the Microsoft cloud.
This is where the problem occurs. Earlier on,
and further down in the header we see this:
Received-SPF: Pass (RDW083A006ED62.bt.com: domain of
REDACTED@m0noc.net
designates 82.70.154.45 as
permitted sender) receiver=RDW083A006ED62.bt.com;
client-ip=82.70.154.45;
helo=tbhext01.m0noc.net
It shows that when it was on its initial
incoming path the BT mail servers correctly validated the SPF rule. However, as
it went on its way out to the cloud this test was re-done (see the earlier
trace, key part of SPF highlighted in red), along with the embedded DKIM
signature and the DMARC policy for my domain. Accordingly, it all failed as it
was basing its results on one of the internal BT relays being authorized to
generate mail for my domain and had already munged some of the other fields
which broke DKIM as well, so DMARC then also failed.
Epic fail.
Raynet
So what could be worse? Perhaps ensuring that
any sender with an SPF fail policy is marked as spam just because you evaluated
the SPF policy against your Internet
facing email relay server?
This is what the Raynet mail forwarding service
appears to be doing; lets see if you
concur...
The reason I was looking at this one is that
none of my emails were getting through when sent to this account when it was
set up to forward to my ISP.
First, the original failure was because the
forwarder forwarded the email as-is, without changing the From: field. Accordingly my ISP
was discarding the email since the Raynet server isn't one of my ISP's email
servers, and the mail claims it originated from my ISP.
How do I know? Well I changed the forwarding to
my m0noc.net address and did the same thing. A mail sent came back (in the logs
at least); with two SPF fails. On marking it as SPAM by the second Raynet
server, and then me as the From:
field asserted it was from m0noc.net, but my SPF rules stated 'computer says
no'/hard fail.
The key headers (cut for brevity) are:
Received: from mail.raynet-uk.net (www.raynet-uk.net
[88.208.249.159])
Received: from amg1.gbse.gb.net (amg1.gbse.gb.net [91.234.185.121])
by mail.raynet-uk.net
Received: from tbhext01.m0noc.net ([82.70.154.45]:33182) by
amg1.gbse.gb.net
with esmtp (Exim 4.82_1-5b7a7c0-XX)
Subject: [SPAM] test
X-hMailServer-Spam: YES
X-hMailServer-Reason-1: Blocked by SPF () - (Score: 3)
X-hMailServer-Reason-Score: 3
As we can see, there are X-hMailServer headers that state that this
email is spam due to an SPF failure.
Now, given that I'm not running hMailServer (a
freeware mail server) and Raynet's first mail server is running Exim 4.82, we
can reasonably assume that 91.234.185.121 is relaying it to 88.208.249.159, and
it is this second server that is running hMailServer (we can
research that online).
Secondly, as I know my SPF records are valid
(others such as Google, Yahoo, etc all confirm it is OK), we can reasonably
assume that the second Raynet server is performing the SPF check against the
first Raynet server; which will of cause always fail where there is an SPF
record for the sending domain.
Whois is also interesting with this one.
No comments:
Post a Comment