Having recently built my home email server and
wanting to be a good MTA I decided to look a number of anti-spam mechanisms.
Whilst host-based anti-virus solutions and the like offer anti-spam engines,
there also exist a number of other technologies to help MTAs determine if email
is legit and which can have unintended consequences. There is also a brief look
at understanding the internals of an org without access it (recon).
Here I'll briefly cover four of them: SPF,
DKIM and DMARC.
The fourth is a a common practice that if you see email originating from you
(as defined by the From:
address) which did not originate from your server, you just discard the email.
A good number of organisations do this.
Finally, we will have a look at a couple of
The first three listed aren't always enforced.
However, that doesn't infer your email is going to get through. Those anti-spam
engines can and do use the results to rate the email, so a misconfigured setup
can result in mail been rated as spam or discarded, not by spf/dkim/dmarc
directly, but by the anti-spam mechanism.
Further, as you will see, to have a good
all-round solution, you will need DNSSec fully implemented.
There is a wealth of information on this
subject, including dmarc.org and Google itself. Just search
At a basic level, Sender Policy Framework (SPF)
works by a domain publishing a DNS TXT record in it's domain to describe who
sends mail on the behalf of the domain. Typically this ends with an all action which describes how
the receiving MTA should handle unmatched senders. For example:
m0noc.net text =
"v=spf1 +mx -all"
silentcircle.com text =
"v=spf1 include:spf.protection.outlook.com -all"
google.com text =
"v=spf1 include:_spf.google.com ~all"
yahoo.com text =
"v=spf1 ptr:yahoo.com ptr:yahoo.net ?all" # NB: the main rec is a
A minus preceding the all infers that you should
perform a a 'hard fail', a tilde infers a 'soft fail' and a question mark
infers treat it as if the SPF record didn't exist (i.e. is not a positive
validation; it is 'neutral').
Oftentimes, the receiving MTA does not discard
a mail, even if it is a hard fail. However, an anti-spam solution may make a determination or adversely
rank it. It is up to the policy that the receiver chooses to implement as to
how to proceed. My server rejects hard fails but currently allows soft fails
(which may get rated as spam by either my mail server or mail client).
As a result of this, and organisations
rejecting email that states it came from them, but from an unknown host,
mailing lists will often rewrite the From:
address so you see the on
behalf of statements in your email.
At a basic level DKIM works by adding a mail
header to outgoing messages. This is an assertion to protect certain header
fields from being altered and to provide an assertion that it was signed by an
authorized host; for example protecting the From:
The linkage to the authorized host is via a selector. This is a hook to a DNS TXT
record. For example, a yahoo.com originated email may have the selector s2048,
so we prepend this to domain and the subdomain _domainkey to get published bit including the public key.
The following is truncated for brevity.
= "k=rsa\; p=MIIBIjANBgkqhkiG.....”
With DKIM there is a lot of debate about what
mailing lists and forwarders do with this. As previously mentioned, due to a
number of reasons the From: field will get re-written. However, this will then
break the DKIM signature. So, do you keep it and risk a receiving MTA or
end-user anti-spam solution treating it as SPAM, or not.
Domain-based Message Authentication, Reporting
& Conformance (DMARC) sits on top of SPF and DKIM. It allows a domain to
publish a policy on how to handle messages that fail SPF and DKIM tests, and
also publish a reporting function.
Like SPF and DKIM the policy is published as a
DNS TXT record. For example:
_dmarc.m0noc.net text =
"v=DMARC1\; p=reject\; pct=100\; rua=mailto:firstname.lastname@example.org\;
_dmarc.gmail.com text =
"v=DMARC1\; p=none\; rua=mailto:email@example.com"
_dmarc.google.com text =
"v=DMARC1\; p=reject\; rua=mailto:firstname.lastname@example.org"
_dmarc.yahoo.com text =
"v=DMARC1\; p=reject\; pct=100\; rua=mailto:email@example.com\;"
part is the policy: none
infers to just report on it, quarantine
infers mark failed messages as spam, and reject
infers reject the message at the SMTP layer.
Again, it is up to the receiving MTA to
determine if they honour the policy or not.
Sending to (some) BT.com addresses
A recent email I sent to someone at bt.com had
a delivery receipt set. The receipt will contain an attachment which is the
header of the original message that
was sent, and revealed a few interesting things about the path it (most likely)
took. It also reveal a bit of information about how the network is set up and
what's running on them. Useful information you can publicly get hold of;
remember that the organisation voluntarily gave this information as part of a
Here is an edited version containing the Received: headers.
Received: from HE1PR06CA0108.eurprd06.prod.outlook.com
(10.164.40.141) with Microsoft SMTP
Server id 15.1.557.21
Received: from VE1EUR02FT011.eop-EUR02.prod.protection.outlook.com
with Microsoft SMTP Server id 15.1.587.13 via Frontend Transport
Authentication-Results: spf=fail (sender
IP is 22.214.171.124)
(signature did not verify)
header.from=m0noc.net;bt.com; dkim=fail (signature did not
Received-SPF: Fail (protection.outlook.com: domain of m0noc.net does not
126.96.36.199 as permitted sender)
Received: from RDW083A006ED62.bt.com (188.8.131.52) by
(10.152.12.150) with Microsoft SMTP
Server id 15.1.587.6 via
Received: from EVMHT66-UKRD.domain1.systemhost.net (10.36.3.103) by
(10.187.98.11) with Microsoft SMTP Server id
Received: from smtpe1.intersmtp.com (10.187.98.11) by
(10.36.3.103) with Microsoft SMTP Server id 8.3.342.0
Received: from tbhext01.m0noc.net (184.108.40.206) by
Microsoft SMTP Server id 220.127.116.11
We have to read these bottom up to track the
We also note that 18.104.22.168 is Update Rollup 5 for Exchange Server 2010 SP3
and 8.3.342.0 is probably Update Rollup
12 for Exchange Server 2007 Service Pack 3.
Now for the interconnect.
22.214.171.124 is my mail server; the origin of
the email. This connected to smtpe1.intersmtp.com (ip address 126.96.36.199).
The next line up then states the message was received
from smtpe1.intersmtp.com (10.187.98.11) by a different host. This tells us
that the public interface for smtpe1.intersmtp.com has an IP address of
188.8.131.52 and an internal interface of 10.187.98.11 (which is also an
RFC1918 address; not valid on the Internet; hence internal – or there is a bit
of NAT). We can repeat this up the chain to reveal the likely path.
We can also see that smtpe1.intersmtp.com is
also known internally as RDW083A006ED62.bt.com (the reverse path with the same
IP's and the receiving path).
Finally, the outgoing path goes from a public
address to an rfc1918 address. This suggests some DMZ with a B2B link to
Microsoft (there are other possibilities).
In some cases, viewing whois records can also
offer some intelligence.
I will leave it as an exercise for the reader
to produce a diagram of the interconnect.
What is of particular interest is that the
email goes into BT then back out again to the Microsoft cloud.
This is where the problem occurs. Earlier on,
and further down in the header we see this:
Received-SPF: Pass (RDW083A006ED62.bt.com: domain of
designates 184.108.40.206 as
permitted sender) receiver=RDW083A006ED62.bt.com;
It shows that when it was on its initial
incoming path the BT mail servers correctly validated the SPF rule. However, as
it went on its way out to the cloud this test was re-done (see the earlier
trace, key part of SPF highlighted in red), along with the embedded DKIM
signature and the DMARC policy for my domain. Accordingly, it all failed as it
was basing its results on one of the internal BT relays being authorized to
generate mail for my domain and had already munged some of the other fields
which broke DKIM as well, so DMARC then also failed.
So what could be worse? Perhaps ensuring that
any sender with an SPF fail policy is marked as spam just because you evaluated
the SPF policy against your Internet
facing email relay server?
This is what the Raynet mail forwarding service
appears to be doing; lets see if you
The reason I was looking at this one is that
none of my emails were getting through when sent to this account when it was
set up to forward to my ISP.
First, the original failure was because the
forwarder forwarded the email as-is, without changing the From: field. Accordingly my ISP
was discarding the email since the Raynet server isn't one of my ISP's email
servers, and the mail claims it originated from my ISP.
How do I know? Well I changed the forwarding to
my m0noc.net address and did the same thing. A mail sent came back (in the logs
at least); with two SPF fails. On marking it as SPAM by the second Raynet
server, and then me as the From:
field asserted it was from m0noc.net, but my SPF rules stated 'computer says
The key headers (cut for brevity) are:
Received: from mail.raynet-uk.net (www.raynet-uk.net
Received: from amg1.gbse.gb.net (amg1.gbse.gb.net [220.127.116.11])
Received: from tbhext01.m0noc.net ([18.104.22.168]:33182) by
with esmtp (Exim 4.82_1-5b7a7c0-XX)
Subject: [SPAM] test
X-hMailServer-Reason-1: Blocked by SPF () - (Score: 3)
As we can see, there are X-hMailServer headers that state that this
email is spam due to an SPF failure.
Now, given that I'm not running hMailServer (a
freeware mail server) and Raynet's first mail server is running Exim 4.82, we
can reasonably assume that 22.214.171.124 is relaying it to 126.96.36.199, and
it is this second server that is running hMailServer (we can
research that online).
Secondly, as I know my SPF records are valid
(others such as Google, Yahoo, etc all confirm it is OK), we can reasonably
assume that the second Raynet server is performing the SPF check against the
first Raynet server; which will of cause always fail where there is an SPF
record for the sending domain.
Whois is also interesting with this one.
I'm sure you have seen many
other examples of anomalies that are misconfiguration or logic issues rather
than a suspect email.