Debugging 'The message failed SPF checks' when everything looks okay

So the other day we had a customer who complained that they were not able to send us mail because we blocked there emails because of SPF(https://en.wikipedia.org/wiki/Sender_Policy_Framework).

[root@mailwatch exim]# host -t TXT foo.tld 
foo.tld descriptive text "v=spf1 include:_spf.google.com ~all"

Everything looks good ... gooing though the "include" from google.com also seems fine ... well, maybe a caching issue or some issues at google ... but ... did not really believe in that.

Later we checked again to see if it fixed itself ... but NO ... WTF.

Again ... it was still failing.

Then someone asked me to check the actual SPF DNS Records and not the TXT records ... well ... after reading the SPF history from the Wikipedia link it's been deprecated in 2014: https://mxtoolbox.com/problem/spf/spf-record-deprecated

When doing a query for SPF records i saw this:

host -t spf foo.tld
foo.tld has SPF record "v=spf1 a include:spf.unoeuro.com -all"

So when they migrated there email service to GSuite, there DNS provider to https://www.cloudflare.com/dns/ they actually also copied all the old records including the DNS SPF records, but in the same time also created the new entries in the TXT records for GSuite. This is just ... one of these days when you love to be a email/system administrator.

MailScanner, RTF(Rich Text Format), Exchange, Office 365 and winmail.dat ... craziness

RTF was discountiuned in 2008 but still lives in the hidden(https://en.wikipedia.org/wiki/Rich_Text_Format)

MIME (https://en.wikipedia.org/wiki/MIME) has now taken over for the RTF format.

We use MailScanner to scan for virus and spam using https://www.baruwa.com/ whcih most of the time works great, when there is a problem is often caused but some other factors as in this example.

We had a customer whcih could not receive mails in HTML format from some customers in the original format. Some of the mails was displayed correctly so really odd. There were also a mismatch in the mail size, which led the investigation on too ... WTF ... why were there any diffence in the size of the mail ... so ... ofcause we blaimed the sending system for this, which was ofcause also correct.

In RTF messages the attachments/content in put into a winmail.dat file ...

MailScanner has this option:

# When the TNEF (winmail.dat) attachments are expanded, should the
# attachments contained in there be added to the list of attachments in
# the message?
# If you set this to "add" or "replace" then recipients of messages sent
# in "Outlook Rich Text Format" (TNEF) will be able to read the attachments
# if they are not using Microsoft Outlook.
#
# no      => Leave winmail.dat TNEF attachments alone.
# add     => Add the contents of winmail.dat as extra attachments, but also
#            still include the winmail.dat file itself. This will result in
#            TNEF messages being doubled in size.
# replace => Replace the winmail.dat TNEF attachment with the files it
#            contains, and delete the original winmail.dat file itself.
#            This means the message stays the same size, but is usable by
#            non-Outlook recipients.
#
# This can also be the filename of a ruleset.

Use TNEF Contents = add

With the default value of "replace" ... which means it removes the winmail.dat file and replaces it with the content found in the file. But as this is done, the format of the mail is still "RTF", which means that MS Clients no longer can show the mail correctly. This however does is to support is Customers moves away from MS products and let Apple clients still be able to view the attachments.

We have now set this to "add" so we both get the attachments and the winmail.dat file ... so everything is working now.

But why is this still a problem in 2017 and why us there written so much about it ... why does Office 365 has the option to convert all messages to MIME ...

The Outlook client actually still supports RTF and old contacts added might have been forced to use RTF.

You can still do it, if you really want to test

  • In the [To] field write: [smtp:noreply@syska.dk]

  • Click the "Check names", it will now be converted to "noreply@syska.dk"
  • Right click the email address and click the "Open Outlok Properties"

  • Here you can force Outlook to use RTF, which might be the default for some old Outlook contacts.

 

I'm out of words ... Why is this still an option and why does MS not just remove this option and migrate all the old contacts which for some reason have been set to use this old crappy format.

Well ... I used this to replicate the actually issue ... I'm not sure if this is the same for all old contacts ... but it might be ... and if the customer actually moved from MS Exchange/Office 365 to GSuite or other email provider ... I guess you are kind of out of luck ... you then need to tell the clients to change the format of the email or use the option in MailScanner ... but then the mail will still look kind a weird.

You might be able to solve this ... basically just search for "Exchange/Office 365 convert RTF to MIME"

Let me know if you have any problems or questions.