Subject: [Asrg] Unintended consequence of SPF

From: [email-masked]
Date: Thu, 16 Mar 2006 08:41:18 -0800

---------------
Hi,

I just attempted to put in a work request for my Sirius radio subscription,
via their web site.  Whaddya know, I got an E-mail bounce back from them.
Turns out that Sirius uses electric.net behind the scenes, and your web
submission turns into an E-mail submission with a From: which is filled in
to look like a request from your own E-mail address.

It also turns out that electric.net uses SPF, and it looks at that From:
address, asks my vsta.org where E-mail can come from, and bounces the
(internally generated) E-mail.  Thank god the bounce came back to me,
otherwise it would've been just another black hole customer service
scenario.

Word is going to have to get out that spoofing From addresses--even for
internal purposes--is a Bad Idea.  At least in an SPF-ish kind of world.

Regards,
Andy Valencia

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Bill Cole <[email-masked]>
Date: Thu, 16 Mar 2006 16:31:07 -0500

A bad practice. One that is extremely common, but which has always 
been lazy and unwise, even before SPF and even before spam was a 
significant problem.

>(internally generated) E-mail.

This is not an SPF side-effect, it is as "too stupid to run any 
service involving mail" side effect.

>Thank god the bounce came back to me,

We do not now and are likely never to live in an SPF-ish kind of world.

That said, SPF is known to have those sorts of issues: it makes 
various types of stupidity fragile and also breaks generally workable 
tricks like transparent forwarding that are commonplace and accepted 
but are not consistent with the SPF world-view.

It is not really accurate to all that sort of breakage an unintended 
consequence, since the people who defined SPF as it is today were 
fully aware of the sorts of behaviors that SPF is inconsistent with.

-- 
Bill Cole
[email-masked]

---

From: "Martin Hannigan" <[email-masked]>
Date: Thu, 16 Mar 2006 23:20:12 -0500

> been lazy and unwise, even before SPF and even before spam was a

Which RFC does it violate though? I think it's a fair use case.

-M<

---

From: Gunnar Lindberg <[email-masked]>
Date: Fri, 17 Mar 2006 08:17:54 +0100 (MET)

RFC0, aka "Use Common Sense".

Don't leave to outsiders to figure out when you f*ck up inside.

I would say it could be a fair use case, *if* MAIL_From, Errors-To:
etc are set properly, i.e. to an internal address where people fix
internal mail problems. Then you could keep From:, making Support's
MUA sending responses back to the custommer.

	Gunnar Lindberg

---

From: Bill Cole <[email-masked]>
Date: Fri, 17 Mar 2006 09:06:26 -0500

At 11:20 PM -0500 3/16/06, Martin Hannigan wrote:
>On 3/16/06, Bill Cole 
><<mailto:[email-masked]>[email-masked]> 
>wrote:
>
>At 8:41 AM -0800 3/16/06, 
><mailto:[email-masked]>[email-masked] wrote:
>>---------------
>>Hi,
>>
>>Turns out that Sirius uses <http://electric.net>electric.net behind 
>>the scenes, and your web
>Which RFC does it violate though?

I can't fathom why anyone would think that is a useful question. It 
is a practice that leads inevitably to bad outcomes. Whether an RFC 
prohibits an operationally unwise practice is a trivial issue. RFC's 
don't define the boundaries of good or wise behavior.

Less trivially, such a system opens the operator to abuse by unknown 
third parties that could put them in arguable violation of various 
email forgery laws including CAN-SPAM. Violating an RFC is something 
many of us do every day without any negative consequences, but 
violating laws is something one should take real care about.

>I think it's a fair use case.

It causes the mishandling of any transport failures by design. 
Calling it a "fair use" is meaningless. As a matter of principle, 
generating a piece of email  and transporting it via SMTP with an 
envelope sender address whose owner is not explicitly made aware of 
that use is ethically wrong, but even if that use is disclosed there 
are problems with user error and intentional abuse that cause bad 
outcomes with systems like the one described.

-- 
Bill Cole                                  
[email-masked]

---

From: Bart Schaefer <[email-masked]>
Date: Fri, 17 Mar 2006 08:16:30 -0800

On Mar 17,  9:06am, Bill Cole wrote:
} Subject: Re: [Asrg] Unintended consequence of SPF
}
} Calling it a "fair use" is meaningless. As a matter of principle, 
} generating a piece of email  and transporting it via SMTP with an 
} envelope sender address whose owner is not explicitly made aware of 
} that use is ethically wrong

I realize this is not your point, but as this is a relatively technical
mailing list a distinction is important:  The "From:" field is not the
envelope sender, and we have no indication here that the envelope sender
was altered to match the "From:".

---

From: John Levine <[email-masked]>
Date: 17 Mar 2006 16:24:12 -0000

>is a practice that leads inevitably to bad outcomes.

Only if implemented by nitwits.  I have web sites that do more or less
the same thing, submit a request by e-mail, but I take care that the
mail can be delivered like it's supposed to.  

In this case, the problem is that the recipient system is using a
broken authentication scheme that rejects useful mail that it should
accept.  Fans of SPF keep trying to redefine SPF's failure cases to be
the fault of people who send mail in ways that SPF can't handle,
rather than as a failure of SPF.

R's,
John

---

From: "Chris Lewis" <[email-masked]>
Date: Fri, 17 Mar 2006 11:43:21 -0500

> was altered to match the "From:".

Correctly implemented SPF doesn't care what the From: is.  Assuming SPF
was implemented correctly (which is probably a leap ;-), that basically
means that the envelope sender _was_ generated in sync with the user
supplied email address, and we can't be sure what the From: was, tho,
the bounce might make that clearer.

---

From: Bill Cole <[email-masked]>
Date: Fri, 17 Mar 2006 11:47:19 -0500

At 8:16 AM -0800 3/17/06, Bart Schaefer  imposed structure on a 
>was altered to match the "From:".

There is no reference whatsoever to the RFC822 From header. 
Everything about the original message indicated that the broken 
system in question used the user-provided address as the envelope 
sender, as that is the only way any nominally correct mail system 
would be checking SPF records against that value or sending DSN's to 
it.

-- 
Bill Cole
[email-masked]

---

From: "Chris Lewis" <[email-masked]>
Date: Fri, 17 Mar 2006 11:51:10 -0500

> rather than as a failure of SPF.

I have as poor opinion of the SPF koolaid as even you, John.

But, DKIM and virtually any other message authentication method would
fail in this too.

This is really no different than sticking a spam filter up in front of a
spam reporting address, or content munging inbound signed email, and
expecting it to pass signature checking.

Application design error.

---

From: Bill Cole <[email-masked]>
Date: Fri, 17 Mar 2006 12:08:36 -0500

>mail can be delivered like it's supposed to.

It's not a bad thing if you control the path of delivery and so can 
effectively take that care, it is a bad thing for nitwits who apply 
it in a more general way when sending mail through a system whose 
authentication mechanisms they do not control.

-- 
Bill Cole
[email-masked]

---

From: Bart Schaefer <[email-masked]>
Date: Fri, 17 Mar 2006 09:19:36 -0800

On Mar 17, 11:47am, Bill Cole wrote:
}
} >I realize this is not your point, but as this is a relatively technical
} >mailing list a distinction is important:  The "From:" field is not the
} >envelope sender, and we have no indication here that the envelope sender
} >was altered to match the "From:".
} 
} There is no reference whatsoever to the RFC822 From header. 

I looked the original message up in the archives before I sent my response.
Did you?

There are plenty of systems that do SPF checks against the From: header.
MSN/Hotmail, for example, will file a message in the junk folder if the
RFC822 From header gets an SPF fail (fortunately the Sender: header is
checked preferentially).

---

From: "william(at)elan.net" <[email-masked]>
Date: Fri, 17 Mar 2006 10:05:38 -0800 (PST)

> rather than as a failure of SPF.

John,

I know you don't like SPF, but you're missing the point here.

SPF is a policy record. If somebody does not want email listed as
originating from them (or listing their address for bounces if
you like) being sent in certain way where their system is not
original of such an email, they specify such a policy.

This does not account for some cases which are valid valid in
current email architecture, but similar would happen for DKIM
when somebody is using SSP and puts restrictive record there and
then email goes through mail list - this is a valid case that
DKIM would not be able to account.

You on the DKIM list seem to be arguing that SSP is not needed
(or is harmful, although I'm not sure you ever went that far
publicly). You're obviously entitled to that opinion which is
probably based on the possibility of such failure case, but I do want
to note that some want such policies (to prevent their domain from
being abused i.e. phishing) and if we put parrallels to SPF. you
should notice that SPF does not have to be used with restrictive "-all" 
policies either and could be used with record ending with "?all"
which would have similar effect to DKIM being used without SSP.

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: John Levine <[email-masked]>
Date: 17 Mar 2006 18:15:08 -0000

>But, DKIM and virtually any other message authentication method would

Actually, the problem is SPF's -all option, in which senders declare
that they know all the paths that a message could possibly take.  (Or,
I suppose, wacky SPF implementations that pretend they saw -all even
if it's not there.)  We have a fairly clear understanding in the DKIM
group that you treat a message with no signature or a broken signature
the same as you would treat it now, and the lack of signature doesn't
say anything bad about a message.  There's an optional add-on called
SSP which is the moral equivalent of -all, but it's not part of the
base spec and I don't expect anyone to use it other than a few
optimists who think that somehow it'll solve the phishing problem.

In this particular application, though, the problem is not SPF but
just ineptness.  If you're using mail to send a message from one part
of your application to another, I would think that regardless of
what techniques you use, you'd set the receiving end of the mail to
whitelist the source.  That's what I do.

R's,
John

---

From: John Levine <[email-masked]>
Date: 17 Mar 2006 18:16:42 -0000

>RFC822 From header gets an SPF fail (fortunately the Sender: header is

That's called Sender-ID.  People in the SPF community assure me that
the difference is very, VERY, VERY important.

R's,
John

---

From: Douglas Otis <[email-masked]>
Date: Fri, 17 Mar 2006 12:01:34 -0800

On Mar 17, 2006, at 10:16 AM, John Levine wrote:

>> There are plenty of systems that do SPF checks against the From:  
>> header. MSN/Hotmail, for example, will file a message in the junk  
>> folder if the RFC822 From header gets an SPF fail (fortunately the  
>> Sender: header is checked preferentially).
>
> That's called Sender-ID.  People in the SPF community assure me  
> that the difference is very, VERY, VERY important.

While the difference between SPF/Sender-ID is important, it may not  
be discernible by examining the SPF scripts.  When SPF qualifies the  
return-path, a failure may be indicative of a Mediator, rather than a  
MSA known by the email-address domain.  The message may still be  
valid for many reasons, but such failure is not readily corrected.   
In the case of a sender acting on behalf of an individual that  
directs DSNs back to the individual, this works without a problem  
until the message is accepted and then rejected.  With DKIM making  
declarations that all messages are signed, BATV offers a compatible  
solution to counter any DSN exploit.  Conventions that do not return  
the message content in the DSN should also discourage this exploit as  
well.  Sender-ID uses the _same_ SPF scripts to also qualify a header  
field deemed representative of the latest sender.  The selection  
process anticipates a violation of RFC2822 with respect to the  
inclusion of Resent-From header fields.

RFC2822:
,----
|Resent fields are used to identify a message as having been
|reintroduced into the transport system by a user.
|...
|
|Note: Reintroducing a message into the transport system and using
|resent fields is a different operation from "forwarding".
|"Forwarding" has two meanings: One sense of forwarding is that a mail
|reading program can be told by a user to forward a copy of a message
|to another person, making the forwarded message the body of the new
|message.  A forwarded message in this sense does not appear to have
|come from the original sender, but is an entirely new message from
|the forwarder of the message.  On the other hand, forwarding is also
|used to mean when a mail transport program gets a message and
|forwards it on to a different destination for final delivery.  Resent
|header fields are _not_ intended for use with either type of
|forwarding.
'___

For each domain-name qualified using SPF in the message, 100 DNS  
transactions can be directed to a victim domain by each Mail Handling  
System in the path of the message.  Each MHS that evaluates messages  
using SPF foster a potential hazard that can not be mitigated without  
blocking SPF records.  While SSP walking up DNS label trees represent  
less of a hazard, it adds to an already high overhead incurred  
verifying the signature for substantially little benefit.  Even Mike  
said that Cisco annotates messages.  When messages are annotated,  
there is no need for SSP.  SSP should not be seen as an anti-spam  
measure and annotation makes the anti-spoofing property of SSP  
redundant.

The value, if there is any, would be in modifying the handling of  
messages that fall into exceptional cases. This could condition the  
use of white-lists, delayed acceptance, and partial vetting of the  
message prior to acceptance, for example.  Before committing  
resources, being able to quickly resolve which DNS is associated with  
the sending system provides greater information about the sender than  
investigating an IP address or domain-name in isolation.  The easiest  
way this can be accomplished is by an SRV record verifying the EHLO.

Once the EHLO is verified, defining paths for DKIM, the PRA, the  
MailFrom would be a simple matter of going to that domain and  
requesting a PTR list of acceptable EHLO parent domains.  By using  
special domain labels "*." for open-ended and "." for no additional  
member (closed-ended) set, the following convention could replace  
both SPF and SSP.  This removes the potential for hundreds of DNS  
transactions with network amplification problem to a single DNS  
transaction:

  _client._smtp.mx1.example.com.  IN SRV 1 2 1 mx1.example.com.
   ; verifies the mx1.example.com EHLO

  _oa._smtp.alumni.example.edu.   IN PTR *.
   ; we don't offer outbound smtp services

  _oa._smtp.example.biz.          IN PTR .
   ; no other domain is used to send our mail

  _mf._smtp.example.net.          IN PTR example.com.
  _mf._smtp.example.net.          IN PTR *.
   ; we use example.com and others to send our mail

  _dkim._smtp.example.gov.        IN PTR example.com.
  _dkim._smtp.example.gov.        IN PTR example.net.
   ; these domains also send our signed messages

"_oa" stands for originating address which should encompass all  
headers utilized by the PRA algorithm and in most cases only this  
list should be needed.  When there is a failure, the optional  
specific identity lists (_mf, or _dkim) could be queried for possible  
exceptions.  This sole resource used by subsequent MHS would only be  
made following verification of the MTA EHLO, a method that does not  
cause network amplification.  The reason for these lists is to help  
in deciding the resource priority assigned to the message.  Whether  
the message is accepted immediately, or delayed could easily make the  
difference between a block-list having timely information about the  
particular sender or not.

This strategy should also permit the safe use of white-lists with  
DKIM signatures, as this permits exceptional handling for messages  
that might be an abusive replay.  Once annotation is assumed, there  
should be no reason to search for DKIM policies.  Including policy  
directly within the message and caching this information could be a  
reasonable alternative.  Those messages that can not be associated  
don't get the smiley face annotation.  There is a greater need to  
indicate which keys within a domain are being used by trustworthy  
sources, which can be done with a simple tag in the key or a selector  
naming convention.

-Doug

---

From: Steve Atkins <[email-masked]>
Date: Fri, 17 Mar 2006 23:48:24 -0800

On Mar 17, 2006, at 8:51 AM, Chris Lewis wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> John Levine wrote:
>

I can't see how that would happen.

This was a webform sent by electric.net to itself. It wouldn't
have signed the message with DKIM (and even if it had, it
wouldn't have signed it claiming to be someone else.)

Even if it did manage to create a broken signature... no
big deal. Nobody should be rejecting on an invalid DKIM
signature, ever. If they do, their implementation is broken.

(Now, if you pull the concept that is SSP out of your
back pocket you might well be able to break DKIM in this
situation. But SSP is primarily an attempt to add the flaws
of SPF to DK, AFAICT, not a flaw in the DK/DKIM concept
itself.)

Cheers,
   Steve

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 23 Mar 2006 08:47:48 +0100

[email-masked] wrote:

> turns out that electric.net uses SPF, and it looks at that
> From: address, asks my vsta.org where E-mail can come from,

Something's odd here, SPF doesn't look at 2822 From-addresses,
it uses the SMTP MAIL FROM.

> Thank god the bounce came back to me, otherwise it would've
> been just another black hole customer service scenario.

If the receiver tests SPF at the wrong place (i.e. behind his
border) any bounce is sent to the MAIL FROM.  That's how the
"return-path" works.

If you got the bounce, it was still your original MAIL FROM.
If you got an SPF FAIL bounce you have an SPF FAIL policy, and
they tested it at the wrong place.  What they did with your
2822 From won't affect this outcome.  

But from what you wrote I'd guess that you have no SPF FAIL
policy, and then you can't get an SPF FAIL bounce.

It could be an SenderID PRA issue, that's at least related to
the 2822 From.  But why should they modify this mail header in
a way that causes a PRA FAIL later, makes also no sense.  Bye