Subject: [Asrg] Default SPF Enablement?

From: Douglas Campbell <[email-masked]>
Date: Mon, 30 Jan 2006 20:34:48 -0800

>Claus Assman said:
>
>  
>
>>So why don't the major MTA distros do this? 
>>    
>>
>
>man forward
>  
>
>  
>
Claus,

Here is the result:

C:\Documents and Settings\admin>man forward
'man' is not recognized as an internal or external command,
operable program or batch file.

and

[root@mooncake ~] man forward
No manual entry for forward

Seriously, if I interpret your two words accurately, you
are saying that SPF breaks forwarding (and aliasing)
to foreign e-mail addresses. But  does it?

What should the RCPT FROM be when an e-mail
is forwarded?  The original foreign sender or the local user
of the forward database?  In other words, if I forward e-mail,
shouldn't the envelope and headers change to reflect that I
forwarded?  An MTA that does this appears to be
SPF-friendly.  Actually, it appears that an MTA called
"qmail" does this, judging by the top results in a
google search for "forward man page".  The sendmail
version seems more capable, but the example uses a
pipe to the "vacation" program, use of which I
understand to be bad etiquette. forward under sendmail also states
nothing about modifications of the envelope and
headers; therefore I must assume that sendmail keeps
them intact.  But why?  I'm trying to understand
history, in an effort not to repeat it.

Cheers,
Douglas Campbell

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Claus Assmann <[email-masked]>
Date: Mon, 30 Jan 2006 20:57:59 -0800

> to foreign e-mail addresses. But  does it?

See the SPF documentation about "SRS".

> What should the RCPT FROM be when an e-mail

"RCPT FROM"? Do you mean the envelope sender?

> is forwarded?  The original foreign sender or the local user

The original sender of course.

> forwarded?  An MTA that does this appears to be

Do you change the header in a snail mail when it is forwarded?

What will happen if the mail can't be delivered (after forwarding)?
Do you want the bounce to end up on the forwarding system? Someone
has to inform the "original" sender, right?

PS: we are talking about forwarding (at the MTA level), not resending
(at the MUA level).

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 31 Jan 2006 14:19:34 +0100

Douglas Campbell wrote:

> you are saying that SPF breaks forwarding (and aliasing)

Generally receivers cannot check SPF behind their border MX,
and 1123 5.3.6(a) aka "251 user not local" _is_ behind their
border.

Something has to give.  Otherwise they'd get an emulation of
"551 user no local".  Calling that "breaks forwarding" is a
simplification.  Actually 1123 broke it 16 years ago, it was
fine in STD 10 with its (literally) reverse paths:

251-forwarders knew that they get the mail back if it doesn't
work out, therefore they always had the option to reject (551).

All "MSAs" (at that time "smart hosts") and relays knew that
they cannot accept arbitrary Return-Paths, because they were
responsible to return this if it doesn't work out.  SMTP was
a simple and robust protocol, designed by Jon Postel, it took
the 1123-committee to "break" it.  SPF allows to "fix" this.

After the spammers found and abused this loophole for their
own purposes (they need MAIL FROMs surviving call back tests)
about twelve years after 1123 was published.

Probably nobody expected in 1989 (when 1123 was published)
that today most mails are spam with forged Return-Paths.  Or
maybe they thought that "MX" should cover the concept "RMX",
some kind of default "v=spf1 mx a ~all".

It's a subtle bug, all reasons given in RfC 1123 for dropping
the source routes are convincing.  Only the side-effect "this
allows to use arbitrary Return-Paths" turned out to be very
bad, more than a decade later.

> In other words, if I forward e-mail, shouldn't the envelope
> and headers change to reflect that I forwarded?

Yes, if you do that - aka 1123 5.3.6(b) - there's no problem
with SPF at whatever next hop:  It would check your policy
against your "RMX" IPs and get a PASS.  Or if you have no SPF
policy it would get NONE, but not a FAIL as with the original
MAIL FROM policy checked against your IPs.

> trying to understand history, in an effort not to repeat it.

The problem is that some forwarder claim that 251-forwarding
is a "feature" (it really was before 1123) and don't accept
that it's now a "bug".  Same situation as with "open relays",
that was a good (or at least possible) idea for some time.

                             Bye, Frank

---

From: Daniel Feenberg <[email-masked]>
Date: Tue, 31 Jan 2006 08:59:59 -0500 (EST)

> bad, more than a decade later.

Can you be more specific about where the reasons are given? I can 
find MUST and SHOULD - no rationale. Was that given elsewhere?

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 31 Jan 2006 16:27:21 +0100

Daniel Feenberg wrote:

>> It's a subtle bug, all reasons given in RfC 1123 for
>> dropping the source routes are convincing.
[...]
> Can you be more specific about where the reasons are given?

<http://tools.ietf.org/html/1123#section-5.2.6>

| The intent is to discourage all source routing and to
| abolish explicit source routing for mail delivery within
| the Internet environment.  Source-routing is unnecessary;
| the simple target address "user@domain" should always
| suffice.  This is the result of an explicit architectural
| decision to use universal naming rather than source
| routing for mail.  Thus, SMTP provides end-to-end
| connectivity, and the DNS provides globally-unique,
| location-independent names.  MX records handle the major
| case where source routing might otherwise be needed.

With end-to-end connectivity (that excludes 251-forwarding)
source routes are unnecessary, the MX is always good enough.

| Some have suggested that source routing may be needed
| occasionally for manually routing mail around failures;
| however, the reality and importance of this need is
| controversial.  The use of explicit SMTP mail relaying for
| this purpose is discouraged, and in fact it may not be
| successful, as many host systems do not support it.  Some
| have used the "%-hack" (see Section 5.2.16) for this purpose.

Besides they offered worse ideas for forward-routing issues.
Of course the "%-hack" is now deprecated (2821).  Missing the
point of _reverse_ paths for the 5.3.6(a) case.

<http://tools.ietf.org/html/1123#section-5.2.19>

| In an understatement, RFC-822 says "The use of explicit
| source routing is discouraged".  Many hosts implemented
| RFC-822 source routes incorrectly, so the syntax cannot be
| used unambiguously in practice.  Many users feel the
| syntax is ugly.  Explicit source routes are not needed in
| the mail envelope for delivery; see Section 5.2.6.  For
| all these reasons, explicit source routes using the RFC-
| 822 notations are not to be used in Internet mail headers.

| As stated in Section 5.2.16, it is necessary to allow an
| explicit source route to be buried in the local-part of an
| address, e.g., using the "%-hack", in order to allow mail
| to be gatewayed into another environment in which explicit
| source routing is necessary.  The vigilant will observe
| that there is no way for a User Agent to detect and
| prevent the use of such implicit source routing when the
| destination is within the Internet.  We can only
| discourage source routing of any kind within the Internet,
| as unnecessary and undesirable.

In other words source routes plus %-hack plus bang paths were
messy and often didn't work as expected.  The MX concept is
better.  For no-nonsense Return-Paths, they had more pressing
needs than forged Return-Paths in 1989.  My bat-book printed
1994 has 792 pages, and editing sendmail.cf was a royal PITA.

In 1123 section 5.3.7 you might "see" why 5.3.6(a) was a bad
idea to start with, unfortunately too little too late.

> I can find MUST and SHOULD - no rationale.

Their rationale often starts with lines "DISCUSSION:" in 1123.

                             Bye, Frank

---

From: Bill Cole <[email-masked]>
Date: Tue, 31 Jan 2006 11:06:42 -0500

>to foreign e-mail addresses. But  does it?

Yes. This is a well-known problem with SPF. Meng Weng Wong devised a 
mechanism called SRS so that forwarders could *change* how they do 
forwarding and not break SPF. SRS has not been widely adopted. 
Unfortunately, forwarders do indeed *need* to change their behavior 
if they wish to avoid breaking SPF, but few have bothered to do so, 
as apparently they don't much care about whether SPF works.

>is forwarded?

"Should" is a loaded term.

What it *is and always has been* for functional forwarding systems is 
the original value, unmodified by the forwarder.

>  The original foreign sender or the local user
>of the forward database?

That latter would be a very bad idea. A far worse idea than the 
people devising SPF managed to come up with, and theirs is a mess.

The primary function of the envelope sender is as a return path: 
where to send notices of transport and delivery status, including 
bounces. If you change that to an address which was originally a 
target of the message, any DSN  will loop rather than go back to the 
actual sender.

SRS addresses that, but it demands more of forwarders than I think 
they can be expected to do reliably. Call me cynical.

>In other words, if I forward e-mail,
>forwarded?

Headers? Maybe added to, but existing headers should not get changed. 
Envelope sender? No, at least not according to any existing practice 
in broad deployment that is not known to be harmful.

In the best possible mail transport world, there might be no 
forwarding or forwarders would have a standard mechanism for 
modifying the envelope to indicate a chain of responsibility in a 
trustworthy and secure manner. We don't live in that world. We 
actually have decades of habits and standard practices that work, but 
imply a trust of others which no longer is based in reality. SPF  has 
provided a demonstration that breaking widespread practices (whether 
they "should" exist in the modern world or not) is a path to 
footnotehood.

>An MTA that does this appears to be
>google search for "forward man page".

You have not done your research adequately. Sendmail remains the 
dominant MTA and mail tool suite, while qmail is abandonware. On many 
Unix and Unix-like systems, 'man forward' will show you something 
like the page at 
http://www.rahul.net/cgi-bin/userbin/man?topic=aliases&section=5 
although that looks suspiciously like the Solaris page and has hints 
of Sun in it.  I see that the most current Sendmail has demoted 
mention of the .forward file to the body of the aliases man page 
rather than as another name for that page.

>The sendmail
>understand to be bad etiquette.

I disagree.  Many programs (including MS Exchange) that try to do the 
same thing as 'vacation' do so quite poorly without considering any 
edge cases and demonstrate the truth that egotism and coding don't 
mix well. Anyone can use the concepts and even the code in Sendmail's 
vacation program, but many who should have done so have chosen not to 
and make their software a public nuisance.

Even if you think using vacation per se is a bad thing, I'm not sure 
how that is relevant. The example in Sun's aliases man page is there 
to show how one would use the .forward file (one place Sendmail looks 
for forwarding information) to pipe mail into a program, and the fact 
that it happens to be 'vacation' is trivial.

>forward under sendmail also states

You would do well to do your research first rather than state what 
you believe should be as if it is fact.
-- 
Bill Cole                                  
[email-masked]

---

From: Kari Hurtta <[email-masked]>
Date: 31 Jan 2006 18:35:23 +0200

> history, in an effort not to repeat it.

Some MTAs alter MAIL FROM when forwarding.
I have experience about Communigate Pro.

User was forwarded mail to another site.

Mail bounced. Communiage Pro's user 
account got that mail (because Communnigate Pro
was altered MAIL FROM). 

What happened that bounce mail?

Answer: Communigate Pro forwarded that to 
that address what user was configured.

What happened that forwarded bounce ?

Answer: It bounced and was returned to
that Communigate Pro's account (because 
Communnigation Pro was altered MAIL FROM
even when original MAIL FORM was <>).

What happened that bounce mail?

Answer: Communigate Pro forwarded that to 
that address what user was configured.

What happened that forwarded bounce of bounce ?

Perhaps reader can already guess that.

So is altering MAIL FROM very wise when forwarding?

/ Kari Hurtta

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 31 Jan 2006 18:15:05 +0100

Bill Cole wrote:

> Unfortunately, forwarders do indeed *need* to change their
> behavior if they wish to avoid breaking SPF, but few have
> bothered to do so, as apparently they don't much care about
> whether SPF works.

They will have to change even without SPF FAIL.  Something is
awfully wrong if accepted (not limited to forwarded) mails are
rejected later.  For a forwarder the only way to get a clue
about such issues directly is to use its own error handling
MAIL FROM addresses.  Some kind of RfC 1123 5.3.6(b) or SRS.

It makes no sense to forward mails that are rejected later.

> What it *is and always has been* for functional forwarding
> systems is the original value, unmodified by the forwarder.

That's wrong, STD 10 clearly demanded to add the host to the
reverse path.  

For a host forwarder.example and MAIL FROM:<[email-masked]>
that became MAIL FROM:<@forwarder.example:[email-masked]>

The reverse path _was_ modified before 1123.  That's why it's
still called a "path" as in Return-Path: and not an "address".

> If you change that to an address which was originally a
> target of the message, any DSN  will loop rather than go
> back to the actual sender.

Yes, MAIL FROM = RCPT TO can't work, worse than 1123 5.3.6(a).

MAIL FROM:<[email-masked]>
could do the trick (ignoring some technical details) from the
POV of a forwarder, same idea as for mailing lists.  

After all the final receiver shouldn't reject (or worse bounce)
mails forwarded to her/him.

> SRS addresses that, but it demands more of forwarders than I
> think they can be expected to do reliably. Call me cynical.

No issue, let them emulate 551 and die in peace.  Bye, Frank

---

From: "Peter J. Holzer" <[email-masked]>
Date: Tue, 31 Jan 2006 20:30:23 +0100

On 2006-01-31 18:15:05 +0100, Frank Ellermann wrote:
> The reverse path _was_ modified before 1123.

Was it? Just because RFC 822 demanded it, doesn't necessarily mean it
was ever widely implemented. 

	hp

-- 
   _  | Peter J. Holzer    | Ich sehe nun ein, dass Computer wenig
|_|_) | Sysadmin WSR       | geeignet sind, um sich was zu merken.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Holger Lembke in dan-am
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 31 Jan 2006 23:21:19 +0100

Peter J. Holzer wrote:

>> STD 10 clearly demanded to add the host to the reverse path.
[...]
 
> Was it? Just because RFC 822 demanded it, doesn't necessarily
> mean it was ever widely implemented.

s/822/821/  I can't tell, when I tried to figure out sendmail
it was for a UUCP route to some kind of "emergency smart host",
and some years _after_ 1123 was published.

Today my general impression is that Postel wrote good stuff
not limited to STD 10.  He already had the very same concept
in RfC 788 (obsoleted by 821), so I guess it's no nonsense.

RfC 821 was published in August 1982, 788 in November 1981.

In RfC 780's "MTP" (May 1981, obsoleted by 788) it was also
the same concept (apparently, better check this).  In RfC 772
(September 1980, obsoleted by 788) also about "MTP" I find no
reverse path / error handling concept (again please check it -
I never had any reasons to look into this old stuff before ;-)

So _maybe_ he first tried to get away without reverse path and
error handling in 772.  Nine months later he found that this
is not good enough, and added reverse paths and error handling
in 780 to MTP.

The he sticked to this concept up to STD 10 as we know it, for
16 months.  "Nobody implemented it" doesn't sound plausible
for a full Internet Standard, there was enough time to modify
this "reverse path" concept if it didn't work out in practice.

Okay, seven years later "they" pulled this stunt, in RfC 1123,
after discussing it for 18 months (see chapter 1.4, the list
of credits is very interesting, many folks that are Internet
legends today, 2822 has a similar "who-is-who in the IETF" :-) 

                          Bye, Frank