Subject: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt

From: "John R Levine" <[email-masked]>
Date: 27 Mar 2006 21:10:48 -0500

This document has been submitted to the RFC editor for possible
publication.  They have asked for the ASRG to comment on it.  Comments
could include how accurate it is, changes that would be needed before
publishing it, etc.

Would someone be willing to collect comments and edit them into a
response?   Personally, I think it needs a lot of work but we need a
statement from the ASRG, not just from one person.

This draft is available in many places including
http://www.xyzzy.claranet.de/home/test/draft-church-dnsbl-harmful-01.txt

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, Mayor
"More Wiener schnitzel, please", said Tom, revealingly.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Chris Lewis" <[email-masked]>
Date: Mon, 27 Mar 2006 22:42:46 -0500

> could include how accurate it is, changes that would be needed before

There's a number of logical inconsistencies, false conclusions, and even
just plain wrong arithmetic.  The sample set is grossly too small etc
etc. etc.

I can feed someone remarks, but I don't have the time to write up a
formal document.

[Tho, there's a remote possibility I can get someone off-list to put
pieces together - and write some of his own.  I'll check tomorrow.]

---

From: "Dan Oetting" <[email-masked]>
Date: Tue, 28 Mar 2006 10:08:42 -0700

No one is forced to use any DNS block list. Therefor, any harmful  
effects of the DNSBL can only be because they are the best  
alternative currently available. The response I would prefer to see  
is the building of a better block list system to address any  
shortcomings of the current DNSBL systems.

One particular issue brought up in the draft is that the current  
block lists don't adequately deal with bot-nets. If the block list  
could respond fast enough, each individual bot could be blocked  
before it could deliver a substantial amount of spam. And if used in  
conjunction with grey listing the entire spam run could be stopped  
without inducing collateral damage.

-- Dan Oetting

---

From: der Mouse <[email-masked]>
Date: Tue, 28 Mar 2006 12:35:19 -0500 (EST)

> No one is forced to use any DNS block list.

Well...true to a first approximation, at least.

> Therefor, any harmful effects of the DNSBL can only be because they
> are the best alternative currently available.

Uhh...no.  At least not unless your "best" includes things like "it's
in place and nobody still here understands how to switch" to cover
cases like machines set up by someone who's left, or something similar
to cover cases where someone just doesn't have time to track down the
better alternatives.

> before it could deliver a substantial amount of spam.

That's a nice theory, but - at least as far as I can tell - nobody
knows how to build a blacklist that's that hairtrigger without blocking
a lot of other stuff too.

> And if used in conjunction with grey listing the entire spam run
> could be stopped without inducing collateral damage.

Well, except for any non-spam sent from the host(s) in question.

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       [email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: "Chris Lewis" <[email-masked]>
Date: Tue, 28 Mar 2006 13:00:32 -0500

Dan Oetting wrote:
> effects of the DNSBL can only be because they are the best  alternative
> currently available. The response I would prefer to see  is the building
> of a better block list system to address any  shortcomings of the
> current DNSBL systems.
> 
> One particular issue brought up in the draft is that the current  block
> lists don't adequately deal with bot-nets. If the block list  could
> respond fast enough, each individual bot could be blocked  before it
> could deliver a substantial amount of spam. And if used in  conjunction
> with grey listing the entire spam run could be stopped  without inducing
> collateral damage.

I don't think a remark like that in the Church draft belongs in a BCP.

Furthermore, define "adequately" ;-)

Be that as that may, before taking it seriously, you do have to verify it.

The CBL's reaction time, for example, seems to have improved greatly of
late, and the CBL's particular forte is bots.  Coupled with greylisting,
that should catch just about everything, including that sent before the
CBL notices the IP.  So, we already have what you suggest.

Given that here the CBL alone is responsible for approximately 80% of
our blocking (others variously report from 60-85%), and approximately
only 3% of _all_ spam gets through (the remainder are caught by our
other mechanisms, some DNSBL, some content), I don't think it can be
said that blocklists in general aren't dealing reasonably well with
bots.  Not with perfection - nothing is. But pretty darn good.

Is that "adequate"?  Perhaps.  Perhaps not.  But it's certainly better
than any of the alternatives.

---

From: Douglas Otis <[email-masked]>
Date: Tue, 28 Mar 2006 10:51:22 -0800

> shortcomings of the current DNSBL systems.

May I add, some block-lists require contacts from the subscribers.   
This view of improving upon the block-list is shared by Trend  
Micro.  : )

> block lists don't adequately deal with bot-nets.

Bot-nets are affected by the DUL lists, which for some block-lists,  
involves the cooperation of the ISPs and are not the product of  
guesswork.  Some ISPs however do not do a good job of keeping track  
of their own address space.  These issues necessitates a staff of  
investigators.  In areas where there is good ISP cooperation, the  
related problems are much fewer.

> If the block list could respond fast enough, each individual bot  
> could be blocked before it could deliver a substantial amount of  
> spam. And if used in conjunction with grey listing the entire spam  
> run could be stopped without inducing collateral damage.

Alas, some are also dribbling spam through the provider's servers.   
There are strategies for dealing with this, but this continues to be  
work-in-progress.  The best success seems to be directing affected  
accounts to a free system scrubbing service and port 25 blocking. : )

-Doug

---

From: Tony Finch <[email-masked]>
Date: Tue, 28 Mar 2006 20:34:20 +0100

"John R Levine" <[email-masked]> wrote:
>publication.  They have asked for the ASRG to comment on it.

Sigh. When I first saw it it seemed to me to be yet another idiot
complaining about being blacklisted by an over-aggressive blacklist,
and blaming all blacklists rather than foolish MTA operators who use
them inappropriately. It was so obviously biased and ill-informed that
I thought it would be given all the attention ite deserves, i.e. none.

Sigh.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BISCAY FITZROY: WEST OR SOUTHWEST 6 TO GALE 8. RAIN OR THUNDERY SHOWERS.
MODERATE OR GOOD, OCCASIONALLY POOR IN FITZROY.

---

From: John L <[email-masked]>
Date: Wed, 29 Mar 2006 08:47:25 +1200 (NZST)

> them inappropriately.

That was my impression, too, but the IETF has processes and if someone 
asks them to consider a draft, the need some basis to blow him off.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://iecc.com/johnl, Mayor
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: "Jim MacLeod" <[email-masked]>
Date: Tue, 28 Mar 2006 17:03:24 -0800

The main arguments in the Church draft are:

1) DNSBL are necessarily out-of-date due to the rapid rate of change in
addressing on the Internet.

2) DNSBL operators use "arbitrary policies".

3) DNSBL are ineffective at blocking spam.

Responses:

1) The first argument relies on the assumption that IP addresses are
reassigned at a frantic pace, which DNSBL operators cannot possibly track.
However, the relative scarcity of IPv4 addresses has led to a common
practice wherein they are meticulously assigned by ISPs.  The lifetime of a
public IP address is arguably at least one month, the minimum duration of
most hosting agreements.  Under closer examination, argument 1 becomes a
special case of argument 2, namely that DNSBL operators are slow to remove
addresses.

2) While it is true that each DNSBL appears to have its own goals and
policies, one can hardly classify a reputable DNSBL's policies as
arbitrary.  Large DNSBLs have documented procedures for removal.  Rather, it
could be said that the diversity of policies among DNSBLs is a strength of
the system, that email admins can choose to use DNSBLs whose attitudes
towards email closely mirror their own.  Church's personal experience of
having an IP address on a DNSBL, a notable inclusion in the draft, may
provide insight into the motives for writing the draft.  It certainly
excuses the presence of the same logical fallacy in the draft as DNSBL
operators are accused of in argument 2: expanding isolated examples into a
general case, which is then tenaciously defended.

3) The statistical sample presented is small, and the analysis lacks rigor.
It begs the question by stating that DNSBLs are prone to false positives,
but uses as evidence a union of results biased towards marking messages as
spam.  Furthermore, there is little to suggest in this sample that valid
email is being delayed or blocked.  A more objective analysis of the results
might imply that inclusion on a DNSBL is strongly correlative evidence that
an email message is spam.

Obligatory ad hominem attack:

The anecdotal evidence in the Church draft, including but not limited to the
suggestion that spam should be filtered by either Hotmail or the MUA,
implies that the author has experience primarily as an end user.  I refer
the reader to RFC 1925 Section 2 paragraph 4.

Summary:

The Church draft focuses strongly on problems with DNSBL use that are
apparent to the author, with the conclusion that DNSBL use should be
shunned.  More useful content for a document dealing with DNSBLs would
include suggested practices for DNSBL operators and users, such as
statements of policy, inclusion criteria, and removal procedures.

It's clear that more research is required in this area before the IETF
should consider publishing a document which suggests abandoning a widespread
application of technology generally considered by knowledgeable users to be
useful, without providing a more reasonable alternative.

Regards,
Jim MacLeod

---

From: "Chris Lewis" <[email-masked]>
Date: Wed, 29 Mar 2006 00:09:17 -0500

> asks them to consider a draft, the need some basis to blow him off.

John, do you think a point-by-point refutation would be useful?

I've got a suc^H^H^Hvolunteer to draft one for comments.  Thomas.

---

From: "Chris Lewis" <[email-masked]>
Date: Wed, 29 Mar 2006 00:22:42 -0500

> 3) The statistical sample presented is small, and the analysis lacks

We plan on presenting a statistical sample over 400 times larger that
shows more realistic numbers.  Eg: .01% FPs on CBL, 0 on SBL.

If our numbers were as lousy as his, we'd not use DNSBLs either.

Hand-managed content filters achieving 97% effectiveness?  That can
hardly be realizeable in a community of any appreciable size.

---

From: John L <[email-masked]>
Date: Wed, 29 Mar 2006 22:53:44 +1200 (NZST)

> John, do you think a point-by-point refutation would be useful?

What we need is a list of things he would need to change to make it 
publishable, i.e., everything.

I realize that it's obvious to us how bad this is, but the RFC editor and 
IAB don't know much about spam so we have to lay it out for them.

R's,
John

---

From: Daniel Feenberg <[email-masked]>
Date: Wed, 29 Mar 2006 07:33:20 -0500 (EST)

May I add the following:

While the author does a good job of listing the negative aspects of 
certain domain name based blacklists, he omits the advantages relative to 
other spam suppression techniques. A balanced discussion would consider 
both advantages and disadvantages.

The first advantage of DNBLs I want to mention is a private benefit for 
the mail transfer operator, his users, and their corespondents.

The typical mailserver using DNSBLs for spam suppression REJECTS suspected 
bad mail, while a typical content based scanner DISCARDS suspected spam, 
or leaves it in a spam folder. In the case of a false positive This is a 
significant advantage to the DNSBL, because the actual sender will get a 
notice of refusal from the DNSBL based refusal, but no notice at all of 
the discard from the content based system. A user or MTA operator might 
place much greater weight on lost mail than rejected mail, as lost mail 
may be the source of ill feeling or financial loss, while rejected mail is 
merely an inconvenience.

Notice that a content scanner can't reasonably return DSNs for discarded 
mail, since the majority of such mail will come from forged addresses, but 
the DNSBL rejection occurs during the SMTP control phase, and is returned 
to the sender by the originating MTA, if the mail is legitimate.

The second advantage for the DNSBL I want to mention is the public benefit 
to all email users when MTA operators administer their sites to discourage 
the output of spam from their shared MTAs. In the present legal 
environment, the existence of DNSBLs is the primary motivation for such 
efforts. Without DNSBLs, many large ISPs and hosting companies would lose 
interest completely in suppressing spam spewage from their MTAs and IP 
addresses. With the resulting increase is spam messages, content analysis 
would become increasing difficult.

Without consideration of the advantages of the DNSBL, or the disadvantages 
of content scanning the author has come to a foregone conclusion. An IETF 
document deprecating a superior solution is unfortunate.

I am aware that some content based scanners are able to reject mail, and
that some MTAs using DNSBLs discard mail, but both situations are unusual,
and the former is technically difficult. In any case, placing suspected
spam in a spam folder seems like more of a way to avoid legal liability
than to improve the user experience.

I am aware that some MTA operators are frustrated by their inability to
get off certain DNSBLs, and I do not have a cost free solution. I have
referred such operators in the past to my page at

http://www.nber.org/sys-admin/smarthost.html

which suggests that they obtain mail relay service from an operator of an 
unlisted MTA and provides some sources. Most complainants are operating 
MTAs on dynamic addresses. It would perhaps make more sense for the IETF 
to deprecate that practice than to deprecate the DNSBL.

Lastly, the draft refers to no quantitative evidence that DNSBLs are more 
likely to reject valid mail than content scanners. This is not my 
experience with Google Mail, or Spamassassin. Perhaps other scanners would 
be more effective, but by the same token, there is variation among DNSBLs 
too.

Perhaps the IETF could publish a "Best Practices" for DNSBL operators 
instead of deprecating them.

Thank you for this opportunity to comment.

Daniel Feenberg
IT Director
National Bureau of Economic Research
Cambridge MA, USA

---

From: William Leibzon <[email-masked]>
Date: Wed, 29 Mar 2006 09:20:51 -0800 (PST)

On Wed, 29 Mar 2006, John L wrote:

> That was my impression, too, but the IETF has processes and if someone asks 
> them to consider a draft, the need some basis to blow him off.

The draft was submitted directly to RFC-Editor. So did this request come 
from IETF (i.e. RFC-Editor asked IETF and IETF then asked ASRG) or directly
from RFC-Editor? Note also that it being sent to RFC-Editor means its not 
IETF that is final judge based on the comments from ASRG but RFC-Editor
(which generally has responsibility to publish where as IETF often prefers
to "blow off" when its indendent submission)

BTW - I seem to remember bringing this draft for discussion here a few months ago...

---
William Leibzon
   mailto: [email-masked]
Anti-Spam and Email Security Research Worksite:
   http://www.elan.net/~william/emailsecurity/
Whois & DNS Network Investigation Tools:
   http://www.completewhois.com

---

From: Douglas Otis <[email-masked]>
Date: Wed, 29 Mar 2006 09:50:21 -0800

On Mar 29, 2006, at 4:33 AM, Daniel Feenberg wrote:

> Lastly, the draft refers to no quantitative evidence that DNSBLs  
> are more likely to reject valid mail than content scanners. This is  
> not my experience with Google Mail, or Spamassassin. Perhaps other  
> scanners would be more effective, but by the same token, there is  
> variation among DNSBLs too

 From experiences with both block-listing and content filtering,  
there is another important aspect with respect to block-lists.   
Without block-lists, many email systems or associated networks will  
not handle all transactions, making block-list protection essential.   
Harmful and essential can not be reconciled.  Valid emails represent  
a small portion of email transactions, where block-lists protects  
both the mail-box and importantly, network resources.  Done after  
exchanging the message, filtering unblocked sources would be done  
where a small percentage of their transactions may be comprised of  
malware or spam.  As many providers block abusive traffic at the  
network (another form of block-list or black-hole list), tests must  
first ensure no other protective mechanism is active.  Block-lists  
are simply essential to retain the utility of email.  Message  
authentication will not supplant the block-list network resource  
protections.

Many of the issues related to block-lists involve the blunt  
application at the IP address.  Adopting a convention of ensuring  
verification of the EHLO, it would then be possible to utilize domain- 
name based block-lists.  Such a strategy would reveal tell-tale  
associations with domain name servers.  The hierarchy of the  
supporting infrastructure is more constraining than IP addresses with  
an expectation of longer histories of good administration.  CSV or A  
records would be a good choice for EHLO verification, whereas SPF  
invites dangerous network amplification.

-Doug

---

From: John L <[email-masked]>
Date: Thu, 30 Mar 2006 09:21:07 +1200 (NZST)

> The draft was submitted directly to RFC-Editor. So did this request come from 
> IETF (i.e. RFC-Editor asked IETF and IETF then asked ASRG) or directly
> from RFC-Editor?

RFC editor, but they talk to the IESG and IAB.

R's,
John

---

From: "Walter Dnes" <[email-masked]>
Date: Thu, 30 Mar 2006 06:38:15 -0500

On Tue, Mar 28, 2006 at 08:34:20PM +0100, Tony Finch wrote
> I thought it would be given all the attention ite deserves, i.e. none.

  I think that a reply of some sort is *ABSOLUTELY NECESSARY*.  It's a
bad idea in so many ways it's not even funny.  Is some vote necessary,
or will it be "accepted by default" if everybody else is too busy rolling
around on the floor, laughing, to refute it?

  Getting back to Church's draft-01, I'm an end user with a personal
domain.  During the past 2 weeks, my remote ISP's end-user-configurable
SMTP-stage filters have rejected 3472 email delivery attempts.  I feel
just as (un)qualified as Mr. Church to submit comments.  If nobody else
else will volunteer, I will.  Even if someone else takes the lead, I'm
willing to help out.

-- 
Walter Dnes <[email-masked]> In linux /sbin/init is Job #1
My musings on technology and security at http://tech_sec.blog.ca