Subject: [arch-d] QoS Signaling Status and Support

From: "Fleischman, Eric" <[email-masked]>
Date: Tue, 16 May 2006 13:14:12 -0700

I've been away from QoS for several years, but am currently trying to
become current again. I am surprised to note that the Next Steps in
Signaling (NSIS; see
http://www.ietf.org/html.charters/nsis-charter.html) WG exists within
the IETF. My surprise stems from my own perception that Int-Serv proved
to be a very unsatisfactory experience for our IETF community.
Certainly, as a user, I personally was very disappointed that nothing
useful (that I am aware of) came from that approach, because, had it
worked, then the path forward for voice-video-data convergence would
have been clearer.

A few years back I started a thread on the ietf mailing list asking
about the current state of RSVP, and I received a lot of feedback from
IETF participants about how valuable RSVP-TE was proving to be for MPLS.
That told me that a certain percentage of our community was excited
about RSVP signaling for MPLS, and apparently were experiencing success
using it, even though we were achieving similar results at that time
through different means. 

Anyway, in my mind, I formed a view that RSVP when used for hard-states
such as Int-Serv didn't scale and had some ugly overheads that "hardened
the network's arteries" while when used with MPLS it had apparently
experienced satisfactory results. Does this perception jive with your
view? If it does, why do you all think that it is successful for MPLS
but not for Int-Serv? What do you believe the key factors are that make
MPLS more successful than Int-Serv?

Anyway, when I read the NSIS material, I can't escape the feeling that
these people have a telephony mentality. Thus, in my own mind, I put
them into the "bucket" of trying to apply hard state into the Internet
to achieve SS7-like results. Is this an accurate interpretation of their
work and mind-set? If so, then why do they think that NSIS will have a
different result than Int-Serv? 

How well accepted is the NSIS work within the IETF? For example, I note
that their RFCs that I have read were all experimental and that they
point to other experimental work such as Seamoby. From this alone I
surmise that it is not mainstream. Is this accurate? If so, then why is
the IETF sponsoring their WG? Most importantly, how can their work fit
into historic IP architectural models? 

_______________________________________________
Architecture-discuss mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/architecture-discuss

---

From: "Tony Li" <[email-masked]>
Date: Tue, 16 May 2006 13:19:02 -0700

Eric,

RSVP worked for TE purposes soley because TE is intra-domain.  It would
not scale to the inter-domain.

Tony

> -----Original Message-----
> From: Fleischman, Eric [mailto:[email-masked]] 
> Sent: Tuesday, May 16, 2006 1:14 PM
> To: [email-masked]
> Subject: [arch-d] QoS Signaling Status and Support
>

---

From: "Fleischman, Eric" <[email-masked]>
Date: Tue, 16 May 2006 13:35:15 -0700

Tony,

Thank you for your quick reply! 

I'd like to get your viewpoint on the following related topic: 
Background: my employer has successfully used MPLS for both intra-domain
and inter-domain. The latter was due to close cooperation and
coordination with our major ISP, SITA. 
Question: Why does MPLS have success stories like this when Int-Serv
didn't? Or does it? What do you think?

--Eric

-----Original Message-----
From: Tony Li [mailto:[email-masked]] 
Sent: Tuesday, May 16, 2006 1:19 PM
To: Fleischman, Eric; [email-masked]
Subject: RE: [arch-d] QoS Signaling Status and Support

Eric,

Tony

> the IETF. My surprise stems from my own perception that Int-Serv 
> proved to be a very unsatisfactory experience for our IETF community.

> IETF participants about how valuable RSVP-TE was proving to be for 
> MPLS. That told me that a certain percentage of our community was 
> excited about RSVP signaling for MPLS, and apparently were
> experiencing success

> to achieve SS7-like results. Is this an accurate interpretation of 
> their work and mind-set? If so, then why do they think that NSIS will 
> have a different result than Int-Serv?
> 
> Architecture-discuss mailing list [email-masked]

---

From: "Tony Li" <[email-masked]>
Date: Tue, 16 May 2006 13:56:05 -0700

Eric,

Well, IMHO, IntServ existed soley for the purpose of killing off ATM.
Not that that succeeded, mind you, it was POS that killed it and MPLS/TE
that drove in the final nail.  In any case, once it was dead, interest
in Int-Serv waned and shifted (appropriately, IMHO) to DiffServ.

For obvious reasons, DiffServ is far preferable from the network's point
of view.

Another reason for MPLS succeeding lies in the old saw: "he who pays the
bills makes the rules".  The corollary for networking is: "ISPs dictate
router features".  ISPs never seemed seriously interested in Int-Serv.

Tony

---

From: Pekka Savola <[email-masked]>
Date: Wed, 17 May 2006 08:20:33 +0300 (EEST)

On Tue, 16 May 2006, Fleischman, Eric wrote:
> My surprise stems from my own perception that Int-Serv proved

You shouldn't be surprised -- the QoS community is in denial, and has 
been for a long time.

The thinking seems to go like this: "IntServ using RSVP was not 
succesful.. so the fault must have been at RSVP.  Let's design a 
better version of RSVP!"

> IETF participants about how valuable RSVP-TE was proving to be for MPLS.

I remember that discussion because I remember criticizing path-coupled 
signalling approaches :-)

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: "Tschofenig, Hannes" <[email-masked]>
Date: Wed, 17 May 2006 13:14:18 +0200

Hi Eric, 

you raise an interesting topic. Please find my response below: 

> -----Ursprüngliche Nachricht-----
> Von: Fleischman, Eric [mailto:[email-masked]] 
> Gesendet: Dienstag, 16. Mai 2006 22:14
> An: [email-masked]
> Betreff: [arch-d] QoS Signaling Status and Support
> 
> these people have a telephony mentality.

What do you consider "telephony mentality" when you read about the NSIS work? 

> Thus, in my own mind, I put
> work and mind-set?

No, it is not. 
I would rather argue that this is the case for RSVP-TE. 
 
> If so, then why do they think that NSIS will have a

NSIS is a protocol suite where as IntServ (CL/GS) is a Quality of Service model. 
There are different QoS models being developed at the NSIS working group. 
 
> point to other experimental work such as Seamoby.

Which documents do you mean? 

The NSIS WG has a number of major documents in WGLC and GIST in 'AD Evaluation' state. 

> From this alone I

You are touching an entirely separate topic with these questions, namely
"Should the IETF work on topics where deployment is not already happing (or imminent)?"

The IETF takes quite long to finish their work on almost any topic (because of many reasons; yet another discussion thread necessary). So, if you start your work in the IETF if a customer is knocking on the door then it will not be ready in time. There are other working groups that just rubber-stamp the work done outside the IETF. 

I have raised this question in past plenaries but I received only unsatisfactory feedback. This is mostly an aspect related to the interaction between the IESG and the IAB about when and how many WGs with a rather long-term workscope should be created. QoS is rather a long-term project, I would argue. 

Ciao
Hannes

PS: I would also like to point to the fact that we also started the Diameter QoS work in NSIS that has, interestingly enough, gained more attraction in the industry. This work has finally found a home in the DIME working group and we are seen interesting thoughts how to harmonize the different flavors being developed by various organizations.

---

From: "Fleischman, Eric" <[email-masked]>
Date: Wed, 17 May 2006 16:19:07 -0700

From: Tschofenig, Hannes [mailto:[email-masked]] 

>What do you consider "telephony mentality" when you read about the NSIS
work? 

When I seek to integrate non-Internet technologies with IP, I first try
to entirely replace them with IP only technologies. If that isn't
possible, then I encapsulate them within IP so that they could use the
Internet via RFC 1001, 1002, and 1006-like mechanisms. Failing that, I
interpose application layer translation gateways. The latter is always
my last choice because of the operational difficulty we have experienced
in maintaining these gateways historically, with significant performance
and scaling implications.

When I read the NSIS work, I believe that I am seeing a body that is
trying to link POTS or similar legacy technology systems with IP as if
they are equals. Specifically, I imagine that I am seeing a group that
is trying to add signaling semantics to IP networks to support
integration with telephony ideas and constructs. However, your email
stated that I am not viewing this correctly, which means that I need to
re-read that material.

>From where I sit, the approach of adding state to the IP layer has
proven to be damaging to IP networks and systems. I rather recommend
that the applications themselves become modified to work in stateless IP
environments, if possible. For voice and video, this may include adding
extensive error correction techniques but definitely means re-writing
the codecs themselves to address IP realities. 

I also recognize a class of applications that aren't likely to work well
over the Internet. What we need to better support them is stochastic
access with deterministic trunking. It may be that Diff-Serv with MPLS
can achieve this, or maybe it can't. I just don't know. I started this
thread to learn, not to preach. What insights can you offer me?

One lesson we learned during the 80s from the OSI debacle is that mixing
resource-reservation lower layers over a CONS-like network layer with
best-effort CLNS network layers just don't work. The infrastructure
either needs to be CONS or CLNS -- mixing them is A Bad Thing.

--Eric

---

From: "Fergie" <[email-masked]>
Date: Wed, 17 May 2006 23:34:45 GMT

Bell-shaped heads, that what we used to call them. ;-)

No offense intended. :-)

Back to the topic at hand:

I firmly believe that we have the building blocks, in diff-serv,
to actually do the things we talk about, and make end-to-end
service levels a reality.

The problem, unfortunately, is not a technical one -- it is
politics in its most base form. And within the Internet that we
know _today_, we cannot get around that hurdle at the moment.

- ferg

-- "Fleischman, Eric" <[email-masked]> wrote:

[snip]

--Eric

--
"Fergie", a.k.a. Paul Ferguson
 Engineering Architecture for the Internet
 [email-masked] or [email-masked]
 ferg's tech blog: http://fergdawg.blogspot.com/

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 May 2006 16:47:44 -0700

> and scaling implications.

Internet - degrades gracefully with additional traffic. Avoids global 
state; typically relies on local state with local decisions.

Telephony - performs call admission to limit to a finite number of users 
each receiving a sufficient level of service; rejects calls beyond that. 
Requires global state and coordination.

There are those who treat Internet and telephony like MPI and shared 
memory - duals that can be interchanged.

While it's trivial to build an Internet on top of a telephone system 
(that's how it's typically done), the converse is nonsensical. Adding 
state and call admission to the Internet means that local state / 
decisions won't suffice, and that graceful degradation won't result.

> The infrastructure

And layering them doesn't work (at least in one direction) either.

;-)

Joe

---

From: John Day <[email-masked]>
Date: Wed, 17 May 2006 20:39:07 -0400

>either needs to be CONS or CLNS -- mixing them is A Bad Thing.

Just for the record, OSI never attempted such a solution and in fact 
I don't believe it really ever considered it.  All it did was attempt 
to legislate their separation.  (OSI should never invited the phone 
companies to the party, but it is doubtful the Europeans could have 
not invited them. The only real lesson learned from the OSI debacle 
is that if you are doing a new architecture, don't invite the legacy 
one to join.)

Actually as already pointed out, the ARPANet was more CONS than CNLS 
for most of its life.  Also it was very likely that the connection 
nature of BBN 1822 and the use of X.25 as its replacement that 
forestalled the congestion collapse as long as it did.

Now that said, I think the CONS advocates have done a pretty lousy 
job of understanding how it can/should be used.  They seemed to be as 
locked in their religion as the CLNS advocates are in theirs.

We have seen one engineering hack proposed after another.  I have 
seen no real attempt in the last 35 years to really understand what 
is going on with CO and CL.  All I have seen are two warring factions 
with the strong belief that the thing they are currently working on 
will spell the demise of the other.

At this point, it should be clear that neither is going to happen. 
That should tell us that there is something about this that we don't 
yet get.  But so far all I hear on this list is how sure we are of 
what we know.  Given how little science we have to base that on, it 
should give us considerable pause.

Take care,
John

---

From: Fred Baker <[email-masked]>
Date: Thu, 18 May 2006 08:34:55 +0200

> succesful.. so the fault must have been at RSVP.  Let's design a  

OK, so let me ask what must be a dumb question. What was in fact the  
problem?

The problem that int-serv went out to solve is one that we  
increasingly find ourselves needing to solve in the commercial  
backbone, says the guy who met yesterday and has been meeting for  
quite a while with half a dozen phone-mobile-and-internet companies  
and several government agencies, all of whose names you would  
recognize, who are looking at exactly this issue. There is a set of  
applications whose defining characteristics include the fact that  
they send data at something resembling a fixed rate. The archetypal  
ones are voice and video of various types, but these also including  
telemetry feeds and related sensor applications. The "fixed" speed  
may be variable-within-a-range, there may be a way for the codec to  
select one of several speeds or speed ranges, and in some cases, the  
codec may be silent for periods of time. However, when transmitting  
there are a minimum and a maximum rate in any case. The list of these  
applications include unidirectional, bidirectional, and multicast  
versions. In order to reliably deliver a certain QoS, I have to know  
how much is in the network; if the application cannot or will not  
adapt, I am forced to limit the number of instances of it in the  
network to maintain an SLA.

Internet Routing does not route circuits/LSPs/microflows/whatever-you- 
want-to-call-them at the IP layer, so we don't have that option  
available to us for solving this. Measurement-based admission  
techniques performed in the end system can be helpful in places where  
the law of large numbers applies, but are of limited utility on  
slower links, and in any event from a provider perspective require  
trusting the host to do the right thing for the network. They have  
significant problems: past performance, to quote the people that want  
to sell me stocks, is not a good predictor of future performance.  
Even if changes in routing that dump engineered data streams into a  
common hopper are not a real problem, the probing involved in  
measurement-based techniques like Briscoe is suggesting can itself be  
a denial of service attack, and doesn't allow for provider  
implementation of policies like MLPP or its civilian counterpart. If  
one discards path-coupled control plane protocols, that doesn't leave  
a lot of choices.

Absent a mechanism that can dynamically ask the network whether  
critical points have a certain capability, we do it by static  
engineering, whether done in network planning or in MPLS-TE and  
related approaches. Sometimes that is the right approach; I am  
working with an astronomer at Swinburne University on a matter  
related to a telemetry feed, and for an 8 GBPS data flow deriving  
from a 512 MHz A/D conversion, a lambda is a wonderful solution. In  
the kinds of networks that you, Eric, are working on these days,  
manual engineering of data flows is a suboptimal solution I should  
think, and yes you know exactly the types of telemetry and video  
feeds and session management policies to which I refer.

Int-Serv gives the network the ability to know what its parameters  
are for these protocols and the ability to say "no" in a manner  
consistent with a management policy that may differentiate between  
sessions of different categories with different thresholds. By  
"parameters", I mean statements of the form "the sum of the codecs-or- 
etc using this class right now is <whatever>, could burst to  
<somethingElse> at any time, and could conceivably sustain that".

I'm quite happy to be told that I am in denial if it can be shown.  
Nevertheless, I have a set of people who find themselves stuck  
between the proverbial rock and hard place. Their applications have  
limited ability to adapt to the network, they are required (in some  
cases by the local equivalent of law) to implement network policies  
that differentiate in the network between different sessions, and  
they have commercial requirements both to deploy the services and to  
not permit new calls to unnecessarily disrupt existing calls. If  
infinite* bandwidth is not available or is costly, and asking the  
network at bottlenecks to state whether it currently has available  
capacity to accomplish a stated goal is off limits, and end system  
measurement fails to reliably deliver the goods, what would you  
suggest? Since we are the ones in denial, you must have a  
constructive suggestion in mind that we are overlooking. If you  
don't, then I submit that it is at least possible that I am not the  
one in denial.

By the way, the assertion that NSIS is in fact a better protocol than  
RSVP is at this point unproven. It is probably not worse (although it  
is in fact two protocols where RSVP has one, which in my opinion is a  
debatable call). I don't see any proofs on the table that it is  
across-the-board better. It is mostly different, and from the  
perspective of RFC 4542, incomplete.

* - "Infinite bandwidth", as I use the term, is "sufficient bandwidth  
that traffic inter-arrival jitter is nominal compared to the  
requirements of any current application or transport and congestion- 
related loss is negligible along the entire path or set of paths in  
question". ISPs in the "lit" portion of the world routinely deliver  
core networks that have this characteristic; it is not necessarily a  
characteristic of access paths, or of network types that are not  
easily implemented using fiber (see http://www.skyimagelab.com/ 
earatnitlar.html for a notion of what parts of the world are "lit"  
and "unlit", but note that network edges are often bandwidth limited  
even in the "lit" world). Answers of the form "the core of my network  
has infinite bandwidth under that definition" are non-responsive to  
the question from an end-to-end perspective.

---

From: <[email-masked]>
Date: Thu, 18 May 2006 09:57:25 +0300

Hi Fred,

>OK, so let me ask what must be a dumb question. What was in 
>fact the problem?

Thanks for the question.  When we were talking with Scott & Allison
about NSIS, the problem basis was two-fold:

1) There's a lot of transparent entities on the Internet that
effect the treatment of packets - NATs, FWs, admission control
boxes, etc.  It would be good to be able to allow the the end
node to be able to signal these entities, as these transparent
entities cause a lot of trouble.

2) The IETF has spent a lot of energy on QoS - IntServ (CL, Guarenteed
Service) and DiffServ - but there has been a general feeling 
that more needs to be done to enable some services and deployments.
A lot of other SDOs are specifying QoS and planning on deploying
their QoS over the Internet.  Having a general signaling mechanism
to signal IETF QoS (IntServ, DiffServ) as well as having a framework
for other groups to signal their QoS over the Internet could be a good 
thing.  At a high-level, this MAY promote some level of
interoperability.

Since kicking that work off, there has been some other issues that 
have come up, but I'll get to that in a separate mail.

>> succesful.. so the fault must have been at RSVP.  Let's design a 

Not exactly - the thinking was the tight coupling of RSVP with
IntServ has created some problems, so it might be good to 
try to decouple them.  RSVP-TE works quite well, so I don't
think that RSVP is the problem.

John

---

From: <[email-masked]>
Date: Thu, 18 May 2006 10:09:14 +0300

Further responses in-line.

>backbone, says the guy who met yesterday and has been meeting 
>for quite a while with half a dozen phone-mobile-and-internet 
>companies and several government agencies, all of whose names 
>you would recognize, who are looking at exactly this issue. 
>There is a set of applications whose defining characteristics 
>include the fact that they send data at something resembling a 
>fixed rate. The archetypal ones are voice and video of various 
>types, but these also including telemetry feeds and related 
>sensor applications. The "fixed" speed may be 
>variable-within-a-range, there may be a way for the codec to 
>select one of several speeds or speed ranges, and in some 
>cases, the codec may be silent for periods of time. However, 
>when transmitting there are a minimum and a maximum rate in 
>any case. The list of these applications include 
>unidirectional, bidirectional, and multicast versions. In 
>order to reliably deliver a certain QoS, I have to know how 
>much is in the network; if the application cannot or will not 
>adapt, I am forced to limit the number of instances of it in 
>the network to maintain an SLA.

I agree with the above scenario - but also a small tweak - 
some applications can adapt to smaller pipe, but they behave
best if they have some rough constraints, in that case, 
the rates can change, but the goal is to have the change more
slowly, in order to be able to adapt or switch codecs.

>want-to-call-them at the IP layer, so we don't have that 
>option available to us for solving this. Measurement-based 
>admission techniques performed in the end system can be 
>helpful in places where the law of large numbers applies, but 
>are of limited utility on slower links, and in any event from 
>a provider perspective require trusting the host to do the 
>right thing for the network. They have significant problems: 
>past performance, to quote the people that want to sell me 
>stocks, is not a good predictor of future performance.  
>Even if changes in routing that dump engineered data streams 
>into a common hopper are not a real problem, the probing 
>involved in measurement-based techniques like Briscoe is 
>suggesting can itself be a denial of service attack, and 
>doesn't allow for provider implementation of policies like 
>MLPP or its civilian counterpart. If one discards path-coupled 
>control plane protocols, that doesn't leave a lot of choices.

Agreed, I also have been hearing more of a need from different
service providers that having a general mechanism to have
interdomain solutions - measurement-based admission control is
OK in a single domain - but from an end-to-end or interdomain
deployment, the results are far less satisfying. 

[Editing out the stuff that I agree with]

>By the way, the assertion that NSIS is in fact a better 
>protocol than RSVP is at this point unproven. It is probably 
>not worse (although it is in fact two protocols where RSVP has 
>one, which in my opinion is a debatable call). I don't see any 
>proofs on the table that it is across-the-board better. It is 
>mostly different, and from the perspective of RFC 4542, incomplete.

The goal is to try to overcome some of the current shortcomings in
the RSVP + IntServ approach.  There is on-going work in using IntServ
QoS over NSIS signaling, which looks like it should work quite well.
Some early tests, using well-known RSVP implementations have shown
that a few NSIS implementations scale a bit better that RSVP, at around
60k sessions, if I remember right.  I think there is some published 
results about this, that I could post a link to if anyone is interested.

Anyhow, I'd probably stay away from 'better protocol' at the moment,
as I generally tend to see what NSIS is doing is trying to make
some additional tools to allow better signaling across the Internet,
especially when different deployments models are traversed.

John

If anyone is interested, I'd thought I'd plug a couple of
implementations,
that people could check out.
	
http://nsis.dei.uc.pt/
http://user.informatik.uni-goettingen.de/~nsis/

---

From: <[email-masked]>
Date: Thu, 18 May 2006 10:15:22 +0300

Ferg,

>Bell-shaped heads, that what we used to call them. ;-)

Not really sure that the above helps.  If you look at the current
state of the Internet, with consolidation of ISPs, backbone carriers,
telcos and mobile nets - the clear division between bell heads
and net heads is gone.  Heck, SCTP has been discussed on this list
as a potential new way of doing things, but SCTP was originally
designed to carry SS7 over IP.

>I firmly believe that we have the building blocks, in 
>diff-serv, to actually do the things we talk about, and make 
>end-to-end service levels a reality.

RFC2996 does allow you to use RSVP to signal DiffServ end-to-end,
but AFAIK, it isn't widely deployed.  DiffServ is not designed
to be end-to-end, so I think we need to be able to tie things
up on an end-to-end basis.

>politics in its most base form. And within the Internet that 
>we know _today_, we cannot get around that hurdle at the moment.

What level of politics do you mean?  Net Neutrality type of politics,
SDO level of politics (different SDOs defining very different QoS
solutions), or something else.

thanks,
John

---

From: <[email-masked]>
Date: Thu, 18 May 2006 10:22:38 +0300

Hi Eric,

>When I read the NSIS work, I believe that I am seeing a body 
>that is trying to link POTS or similar legacy technology 
>systems with IP as if they are equals. Specifically, I imagine 
>that I am seeing a group that is trying to add signaling 
>semantics to IP networks to support integration with telephony 
>ideas and constructs. However, your email stated that I am not 
>viewing this correctly, which means that I need to re-read 
>that material.

I will definitely agree that many of the people bringing requirements
to NSIS are coming from a telco deployment mindset.  AFAIK,
the goal isn't strict interworking with POTS, but how they can
move their networks to be fully IP based.  However, for many of 
service providers, their value proposition to their customers
is a very high-level commitment to quality - in terms of 
predicability (roughly same level of service provided);
good session setup time; high service availability.  Additionally,
many of these deployments have SLA agreements with other companies
and they are looking for solutions on how to tie these things together.

>proven to be damaging to IP networks and systems. I rather 
>recommend that the applications themselves become modified to 
>work in stateless IP environments, if possible. For voice and 
>video, this may include adding extensive error correction 
>techniques but definitely means re-writing the codecs 
>themselves to address IP realities. 
>
>I also recognize a class of applications that aren't likely to 
>work well over the Internet. What we need to better support 
>them is stochastic access with deterministic trunking. It may 
>be that Diff-Serv with MPLS can achieve this, or maybe it 
>can't. I just don't know. I started this thread to learn, not 
>to preach. What insights can you offer me?

I think that this is what a lot of folks are doing, but there 
are issues when looking at this end-to-end, across different network
providers, or at least that's what people are telling me.

John

---

From: <[email-masked]>
Date: Thu, 18 May 2006 10:28:35 +0300

Eric,

>Question: Why does MPLS have success stories like this when 
>Int-Serv didn't? Or does it? What do you think?

My (cynical) feeling is that if you give tools to people who
are running and maintaining networks that feel like something
that they are familiar with, they are always more happy than
with tools that are wildly different.  

John

When Internet access came to all employees at a place I was working
at, NATs provided a great way for the sysadmins to deploy connectivity,
as it looked like how they deployed T1 lines to interconnect different
office networks between remote sites, so they thought that NATs
were perfect for them.

---

From: <[email-masked]>
Date: Thu, 18 May 2006 10:47:26 +0300

Hi Eric,

>How well accepted is the NSIS work within the IETF? For 
>example, I note that their RFCs that I have read were all 
>experimental and that they point to other experimental work 
>such as Seamoby. From this alone I surmise that it is not 
>mainstream. Is this accurate? If so, then why is the IETF 
>sponsoring their WG? Most importantly, how can their work fit 

Actually, none of the RFCs are experimental - the protocol
work is all standards track.  There's no normative references
to Seamoby - that stuff is all purely informative.

There have been a couple of protocol interops that have been
very successful - and a couple of open source implementations
of the protocols, I sent links earlier.

RSVP (especially in the -TE sense) is a very successful 
protocol; but there are a number of issues with RSVP 
and RSVP's extensibility that impact it's further use.  I think
Tony mentioned the issue about scalabililty for inter-domain
use.  

I don't see a lot of what NSIS is doing as being outside of
standard IP architecture.  Sure, some people might complain
about different some of the different QoS models, but I view
running different QoS models (ITU-T QoS models for example) over
NSIS protocols as not being incompatible with historic IP 
architecture models, like you said earlier, tunneling non-IP 
traffic over IP is well accepted practice.

For the NSIS QoS work, the driving goal is to be able to 
signal IETF QoS models on an end-to-end basis; because of
some deployment cases the signaling might have to be end-to-
middle - but this is better than some other strategies, like
having a box that sits inside of network, making QoS decisions for
entity for the end hosts. 

As a secondary goal of the QoS NSIS work, we want to be able to
support other non-IETF QoS models, so that they can signal 
their QoS over IP in an Internet-friendly way.  Probably at a
later date, we'll address how to interwork these different
QoS regimes.

I look at QoS signaling as more oportunistic than any thing else.
If you and I deploy similar QoS in our networks, it would make
sense to be able to signal this end-to-end; most likely the back
bone shouldn't know or care about QoS, as much of the QoS-related
bottlenecks happen in the local / access networks.  If / when
QoS is more widely deployed, my basic assumption is it will
be tied more towards the network where you are accessing it 
from, especially in the wireless domain.  The implication is
that there are some QoS islands at first, and opportunistically
signaling some packet treatment in these islands could be useful,
and being able to signal this to the other end would be good.

Having a default Internet QoS mechanism seems like much more difficult
issue, IMO.

John

---

From: <[email-masked]>
Date: Thu, 18 May 2006 10:51:43 +0300

Pekka,

>I remember that discussion because I remember criticizing 
>path-coupled signalling approaches :-)

I remember as well, but the idea of having having the network do
this on an end-node's behalf seems a bit wonky to me, especially
if you deploy it as some type of QoS oracle box that knows
exactly what is going on in the network and makes decisions
based upon that. 

If I remember correctly, you were suggesting a bandwidth broker style
approach, but I don't really see how to do end-to-end with such
boxes, outside of limited deployments.

John

---

From: <[email-masked]>
Date: Thu, 18 May 2006 11:30:01 +0300

Fred Baker <[email-masked]> wrote:
...
>I'm quite happy to be told that I am in denial if it can be shown.  
...
> asking 
>the network at bottlenecks to state whether it currently has 
>available capacity to accomplish a stated goal is off limits, 
...

Isn't IntServ style QoS reservation equivalent to this? If this is off
limits, are you saying there is nothing that can be done?

	Jarno

---

From: Fred Baker <[email-masked]>
Date: Thu, 18 May 2006 12:01:48 +0200

On May 18, 2006, at 10:30 AM, <[email-masked]>  
<[email-masked]> wrote:

>> asking the network at bottlenecks to state whether it currently  
>> has available capacity to accomplish a stated goal is off limits,
> ...
>

Integrated Services, aka NSIS or RSVP equivalently and presumably  
other protocols if one wants to build them, allows a bottleneck  
interface in the network to know what its requirements are and say  
"no" when it needs to. What I am saying is that if you say "I don't  
allow the network to do that", which is how I read the comments that  
opened this discussion, and the application is non-adaptive or  
insufficiently adaptive, and edge measurement models don't work well,  
I'm not aware of very many additional options. dccp has been  
suggested, but unlike TCP/SCTP-based applications (whose behavior is  
defined by their transports), putting a semi-adaptive transport under  
a non-adaptive application doesn't make the application adaptive.

---

From: Fred Baker <[email-masked]>
Date: Thu, 18 May 2006 12:03:40 +0200

> Bell-shaped heads, that what we used to call them. ;-)

Ah, yes. My children used to descend to name-calling when they  
couldn't win intellectually. They grew up.

> I firmly believe that we have the building blocks, in diff-serv, to  
> actually do the things we talk about, and make end-to-end service  
> levels a reality.

If the applications are adaptive, yes. I have a problem when they are  
not,or not sufficiently so.

---

From: Scott W Brim <[email-masked]>
Date: Thu, 18 May 2006 07:59:09 -0400

Just catching up ...

> Tony mentioned the issue about scalabililty for inter-domain

For clarity ... Are you separating the int-serv model with per-flow
state (which doesn't scale inter-domain) from RSVP as a protocol for
signaling intention and for "admission control" (which seems to work
pretty well)?

> traffic over IP is well accepted practice.

What are the "ITU-T QoS models" versus IETF ones?

Thanks.

swb

---

From: <[email-masked]>
Date: Thu, 18 May 2006 15:37:46 +0300

Scott,

>protocol; but 
>> there are a number of issues with RSVP and RSVP's extensibility that 
>> impact it's further use.  I think Tony mentioned the issue about 
>> scalabililty for inter-domain use.
>
>For clarity ... Are you separating the int-serv model with 
>per-flow state (which doesn't scale inter-domain) from RSVP as 
>a protocol for signaling intention and for "admission control" 
>(which seems to work pretty well)?

That would be part of it, yes.

>> I don't see a lot of what NSIS is doing as being outside of standard 
>> IP architecture.  Sure, some people might complain about different 
>> some of the different QoS models, but I view running different QoS 
>> models (ITU-T QoS models for example) over NSIS protocols as 
>not being 
>> incompatible with historic IP architecture models, like you said 
>> earlier, tunneling non-IP traffic over IP is well accepted practice.
>

ITU-T's Y.1541 vs. IntServ / DiffServ.

John

---

From: "Fleischman, Eric" <[email-masked]>
Date: Thu, 18 May 2006 09:10:42 -0700

If network traffic is consistent at any given network node, so that a
network node could extrapolate future events based on past or present
behaviors, then I can readily resonate with what you are saying (i.e.,
that the node could say "yes" or "no" to received QoS signals in an
intelligent fashion). However, if what the node is seeing is
self-similar traffic, such that there are no discernable patterns in the
past or present, then I wonder how that node could determine how it
should behave in the future. 

On the other hand, one could push back that nodes currently have very
limited intelligence to make such judgements. Therefore, this really is
a policy issue. The effectiveness of the policy system (when measured in
terms of unused capacity) is a function of how self-similar the traffic
patterns are. 

Does this correlate with your own insights? If not, how do you see it?

I believe that another relevant dimension is admission control. If one
has the POTS mentality in which E2E admission control occurs at the
physical or data link layers then one has an entirely different
algorithm having very different properties than historic IP.

Nevertheless, I see many attractive features within the POTS mentality
for establishing deterministic trunking. In my own mind, the ideal
system would have stochastic access in the network periphery and
deterministic trunking in the backbone core. Regardless, my question for
this list is whether or not signalling (whether it is NSLS or RSVP or
something else) can *systematically* provide deterministic trunking in
*IP environments*? What are the pros and cons that people have
discovered thus far?

-----Original Message-----
From: Fred Baker [mailto:[email-masked]] 
>I'm not aware of very many additional options.

---

From: "Fleischman, Eric" <[email-masked]>
Date: Thu, 18 May 2006 09:27:43 -0700

John,

It seems that our views of history are congruent and that we have formed
similar observations. I have three additional questions that I'd like to
ask:

Do you believe that QoS signaling is an attempt to introduce CONS
concepts into CLNP networks or do you view that to be an orthogonal
issue?

A second question is does QoS signaling in IP environments actually
achieve its goals or not? Specifically, does QoS signaling in real
networks enable support for latency sensitive, real time, and highly
interactive applications or do IP networks still fall short in real life
when compared to CONS?

Third question: We've dealt with "ships in the night" multi-protocol
deployments for years. Can the same network infrastructure deploy "ships
in the night" CONS and CLNS systems?

--Eric

-----Original Message-----
From: John Day [mailto:[email-masked]] 
Sent: Wednesday, May 17, 2006 5:39 PM
To: Fleischman, Eric; Tschofenig, Hannes; [email-masked]
Subject: RE: [arch-d] QoS Signaling Status and Support

>
>
>One lesson we learned during the 80s from the OSI debacle is that 
>mixing resource-reservation lower layers over a CONS-like network layer

>with best-effort CLNS network layers just don't work. The 
>infrastructure either needs to be CONS or CLNS -- mixing them is A Bad 
>Thing.

Take care,
John

---

From: "Stephen Sprunk" <[email-masked]>
Date: Thu, 18 May 2006 09:13:31 -0500

Thus spake "Tony Li" <[email-masked]>
> not scale to the inter-domain.

That's an interesting take, and one I'm not inclined to argue with, but 
RSVP-TE solves an entirely different problem than plain RSVP, however 
similar they may look on the surface.

RSVP (and IntServ in general) failed because it requires every hop to 
maintain per-flow state.  Even ignoring the problems with variable paths, 
per-flow state has been shown time and time again to be fundamentally 
unsuitable for large-scale networks.  ISPs found this out a decade ago, and 
even large enterprises have been seeing it for 5+ years.

RSVP-TE works on pairs of routers, which aggregate arbitrary numbers of 
flows.  Since there's a finite number of pairs, one can design the topology 
to stay within the constraints of the amount of state routers can handle.

Another thing is that RSVP-TE doesn't require the forwarding plane to 
actually pay attention to the bandwidth "reserved" to an LSP for thinks to 
work.  If one updates each LSP to reserve current usage plus X% every Y 
seconds, the forwarding plane will never drop packets except in extreme 
circumstances (which can be dealt with by increasing X or reducing Y).

The short answer, though, also explains why DiffServ is also generally seen 
as a failure: it's almost always cheaper to throw bandwidth at the QoS 
problem than to provide a real solution.  It's only really tiny pipes (e.g. 
the last mile) where any form of QoS sees even a modicum of success.

This last is why the "network neutrality" debate in the US is ultimately a 
moot point; sooner or later the Bells will find it cheaper to give 
everything best effort treatment than to extract money from people for 
better QoS.  It's been tried over and over, and every time it has failed; 
nothing has changed except the press coverage.

S

Stephen Sprunk        "Stupid people surround themselves with smart
CCIE #3723           people.  Smart people surround themselves with
K5SSS         smart people who disagree with them."  --Aaron Sorkin

---

From: "Tony Li" <[email-masked]>
Date: Thu, 18 May 2006 10:55:32 -0700

Scott,

> signaling intention and for "admission control" (which seems to work

For clarity...  I'm not sure that the two are separable.  Our experience
with RSVP is that even when used for signalling LSPs, the ISPs
immediately wanted to instantiate an N^2 mesh of LSPs, quickly leading
to enormous scalability issues, even intra-domain.  This was also with
some of the most knowledgeable ISPs who understood all of the
scalability issues and yet were still unwilling to only instantiate
flows as exceptions to normal forwarding.

We dealt with the scalability issues by scrambling and doing the refresh
reduction changes to the protocol, but even with these, the fact of the
matter is that the overhead of per-flow signalling is non-trivial and
that even intra-domain flows quickly end up being challenging.

Put another way, since per-flow state doesn't scale, what's the value in
a protocol that signals per-flow state?

Tony

---

From: John Day <[email-masked]>
Date: Thu, 18 May 2006 13:27:51 -0400

>designed to carry SS7 over IP.

I was going to challenge this claim, but the more I think about it 
the more I think you are right.  The phone company model *is* well on 
its way to taking over the IETF.  We have certainly drifted a long 
way from the fundamentals the 'Net was founded on.  But then we did 
not really keep them current, more enshrined them.

ahhh, lost opportunities.

Take care,
John

---

From: "Fleischman, Eric" <[email-masked]>
Date: Thu, 18 May 2006 13:15:40 -0700

From: [email-masked] [mailto:[email-masked]] 
>I will definitely agree that many of the people bringing requirements
to NSIS 
>are coming from a telco deployment mindset.  AFAIK, the goal isn't
strict 
>interworking with POTS, but how they can move their networks to be
fully IP 
>based.  However, for many of service providers, their value proposition
to 
>their customers is a very high-level commitment to quality - in terms
of 
>good session setup time; high service availability.  Additionally, many
of 
>these deployments have SLA agreements with other companies and they are

>looking for solutions on how to tie these things together.

Thank you for your insightful postings. However, I am concerned about
the excerpt above. The way I read it, these (telephony) entities are
trying to modify IP to preserve QoS attributes in the SLAs that IP isn't
designed to support. One cannot do this without changing other
properties of IP, which has implications to the Internet.

If the target they are trying to achieve is to provide telephony QoS
service attributes over an IP network interface to their customers then
there are more obvious ways to do this without changing the Internet in
possibly undesirable ways. What would make more sense to me is for them
to provide a VPN service to their customers having IP interfaces with
legacy QoS. Specifically, establish the required QoS at the sub-IP layer
using whatever protocol system they have already deployed that support
those QoS attributes and then run IP on top of it. This way they would
only use the IP layer as an interoperable transport without having to
struggle with issues associated with making fundamental changes to IP
itself.

The approach I'm suggesting can be locally done with minimal changes to
current infrastructures. It also doesn't require the establishment of
possibly dubeous signaling translations between protocols having very
different state machines and semantics nor does it necessitate a radical
re-design of IP. Wouldn't this be far easier for everyone? Is there
really a need for NSIS? If so, what is it?

>Some early tests, using well-known RSVP implementations have shown that
a few NSIS
>implementations scale a bit better that RSVP, at around 60k sessions,
if I 
>remember right.  I think there is some published 

I am definitely interested in seeing these performance and scaling
results. Thanks!

---

From: <[email-masked]>
Date: Fri, 19 May 2006 09:03:48 +0300

Eric,

>Do you believe that QoS signaling is an attempt to introduce 
>CONS concepts into CLNP networks or do you view that to be an 
>orthogonal issue?

Orthogonal - IP is connection-less (for the most part) but above IP is
TCP, which is connection-oriented, at least on an end-to-end basis, but
it doesn't require static state along the path.  Toss in another dozen
or so protocols, some that are more-or-less hop-by-hop, and things are
more murky.  However, I think the networking layer should be
connection-less.

>A second question is does QoS signaling in IP environments 
>actually achieve its goals or not? Specifically, does QoS 
>signaling in real networks enable support for latency 
>sensitive, real time, and highly interactive applications or 
>do IP networks still fall short in real life when compared to CONS?

QoS signaling still is not mature, IMO, so I cannot categorically answer
this, but let's say that I have seen enough results that make me think
that you can good results, even under heavy traffic.  As you could
probably tell  by my email address, I deal with a lot of a lot of
wireless topics, and it seems that with soon-to-be deployed wireless
l2s, we're seeing a lot better performance with VoIP than with
traditional circuit-switched voice - better on an end-to-end basis and
also better spectral usage and network usage.

>Third question: We've dealt with "ships in the night" 
>multi-protocol deployments for years. Can the same network 
>infrastructure deploy "ships in the night" CONS and CLNS systems?

Hard to say, actually.  I favor a more 'free-market' approach towards
networking - people are going to deploy what they think they need - and
if their deployment doesn't meet market / consumer needs, then it goes
away.  It currently seems like there is a lot of incentive to move to
CLNS systems, but the leap-of-faith needed believe that they can achieve
their needs with CLNS.

To put an arch-d spin on this, we are really seeing the development of a
hybrid network, where tradition PSTN and Cellular systems are migrating
to IP networks.  However, the more difficult part is where people are
grafting their PSTN business models to IP, and trying to ensure that
they have control points in the IP network, to ensure their business
models.

I've lately been talking to a number of large mobile network providers,
and when you talk to the people running their networks, they are
actually very IP friendly and know what it takes to meet user needs -
but it is usually back in the corporate office that IP knowledge is
lacking.

John

---

From: Roland Bless <[email-masked]>
Date: Fri, 19 May 2006 10:08:21 +0200

Hi Tony
> a protocol that signals per-flow state?

I would not generally say per-flow state doesn't scale.
It actually depends on _where_ you keep/manage state.
If it is not in the core and only at the boundary/first-hop
then I don't expect scalability problems. The tool for this is
aggregation. When considering QoS for data forwarding that's
the main difference between the DiffServ and IntServ approaches.
However, you also need this type of aggregation in the "control
plane", e.g. for signaling/admission control, otherwise you run
into scalability problems there, too.

Regards,
 Roland

---

From: Scott W Brim <[email-masked]>
Date: Fri, 19 May 2006 11:52:18 -0400

> flows as exceptions to normal forwarding.

So if they don't separate them in practice, then our claim that a "dual
mode" architecture has the lowest overall complexity doesn't hold up in
reality, and we really do need QoS routing?

---

From: David Meyer <[email-masked]>
Date: Fri, 19 May 2006 09:41:45 -0700

> reality, and we really do need QoS routing?

	What was the "claim" around "dual mode" architecture (and
	what is a dual mode architecture)?

	Thnx,

	--dmm

---

From: "Stephen Sprunk" <[email-masked]>
Date: Fri, 19 May 2006 11:58:13 -0500

Thus spake "Roland Bless" <[email-masked]>
> then I don't expect scalability problems.

Per-flow state doesn't even work at the first-hop router these days.  Larger 
end-user networks see problems with per-flow state in normal traffic 
scenarios, and even small sites can see per-flow state melt their networks 
when they get hit with the virus/worm/DDoS du jour.

If we have a "solution" that doesn't even scale if only used at the edge, 
what hope is there of it working in the core?

S