I am thinking that protocols that meet the collective 2/3'ds majority
approval are allowed to carry a cute little California Garibaldi
seal of approval with extra gold stars for making a robust economic
impact and working in a bi-partisan political manner with other
protocol organizations.
One thing I've learned from Seventh Heaven over the months is
that every protocol needs a killer app. So, sure you can stick everything
you need into a GUP, but how many applications truly need it? Sort
of a factored protocol language, speaking of which, what's the latest on PEP?
Greg
Rohit Khare wrote:
>
> Any and all editorial thoughts would be appreciated -- but please
> reply to me personally. It seems flabby to me, and I'm too bleary to
> tell if it's too lame or too technical at times... RK
>
> This column: http://www.ics.uci.edu/~rohit/IEEE-L7-applcore.html
> First column: http://www.ics.uci.edu/~rohit/IEEE-L7-1.html
> General info: http://www.ics.uci.edu/~rohit/
>
> Seventh Heaven appears in each issue of IEEE Internet Computing:
> http://computer.org/internet
>
> ---
> Rohit Khare -- UC Irvine -- 4K Associates -- +1-(626) 806-7574
> http://www.ics.uci.edu/~rohit -- http://xent.ics.uci.edu/~FoRK
>
> ========================================================================
>
> Building the Perfect Beast
>
> Dreams of a Grand Unified Protocol
>
> Rohit Khare // March 5, 1999
> IEEE Internet Computing // Seventh Heaven
>
> Seventh Heaven was inaugurated one year ago to chart the evolution of
> Internet application protocols. Perched upon the heights of the
> network stack, we paged back through history to trace the lineage of
> Telnet, FTP, SMTP, NNTP, Gopher, and HTTP. With Jon Postel's
> fingerprints to guide us, the arc of thirty years of Transfer Protocol
> (TP) development brought us to the present day -- where the pages of
> our history books go blank.
>
> Leaving us, gentle reader, with but one option for our second season:
> to turn from documenting the past to divining the future. After all,
> unlike designs past, there are infinite number of designs yet-to-be:
> XML-RPC, HTTP-NG, DIAMETER, IMPP, SIP, ISEN, SWAP, WAP, SDMI, ...
> and every other acrynomious concoction we can get our hands on(*). And
> that's to say nothing of how protocols are actually adopted and
> history actually gets written: the mythical eighth (Economic) and
> ninth (Political) layers of the stack. These are the kinds of new
> topics we'll tackle in 1999: new protocols, economic models, and
> standardization processes.
>
> (*) [Footnote: For the record, that's Remote Procedure Calls in
> Extensible Markup Language, HTTP-Next Generation, an extensible
> successor to RADIUS (Remote Dial-In USer) authentication, Instant
> Messaging and Presence Protocol, Session Invitation Protocol,
> Internet-Scale Event Notification, Simple Workflow Access Protocol,
> Wireless Access Protocol, and the Secure Digital Music Initiative.]
>
> Johnny APPLCORE
>
> Conveniently, this raises a very timely topic to the head of our
> agenda: the IETF Applications Directorate's debate over the
> possibility and desirability of an Application Core Protocol
> (APPLCORE). This latest outbreak of an ancient meme bloomed rapidly,
> from its first mention in January to a Birds-of-a-Feather session at
> IETF-44 in Minneapolis by March. The proliferation of new applications
> derived -- or worse, cut-and-pasted by parts -- from SMTP and IMAP
> (Interactive Mail Access Protocol) has prompted an interest in
> documenting the common challenges of such extensions, and perhaps in
> developing an entirely new set of command building-blocks.
>
> Another variant of the same infectious hope motivates the HTTP-Next
> Generation project, sponsored by the World Wide Web Consortium with
> participation from Xerox and Digital (profiled in detail by Bill
> Janssen last issue). From their vantage point, the Grail looks like a
> programmable distributed object protocol, atop an efficient new
> multiplexing transport. They look ahead to interoperating with Java
> RMI, CORBA IIOP, and RPC protocols.
>
> And of course, HTTP/1.1 itself has proven flexible enough for a host
> of communities to (ab)use it to their own ends. From distributed
> authoring and versioning (WebDAV) to the Internet Printing Protocol
> (IPP), the freedom to define new methods, headers, and Internet Media
> Types at will has led to at least three schools of thought on
> extending the hypertext Web to other information transfer problems.
> Its proposed Mandatory extension mechanism aims to rationalize the
> patchwork of existing rules.
>
> These three camps -- Mail, Web, and Objects -- are each vying to
> establish a new order for Application Layer Protocol design. The quest
> for universal applicability, though, crosses several common fault
> lines in the design space. Each camp tends towards different tradeoffs
> between human- and machine-readability, command-oriented and stateless
> transactions, end-to-end and proxiable connections, inspectability and
> security, and whether to exploit the unique properties of UDP,
> multicast, broadcast, and anycast transmission.
>
> While it may seem quixotic to hope for a Grand Unified Protocol, part
> of the IETF ethos is that you never know unless you try. However slim
> the opportunity for clean reengineering, it is still appears
> worthwhile to each camp to seek out an "application-layer TCP," a
> second neck in the Internet hourglass.
>
> Or, as mail protocol and security maven Chris Newman put it when
> calling for APPLCORE in the first pace, "What protocol facilities are
> common to most of FTP, HTTP, IMAP, LDAP, NNTP, POP, SMTP/ESMTP, Telnet
> and our other successful protocols? ... I can't predict what a
> careful study of IETF protocol history and comparison of candidate
> solutions will suggest." Since that's the very mission statement of
> our column, let's take a shot at evaluating these camps'
> prospects...
>
> Diversity vs. the Melting Pot
>
> While "transport" protocol merely deliver piles of bits, TPs need to
> 'tag-and-bag' their information, almost invariably using MIME syntax.
> IETF's eventual success in hammering out a standard for packaging
> information objects is a powerful inspiration for today's protocol
> unification.
>
> It's not much of a stretch to think of today's TPs as merely different
> strategies for delivering MIME objects -- real-time vs batch, push vs.
> pull, end-to-end vs. relayed, reliable vs. unreliable, and so on. Over
> the past year, this column has fleshed out a model of Transfer
> Protocols (TPs), as summarized in Table 1. We classify three aspects
> of TPs: addresses that identify participating nodes; distribution
> rules controlling transfers; and the message formats on the wire.
>
> Before proposing a universal solution by picking winners and losers,
> it behooves us to investigate the tensions and tradeoffs that
> motivated such diverse solutions in the first place.
>
> Engineering for Humans or Machines?
>
> While all of these protocols are executed by machines, they are
> created by programmers -- two actors with rather different priorities
> for standardization. Will the messages be binary (and faster for
> computers to parse) or English text (and easier for humans to debug
> and extend)? Will the application semantics be precisely defined, a la
> RPC, or open to extensible interpretation, like HTTP?
>
> Syntax. Two archetypal alternatives are RFC822 header fields and X
> Window System events. Email, news, and web headers from any
> information service are reasonably easy to understand, at the cost of
> marshaling complex data structures like dates and numbers as ASCII
> text and parsing all sorts of variations of case, line folding,
> comments, character sets and other gremlins to uphold the maxim "be
> liberal in what you accept". X Protocol messages are tightly packed
> binary data aligned on machine boundaries -- and can be processed
> several orders of magnitude more efficiently because the control flow
> through a server is clearly delineated by X Window System semantics
> and a strict extension mechanism.
>
> Furthermore, both alternatives are relatively brittle.
> Internationalizing header text in various human languages is
> notoriously difficult because of the number of techniques available
> (RFC 2277, "IETF Policy on Character Sets and Languages"). IETF also
> has some (exasperating) experience with ISO's Abstract Syntax Notation
> (ASN.1) self-describing binary marshaling format for SNMP and X.500
> public-key certificates. The tools for compiling such data structures
> are expensive and still can't guarantee interoperability amongst all
> the possible Encoding Rules (Basic, Defined, Text, ...).
>
> The silver bullet of the moment is XML. Roll-your-own tagsets offer
> some hope of jointly guaranteeing machine-readable validity and
> human-readable message text. As WebDAV encountered, using XML in a
> wire protocol has its own costs: complex machinery for XML text
> encoding ("entities"), whitespace, namespaces, and bloated message
> text. And for all that, there's still no standard data-formatting
> rules yet for basic data types like dates, numbers, arrays, and so on.
>
> The ability to simulate a protocol exchange with Telnet is another
> litmus test whether it's catering to machines or humans. If the
> application itself expects to be supported by individual users and
> system administrators, it's useful to experiment and debug the system
> with a bare-minimum tool available on almost any host. If it's a core
> OS service supported by the vendor, then it's reasonable to expect
> libraries for inspecting binary messages and APIs to support it.
>
> Semantics. Another way of justifying the "Telnet test" is to estimate
> how many independent implementations are expected. While there are
> only a few X servers and client libraries -- including definitive
> editions from the X Consortium -- there are uncountable little
> HTTP-driven scripts and hacked applications babbling away in pidgin
> dialects developers learned by imitating other clients and servers.
> The smaller the community of developers, the greater the likelihood of
> establishing a common ontology. A system as multilateral as the Web
> dilutes the semantics of "GET" to the point it could be opening a
> database, running a Turing-complete program, instructing a robot, or
> any other process that eventually generates a MIME entity.
>
> Turn the argument on its head, and it illuminates the tension between
> APIs and protocols. If there is a suite of clearly defined operations,
> there's more ease-of-reuse by standardizing programming interfaces
> ("the Microsoft way," Generic Security Services (GSSAPI, RFC 1508)).
> If implementations are expected to diverge, just focus on the
> bytes-on-the-wire and the message sequence ("the IETF way," Simple
> Authentication and Security Layer (SASL, RFC 2222)).
>
> It's the Latency, Stupid!
>
> Performance constraints also vary with human-driven or machine-driven
> transactions. Interactive use must minimize latency, leading to
> stateless protocols, while batched server-to-server communications can
> optimize bandwidth utilization with stateful command protocols.
>
> In the messaging arena, POP and IMAP provide client-server access,
> while SMTP and NNTP relay between stores. POP optimizes latency by
> selectively listing headers and bulk data separately; IMAP further
> offers concurrent operations. SMTP and NNTP, though, operate in modal
> lockstep, pipelining transmission and reception of new messages.
>
> On the Web, last issue's column suggested that the earliest HTTP/0.9
> spec was even less powerful than FTP -- but it had a critical
> advantage of lower latency. While FTP requires separate commands (and
> round-trips) to login, authenticate, navigate to a path, and request
> transmission, an HTTP download begins within one round-trip after
> connection establishment. That's because its request message
> encapsulates all those commands in a single message (URLs for path and
> filename; headers for authentication information and media-type) --
> and each request can be processed on its own, without reference to the
> state of a connection.
>
> As the 'bandwidth-delay product' gets larger and larger for wireless,
> fiber, and satellite links, stateless protocols become more valuable
> -- it takes less time to pickle the added state information and
> transmit it than to wait for a command to complete.
>
> Relay Races
>
> End-to-end support is another contentious design decision facing our
> conquering heroes. Stateful command sequences are harder to proxy
> through firewalls or cache, but can manage concurrency explicitly.
> Store-and-forward messaging (TPs) typically operate across a chain of
> relays, in contrast to interactive query and access protocols (APs)
> directly connecting clients and servers.
>
> Consider, then, the Calendaring and Scheduling WG's dilemma. Their
> requirements included connected and disconnected operation, queries
> against multiple stores, and low-bandwidth operation
> (draft-ietf-calsch-capreq-02.txt). There are some arguments for
> modeling a calendar as a Web resources, and operations upon it as HTTP
> transactions. In particular, the DAV and DAV Searching and Locating
> (DASL) extensions cover some of their requirement space. It's unclear,
> though, if the HTTP caching model is sufficient for nomadic use,
> though. An access protocol like IMAP has richer support for reflecting
> local actions immediately, chains of actions, and conflict resolution,
> but at the expense of custom development (about four years in this
> case).
>
> Concurrency is also represented differently in each school. An AP can
> tag its requests by ID and thus reshuffle its responses to several
> outstanding operations in the same session. APs like SMTP and NNTP can
> also "turn" the connection around to make requests in the opposite
> direction. Stateless TPs typically require synchronous response,
> tacitly pushing concurrency control -- the timing and priority of
> responses -- to the transport layer. Thus, Web browsers that open
> multiple TCP connections; and the Message Multiplexing (MEMUX) effort
> within HTTP-NG.
>
> Unlike end-to-end APs, proxying permits intermediaries to offer
> sophisticated services, from caching to Japanese translation. While
> HTTP cannot explicitly model the side-effects of a chain of operations
> (does the server reply to the outstanding GET or PUT first?), its
> statelessness does enable a rich caching model (I don't care as long
> as the reply's no older than five minutes).
>
> Checkpoint Charlie
>
> Firewalls are another kind of proxy. Network administrators have a
> right to inspect the contents of Internet connections. Today's
> baseline is filtering services by TCP port number, but as more and
> more applications attempt to extend HTTP or other APs, the port number
> isn't precise enough to enable or disable specific services. It's
> intellectually dishonest to use an existing, popular protocol as mere
> transport "because it gets through firewalls" -- they're there for a
> reason.
>
> For example, one totally flexible universal protocol is to map remote
> procedure calls to HTTP, whether in XML or directly coding, say
> Distributed COM (shipped with NT5, though thankfully off by default).
> If Web traffic could now conceal information leaking out or hackers
> coming in...
>
> Even multiplexing makes firewalls more resource-intensive. One TCP
> connection running MEMUX could have several subchannels, each of which
> must be judged individually.
>
> End-to-end encryption of an AP is another tough scenario: while a TP
> proxy might be able to decrypt and then forward an entire information
> object, it can be a violation of the standard to interpose a proxy in
> the middle of a stateful conversation.
>
> Security is more than transport-layer encryption alone. There are far
> too many application-layer Authentication and Authorization schemes.
> SASL succeeded by providing a simple building block -- a sequence of
> challenge-response messages and status codes -- that allow developers
> to mix-and-match authentication algorithms.
>
> Making the Medium the Message
>
> The layers below also offer application designers unique capabilities
> often overlooked. Telnet, for example, relied on TCP's urgent data and
> interrupt facilities, but HTTP serenely floats atop any 8-bit clean
> channel (even half-duplex!). Since there are so few applications that
> take advantage of broadcast, multicast, and anycast semantics, it's
> hard to plan ahead for a core protocol that could bridge those modes.
> And even though most services reserve TCP and UDP ports, datagrams are
> typically used for "small enough" messages -- it's the rare protocol
> that intelligently copes with lost packets.
>
> Link-layers also affect the evolution of application protocols. The
> Wireless Access Protocol suite is founded on a belief that every
> Internet layer must be reinvented for the cellular environment.
> Very-low-bandwidth and very-high-latency environments call for compact
> message encoding and pipelining, among other features.
>
> Camp the first: Mail
>
> The initial call for APPLCORE came from folks in the email and
> directory communities. A strawman proposal like Application Core
> Protocol (draft-earhart-acp-spec) can trace its heritage back to
> Postel's SMTP and FTP state machines and his theory of reply codes. It
> defines a framework for transitioning to an authenticated connection,
> issuing commands, and receiving interleaved, tagged responses a la
> IMAP.
>
> While it's easy to imagine, say, NNTP's current-group and
> current-article pointers and commands in this vein, the APPLCORE
> charter per se does not call for a stateful solution. It's just a
> coincidence that its author believes the proposed WG should " focus on
> a single "core protocol" based on the connection-based stateful
> client-server structure that most successful IETF application
> protocols follow."
>
> Camp the second: Web
>
> HTTP proponents would beg to differ at Chris Newman's imprecation that
> it's one of the "protocol models with which we have far less
> experience...research problems and thus out-of-scope." They can
> point at a handful of significant, standardized extension packages for
> HTTP/1.1 -- and a massive set of informal experiments. Its stateless,
> textual model is particularly hackable, even by shell scripts and
> programs without any internal model of the Web.
>
> HTTP/1.1 enables all this by permitting new methods, header fields,
> and content types. WebDAV, for example, modified the PUT method with
> lock fields; added MOVE and COPY (limited by a Depth: header); and
> manipulates metadata with XML-coded requests and responses.
>
> Complexity is the natural consequence of such freedom. Consider how
> byte-ranges, the ability to request only a portion of the expected
> response (typically, for partial rendering, e.g. a single page of a
> PDF file), interact with all other possible extensions.
>
> This community's best solution is the Mandatory extension mechanism
> (draft-frystyk-http-mandatory), which provides a namespace to discover
> more about an extension, and switches to indicate whether to succeed
> or fail if an extension isn't recognized. This permits multiple
> extensions to interoperate by marking off their own method names,
> header fields, and error codes. Extensions are identified by URIs and
> marked as Mandatory or Optional obligations on the next server (proxy)
> or only the origin server.
>
> The risk of continuing to subsume more application services within
> HTTP is the tendency to abuse it as an opaque container for their own
> traffic. One of its authors, Roy Fielding, declaimed: "The Web uses
> HTTP as a transfer protocol, not a transport protocol. HTTP includes
> application semantics and any application that conforms to those
> semantics while using HTTP is also using it as a transfer protocol.
> Those that don't are using it as a transport protocol, which is just a
> waste of bytes."
>
> Camp the third: Objects
>
> The third tack is to directly model Internet applications as
> distributed programs, and the client-server protocol follows as a
> consequence of the API. The IETF's standards-track Remote Procedure
> Call v2 specification (RFC 1831) was issued four years ago, based on
> fifteen years' experience at Sun and elsewhere. The popularity of
> CORBA and Java RMI interfaces underscores the power of this approach.
>
> So while the Web camp could be said to provide OO interfaces through
> document transfer (FORMs and TABLEs and so on), the HTTP-NG effort
> aims to provide document transfer over an OO interface. Its three
> layers begin with MEMUX to optimize transmission for the Web by
> composing compression, encryption, and other services for its virtual
> channels; then a messaging layer that can marshal binary request and
> response messages; and its own upper layer for services like The
> Classic Web Application (TCWA), WebDAV, printing, and so on.
>
> It's certainly appealing to reuse application protocols by subclassing
> and adding your unique functionality, then composing it with other
> off-the-shelf modules for security and performance. But Chris Newman
> invokes the conventional wisdom at the IETF: "RPC mechanisms are a
> poor choice in general for standards-based protocols. It's much harder
> to design an extensible and simple API than it is to design an
> extensible and simple wire protocol."
>
> Prospects
>
> So with each camp set against the others, how likely is any compromise
> core protocol -- or one camp's hegemony? The answers are not at layer
> 7 alone: we must climb higher. Technical analysis alone does not point
> to a clear intersection of services, nor justifies any protocol's
> universal utility.
>
> Layer 8: Economics
>
> First, investigate whose scarce resources are being reallocated.
> "Reuse" claims litter the 120+ message archive of the debate
> (http://lists.w3.org/Archives/Public/ietf-discuss/) -- but whose
> efforts are being reused?
>
> There are at least three plausible actors: programmers, spec writers,
> and standards committees. The first group doesn't benefit until
> there's a single core protocol on the wire and a code library -- and
> then only for developing 'multi-protocol' tools. The second benefits
> by consulting the accumulated wisdom of APPLCORE to navigate the
> archives and cite prior art. The third benefits from rigorous modeling
> of the design alternatives and a framework to classify inter-extension
> dependencies in order to judge which protocols ought to be blessed.
>
> Now we can identify institutions that would be motivated to invest in
> this effort. Look for vendors who implement lots of protocols within a
> single product. Microsoft, for example, promotes a single "Internet
> Information Server" with the vision of vending the same information by
> HTTP, DAV, and other protocols. Netscape has a new app for every
> protocol. Apache, focused purely on HTTP, has little interest in
> implementing printing, conference calls, or other whiz-bang
> applications. In the second group are authors with experience from
> multiple working groups. Such IETF luminaries are leading each camp,
> respective examples such as Chris Newman, Henrik Frystyk Nielsen, and
> Mike Spreitzer.
>
> Layer 9: Politics
>
> To identify the third class of actors, though, requires understanding
> motivations at the highest level: Politics. The standardization
> process is an institutional struggle, in this case to establish order
> within the IETF Directorates, and to defend its turf from ISO and
> other bodies.
>
> APPLCORE intends to "go only in directions that at least two
> successful IETF protocols have gone," Newman claims. This extends the
> "rough consensus" maxim to bless reuse of solutions as guidance to
> spec developers. But to envision and APPLCORE robust enough to measure
> which protocols are worthy or not is another order entirely -- and one
> the IESG may not support.
>
> For example, SG member Brian Carpenter commented "that approach could
> imply that HTTPng is the basis for all future applications
> protocols...I doubt that the IETF is ready to make that step."
> There are similar interests against defining the Internet as mail-like
> or web-like.
>
> Instead, if no dominant camp emerges, a compromises which aims for
> their intersection raises the scepter of another bogeyman: ISO. IETF
> culture alternately derides for lowest-common-denominator designs with
> lots of options required for the useful cases; and for wasteful
> layering. Charges of ISOism have been leveled by all sides in this
> debate. Grand Unification Theories from the Object Management Group
> (OMG) don't go over well, either.
>
> All of these considerations feed into the decision whether IETF even
> charters a WG on this topic. Without viable candidate protocols, it
> smacks of design-by-committee -- which triggers quite an allergic
> reaction in this community.
>
> Implication
>
> But if -- just suppose -- APPLCORE succeeded, what a wonderful world
> it would be! An application designer would only have to focus on
> understanding the underlying interaction pattern. Leave it to this bit
> of middleware to choose the right addressing system, syntax, and
> distribution algorithm in order to get the right bag of bits to the
> right people by the right time. Internet application design seem more
> like "rational drug discovery," adapting a protocol tuned to its
> requirements rather than merely chasing popular implementations and
> patching them to match.
>
> We expect new kinds of hybrid applications: Event notification with
> server-initiated delivery, not just polling. Smart pages that
> represent live processes. Multiprotocol message archives accessible
> over the web, on the phone, as a fax, by e-mail...
>
> Sure, it would only be a 'good enough,' but a single interface would
> be immensely valuable. TCP isn't ideally tuned for many applications,
> but its ubiquity has a quality all its own. Its twenty-year dominance
> has relied on technical ingenuity to manage its evolution and explicit
> political commitment. We have IP for packets, TCP for streams, and a
> clear vacancy for a message middleware standard.
>
> The Perfect Beast
>
> Realistically, though, we suspect this outbreak of the Perfect Beast
> meme will only take us as far as cataloging design patterns of
> Internet application layer protocols. Thirty years' experience is long
> enough to write a textbook, even if the ultimate solution isn't at
> hand. We can capture common solutions to common problems along with
> the rationale. Guidance like "measure compatibility with capability
> lists rather than version numbers" would greatly reduce design-time
> costs -- and guide repairs to today's protocols.
>
> It's instructive to reflect upon other outbreaks. Mightn't the search
> for One True Application Protocol lead to the same cul-de-sac as the
> One True Programming Language? That community has also played a game
> of turtles: whether functional, imperative, or procedural ought to be
> the most fundamental representation. LISP has been the language of the
> future since 1959 -- even as it has grown to accommodate procedures
> (Scheme), objects (CLOS), and a vast array of time-tested tools (the
> Common Lisp environment). The risks of sprawling unification were
> incoherence, poor performance, and refragmentation into sublanguages.
>
> There is a fundamental phenomenon at work: ontology recapitulates
> community. Ask how many people need to understand a given sentence?
> -- whether source code or protocol message. That's how universal
> the solution needs to be. If we want every Internet-accessible
> resources in a global hypermedia system, you can't expect more
> than a snapshot representation; if you expect to manipulate a datebook
> to automatically schedule a meeting, then only special-purpose
> calendaring tools will be cognizant of other conflicts.
>
> All we can hope for is to make the common case easy and keep the hard
> stuff possible...
>
>
> Rohit Khare is a graduate student in the WebSoft group at the
> University of California, Irvine; and a principal of 4K Associates,
> a standards strategy practice.
>
>
>
> [Word count:~4,000..Note to the editors: feel free to elide or
> substitute the intra-document headings Note: Table 1 can be copied
> directly from the March/April 1998 issue's Table 1, with a suitable
> change of tense in the caption.]
>
>
> <PRE>
> CAPTION: Three key aspects of the Transfer Protocols (TPs) Seventh
> Heaven covered in 1998
>
> Transfer Protocol Addressing Distribution Content
> ------------------------------------------------------------------------
>
> Terminal Telnet Host Port 1-1 Sync Both Bytestream w/interrupts
> Files FTP Host Path 1-1 Sync Both Text / Binary Files
> E-mail SMTP Mailbox Msgid 1-N Async Push 822 + MIME
> Usenet NNTP Newsgroup Msgid N-N Async Push 822 + MIME
> Web HTTP Host URL Path 1-N Sync Pull 822 + MIME +
> HTTP caching
>
> </PRE>