Starting off, then,
http://www.w3.org/pub/WWW/Protocols/HTTP-NG/http-ng-status.html
contains, I assume, the current plans for the next generation of
the HTTP protocol. Below is an opinion piece I've drafted based
on discussions I've had with Rohit on why this effort might be
moving in the wrong direction.
The note I wrote last night about the Web-Based Enterprise
Management Standards touches on the two things that I think
would be most useful to add to the current incarnation of HTTP:
TRANSACTIONALITY and AUTOMATABILITY. In case you missed that
memo, it's archived in section 8 of
http://xent.w3.org/FoRK-archive/current/0041.html
which is summarized as follows:
> This system could really become powerful, if they add two notions:
>
> A. TRANSACTIONALITY. By transactionality, I mean that any system
> interactions should be operations with one of two outcomes: commit or
> abort. Transactions with the system are recoverable, replicable,
> time-stampable, and secure. The transactions are also be nestable in a
> manner that preserves consistency of operations.
>
> B. AUTOMATABILITY. Right now, a human operator is required to make
> use of this system. But one of the tremendous powers of the Web (and
> especially the next generation of HTTP) is the concept of the ability to
> automate transaction operations. If you looked at the demo, think how
> great it would be if we could have machines in front of those forms and
> widgets, instead of a human, to do things like fault-tolerance, logging,
> system checking, and automation of tasks.
>
> It's unclear that the industry standards board is working on either
> of these, or even that they should. Perhaps both of these concepts
> should just be added to the current incarnation of HTTP, and that
> solution would propogate to other standards such as the one in this memo.
It might be that this is a significant enough insight to be worthy of
publication in and of its own right; see, for example, below:
> Call for papers
> Contributions are invited to a Special issue of
> COMPUTER COMMUNICATIONS
> on
> "Internet: State-of-the-art"
>
> Guest Editor: Dr Gurdeep Singh Hura, SAS/SS
> Nanyang Technological University, Singapore 639798
> Phone: (65) 799-1271, Fax: (65) 792-6559
> E-mail: g.hura@ieee.org or ashura@ntu.ac.sg
> http://www2.ntu.ac.sg:8000/~ashura/hurama.htm
>
> Publication: Summer 1997
> Submission due date: Nov 15, 1996
>
> Since the early 1990s, there has been a considerable and
> ever-growing interest among computer professionals in introducing a
> variety of services on the Internet. This aspect of IT has generated a
> wide spectrum of technologies, ranging from computing, communication and
> multimedia applications to the exchange of information across the world,
> and this has made a tremendous impact on the number of Internet users.
> The services offered on the Internet range from email, Telnet, FTP and
> Gopher through to WWW, Usenet, etc., and the underlying architecture
> (NSFnet as the backbone) and communication protocol suite (TCP/IP) have
> provided a standardized platform. HTML has provided yet another added
> enterprise and business oriented tool, and the introduction of Java and
> JavaHot browsers will further allow users to access documents live.
> While enormous amounts of data flow over the Internet, only a
> few computer professionals really understand how the Internet works -
> the underlying architecture and information flow, connectivity for a
> number of internetworking interfaces and components, etc. As a result,
> although Internet seems to be one of the 'hot' topics, the lack of a
> basic understanding of its structure and the tools available will force
> users to make mistakes and misuse the Internet.
> The objective of this special issue of Computer Communications
> is to present the current state-of-the-art of Internet technology in a
> structured manner to a wide range of readers. The issue will include
> original research papers, case studies and survey articles from both
> industry and academia. The following topics will be covered (but not
> limited to):
> - Internet evolution and growth
> - Internet architecture
> - On-line services on Internet and information retrieval tools
> - Network service options and new functionality (audio, vc, etc.)
> - Case studies
> - New innovations in WWW
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> - Future research directions for Internet
>
> Four copies of complete manuscripts (not to exceed 25 double spaced
> pages) should be sent to:
>
> Dr Gurdeep S. Hura
> Division of Software Systems, School of Applied Science
> Nanyang Technological University, Singapore 639798
>
> Deadline for receipt of papers: Nov. 15 1996
> Notification of acceptence: Feb. 24, 1997
>
> Please include a title page containing author(s) names and affiliations,
> postal and email addresses, phone and fax numbers. Contributions should
> be original and should not have appeared in any other journal. Authors
> will be required to assign copyright to Elsevier Science BV. Authors are
> advised to consult the journal's 'Notes for Authors' and 'Notes for
> Authors Submitting on Disk', published in the journal or available from
> the publishers, before submitting their papers. Further information on
> paper submission is available from the Guest Editor, or the General
> Editor (Email: 100113.2636@compuserve.com).
Now, let's move back to the "Progress on HTTP-NG" document.
I'll take this section by section. Rohit, feel free to elaborate
on anything I've missed here.
The document is separated into 3 parts: architecture, implementation
experience, and transition.
> ARCHITECTURE
>
> HTTP-NG is designed to be a replacement for HTTP 1.0 offering much
> higher performance and adding some extra features needed for use in
> commercial applications. The protocol is designed to make it easy to
> implement the basic functionality needed for a simple browser whilst
> making the addition of more powerful features such as security and
> authentication much simpler than it is for HTTP.
It's unclear that you have to radically change HTTP in order to get the
things he wants: performance, functionality increase, security,
authentication. In fact, I believe that the current HTTP, when
supplemented by a Protocol Extension Protocol (PEP) such as the one
Rohit is working on, may serve all these needs well.
> HTTP-NG uses a different basic model to HTTP. HTTP sets up a new
> connection for every request, which causes a lot of severe performance
> problems both in the time taken for each transaction, and the load
> placed on both networks and servers.
It is true that HTTP 1.x sets up new connections for every request,
but you can come up with cleverer ways of caching, transferring,
proxying, and decoupling without changing the basic model. In this
way, higher performance can be achieved.
> To avoid these problems, HTTP-NG allows many different requests to be
> sent over a single connection. These requests are asynchronous - there's
> no need for the client to wait for a response before sending out a
> different request. The server can also respond to requests in any order
> it sees fit - it can even interweave the data from multiple objects,
> allowing several images to be transferred in "parallel".
In a sense, this transforms HTTP from a transaction-based model to an
asynchronous RPC-based model.
Transactions are good because they are simple: you have a request, and
you have a response, and that's all the messaging required. With an
asynchronous RPC, you have all kinds of new problems: marshalling and
demarshalling, acknowledgements and cancellations, and you don't get
the benefits of recovery, replication, time stamping, and security.
To be fair, there's lots to be gained by moving to an RPC, but the
fundamental problem is that you LOSE transactions. As an example of why
this is bad, consider the fact that you're giving away any notion of
aggregtion. As another example of why this is bad, consider the fact
that the CORBA security model looks like SSL because it cannot talk
about transactions.
So maybe HTTP 1.x with PEP is more powerful than an asynchronous RPC;
after all, where would you put an encryption system in an RPC model?
If you try to reinvent it by putting in bracket operators, you're
simply reimplementing what you already have in HTTP 1.x!
> To make these multiple data streams easy to work with, HTTP-NG sends all
> it's messages and data using a "session layer". This divides the
> connection up into lots of different channels. HTTP-NG sends all control
> messages (GET requests, meta-information etc) over a control channel.
> Each object is returned over in its own channel.
Note the misuse of the word "it's" [sic].
I can imagine that for most purposes, you will not want this
multi-channel functionality. And, just as there IS a difference between
the object metaphor and an actual object implementation, there is also a
difference between the channel metaphor and an actual channel
implementation, which is (I think) what they're suggesting here.
> This separation of data and control information also makes redirection
> much more powerful - for example, if the object is a video the server
> can return the meta-information over the same connection, together with
> a URL pointing to a dedicated video transfer protocol that will fetch
> the data for the relevant object. This becomes much more important when
> working with multimedia aware networking technologies, such as ATM or
> RSVP.
But why change the current version of HTTP to do this? Might it not be
possible to achieve this behavior using HTTP 1.x?
> IMPLEMENTATION EXPERIENCE
>
> There are several obvious ways of implementing HTTP-NG. The first
> approach is to use a simple synchronous RPC model. The client
> establishes a connection to the server, then sends out a request. The
> client then waits until it has received all the data corresponding to
> the request, then returns. This approach is very simple to implement,
> and corresponds quite closely to current practice in older HTTP clients.
> As a simple optimization, the implementation could send a number of
> requests before waiting for all their responses.
Quite simply, using an RPC system suggests a lower grain of invocation.
This implies a more brittle client-server relation; what you'd really
rather have is interacting coprocesses that can interact in real time.
I mean, the Web should be less brittle, so it can transfer, cache,
proxy, and decouple in time and space. I hypothesize that the Web needs
to go in direction of increasing the decoupling.
> The second approach is to use an event based model. A simple dispatcher
> can be wrapped around the connection. Each request then adds a callback
> that should be called whenever the response to the request comes in.
> This model works well with event based browsers such as Arena and
> Netscape; it is also quite simple to implement.
Simple to implement does not imply good solution. Can you imagine the
callback loops in some of your more complex connections?
> The final approach is to use a multi-threaded implementation; each
> request and channel gets allocated its own thread, allowing multiple
> requests to be processed in parallel. This model works supremely well
> for servers, especially those serving as proxies. The proxy server can
> keep a cache of open connections to other proxies or to popular sites.
> Since each connection resides within a single process, requests that
> can't be satisfied from the local cache can easily be switched to the
> appropriate connection. This approach also allows efficient gatewaying
> of HTTP 1.0 requests (see next section).
This misses the heart of the problem. See, HTTP 1.1 has just enough
caching that I can do something else to achieve the effect of caching,
but the cache tags themselves form a lattice of types around which I can
do caching. Deploying a new cache protocol ONLY affects the backbone,
not the functionality. The key point here is that we need only figure
out 1) how to cache things, and 2) how to do it efficiently. The
multi-threading issue is completely orthogonal; the point is that you
don't have to change HTTP 1.1 to get the effect that you want.
> TRANSITION
>
> The best transition strategy for moving from HTTP 1.0 to HTTP-NG is
> through the use of intermediate proxy servers. This allows the existing
> base of servers and clients to continues operating as they are now,
> whilst still taking advantage of much of the performance enhancements in
> the new protocol.
>
> The reason that this works is that most of the performance problems in
> HTTP 1.0 are caused by delays in the network. If proxy servers are
> placed close to older clients and servers, then these delays become
> significant. For example, if two servers are placed at either end of a
> transatlantic link, communicating with each other using HTTP-NG, but
> accepting and sending requests to and from other systems using HTTP 1.0,
> all the HTTP 1.0 delays would all occur within a continent, rather than
> spanning the intercontinental links. Further, a cacheing server can
> interpret HTML documents and pre-fetch any inlined objects before an
> HTTP client requests them.
It's unclear that you need this transition strategy, because it's
unclear that the vision of HTTP-NG as it stands in this document is what
the world wants and/or needs. W3C should be driving the direction of
HTTP-NG towards *TP ("Star TP") -- the universal transfer protocol.
At its most fundamental level, *TP is about moving bags of bits around
the world. The model is this: when requesting information to be
transferred to you, you basically get a bag of bits, a piece of
metainformation that tells you what type describes those bits (in most
cases, it's going to be a MIME type), some routing information, and
some related external information (such as a like a PEP bag).
Think about it: all transfer protocols are variations of this. Telnet
is a single point to single point line generation scheme, which
maintains a connection. NNTP is essentially a many points to many
points flood fill. HTTP is a single point to a single point on demand,
with performance unreliable. FTP is a single point to a single point on
demand, with reliable performance. SMTP is a single point to
potentially multiple points, with reliable performance. And so on.
Note that to implement these, you don't need a vast spectrum of millions
of methods. Maybe you only need what HTTP 1.1 provides: HEAD, GET, PUT,
POST, DELETE, PATCH, etc., and everything else is something that goes
through dynamic method invocation. Heck, the metainformation could
include a signature of a method that's already on the Web for use with
the transferred bits.
Now, what good would an HTTP that serves as *TP be? The answer, of
course, is AUTOMATABILITY. For example, think of an interface builder
for the web that is automatable. As one of countless applications, you
could put a palate toegether to make a United Airlines reservation map.
Or, as another example, a vendor's home page could be received as
receive as HTML type, but you could extract other information, such as a
virtual business card, from this. This virtual business card could be a
draggable thing from my homepage, that I could drop into forms, send off
as needed, and so on. Taken to its logical conclusion, I could
annotate every form with what data types (denoted by their URLs) are
compatibe, and I could make assertions based on that, drag it into a
homepage and it fills the right fields -- and here's the key word --
AUTOMATICALLY.
As Rohit says, the problem with programming for the web is that it is
essentially two sides of a sheet of glass. On the server side, you need
to take a legacy application, and use forms to have HTTP/HTML interface
that only knows the MIME type.
But if you have a smart MIME type manager, after you've demarshalled the
bits transferred and recognize them to be a format such as "Postscript",
you could also recognize the actual individual components in there for
use elsewhere. With the proper transaction model, you could do it
AUTOMATICALLY. And HTTP 1.1 with PEP might very well be the proper
transaction model for such a scheme.
Now, this brings us to the land of Microsoft, for, at the low levels,
there is a lot of similarity between *TP and Microsoft's DCOM.
See, CORBA is too brittle for such a typing system to work, because
new types need to inherit from old types, and the hierarchy can become
quite a mess. And although it might very well be the case that Java
Beans also are loose enough to allow for this type play, it will
actually be unknown until Javasoft releases the thing in like 12 months.
But the important thing to note here is that the models afforded by Java
Beans, COM/DCOM, and, for that matter, Objective-C, are actually the
same thing! It is quite possible that COM is far better than we gave it
credit for. Infospheres started along that path, too, but along the
way it seems to have lost this crucial insight.
Rohit can probably go through the United Airlines example a lot better
than I could, but at its core what should be the crucial elements of
HTTP-NG:
1. A better file manager than Plan9, providing OPEN, READ, WRITE,
and CLOSE with caching, and byte ranges.
2. A simple but good DII, which is to say, NOT a functional one.
All I want is gateway through which I can invoke any message,
NOT necessarily understand them.
3. Persistence, as good as DCOM's Istream and Ipersiststream.
Note the continual return to DCOM -- check out Nat Brown's and Charlie
Kindel's spec of the Distributed Object Model Protocol, if you can get
through their jargon:
http://www.microsoft.com/oledev/olecom/dcomspec.txt
You have to translate through their jargon, but when you do,
you realize that they may have hit upon the big ideas. For example,
as Dan pointed out:
> 2.7 Causality ID
>
> Each ORPC carries with it a UUID known as the causality id that
> connects together the chain of ORPC calls that are causally related.
> If an outgoing ORPC is made while servicing an incoming ORPC, the
> outgoing call is to have the same causality id as the incoming call.
> If an outgoing ORPC is made while not servicing an incoming ORPC,
> then a new causality id is allocated for it.
>
> ...
>
> // Extension to implicit parameters.
> typedef struct tagORPC_EXTENT {
> GUID id; // Extension identifier
> unsigned long size; // Extension size
> byte data[]; // [size_is((size+7)&~7)]
> } ORPC_EXTENT;
This recognition of DCOM's potential breakthrough reckons us
way back to Rohit's note on COM vs. CORBA battle scheduled for
Fall 96:
http://xent.w3.org/FoRK-archive/summer96/0485.html
> From khare@pest.w3.org Wed Aug 14 21:42:08 1996
> To: w3t-tech@w3.org, FoRK@xent.w3.org
> Subject: COM vs CORBA vs ?? battle scheduled for Fall'96
>
> Well, the battle is joined. There's probably going to be very interesting
> scuttlebutt next week at Object World/San Jose.
>
> 1) the world realizes the battle lines will be DCOM vs IIOP
> 2) some folks are trying to play neutral party (Oracle)
> 3) some people are innovating around the margins (ILU, RMI)
> 4) radical innovation is still possible, esp in the guise of HTTP/2
>
> What I'm personally not certain of is whether all the lessons from
> message-oriented middleware systems have been learned.
> DCOM/IIOP/ILU-over-SunRPC/RMI are all essentially streaming RPC systems,
> without any transactioning/grouping at the lower levels. HTTP POST has some
> properties that are higher than an individual RPC, and closer to MOM -- one
> can identify an invocation, pickle it, redistribute it, delay it, track it,
> and so on. I would be happier if the world was sure #4 is a dry hole. IMHO,
> someone, somewhere, is going to look at DCOM and say, "I can see the
> higher-level abstraction that makes sense for globally-distributed systems"
> (e.g. Infosphere portlets: http://www.infospheres.caltech.edu/ ).
Infospheres ALMOST did it. It almost had the vision, then flubbed it.
The question is, is this recoverable? Because the world is obviously
waiting for a leader:
> "DCOM will be the market standard--90 percent of the objects written
> today are written to COM and OLE," said Carl Carrie, vice president of
> trading and technology research at Tullet & Tokyo Forex Inc., a New
> York broker.
>
> One user said the ability to create distributed applications using
> IIOP cannot come soon enough.
>
> "This is a big win for us because we have a lot of back-end CORBA
> objects," said Aaron Dutta, principal of banking and capital markets
> practice at Booz Allen Hamilton Inc., in New York. "Now there will be
> a client out there that we can use to create intelligent distributed
> applications that are based on a standard model."
Now, the questions are:
1. Did I do a good job of describing the problem?
2. Did I do a good job of debunking the current solution?
3. Did I do a good job of proposing an alternate solution?
4. Did I articulate the full vision well enough?
I'm throwing the ball to your court, Rohit. Take this document, and
soup it up. Make it better, because as it currently stands, it probably
does a poor job of all four of these things. *YOU* know the vision and
its importance, and *I* know the vision and its importance, but how can
we get the rest of the world to see it?
:) Adam