From: DaveNet email (dave@scripting.com)
Date: Sat Sep 02 2000 - 11:25:35 PDT
DaveNet essay, "What to do about RSS?", released on 9/2/2000; 11:17:45 AM Pacific.
-----------------------------------------------------------------------------------------
***A partial viewpoint
RSS is a simple XML-based syndication format started a little over a year ago. It's become a popular format, supported by thousands of news sources, both amateur and professional. There are serious developers and companies investing in RSS-based technology. And now that it's caught on, it's getting heavily political, which means it's also getting more interesting. I'm right in the middle of this. What you read here is partial, it represents one viewpoint, there are others.
I want to help settle the differences in RSS-land, as soon as possible. In that spirit, here's an essay that represents my current thinking, where RSS came from, what it is, what its strengths and weaknesses are, and raises questions about its evolution. If others feel that RSS is important, I encourage them to write their own essays, and post them on the Web so others can read the different points of view and help us settle the question of what RSS is and where it goes.
Even though this essay is sprinkled with "imho"s I want to be clear upfront that these are my opinions only, and while I speak for UserLand, I don't speak for anyone else. Your mileage may vary. I am not a lawyer. With all due respect, and every other disclaimer in popular use on the Internet.
***How RSS got started
In December 1997 I was looking for a project to do with XML, to get some experience, and to see if there were other developers who wanted to work in XML, because imho that's the purpose of XML, to provide common formats to allow information to flow through different applications over the Internet.
The result was the XMLization of my news site, Scripting News, www.scripting.com. I wrote it up in DaveNet, and some developers showed interest. Josh Lucas did an application that delivered Scripting News in email. Vignette did a trial app to show their customers the benefits of XML-based interchange between content management systems.
Not much more happened between then and the winter of 1999, when we started getting random questions from people at Netscape about our format. They put up a trial app that read Scripting News in XML and rendered it in HTML. A few weeks later they came out with My.Netscape, a personalized news service, an "aggregator" that rendered XML channels in a format called RSS, in HTML, according to user selection. I don't know if they used our previous work as a model, let's assume for the sake of argument that it was a completely independent development.
My feeling at the time was a combination of excitement and frustration. I wished they had worked with us, we had gone first, and shared our ideas freely. Why not build on our work? Why not work with us instead of reinventing the wheel?
At the same time I was excited, finally one of the big names in Internet software was getting behind XML in a pragmatic way. So I chose to ignore the frustration and we got busy writing an aggregator to stand alongside Netscape's. The difference between ours and theirs is that ours would be open, the service list would not be hidden and private, it would be available to any developer who wanted to build an application that aggregated the channels.
Netscape's channel list was not open. If we wanted channels to aggregate, we'd have to open our own registration point, which we did. Because ours was open, no one else had to do a registration point. In retrospect, perhaps we shouldn't have been so open, even though it enabled applications like Headline Viewer and Meerkat, it also made it easy to go forward with RSS without our participation. In the future, UserLand services might not be open in this way.
I also strongly disapproved of Netscape's publisher agreement, which placed huge constraints on the freedom of speech of the channels that flowed through their service. I wrote about this in DaveNet as well. This should be of interest to open source advocates who pay so much attention to agreements and what they allow and don't allow. To me free expression on the Web is the real purpose of the Web, its purpose is not to make huge piles of money for venture capitalists or avoid lawsuits from companies that want to restrict free speech. That one of the richest companies on the Web could put a contractual throttle on free speech was horrible to me. That no one else seemed to care was even more horrible.
At the same time, to give Netscape an incentive to work with us, we added all the features of RSS 0.90 to our XML format, to send them a signal that we felt that this was a competitive space, and to help ease a transition if they decided to work with us. I had already been in contact with them, but was told that RSS was a private Netscape-only format, not something they wanted to work with others on. I had my own opinion why they would do it this way, after a lengthy battle with Microsoft over standards in HTML, I could empathize. Which is more fun -- making software or fighting over standards? Making software, of course.
So we went forward with My.UserLand built around RSS 0.90. Then one day I got a call from Werner Eckhert, the manager of the developers working on RSS at Netscape. He told me that they were going to embrace our format, and come out with RSS 0.91 which was a merger of their format and ours. Of course I thanked him. A few days later they announced the new format and published their spec, they included some elements that even I considered questionable. However with one (important) exception, they embraced everything in our XML syndication format.
I got email from people saying that it was weird to see UserLand's stuff on a Netscape site. They gave us a prominent link in DMOZ right after My.Netscape. They wanted people to know that there was a second aggregator. This is the true Internet culture, let's not be afraid to tell people that there are others are competing. Give people choice. Treat them like they have minds, let them make their decisions based on full information. And when there's compatibility, promote the hell out of it. That's the miracle, when there's agreement. It's even better when the agreement is about something that's as useful and easy as RSS 0.91 is.
Shortly after that I got an email from Werner saying he was leaving Netscape. I think this, more than anything, explains why they embraced and supported our work. After the AOL acquisition, the RSS team at Netscape was on its way out. We were clearly commited to the future of the technology. I think they did the right thing, because over the next year, we worked tirelessly to help RSS become the strong standard that it is now.
***RSS after Netscape
It was hard work to get all the elements of My.UserLand working. There were lots of dead-ends, things that didn't work that never saw the light of day. As it grew there were scaling issues. Many of the publications required technical help and help selling it to their management. UserLand took on that responsibility. We weren't always able to help, for sure, we're a very small company. But we stayed with RSS, and we stopped promoting our own format. This is a key point. Had there been no agreement with Netscape we would have continued evolving our own format. In retrospect that probably would have worked out better, but that's not what we did.
We put our energies behind RSS, even though I feel that RSS is imperfect at modeling Web content, its core philosophy is rooted in the model of print publications where there are articles with titles, descriptions and bodies. The Web, especially weblogs, has evolved a more free-form style. I wanted to try to solve thhis problem. I felt it could only happen if there was a group of developers and content engineers who wanted to do hard, cooperative and creative work to figure out what's really going on on the Web, and instead of forcing the content through a title-description bottleneck, figure out how to widen the pipe to offer more flexibility, yet still allow meaningful aggregation.
At the same time, XML-RPC was taking hold, I wanted to see if there were interesting ways the two technologies could be linked. I'm sure there are ways for this to happen, but again, it takes two (or more) to tango, and people didn't seem to want to work with us, so I was unclear how that could happen in the context of RSS. By July of this year, it seemed that RSS was frozen, and would stay that way until a group could form to take it forward.
While RSS wasn't everything I wanted, it had a lot to offer that I did want. I've been involved in the XML developer community for a few years, and worked with Microsoft and Developmentor on XML-RPC and SOAP, and in those experiences, I think many other XML technologists miss a key reason the technology hasn't caught on with average Web developers, people who are not theorists or scientifically trained, in other words, the vast majority of Web developers. It's too complex and fast-moving for Web developers to catch onto it. The XML frameworks are designed by programmers, very high IQ programmers, the cream of the cream, people with interest in mathematics, computer science, graph theory, database architectures. To stay current in the latest XML technology requires almost a full-time commitment to that. Many companies hire consultants to keep them informed on what's going on with XML.
***RSS is about simplicity
RSS, in its simplicity, is a breath of fresh air. You can understand it fully in a few minutes. You can quickly deploy an application with just a basic understanding of HTML and a bit of experience in a scripting language like Perl, AppleScript or Python. That, imho, is the reason it gained traction, while most other XML formats are still in the working groups or waiting for deployment. In the overworked world of Web development, there's no time to study, there's only time to do.
To me, RSS is not just a syndication format, it's also a fork from the W3C process, a chance for XML to be widely adopted while the minds of the W3C working groups work out details of a network that, imho, will likely not be built. At this point there are even doubts in the minds of people who are commited to the W3C process. My views on this are well-known in the XML working groups and mail lists, and to our development partners. There's a respectful disagreement. And when such disagreements happen, both approaches should have a fair chance in the market.
I believe XML formats should be designed as end-user software is designed. Hack at the details, make every feature justify itself, reduce every three-step process to one if you can. Do it over and over, and then work on the top level. Then and only then does it get simple enough for ordinary people to use. I'm like Steve Jobs on this. I think when you lift the hood you should see a beautifully designed machine that invites you to understand and then use it.
Imho, if you want to work at the top of the pyramid, designing stuff that lots of people use, this is the price of admission. Roll up your sleeves and sweat the details. When you're done, the result you present to the users must be easy to understand and its power should be immediately evident. If you have to understand complex ideas before you even understand the purpose of a format, you lose most busy people, and your format doesn't catch on.
Now, it's OK if others don't want to invest the effort doing this, but I feel I've already paid my dues. I've been making end-user software for 20 years. I want to make end-user level syndication formats that are transparently simple. Formats like RSS.
***Let's get some help
Some people have said that I have an unfair advantage, I can write, so my formats tend to be well explained. There are a couple of reasons for this. First, I work hard at explaining things. It's not easy to make complex things understandable. And second, instead of writing five hard to understand paragraphs, I allow myself to go back and change the software to make it possible to write one easy to understand paragraph.
So to me, as one of the parents of RSS, it's sad to this courageous format be taken directly into the land of complexity. That's why this is such a big thing for me. To be fair, it should have a chance to prove itself. By an accident of history it has the name RSS. If we hadn't joined with Netscape in 1999 it probably would have a UserLand name. We never would have joined with Netscape if their format hadn't been a simple end-user-level format. We would have continued to develop and promote a format that matched our philosophy of simplicity.
I wish it had turned out this way, then the people who legitimately want to do a Namespaces-and-RDF syndication format would have to choose another name. To their credit, the water is muddied by the departure of Netscape from the process. So there's now an identity crisis, what is RSS, and who, if anyone, has the right to evolve it?
I think the answer to this question is totally obvious. But as one of the parties to the dispute it's not up to me to say what it is. Let's get a mediator, or a small group of mediators, and ask them to read our essays, and let's agree to abide by their decision. Then, feeling that our points of view have had an adequate hearing, we can accept their decision. Even if we don't like it, chalk it up to an imperfect world, and go on.
Or maybe there's a win-win, a way to gain exposure for both approaches, and for each way to gain more help from developers on the Internet. It could end up being a totally positive thing. Sometimes things work out that way, if we listen and develop trust, and respect each others' points of view.
***What's ahead?
It could turn out that the complexities I dislike so much are irrelevant. I've been wrong before. I thought desktop publishing was a dud. I thought Yahoo was a bad idea. I thought the Internet would route around the venture capitalists.
Or maybe after a couple of years of separate evolution, we might join up again, lessons learned, and take the best of both approaches and form a new one that could work well into the decade. We don't know what's ahead of us.
We also don't have a peer review or mediation process in Internet development. Perhaps this crossroad can serve as a catalyst for development of such a process. A group of technologists who are ready to mediate disputes fairly, thoughtfully and professionally, and dispassionately. Sometimes there's too much heat and passion, and a calm gentle hand can help get everyone back to work and out of each others' way.
Dave Winer
-----------------------------------------------------------------------------------------
(c) Copyright 1994-2000, Dave Winer. http://davenet.userland.com/.
"It's even worse than it appears."
This archive was generated by hypermail 2b29 : Sat Sep 02 2000 - 11:34:43 PDT