The first version of Ariba's software was written in lisp, although i think
they later rewrote it in something else.
----- Original Message -----
From: <Eugene.Leitl@lrz.uni-muenchen.de>
To: "farkit" <fork@xent.com>
Sent: Saturday, April 28, 2001 12:52 PM
Subject: Lisp, the secret weapon.
>
>
http://www.franz.com/about/conferences/fds2001/presentations/pgtalk-rev.pdf
>
> Beating the Averages
> PAUL GRAHAM
>
> This article is based on a talk given at the Franz Developer Symposium in
Cambridge,
> MA, on March 25, 2001. Copyright 2001 by Paul Graham. Draft, April 12,
2001.
>
> In the summer of 1995, my friend Robert Morris and I started a startup
called
> Viaweb. Our plan was to write software that would let end users build
online
> stores. What was novel about this software, at the time, was that it ran
on our
> server, using ordinary Web pages as the interface.
> A lot of people could have been having this idea at the same time, of
course,
> but as far as I know, Viaweb was the first Web-based application. It
seemed such
> a novel idea to us that we named the company after it: Viaweb, because our
soft-
> ware worked via the Web, instead of running on your desktop computer.
> Another unusual thing about this software was that it was written
primarily in
> a programming language called Lisp.1 It was in fact the first big end-user
applica-
> tion to be written in Lisp, which up till then had been used mostly in
universities
> and research labs. Was Lisp ready for the real world? In our case the
answer
> turned out to be a definite yes: Lisp gave us a great advantage over
competitors
> using less powerful languages.
>
> The Secret Weapon
> Eric Raymond has written an essay called "How to Become a Hacker," and in
it,
> among other things, he tells would-be hackers what languages they should
learn.
> He suggests starting with Python and Java, because they are easy to learn.
The
> serious hacker will also want to learn C, in order to hack Unix, and Perl
for sys-
> tem administration and cgi scripts. Finally, the truly serious hacker
should con-
> sider learning Lisp:
> Lisp is worth learning for the profound enlightenment experience
you will
> have when you finally get it; that experience will make you a
better pro-
> grammer for the rest of your days, even if you never actually use
Lisp itself
> a lot.
> This is the same argument you tend to hear for learning Latin. It won't
get you a
> job, except perhaps as a classics professor, but it will improve your
mind, and
> make you a better writer in languages you do want to use, like English.
> But wait a minute. This metaphor doesn't stretch that far. The reason
Latin
> won't get you a job is that no one speaks it. If you write in Latin, no
one can
> understand you. But Lisp is a computer language, and computers speak
whatever
> language you, the programmer, tell them to.
>
> 1. Viaweb at first had two parts: the editor, written in Lisp, which
people used to
> build their
> sites, and the ordering system, written in C, which handled orders. The
first version was
> mostly Lisp,
> because the ordering system was small. Later we added two more modules, an
image generator
> writ-
> ten in C, and a back-office manager written mostly in Perl.
>
>
>
> 1
>
>
>
> So if Lisp makes you a better programmer, like he says, why wouldn't
you
> want to use it? If a painter were offered a brush that would make him a
better
> painter, it seems to me that he would want to use it in all his paintings,
wouldn't
> he? I'm not trying to make fun of Eric Raymond here. On the whole, his
advice is
> good. What he says about Lisp is pretty much the conventional wisdom. But
> there is a contradiction in the conventional wisdom: Lisp will make you a
better
> programmer, and yet you won't use it.
> Why not? Programming languages are just tools, after all. If Lisp
really does
> yield better programs, you should use it. And if it doesn't, then who
needs it?
> This is not just a theoretical question. Software is a very competitive
business,
> prone to natural monopolies. A company that gets software written faster
and
> better will, all other things being equal, put its competitors out of
business. And
> when you're starting a startup, you feel this very keenly. Startups tend
to be an all
> or nothing proposition. You either get rich, or you get nothing. In a
startup, if you
> bet on the wrong technology, your competitors will crush you.
> Robert and I both knew Lisp well, and we couldn't see any reason not to
trust
> our instincts and go with Lisp. We knew that everyone else was writing
their soft-
> ware in C ++ or Perl. But we also knew that that didn't mean anything. If
you
> chose technology that way, you'd be running Windows. When you choose tech-
> nology, you have ignore what other people are doing, and consider only
what will
> work the best.
> This is especially true in a startup. In a big company, you can do what
all the
> other big companies are doing. But a startup can't do what all the other
startups
> do. I don't think a lot of people realize this, even in startups.
> The average big company grows at about ten percent a year. So if you're
run-
> ning a big company and you do everything the way the average big company
> does it, you can expect to do as well as the average big company-that is,
to grow
> about ten percent a year. The same thing will happen if you're running a
startup,
> of course. If you do everything the way the average startup does it, you
should
> expect average performance. The problem here is, average performance means
> that you'll go out of business. The survival rate for startups is way less
than fifty
> percent. So if you're running a startup, you had better be doing something
odd. If
> not, you're in trouble.
> Back in1995,we knew something that I don't think our competitors under-
> stood, and few understand even now: when you're writing software that only
has
> to run our your own servers, you can use any language you want. When
you're
> writing desktop software, there's a strong bias toward writing
applications in the
> same language as the operating system. Ten years ago, writing applications
> meant writing applications in C. But with Web-based software, especially
when
> you have the source code of both the language and the operating system,
you can
> use whatever language you want.
> This new freedom is a double-edged sword, however. Now that you can use
> any language, you have to think about which one to use. And if you try to
pretend
> that nothing has changed, your competitors will show you that you're
wrong.
>
>
>
>
> 2
>
>
>
> If you can use any language, which do you use? We chose Lisp. For one
> thing, it was obvious that rapid development would be important in this
market.
> We were all starting from scratch, so a company that could get new
features done
> before its competitors would have a big advantage. We knew Lisp was a
really
> good language for writing software quickly, and server-based applications
mag-
> nify the effect of rapid development, because you can release software the
minute
> it's done.
> If other companies didn't want to use Lisp, so much the better. It
might give
> usa technological edge, and we needed all the help we could get. When we
> started Viaweb, we had no experience in business. We didn't know anything
> about marketing, or hiring people, or raising money, or getting customers.
Nei-
> ther of us had ever even had what you would call a real job. The only
thing we
> were good at was writing software. We hoped that would save us. Any
advantage
> we could get in the software department, we would take.
> So you could say that using Lisp was an experiment. Our hypothesis was
that
> if we wrote our software in Lisp, we'd be able to get features done faster
than our
> competitors, and also to do things in our software that they couldn't do.
And
> because Lisp was so high-level, we wouldn't need a big development team,
so
> our costs would be lower. If this were so, we could offer a better product
for less
> money, and still make a profit. We would end up getting all the users, and
our
> competitors would get none, and eventually go out of business. That was
what we
> hoped would happen, anyway.
> What were the results of this experiment? Somewhat surprisingly, it
worked.
> We eventually had many competitors, on the order of twenty to thirty of
them, but
> none of their software could compete with ours. We had a WYSIWYG online
> store builder that ran on the server and yet felt like a desktop
application. Our
> competitors had CGI scripts. And we were always far ahead of them in
features.
> Sometimes, in desperation, competitors would try to introduce features
that we
> didn't have. But with Lisp our development cycle was so fast that we could
some-
> times duplicate a new feature within a day or two of a competitor
announcing it
> in a press release. By the time journalists covering the press release got
round to
> calling us, we would have the new feature too.
> It must have seemed to our competitors that we had some kind of secret
> weapon-that we were decoding their Enigma traffic or something. In fact we
> did have a secret weapon, but it was simpler than they realized. No one
was leak-
> ing news of their features to us. We were just able to develop software
faster than
> anyone thought possible.
> When I was about nine I happened to get hold of a copy of The Day of
the
> Jackal, by Frederick Forsyth. The main character is an assassin who is
hired to
> kill the president of France. The assassin has to get past the police to
get up to an
> apartment that overlooks the president's route. He walks right by them,
dressed
> up as an old man on crutches, and they never suspect him.
> Our secret weapon was similar. We wrote our software in a weird AI lan-
> guage, with a bizarre syntax full of parentheses. For years it had annoyed
me to
> hear Lisp described that way. But now it worked to our advantage. In
business,
>
>
>
> 3
>
>
>
> there is nothing more valuable than a technical advantage your competitors
don't
> understand. In business, as in war, surprise is worth as much as force.
> And so, I'm a little embarrassed to say, I never said anything
publicly about
> Lisp while we were working on Viaweb. We never mentioned it to the press,
and
> if you searched for Lisp on our Web site, all you'd find were the titles
of two
> books in my bio. This was no accident. A startup should give its
competitors as
> little information as possible. If they didn't know what language our
software
> was written in, or didn't care, I wanted to keep it that way.2
> The people who understood our technology best were the customers. They
> didn't care what language Viaweb was written in either, but they noticed
that it
> worked really well. It let them build great looking online stores
literally in min-
> utes. And so, by word of mouth mostly, we got more and more users. By the
end
> of 1996 we had about 70 stores online. At the end of 1997 we had 500. Six
> months later, when Yahoo bought us, we had 1070 users. Today, as Yahoo
Store,
> this software continues to dominate its market. It's one of the more
profitable
> pieces of Yahoo, and the stores built with it are the foundation of Yahoo
Shop-
> ping. I left Yahoo in 1999, so I don't know exactly how many users they
have
> now, but the last I heard there were about 14,000.
> People sometimes ask me if Yahoo Store still uses Lisp. Yes, all the
Lisp code
> is still there. Yahoo has server-side software written in all the
languages Eric
> Raymond recommends to hackers, except Java.
>
> The Blub Paradox
> What's so great about Lisp? And if Lisp is so great, why doesn't everyone
use it?
> These sound like rhetorical questions, but actually they have
straightforward
> answers. Lisp is so great not because of some magic quality visible only
to devo-
> tees, but because it is simply the most powerful language available. And
the rea-
> son everyone doesn't use it is that programming languages are not merely
> technologies, but habits of mind as well, and nothing changes slower. Of
course,
> both these answers need explaining.
> I'll begin with a shockingly controversial statement: programming
languages
> vary in power.
> Few would dispute, at least, that high level languages are more
powerful than
> machine language. Most programmers today would agree that you do not,
ordi-
> narily, want to program in machine language. Instead, you should program
in a
> high-level language, and have a compiler translate it into machine
language for
> you. This idea is even built into the hardware now: since the 1980s,
instruction
> sets have been designed for compilers rather than human programmers.
> Everyone knows it's a mistake to write your whole program by hand in
> machine language. What's less often understood is that there is a more
general
>
>
> 2. Robert Morris says that I didn't need to be secretive, because
even if our
> competitors had
> known we were using Lisp, they wouldn't have understood why: "If they were
that smart
> they'd
> already be programming in Lisp."
>
>
>
> 4
>
>
>
> principle here: that if you have a choice of several languages, it is, all
other things
> being equal, a mistake to program in anything but the most powerful one.3
> There are many exceptions to this rule. If you're writing a program
that has to
> work very closely with a program written in a certain language, it might
be a
> good idea to write the new program in the same language. If you're writing
a pro-
> gram that only has to do something very simple, like number crunching or
bit
> manipulation, you may as well use a less abstract language, especially
since it
> may be slightly faster. And if you're writing a short, throwaway program,
you
> may be better off just using whatever language has the best library
functions for
> the task. But in general, for application software, you want to be using
the most
> powerful (reasonably efficient) language you can get, and using anything
else is a
> mistake, of exactly the same kind, though possibly in a lesser degree, as
program-
> ming in machine language.
> You can see that machine language is very low level. But, at least as
a kind of
> social convention, high-level languages are often all treated as
equivalent.
> They're not. Technically the term "high-level language" doesn't mean
anything
> very definite. There's no dividing line with machine languages on one side
and
> all the high-level languages on the other. Languages fall along a
continuum4 of
> abstractness, from the most powerful all the way down to machine
languages,
> which themselves vary in power.
> Consider Cobol. Cobol is a high-level language, in the sense that it
gets com-
> piled into machine language. Would anyone seriously argue that Cobol is
equiva-
> lent in power to, say, Python?
> It's probably closer to machine language than Python. Or how about
Perl 4?
> Between Perl 4 and Perl 5, lexical closures got added to the language.
Most Perl
> hackers would agree that Perl 5 is more powerful than Perl 4. But once
you've
> admitted that, you've admitted that one high level language can be more
powerful
> than another. And it follows inexorably that, except in special cases, you
ought to
> use the most powerful you can get.
> This idea is rarely followed to its conclusion, though. After a
certain age, pro-
> grammers rarely switch languages voluntarily. Whatever language people
happen
> to be used to, they tend to consider just good enough.
> Programmers get very attached to their favorite languages, and I don't
want to
> hurt anyone's feelings, so to explain this point I'm going to use a
hypothetical
> language called Blub. Blub falls right in the middle of the abstractness
contin-
>
> 3. All languages are equally powerful in the sense of being Turing
equivalent, but
> that's not the
> sense of the word programmers care about. (No one wants to program a
Turing machine.) The
> kind of
> power programmers care about may not be formally definable, but one way to
explain it
> would be to
> say that it refers to features you could only get in the less powerful
language by writing
> an interpreter
> for the more powerful language in it. If language A has an operator for
removing spaces
> from strings
> and language B doesn't, that probably doesn't make A more powerful,
because you can
> probably
> write a subroutine to do it in B. But if A supports, say, recursion, and B
doesn't, that's
> not likely to be
> something you can fix by writing library functions.
> 4. Note to nerds: or possibly a lattice, narrowing toward the top;
it's not the
> shape that matters
> here but the idea that there is at least a partial order.
>
>
>
> 5
>
>
>
> uum. It is not the most powerful language, but it is more powerful than
Cobol or
> machine language.
> And in fact, our hypothetical Blub programmer wouldn't use either of
them.
> Of course he wouldn't program in machine language. That's what compilers
are
> for. And as for Cobol, he doesn't know how anyone can get anything done
with
> it. It doesn't even have x (Blub feature of your choice).
> As long as our hypothetical Blub programmer is looking down the power
> continuum, he knows he's looking down. Languages less powerful than Blub
are
> obviously less powerful, because they're missing some feature he's used
to. But
> when our hypothetical Blub programmer looks in the other direction, up the
> power continuum, he doesn't realize he's looking up. What he sees are
merely
> weird languages. He probably considers them about equivalent in power to
Blub,
> but with all this other hairy stuff thrown in as well. Blub is good enough
for him,
> because he thinks in Blub.
> When we switch to the point of view of a programmer using any of the
lan-
> guages higher up the power continuum, however, we find that he in turn
looks
> down upon Blub. How can you get anything done in Blub? It doesn't even
have y.
> By induction, the only programmers in a position to see all the
differences in
> power between the various languages are those who understand the most
power-
> ful one. (This is probably what Eric Raymond meant about Lisp making you a
> better programmer.) You can't trust the opinions of the others, because of
the
> Blub paradox: they're satisfied with whatever language they happen to use,
> because it dictates the way they think about programs.
> I know this from my own experience, as a high school kid writing
programs
> in Basic. That language didn't even support recursion. It's hard to
imagine writ-
> ing programs without using recursion, but I didn't miss it at the time. I
thought in
> Basic. And I was a whiz at it. Master of all I surveyed.
> The five languages that Eric Raymond recommends to hackers fall at
various
> points on the power continuum. Where they fall relative to one another is
a sensi-
> tive topic. What I will say is that I think Lisp is at the top. And to
support this
> claim I'll tell you about one of the things I find missing when I look at
the other
> four languages. How can you get anything done in them, I think, without z?
And
> one of the biggest zs, for me, is macros.
> Many languages have something called a macro. But Lisp macros are
unique.
> And believe it or not, what they do is related to the parentheses. The
designers of
> Lisp didn't put all those parentheses in the language just to be
different. To the
> Blub programmer, Lisp code looks weird. But those parentheses are there
for a
> reason. They are the outward evidence of a fundamental difference between
Lisp
> and other languages.
> Lisp code is made out of Lisp data objects. And not in the trivial
sense that
> the source files contain characters, and strings are one of the data types
supported
> by the language. Lisp code, after it's read by the parser, is made of data
structures
> that you can traverse. If you understand how compilers work, what's really
going
> on is not so much that Lisp has a strange syntax as that Lisp has no
syntax. You
> write programs in the parse trees that get generated within the compiler
when
>
>
>
> 6
>
>
>
> other languages are parsed. But these parse trees are fully accessible to
your pro-
> grams. You can write programs that manipulate them. In Lisp, these
programs are
> called macros. They are programs that write programs.
> Programs that write programs? When would you ever want to do that?
Not
> very often, if you think in Cobol. All the time, if you think in Lisp. It
would be
> convenient here if I could give an example of a powerful macro, and say
there!
> how about that? But if I did, it would just look like gibberish to someone
who
> didn't know Lisp; there isn't room here to explain everything you'd need
to know
> to understand what it meant. In ANSI Common Lisp I tried to move things
along
> as fast as I could, and even so I didn't get to macros until page 160.
> But I think I can give a kind of argument that might be convincing.
The
> source code of the Viaweb editor was probably about 20-25% macros. Macros
> are harder to write than ordinary Lisp functions, and it's considered to
be bad
> style to use them when they're not necessary. So every macro in that code
is there
> because it has to be. What that means is that at least 20-25% of the code
in this
> program is doing things that you can't easily do in any other language.
However
> skeptical the Blub programmer might be about my claims for the mysterious
> powers of Lisp, this ought to make him curious. We weren't writing this
code for
> our own amusement. We were a tiny startup, programming as hard as we could
in
> order to put technical barriers between us and our competitors.
> A suspicious person might begin to wonder if there was some
correlation
> here. A big chunk of our code was doing things that are very hard to do in
other
> languages. The resulting software did things our competitors' software
couldn't
> do. Maybe there was some kind of connection. I encourage you to follow
that
> thread. There may be more to that old man hobbling along on his crutches
than
> meets the eye.
>
> Aikido for Startups
> But I don't expect to convince anyone (over 25) to go out and learn Lisp.
The
> purpose of this article is not to change anyone's mind, but to reassure
people
> already interested in using Lisp-people who know that Lisp is a powerful
lan-
> guage, but worry because it isn't widely used. In a competitive situation,
that's an
> advantage. Lisp's power is multiplied by the fact that your competitors
don't get
> it.
> If you think of using Lisp in a startup, you shouldn't worry that
it isn't widely
> understood. You should hope that it stays that way. And it's likely to.
It's the
> nature of programming languages to make most people satisfied with
whatever
> they currently use. Computer hardware changes so much faster than personal
> habits that programming practice is usually ten to twenty years behind the
pro-
> cessor. At places like MIT they were writing programs in high- level
languages in
> the early 1960s, but many companies continued to write code in machine
lan-
> guage well into the 1980s. I bet a lot of people continued to write
machine lan-
> guage until the processor, like a bartender eager to close up and go home,
finally
> kicked them out by switching to a RISC instruction set.
>
>
>
>
> 7
>
>
>
> Ordinarily technology changes fast. But programming languages are
differ-
> ent: programming languages are not just technology, but what programmers
think
> in. They're half technology and half religion.5 And so the median
language,
> meaning whatever language the median programmer uses, moves as slow as an
> iceberg. Garbage collection, introduced by Lisp in about 1960, is now
widely
> considered to be a good thing. Runtime typing, ditto, is growing in
popularity.
> Lexical closures, introduced by Lisp in the early 1970s, are now, just
barely, on
> the radar screen. Macros, introduced by Lisp the mid 1960s, are still
terra incog-
> nita.
> Obviously, the median language has enormous momentum. I'm not
proposing
> that you can fight this powerful force. What I'm proposing is exactly the
oppo-
> site: that, like a practitioner of Aikido, you can use it against your
opponents. If
> you work for a big company, this may not be easy. You will have a hard
time con-
> vincing the pointy-haired boss to let you build things in Lisp, when he
has just
> read in the paper that some other language is poised, like Ada was twenty
years
> ago, to take over the world. But if you work for a startup that doesn't
have pointy-
> haired bosses yet, you can, like we did, turn the Blub paradox to your
advantage:
> you can use technology that your competitors, glued immovably to the
median
> language, will never be able to match.
> If you ever do find yourself working for a startup, here's a handy tip
for eval-
> uating competitors. Read their job listings. Everything else on their site
may be
> stock photos or the prose equivalent, but the job listings have to be
specific about
> what they want, or they'll get the wrong candidates.
> During the years we worked on Viaweb I read a lot of job descriptions.
A new
> competitor seemed to emerge out of the woodwork every month or so. The
first
> thing I would do, after checking to see if they had a live online demo,
was look at
> their job listings. After a couple years of this I could tell which
companies to
> worry about and which not to. The more of an IT flavor the job
descriptions had,
> the less dangerous the company was. The safest kind were the ones that
wanted
> Oracle experience. You never had to worry about those. You were also safe
if
> they said they wanted C ++ or Java developers. If they wanted Perl or
Python pro-
> grammers, that would be a bit frightening-that's starting to sound like a
com-
> pany where the technical side, at least, is run by real hackers. If I had
ever seen a
> job posting looking for Lisp hackers, I would have been really worried.
> Back when I was writing books about Lisp, I used to wish everyone
under-
> stood it. But when we started Viaweb I found that changed: I wanted
everyone to
> understand Lisp except our competitors.
>
>
>
>
> 5. As a result, comparisons of programming languages either take the
form of
> religious wars or
> undergraduate textbooks so determinedly neutral that they're really works
of anthropology.
> People
> who value their peace, or want tenure, avoid the topic. But the question
is only half a
> religious one;
> there is something there worth studying, especially if you want to design
new languages.
>
>
>
> 8
This archive was generated by hypermail 2b29 : Sun Apr 29 2001 - 20:26:18 PDT