Open source optimizes everything, even things no one cares about...

Dave Long dl@silcom.com
Sun, 27 Jan 2002 16:28:57 -0800


> Just noticed a posting on freshmeat.net for a "Super sed"  --
> I'm sure it's a fine piece of work, but why?

As plastic bits pop off my current
machine and the LCD shows signs of
age, I find myself contemplating a
replacement box.  Now, the current
box, with all its 133 MHz, manages
to handle my workload responsively;
how am I to soak up all the cycles
a modern machine boasts?

One possibility is to find out how
far the traditional Unix model can
be pushed for graphical use.  (Do
not laugh yet -- my current window
manager is an awk script)

What sorts of things do we do when
we compute?  Functions can be made
in general by first filtering bits
out of the domain, mapping what is
left onto other bits, and finally
interpolating more into the range.
Unix has tools for special cases of
each of those, and pipes to compose
them, but sed alone* suffices for
expressing such functions.

For instance, el cheapo web service
integration, implemented in:
"USe FUll / plate tectonics"
<http://www.xent.com/pipermail/fork/2001-July/001854.html>

What about data structures?  Many
operations on composed data can be
implemented by splicing together
functions that decompose the data,
functions that transform elements
in a useful manner, and finishing
with functions that recompose the
results into a final one.

For instance, el texto fibonacci
calculator (unary), implemented by:
> echo "xxxxx" | sed -e ":s;s/xxx\(x*\)/xx\1gx\1/;ts;s/x[x]*/x/g;s/g//g"
yields: xxxxx
(decimal->unary and unary->decimal
sugaring left as an exercise for
the bor^H^H^Heager reader)

What about state?  Files are one
way functional scripts can keep
state between runs (my filesystem
buffers are much faster than tape
systems ever were); during a run,
sed has the hold space, and oawk
and shells have arbitrary storage.

Feed a stream of input events into
a set of functions to update state
and display, and one has a GUI app.
Can it be done on modern boxes but
with old-school software tools and
yet have sub-100ms visible latency?

(my hunch is that Tcl, Lisp, etc.
are living examples that sometimes
one needs unboxed bits, no matter
how fast the machine.  I have seen
the Green Destiny defeated by just
one stick, well-wielded, but that
was a movie.)

-Dave

* sed's ability to implement lookup
tables with s/// may be why regexps
seem slow compared with the ones in
more specialized tools.