From: Strata Rose Chalup (strata@virtual.net)
Date: Fri Jul 21 2000 - 15:35:46 PDT
I agree that there isn't a "starting point" for Napster, but I also feel
that the concept of starting point for the web is overrated and more
squishy than firm these days. We tend to get fixated on whether
something is "complete", but truly, none of the search engines has a
"complete" map of the web. In some ways Napster is potentially more
complete than the web itself, in that there is a central node point, the
napster server, which can be queried round the clock and to which all
the nodes connect. In many ways this can be less tedious than
portscanning everything from 0.0.0.0 to 255.255.255.255.
I am assuming that there is some logging mode for Napster that tells who
connects to you to download things, and that one's own published
songlist, plus the download log, essentially constitute a start page.
Each of one's own songs represents a virtual link into Napster, via a
search for that song (either solely in the same lexical form one
represents it in, or in that plus several permutations such as group
only, name only, etc). Even if there is no logging mechanism, it is
sufficient to start with one's own songlist and then grow via a search
for those songs, followed by a search for all songs published by each
user who publishes a song which you also publish. Do the recursion over
those results, and you will get lots and lots of data.
Someone who is a fast perl prototyper, unlike myself :-) could basically
spend an afternoon or so and make a tool that would topologically
transform a series of static searches into web pages of that
description, at which point they could then be indexed by conventional
webtools. Of course, developing an intermediate representation would be
vastly more efficient, but all in all the Napster space is merely a
topological transform of web space with some unusual edge conditions.
Cheers,
_Strata
Lucas Gonze wrote:
>
> I'm not sure how that would improve the problem, Strata. It is still
> about entering random search terms and walking the list of results. You
> would still have an incomplete list of results.
>
> The difference between Napster crawling and web crawling is that web
> crawling gives you a starting point - an index page - that you can pick
> the links out of.
>
> That's not to say that you can't write a Napster crawler, but it would
> be brutish even for a brute force approach.
>
> Strata Rose Chalup wrote:
> >
> > To "crawl" Napster, though, all you'd have to do is use the protocol to
> > search on user. You could do some automated crawling to get usernames,
> > and then move from there.
> >
> > _Strata
> >
> > Lucas Gonze wrote:
> > >
> > > Can you talk more how you want to implement this, Dave? No slam
> > > intended, but crawling Napster could be difficult to do in any way that
> > > gave good quality data.
> > >
> > > You'd need to find search terms to match every possible file, and connect
> > > to all of the servers (Napster limits you to a subnet of their total
> > > userbase), and find a way around the limits on number of hits the server
> > > will report. The last issue would be the trickiest. On the other hand,
> > > since Napster clients push the complete local song list up to a central
> > > server, Napster the company could easily compile the listing with 100%
> > > accuracy. ...I'd be surprized if they don't already.
> > >
> > > Gnutella is much easier to program. If the musical tastes of gnutella
> > > users aren't too different than the tastes of Napster users, that would be
> > > a far simpler route to take. I ran across a site that compiled a top-40
> > > list a few days ago; I think I got there from gnutella.wego.com. (I'd
> > > post it again right now but gnutella.wego.com is so _unbefrigginly_
> > > slow!?!!)
> > >
> > > - Lucas
> >
> > --
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > Strata Rose Chalup [strata@knownow.com] | strata@virtual.net, KF6NBZ
> > Director of Network Operations | VirtualNet Consulting
> > KnowNow, Inc [http://www.knownow.com] | http://www.virtual.net/
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-- ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Strata Rose Chalup [strata@knownow.com] | strata@virtual.net, KF6NBZ Director of Network Operations | VirtualNet Consulting KnowNow, Inc [http://www.knownow.com] | http://www.virtual.net/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
This archive was generated by hypermail 2b29 : Fri Jul 21 2000 - 15:36:25 PDT