Smoke Update
Thanks for the comments, everyone. Due to the traffic from Dave Winer, I got several interesting suggestions and we(my team) have had some discussions on how to proceed with this. These are some of the ideas we've been toying with for the last 2 days - none of them are final, of course.
Only Podcasts (enclosures)
Bittorrent doesn't work very well for small files which is what most RSS feeds are (< 200K). Couple this with the fact that only a few blogs get heavy traffic, there's not much use for creating torrent files for everyone. In fact, this would just slow things down as downloading small files over Bittorrent is slow. Right now, we're thinking of making a change to the way we handle feeds- instead of creating a torrent for the entire feed, we create a torrent only for the enclosures (the podcasts, videos,etc).
However, there is a problem with this approach. When some small site suddenly gets a heavy amount of traffic (from a Slashdotting or Scobelizing or whatever), we really can't help with their RSS bandwidth problem.
The update issue
Frankly, we have no idea right now as to how we'll handle blog updates. Whenever someone updates their blog by either posting a new entry or modifying an existing one, we need to detect the change and recreate a torrent file (with the hash for the new piece). We're considering a few ideas as to how to actually detect this but none of them seem elegant. We can either
(a) Ping the server periodically. Doesn't seem a very scalable solution when you have tons of blogs updating constantly
(b) Have the blog ping us (the Smoke server). This goes against our tenet of Smoke just working without any modification to current blogging platforms
(c) Keep track of pings to weblogs.com. This may work - but I have to do some more thinking on this.
Do note that if we go for the only podcast approach, updates to blogs won't cause a problem. The first time a request comes in for a file,we can create a torrent for it.
The big bad server
The question that most people have with our design is this - how do you scale such an app when only one big server in the middle is going to handle all those blogs? The initial idea was to have a bunch of 'known' servers (for e.g, smoke.weblogs.com,smoke.bloglines.com,etc) where the client can pick one randomly from the list. Since the servers don't need to stay in synchronize (that happens automatically), this could possibly work.
If we drop the idea of not making changes to blogging platforms, this could be a lot easier for us as each blog server could act as a smoke server and run a Bittorrent tracker. I suspect that this would happen eventually with blogging platforms anyway though it may not be our Smoke idea that actually causes it.
The Hash
In Bittorrent, a hash is made of the individual pieces by the guy uploading it. In our architecture, since there is no one authoritative uploader, who calculates the hash? If we have the first client do it, we face a problem where the first client could report an incorrect hash, thereby spoiling the download for everyone else. If the Smoke server does it, it would have to download the podcast itself first, which is something we didn't want.
None of these problems are show-stoppers..except the next one.
Not a new idea
Gary Lerhaupt, of Prodigem.com, was kind enough to contact me and offer me an account. The more I read about how his site works(http://www.torrentocracy.com/prodigem/about.php), the more doubts I get on whether I'm just doing something Gary has already figured out. In that case, there really is no point in re-inventing the wheel as Prodigem.com seems to be a pretty good service.