Onyx Cube: My Eventual RSS Reader

Originally published on: 9/16/2005 7:08:05 AM

I'm working on an RSS aggregator that actually aggregates instead of just presenting a directory of feeds and items. Almost every aggregator (all of the ones I've looked at) is making the same hierarchical presentation of feeds that's failed to scale over and over and over again. I'm working on a longer essay that talks about the change that needs to happen to move from "directories" of information to "virtual views". My intent is to build a series of back-end scripts that take inputs like RSS URL's, text content, search terms, blog posts, etc. and do a series of lookups to find keywords, classify my expected interest, look up related products, etc and spit out an aggregated RSS feed. That resulting feed is the only one that the virtual views will be presenting. As each one of these chains of processing will essentially be a black box with an input and an RSS output, I'm naming the whole thing Onyx Cube.

This is pretty much entirely as an itch-driven thing. I'm building it a piece at a time as I need it (meaning I'm squeezing it in between the important stuff because I'm quickly realizing how much I need and want this to solve my own problems and time constraints). At the same time, I'm thinking about the bigger picture and how this fits in with solving the problem of information overload and moving from a least common denominator system of information distribution to a "first choice" model where you get EXACTLY what you are looking for and still have the opportunity to find things you might not know you wanted. To me, that's the promise of Web 2.0.

The first piece is posted in another posting today: sending an RSS feed to an email address. This will actually be the last step in presenting the first virtual view for Onyx Cube. The email account will contain a filtered and manipulated view of the original, raw set of feeds in email form. Other virtual views can easily be built as well: top 10 items for the day on a web page, related feeds not currently subscribed to on a web page, a single email with headlines, etc.. My intent is to use Thunderbird and an IMAP server as my RSS view and my current OPML file as input into the Onyx Cube. What I want on the other end is the stuff I'm actually interested in.

As of this morning, I have 190 feeds in my roster and it seems to be growing quickly. Right now there are 272 items unread and I know from the past few weeks that at least 70% of them will be kicked out without me reading them. Another 20% won't be finished once I start reading them. My goal for Onyx Cubes are that they can help me find the 10%, push the 20% to the bottom and just get rid of the 70%.

Future inputs could be sending email right back through to a web RSS feed, a personal search engine that does a more in depth look at a given search phrase and presents a single report about it, a news report on a given topic, etc.

We'll see if I'm just breathing paint fumes here as things go forward.

Comments

LetterJ
commented on 9/16/2005 I just re-read this and realized that I didn't stress one point as much as I really believe in it. I believe that RSS aggregators that try to focus on building applications instead of just outputting RSS itself are making a mistake. To me, the output of RSS aggregation should be RSS and *not* another complicated application. After the RSS is output, if you still want more complicated interfaces and interactions, you can build a consuming layer to do just that. You can send input back to the configuration portion of your aggregator, etc. but the aggregator itself should just output RSS. Anything else is *not* loosely coupled.

Why I’m Loving IMAP and Thunderbird as my RSS-- The Glass is Too Big - J Wynia
commented on 10/4/2005 [...] A recent posting from the SearchFox folks talked about 2 models for RSS reading. However, the one I’ve been using: IMAP and Thunderbird is proving to be my favorite and a 3rd option that makes the best of the current state of the tools. I’ve written about the bits that contribute to this current system here, here and here. [...]