My name is Brian Whitman. I am a lapsed scientist and sound artist currently co-founder/CTO at The Echo Nest, a music intelligence company in Somerville, MA. As I work on various scaling and media search problems with detours into art projects I'll be posting details here in the hopes that I can learn from others. I'd always like to hear from you if you are working on similar things.
LEXATEXY was away for awhile in other countries much of January, but constantly furrowed his brow at a classic but old-new-again problem with data and Retrieving It. How many times has this happened to you:
CUSTOMER: Here’s some data, take it, immediately!
LEXATEXY: Thank you for that data.
CUSTOMER (Within 80ms): Where is my data?
LEXATEXY: Oh, I have it, but i need to do Things with it until you can see it. You know, the things you pay me to do.
CUSTOMER: I don’t understand. I gave you something, now I want to see it. Immediately. I also want to see it in relation to my older data.
That’s a bit much, especially when CUSTOMER and LEXATEXY are the same person (real problems on the fabric farm some days…) But it’s a common issue, right? Indexing takes time. A computer program has to go through the data you injected and figure out where to put it. When you add lots of data this may take a lot longer than your average web page refresh. Especially if lots of people are doing it too.
LEXATEXY has noticed a few proposed solutions to this over the years:
What do you do?
