April 2010
1 post
The Echo Nest Musical Fingerprint (ENMFP)
Tomorrow begins MHD Amsterdam and at it The Echo Nest is releasing a few new things. Some of our engineering team (who deserve a severe callout for all their work, let me stick with their codenames) have been working tirelessly to get “songs” to be a first-class member of our API, and as of today, they are — we now track many millions of songs and you can query for them by name...
February 2010
1 post
January 2010
2 posts
Primer on new Echo Nest search_tracks, capsule,...
Note: if you are interested in these APIs know that they are no longer alpha — see our new Song APIs for the latest information.
At Stockholm Hack Day we’re announcing three or four new APIs that are going to stay in our “alpha” sandbox for now. These are officially unsupported but we will work with anyone who has a use case for them. For now, the instructions will stay...
Moonbelly
Geocoded place names mentioned in the entirety of “40 Stories”, via Google Books, some sort of future library that no one talks about as we are too scared of it:
Moonbelly sang a new song called “The System Cannot Withstand Close Scrutiny.”
The system cannot withstand close scrutiny
The system cannot withstand close scrutiny
The system cannot...
December 2009
4 posts
WatchKFW’s excellent “Modena” video.
The Echo Nest Remix API talk at Dorkbot NYC, March...
About nine months ago I gave a funny talk with my main man Douglas at his world-dominating Dorkbot NYC about Echo Nest Remix. Slideshare’s not so good with the audio part of things so it may be kinda dry, but here it is:
The Echo Nest Remix at Dorkbot NYC, March 4 2009
I remembered to post it because Paul made something hilarious with it today in just a few lines of code. I...
November 2009
5 posts
A Singular Christmas (2004)
Five years ago today I released “A Singular Christmas,” the rendered output of a piece of software that listened to hundreds of Christmas songs and tried to compose its own new holiday standards. It ended up as my most successful thing ever by a few orders of magnitude: 600,000 people downloaded it over the space of three weeks. I was on BBC Radio on Christmas Eve; Wired Magazine...
Music Hack Day Boston
Last weekend I was happy to be a part of Music Hack Day Boston. The Echo Nest co-organized it and sponsored it; I sat on a panel, helped with some “local coordination,” put up some awesome people at my place nearby, and generally did everything but make a hack myself. (I tried, really.)
Below is an unordered list of “things I learned” last weekend.
Elissa is a...
“I’m not interested in writing short stories. Anything that doesn’t...”
– my man cormac in this excellent interview
(although i would assert that writing short stories can also take years of your life / drive you insane)
KFW @ On Land, Sep 19 2009
Live Generator (1) @ On Land by kfw
October 2009
4 posts
Porting over the archive...
I had a lot of posts sitting in the old Wordpressy blog that is due for a huge SQL injection attack some day soon, so I thought I’d drag the kittens out of the burning house while I could. I backdated them so they are at the end, but here’s some permalinks if you want to read them:
The Rainer Buchty Ratio [oft-referenced mathematical model]
Pushpin is Real [true tales of...
Highlights of Last Week At Music and Bits
I had the pleasure of attending and speaking at the Music and Bits conference in Amsterdam last week that kicked off the Amsterdam Dance Event.
The talk
The morning of my speaking slot, I lost my voice. Those reading this know I don’t speak much to begin with (unless severely inspired.) After practicing saying “eigenvector” five times in a row fast and failing (I’ve...
Music and Bits
I’ll be at the Music and Bits conference in Amsterdam next week that kicks off the excellent Amsterdam Dance Event. There’s talks all day on the 21st and also a hack day that EN is supporting. I’ll be sharing the stage with some serious players: Soundcloud, Justin from Muxtape, RjDj, Last.fm, VOLODKIN …
I’m most excited that STEIM will be there during the hack...
August 2009
3 posts
Install python MySQL (MySQLdb) in Mac OS X Snow...
I hate MySQL but I have some … employees … who think it’s a perfectly fine solution when you don’t want to boot a Solr server. Savages. Anyway, of course I expected the MySQLdb connector in Python to totally break and it did not disappoint. You’re dealing with a 64-bit version of Python now and a computer that gets confused as to its architecture (why does...
The SmartQ
Leonard sent over a must-have inclusion to my Small but powerful devices that run your code roundup, the SmartQ 5 (dealextreme link, please heed my warning about at least 30 day ship times.)
I’ll fill in the chart later this week. Still not sure if I’ll get one on order as nothing I have in my project queue needs a touchscreen. But the specs/price coefficient looks amazing:...
July 2009
9 posts
Chiplotle! →
Chiplotle has been released. I have never thought I would have lived to see this day. Not sure I like my gallery representation. But congrats to Victor and Douglas!
Small but powerful devices that run your code
Sometimes I work on art projects that require a little computer inside to do stuff — audio synthesis or analysis, video projection, motor control or sensors. Recently I’ve been a bit unnervingly interested in this new mostly ARM-based category of “small but powerful devices that will run your code.” While my peers are happy using Arduino or otherwise Atmel-type 8 or 16-bit...
FIF Capacity Statistics For April 2009 →
While I may be excited about my 2000 messages per second, “NYSE ArcaBook Options” is rocking 565,522 mps. Reminder to self: what you do is not that hard.
Broadcasting your logs with RabbitMQ and Python
Update — thanks to Matthias at LShift I’ve updated this code so it is much better and runs without crashing :)
You’ve got sixteen computers running all spitting out various messages. Some of them are useful, some you should ignore, and then there’s errors— you need to know when they happen. But what do you do? Spawn sixteen ssh machine tail -f /var/logs? Crazy time....
WatchDJ API totally destroying it via EN Remix.
The disk backed rabbitmq queue and you
RabbitMQ is still doing great but had one very bad flaw: more than 1.5m un-eaten messages on a single “small” type server (1-2GB RAM) would just take the whole thing down. And the current solution involves giving your machine more swap. When you’re not in a real time scenario (worker messaging on “fake M/R tasks” is ours), you want your messages to stick around and...
EBS won’t mount, bad superblock →
Why you can’t mount an EBS snapshot alongside its original on AWS. Fix: add nouuid to the mount options.
June 2009
1 post
May 2009
3 posts
A list of things that are about parallel matrix...
I want a thing that stores sparse matrices over N computers and I can do math on those matrices.
My Dream API:
matrix = new(matrix_name, cols, rows)
matrix.put(col, row, value) # (or matrix.put_col(row, data) , matrix.put_row(col, data) )
value = matrix.get(col, row)
new_matrix = matrix.multiply(matrix_B)
matrix.transpose()
new_matrix = matrix.invert(iterations=0)
[U, S, V] =...
MPI in python on EC2 talk →
About to tread back into these waters. It’s been 4 years since I last ran anything in MPI, LAM or SCALAPACK. A bit nervous.
Why is NLTK so slow people
AMZN small instance (snail style)
### Took 92.35s to parse 10005 words 351 sentences (76.64% passed.) 0.26s per sentence.
Mac Pro
### Took 26.47s to parse 10005 words 351 sentences (76.64% passed.) 0.08s per sentence.
Where “parse” is pos_tag, an NP chunker (RegexpParser w/ our own grammar.) Most of the work is in pos_tag.
April 2009
2 posts
Some notes on distributed key value stores →
Leonard lays down the law on what to use in April 2009.
Don't use S3 as a key-value store.
Really, don’t. I know it sounds appealing.
Much love, Brian
February 2009
4 posts
Make and attach an EBS volume on EC2 instance boot
This is all you have to do…
create_mount_ebs() {
# Make and mount an EBS. Note: your instance has to be on us-east-1a.
SIZE="$1" # in GB
log "creating and mounting a $1 GB EBS to /vol"
VOL=`ec2-create-volume -z us-east-1a -s $SIZE | awk '{print $2}'`
# Check for it on a loop..
AVAILABLE="unavailable"
while [ "$AVAILABLE" != "available" ]
do
AVAILABLE=`ec2-describe-volumes $VOL |...
Response to: "Among the stupidest tech articles... →
mrgan:
I’m sorry, did you just defend the use of a web app instead of a desktop app by pointing out a desktop app you use to run your web app?
I sure did. Why does it matter what layout engine is rendering your list view? Does [NSClunkyAssSortInboxBySenderWithinMinutes:10 crashProbability:0.7] imbue some magical properties I wasn’t aware of? Can you tell by the pixels? (PS: open bet:...
The Read Write Problem
LEXATEXY was away for awhile in other countries much of January, but constantly furrowed his brow at a classic but old-new-again problem with data and Retrieving It. How many times has this happened to you:
CUSTOMER: Here’s some data, take it, immediately!
LEXATEXY: Thank you for that data.
CUSTOMER (Within 80ms): Where is my data?
LEXATEXY: Oh, I have it, but i need to do Things with...
January 2009
12 posts
Response to: "Among the stupidest tech articles... →
mrgan:
As of this week, Gmail has reached perfection: You no longer have to be online to read or write messages.
Like every other desktop email client ever, that is.
If you’re still tied to a desktop app—whether Outlook, the Mac’s Mail program, or anything else that sees your local hard drive, rather than a Web server, as its brain—then you’re doing it wrong.
Well, I’m convinced. I guess I’ll...
Consistent Hashing and Data Servers
It’s time for LEXATEXY to boot their N solr, sql and BDB servers. N is such an alluring number, isn’t it? You can tell someone that if N isn’t enough, we can add one to it. It’s still N! But when you’ve got a bunch of data sitting on A, B, and C, and then you add a D, what do you do? You’d have to “re-balance,” which sounds like something my car...
Bulk Upload to Amazon SimpleDB →
So… you still can’t actually do this.
John Resig - OCR and Neural Nets in JavaScript →
Awesome. (Is CBLAS for javascript?)
Ascription is an Anathema to any Enthusiasm ›... →
APIs, (specifically ours), what are they good for?
Things that have happened to me on Amazon's EC2
(Dark Cloud II by Daniel Staver)
Love hate relationship with the Amazon EC2 service blossoms deep in LEXATEXY’s veins. Are these problems endemic of anyone trying to do things on more than a few computers? We’re at a point now where people will slowly forget what it’s like to have to call a guy to pull the power on your computer. Remote hands! Paying a $200 setup fee to...
Distributing Solr Nicely
Sometime in early 08 it became clear to LEXATEXY that our biggest client’s solr servers had too many documents on them. [Rule of thumb for “how many documents is too many?”: 15 million. I can upgrade, you bargain. What if I have a super-machine bought with my recently closed VC round?
15 million.]
You’ve already got a pretty reliable thing handling 15 million...
Solr Tips Of The Day
(Part 1 of many of my Fake Solr Consultancy Service, I plan to call it Lexatexy or BARISMO) oh and I will only consult to coffee companies or yacht club membership sites. In my dream world LEXATEXY does not do actual work, he says Hm a lot and is not under any deadlines or commitments. He is the text retrieval kin of Steve Miller’s titular “Joker”
YOUR COMMIT RULE OF THUMB
...
Profiling Java methods with the heap profiling... →
When something in java takes a long time, you can do jmap -histo pid to see where the memory is, kill -QUIT pid to see what it is currently doing, and this to see the % of where all the calls are. (I am currently debugging Solr commit slowness on 2 of our index shards, they are 20x as slow to commit as the others and I don’t know why. More later.)
APPROXIMATION OF A CUBIC BEZIER CURVE BY CIRCULAR ... →
The RBR of this paper is through the roof. Anyone got an HPGL/2 to HPGL converter that does something like this?