Wednesday, December 29, 2010

Debugging MapReduce in MongoDB

On a project that I am working on, we are doing some pretty intense MapReduce work inside of MongoDB. One of the things we've run up against is the lack of solid debugging tools. Some Googling basically tells you that print() is all you've got.

We've decided to take a different approach and debug our MapReduce code in the browser. Since the code is JavaScript and modern browsers have really excellent support for debugging (breakpoints, variable inspection, etc.) it's pretty easy to do.

All you need is a web app (or even static HTML file) that will:

  1. Load up one of your documents that you would like to map in the browser. Since the documents are JSON, this is easy. In our project, we have JSON fixture files and a small web app that allows you to choose which fixture to use for testing.
  2. Mock the emit() method. You can just have it write to a Hash that you can inspect later.
  3. Load up the Map and Reduce functions. If you keep these in separate .js files, you can pull them in with a simple script tag.
  4. Bind the map function to the document so that it has the correct context. In a MongoDB mapper function "this" is set to the document that you are mapping. You can easily do this with the bind() function in Underscore.js. I'm sure that other JavaScript frameworks provide a similar function.
  5. Put a link on the page that will let you run the bound function.
This will emulate the MongoDB MapReduce environment, but you can now use the browser's debugging tools.


Maisam said...

If you want to make you free flash website I think you need free flash templates .
Free flash intos is of the best free flash effects that you need.

Angelo said...

Hi Andy,

Many thank's for your great idea and article.
I have decided to implement your idea in the Mongo MapReduce WebBrowser project.

You can play with it with the live demo.

Hope you will like it.

Regards Angelo

Harding Jamie said...

To tell the certainty this is a kind post concerning this topic. I got an unpolluted solution from here. Also I can refer to you all here to get educational helps. However will wait for more posts from the blogger. Thanks a lot…….

Mari said...

Interesting information and cute writing style.This is a cool read ..

Алексей Бондарчук said...

Great post, I enjoyed ready reading it, Keep posting good stuff like this.

Fred Duffler said...

Such interesting theme. I like it.

steve7876 said...

Very Interesting Nichols story Thanks for nice done :)

yakenzu toby said...

Thanks for the amazing content on your blog I am very interested in this article and you have really helped me. I have just told a few of my friends about this on FaceBook and they love your content just as much as I do. ibcbet, judi bola, sbobet, agen bola, ibcbet, sbobet

Sohidul Islam said...

If you'd want to use on-line associated speed up the tactic even faster you will be able to fill out AN application presently and easily hit “send” auto title loans chicago. one among our enticing loan specialists will call you back shortly and assist you finalize your loan. As shortly as you pay off your automotive title loan, we've an inclination to send your title right back to you. start.

Sohidul Islam said...

This is the foremost bit I even have glimpsed you’re cozy and do favour to broadcast you – it's actually fastidious to glimpse that i absorb your powerful paid work. whereas if you anticipated did it terribly} clear-cut kind which will be terribly persuading fast auto title loan. whereas over all I unquestionably planned you and clear in your brain can gaze ahead to extra mails like this. Thanks most.

Ken Michale said...

Nice post and informative one.Awesome ideas are post. Friv | Friv 1 | Friv 3

Vladimir Petriko said...

I've implemented a simple opensource online tool based on this idea. You can get a final MapReduce results or go and debug Map / Reduce / Finalize functions step-by-step right in your browser.

Will appreciate a comments / critic / requests.