Wednesday, December 29, 2010

Debugging MapReduce in MongoDB

On a project that I am working on, we are doing some pretty intense MapReduce work inside of MongoDB. One of the things we've run up against is the lack of solid debugging tools. Some Googling basically tells you that print() is all you've got.

We've decided to take a different approach and debug our MapReduce code in the browser. Since the code is JavaScript and modern browsers have really excellent support for debugging (breakpoints, variable inspection, etc.) it's pretty easy to do.

All you need is a web app (or even static HTML file) that will:

  1. Load up one of your documents that you would like to map in the browser. Since the documents are JSON, this is easy. In our project, we have JSON fixture files and a small web app that allows you to choose which fixture to use for testing.
  2. Mock the emit() method. You can just have it write to a Hash that you can inspect later.
  3. Load up the Map and Reduce functions. If you keep these in separate .js files, you can pull them in with a simple script tag.
  4. Bind the map function to the document so that it has the correct context. In a MongoDB mapper function "this" is set to the document that you are mapping. You can easily do this with the bind() function in Underscore.js. I'm sure that other JavaScript frameworks provide a similar function.
  5. Put a link on the page that will let you run the bound function.
This will emulate the MongoDB MapReduce environment, but you can now use the browser's debugging tools.

22 comments:

Maisam said...

If you want to make you free flash website I think you need free flash templates .
Free flash intos is of the best free flash effects that you need.

Angelo said...

Hi Andy,

Many thank's for your great idea and article.
I have decided to implement your idea in the Mongo MapReduce WebBrowser project.

You can play with it with the live demo.

Hope you will like it.

Regards Angelo

Harding Jamie said...

To tell the certainty this is a kind post concerning this topic. I got an unpolluted solution from here. Also I can refer to you all www.hwsolvers.com/programming_project here to get educational helps. However will wait for more posts from the blogger. Thanks a lot…….

Mari said...

look.....http://www.promptessay.com/
Interesting information and cute writing style.This is a cool read ..

Алексей Бондарчук said...

Great post, www.masterpapers.com I enjoyed ready reading it, Keep posting good stuff like this.

Fred Duffler said...

Such interesting theme. http://www.online-essay-writer.org/ I like it.

steve7876 said...

Very Interesting Nichols story Thanks for nice done :)

yakenzu toby said...

Thanks for the amazing content on your blog I am very interested in this article and you have really helped me. I have just told a few of my friends about this on FaceBook and they love your content just as much as I do. ibcbet, judi bola, sbobet, agen bola, ibcbet, sbobet

Sohidul Islam said...

If you'd want to use on-line associated speed up the tactic even faster you will be able to fill out AN application presently and easily hit “send” auto title loans chicago. one among our enticing loan specialists will call you back shortly and assist you finalize your loan. As shortly as you pay off your automotive title loan, we've an inclination to send your title right back to you. start.

Sohidul Islam said...


This is the foremost bit I even have glimpsed you’re cozy and do favour to broadcast you – it's actually fastidious to glimpse that i absorb your powerful paid work. whereas if you anticipated did it terribly} clear-cut kind which will be terribly persuading fast auto title loan. whereas over all I unquestionably planned you and clear in your brain can gaze ahead to extra mails like this. Thanks most.

Ken Michale said...

Nice post and informative one.Awesome ideas are post. Friv | Friv 1 | Friv 3

Vladimir Petriko said...

I've implemented a simple opensource online tool based on this idea. You can get a final MapReduce results or go and debug Map / Reduce / Finalize functions step-by-step right in your browser.

Will appreciate a comments / critic / requests.

http://targetprocess.github.io/mongo-mapreduce-debug-online/

Sang Pemikir said...

I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful. Agen Bola

Zacky van Hardi said...

A very awesome blog post. We are really grateful for your blog post. You will find a lot of approaches after visiting your post. I was exactly searching for. Thanks for such post and please keep it up. Great work. Agen Sbobet

Ethan Harry said...

Really important written content. the information that you shown is hard to faith and many superbly i liked the way you afford things here

Accredited GED Online

Lan Linh said...

Great! Thanks for sharing the information. That is very helpful for increasing my knowledge in this field.
pacman |happy wheels |my little pony games | unblocked games
friv4 kizi 4| unblockedgames unblocked games for kids

marko said...

We have to acquire money from any accessible source. Individual money credit is such a source which helps in giving money to these needs. https://www.aaa1autotitleloans.com/chicago

Facebook Descargar said...

What a great online source of information about this topic. you have done great work. keep continue to sharing such kinds of post. keep it up.
Signature:
download descargar facebook gratis para Android celular and download free descargar facebook apk and descargar facebook gratis , descarga facebook

Whatsapp Descargar said...

I understand what you bring it very meaningful and useful, thanks.
Signature:
download free descargar whatsapp and download baixar whatsapp online and descargar whatsapp gratis , baixar whatsapp gratis

Frozen Juegos said...

Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing.
Signature:
Jugar juegos de frozen en línea gratis, los nuevos de princesa de Disney juegos frozen - la princesa encantadora y linda. Divertirse frozen!

marko said...

The organization you choose to manage for the least expensive auto advance rates ought to have an advance frame that is easy to round out, with a without toll telephone number promptly available on the off chance that you have any extra inquiries. usacheckcashingstore.com/costa-mesa

marko said...

Their issues in taking a credit for auto raise additionally on the grounds that they don't claim a property to take an advance against. Moneylenders have in this way composed new auto credits for inhabitant. The advance is given in an obstacle free way to occupants or non-property holders. check cashing nearer in fresno ca