ruby - Querying delayed_jobs in Mongo blocks while map reduce is running -


i'm using ruby's delayed_job_mongoid gem queue map/reduce jobs in background. jobs process fine, completing in minute.

now i'm trying query status of jobs while processing, i'm finding query perform on delayed_jobs table while map/reduce running hangs there, blocking until of jobs have finished.

so example, if execute db.delayed_jobs.find() while map/reduce running, sits there until every last job finished, displays contents of table (which empty point). it's if entire table getting locked while jobs running. isn't expect.

i've checked , i'm not running out of database connections. know what's going on?

this limitation of how mongodb's map reduce works @ moment. time being, mongodb still has db level locking , map reduce jobs in particular take out both read , write lock unless job explicitly non-atomic.

you can read more how make mapreduce command non-atomic here (caveat: available in mongodb v2.2+).

if nonatomic true, post-processing step prevent mongodb locking database; however, other clients able read intermediate states of output collection. otherwise map reduce operation must lock database during post-processing.

mongodb's aggregation framework bit more performant map reduce. however, new , can't quite same things yet. if can accomplish same task via aggregation framework i'd recommend using instead.

for reference, here's list of operation types , kinds of locks each operation requires:

http://docs.mongodb.org/manual/faq/concurrency/#which-operations-lock-the-database


Comments

Post a Comment

Popular posts from this blog

basic authentication with http post params android -

i doing basic authentication passing username , password , using basicnamevaluepair sending post params response service. my method: public stringbuilder callservicehttppost(string username, string password, string type) { // create new httpclient , post header httpclient httpclient = new defaulthttpclient(); httppost httppost = new httppost(webservice + type); httpresponse response = null; stringbuilder total = new stringbuilder(); try { url url = new url(webservice + type); /*string base64encodedcredentials = base64.encodetostring((username + ":" + password).getbytes(), base64.url_safe | base64.no_wrap);*/ string base64encodedcredentials = "basic " + base64.encodetostring( (username + ":" + password).getbytes(), base64.no_wrap); httppost.setheader("auth...
Read more

vb.net - Virtual Keyboard commands -

i wondering difference between virtual these keyboard commands is: keyeventf_extendedkey , keyeventf_keyup is. everywhere have looked gives me description based off integers , not want know each of them does. you've tagged question vb.net, these have nothing @ vb.net. they're constants defined in windows header files, use win32 api functions. as far difference, can't tell looking @ values. individual values not particularly important, that's why named identifiers used. what's important used , documentation functions tells mean. the first one, keyeventf_extendedkey , used keybdinput structure (which used along e.g. sendinput function) pass information synthesized keyboard input. if flag used, means scan code should interpreted extended key. technically, means scan code preceded prefix byte value 224 (&he0 in hexadecimal notation). the second one, keyeventf_keyup , 1 of flags available use structure. means key being released (going up)...
Read more

How to get multiresult with multicondition in Sql Server -

i need solve this. hopefully can giving me advices. for sample, i've got data : proclib.march 1 first 10 rows flight date depart orig dest miles boarded capacity ----------------------------------------------------------------- 114 01mar94 7:10 lga lax 2475 172 210 202 01mar94 10:43 lga ord 740 151 210 219 01mar94 9:31 lga lon 3442 198 250 622 01mar94 12:19 lga fra 3857 207 250 132 01mar94 15:35 lga yyz 366 115 178 271 01mar94 13:17 lga par 3635 138 250 302 01mar94 20:22 lga 229 105 180 114 02mar94 7:10 lga lax 2475 119 210 202 02mar94 10:43 lga ord 740 120 210 219 02mar94 9:31 lga lon 344...
Read more