architecture - .NET processing queue communication between master and worker servers -

-

i'm trying figure out best way create master - worker architecture simple job delegation.

1 master process delegates jobs several worker processes. - master needs continually run , delegate jobs workers (and other tasks). - workers (on different servers) need receive job, process , report back. - master process receive queue of jobs , delegate them worker nodes process request , notify master job has been processed. master not need wait workers complete can delegate job , receive update worker when has completed.

what best way facilitate communication in .net? have class libraries job processing looking communication method.

msmq? windows service? remoting?

thanks

for purpose, used wcf net.tcp bindings, callback interface let master control program know job done (yep, called "the mcp", processes initiated jobs called "sark", , network referred "game grid", go figure).

"sark" implemented both console app , windows service (for ease of development , "dip our toes" new worker machine), while mcp long-running gui. if re-implement it, i'd make master control windows service well, advantageous department see jobs scheduled , pause mcp when backup job or other maintenance task due. today, i'd still make mcp service , provide gui "remote control" it.

the jobs written .net dll assemblies interface invoking task. "sark" copy latest version of binaries on fileshare, create new appdomain, load , run job within appdomain, shut down when done. made possible update jobs without having restart either mcp or "sarks".

in addition, each "sark" instance used msmq short-lived messages (a timeout of 10 seconds) report load on each worker machine. mcp use weighted random pick chose worker machine dispatch job to. is: if machine reported 80% idle, 80 'votes' take next scheduled task, meant more machine 10% idle. reasonably effective way of distributing load evenly while avoiding hotspots.

i chose net.tcp wcf binding dispatch jobs , receive results because "fire , forget" did not work in practice. failure happens: there out-of-memory exceptions, problems runtime on server (a major 1 when task needed work foxpro tables, , odbc foxpro driver microsoft didn't come in 64-bit version--a problem when had mix of 32-bit , 64-bit worker machines), or because of hundred unexpected contingencies such when worker on ip address firewalled machine wanted talk to. immediate response on callback gave mcp option re-assign job worker on different machine.

msmq, however, perfect reporting health of worker machines, because mcp didn't have running in order workers report burden, , short timeout on messages meant mcp wouldn't receive information stale.


Comments

Post a Comment

Popular posts from this blog

linux - Mailx and Gmail nss config dir -

-
i trying send mail linux command line using mailx command. can send local domain no problem want set mail send gmail account receive mail sent gmail account. after configuring mail.rc so: account gmail { set smtp=smtps://smtp.gmail.com:587 set smtp-auth=login set smtp-auth-user=username@gmail.com set smtp-auth-password=password set ssl-verify=ignore } i error: resolving host smtp.gmail.com . . . done. connecting 74.125.25.109 . . . connected. missing "nss-config-dir" variable. "/home/username/dead.letter" 11/354 . . . message not sent. after looking "nss-config-dir" here , located certn.db , keyn.db files , added mail.rc so: account gmail { set smtp=smtps://smtp.gmail.com:587 set smtp-auth=login set smtp-auth-user=username@gmail.com set smtp-auth-password=password set ssl-verify=ignore set nss-config-dir=/home/user/.mozilla/firefox/location.default } now when try send mail using command: echo "sent gmail account" | mailx -v -a
Read more

c# - Is it possible to remove an existing registration from Autofac container builder? -

-
something along lines: builder.registertype<mytype>().as<itype>(); builder.registertype<mytype2>().as<itype>(); builder.deregistertype<mytype>().as<itype>() var container = builder.build(); var types = container.resolve<ienumerable<itype>>(); assert.istrue(types.count == 1); assert.istrue(types[0].gettype == typeof(mytype2)); scenario: go through bunch of assemblies , go register types want make sure have 1 implementation of given type. need before create container. track on own nice if autofac me bit. this cannot done directly using containerbuilder , unless start on new one. mind you, having first built container should able construct new container filtering away unwanted types , reusing registrations first container. this: ... var container = builder.build(); builder = new containerbuilder(); var components = container.componentregistry.registrations .where(cr => cr.activator.limittype != typ
Read more

php - Mysql PK and FK char(36) vs int(10) -

-
i have read mysql can handle indexes , searches better if tables keys integers rather char(36) uuid's. is worth converting int(10)? note: use 1 server , have have plans use multiple databases concurrently therefore removing 1 of big reasons using uuid. our tables innodb if makes difference. thanks in advance. usually rdbms can handle integer keys pk more efficient, other datatypes. reason how build index column, yes: long dont require string (or other typed) keys, should use integers. however: char(36) , int(10) far away being equal, because int(10) much smaller, char(36). dont know, if require many different keys, should keep in mind. update, complete last paragraph: int(10) 32-bit, char(36) 36 - ^byte^ (= 288-bit). not mean, int consumes less space, means, char(36) has 4 times more different keys.
Read more