MT tasks count only one thread time

log in

Advanced search

Message boards : Number crunching : MT tasks count only one thread time

1 · 2 · Next
Author Message
zombie67 [MM]
           
Avatar
Send message
Joined: 30 Mar 10
Posts: 163
Credit: 7,440,049
RAC: 1,240
Total hours: 24,566,636
Message 1042 - Posted: 12 Mar 2013, 4:24:43 UTC
Last modified: 12 Mar 2013, 4:25:19 UTC

This was bought up in another thread. But since we haven't heard anything back from the admin, I am hoping a new, separate thread would get better results.

MT (multi-threaded) tasks are becoming more common. They are currently being sent by yafu, renderfarm, BURP and T4T. Also, I have heard that MW is working on one. Currently, the WUprop app counts either the task wall time, or the CPU time of only a single thread per task. Not sure which. But it needs to be counting the MT apps as "time x thread count". Other wise, the more threads are used, the less WUPROP badge credit you receive. It is an anti-incentive as it is today, to run MT tasks.

Is anything being done, or planned to be done, to address this issue?
____________
Reno, NV
Team: SETI.USA

Profile STE\/E
           
Avatar
Send message
Joined: 28 Mar 10
Posts: 488
Credit: 3,503,378
RAC: 223
Total hours: 17,998,535
Message 1043 - Posted: 12 Mar 2013, 11:14:40 UTC

It is an anti-incentive as it is today, to run MT tasks


Right Z, so far I haven't run any of the MT Projects & I won't as long as they only count 1 Tread when your using 2-12 Treads depending on the CPU ...
____________

zombie67 [MM]
           
Avatar
Send message
Joined: 30 Mar 10
Posts: 163
Credit: 7,440,049
RAC: 1,240
Total hours: 24,566,636
Message 1078 - Posted: 20 Mar 2013, 4:25:53 UTC

What does it take to get the admin to respond to problems? Seriously. I see he responds to some problem threads, but not others.

This is a simple problem statement: The hours for MT tasks hours are counted as if they used only a single thread (run time), when they should be counted by total thread time (CPU time).
Here is a simple example from yafu, with an 8 thread machine: http://yafu.dyndns.org/yafu/result.php?resultid=1003666

Run time 174.58
CPU time 873.23 <-- This is the field that should be counted for MT tasks. Not Run time.

Is the admin aware of the problem? And if so, what is his plan to address? Fix the app now, or in a week, or in a year? Even if the plan is "do nothing", we should to be told. Please, just let us know something. Thanks!
____________
Reno, NV
Team: SETI.USA

Profile Neil Polson
   
Avatar
Send message
Joined: 20 Apr 10
Posts: 20
Credit: 81,989
RAC: 0
Total hours: 33,181
Message 1105 - Posted: 26 Mar 2013, 17:48:57 UTC

You can add Edges@Home Autodock VINA app to the list too.
____________

Profile STE\/E
           
Avatar
Send message
Joined: 28 Mar 10
Posts: 488
Credit: 3,503,378
RAC: 223
Total hours: 17,998,535
Message 1106 - Posted: 26 Mar 2013, 19:46:47 UTC - in response to Message 1105.

You can add Edges@Home Autodock VINA app to the list too.


??? The Edges VINA only use 1 Thread Per CPU & isn't a MT Wu ...

____________

Profile Neil Polson
   
Avatar
Send message
Joined: 20 Apr 10
Posts: 20
Credit: 81,989
RAC: 0
Total hours: 33,181
Message 1109 - Posted: 27 Mar 2013, 7:26:36 UTC - in response to Message 1106.

You can add Edges@Home Autodock VINA app to the list too.


??? The Edges VINA only use 1 Thread Per CPU & isn't a MT Wu ...

That's what i thought until I ran it on my Winx64 dual core host. It immediately used both cores and the run time was 2x cpu time! Maybe it's a bug? I will post on their forum.
____________

Profile STE\/E
           
Avatar
Send message
Joined: 28 Mar 10
Posts: 488
Credit: 3,503,378
RAC: 223
Total hours: 17,998,535
Message 1110 - Posted: 27 Mar 2013, 8:01:47 UTC
Last modified: 27 Mar 2013, 8:10:33 UTC

I only run Edges on my Linux Box's, the Wu's only take 1 Thread Per Wu on Linux or at least my Linux Box's anyway, will try some on a Windows Box & see what happens ...

EDIT: Looks like you were right, the Edges VINA Wu was taking between 2 & 3 Threads to run, it kept jumping back & forth between 2 or 3 Threads, I aborted it, I won't run the MT Wu' s as long as we're only getting Credit for 1 Thread here at WUProp ... :(
____________

zombie67 [MM]
           
Avatar
Send message
Joined: 30 Mar 10
Posts: 163
Credit: 7,440,049
RAC: 1,240
Total hours: 24,566,636
Message 1115 - Posted: 28 Mar 2013, 3:11:47 UTC - in response to Message 1110.

I only run Edges on my Linux Box's, the Wu's only take 1 Thread Per Wu on Linux or at least my Linux Box's anyway, will try some on a Windows Box & see what happens ...

EDIT: Looks like you were right, the Edges VINA Wu was taking between 2 & 3 Threads to run, it kept jumping back & forth between 2 or 3 Threads, I aborted it, I won't run the MT Wu' s as long as we're only getting Credit for 1 Thread here at WUProp ... :(


Yeah, I noticed this about a week ago too. I was looking into why a GPU was underperforming so much. So I restricted a thread....and still at 100% CPU....so a second thread...still 100% CPU. WTF?! After some more digging, I saw vina at like 30% on an 8 thread CPU. But I didn't have time to really look into it at the time, and forgot about it.

Now here are the parts that needs further investigation:

- does the EDGeS@Home project realize it? They really need to identify the app as MT in their app page.
- I see they *also* have an app called "Generic BOINC Application Client" which is a vbox app. And like T4T, it may be MT.
- fightmalaria@home also uses vina. Does that have the same "feature"?
____________
Reno, NV
Team: SETI.USA

Profile skgiven
       
Avatar
Send message
Joined: 7 Sep 10
Posts: 453
Credit: 943,530
RAC: 12
Total hours: 2,098,949
Message 1116 - Posted: 28 Mar 2013, 15:11:06 UTC - in response to Message 1115.
Last modified: 28 Mar 2013, 15:11:46 UTC

I haven't seen a fightmalaria@home tasks use more than one thread. They are generally very short (~100s), so there is no need for MT.

On the Main Edges web page it says "also hosts several other applications at beta (experimental) level". Fine, but we need to know what we are crunching; I don't want my GPU to sit idle because of an erroneous/or unannounced Beta MT CPU app. I also noted that while you can select apps to run, the Edges server_status page doesn't reveal which tasks are available.

Boinc-wide there has been far too much live alpha/Beta testing recently, often ill-thought-out experiments by inexperienced researchers. The ultimate consequence of this is that crunchers are deterred from crunching (all projects).

Profile Steve Hawker*
       
Send message
Joined: 31 Oct 12
Posts: 125
Credit: 655,634
RAC: 0
Total hours: 1,576,278
Message 1154 - Posted: 8 Apr 2013, 12:38:58 UTC - in response to Message 1116.

Bump!

This really needs an answer....

Profile KPX
       
Send message
Joined: 6 Apr 10
Posts: 41
Credit: 749,965
RAC: 260
Total hours: 1,387,655
Message 1179 - Posted: 29 Apr 2013, 14:23:32 UTC

Admin, please, this needs a solution. MT apps can't be counted as if they ran on one thread only.

Profile [AF>WildWildWest]Sebastien
   
Dictator
Avatar
Send message
Joined: 28 Mar 10
Posts: 1806
Credit: 391,419
RAC: 95
Total hours: 816,106
Message 1185 - Posted: 4 May 2013, 19:03:52 UTC

The new version corrects this problem.
____________

zombie67 [MM]
           
Avatar
Send message
Joined: 30 Mar 10
Posts: 163
Credit: 7,440,049
RAC: 1,240
Total hours: 24,566,636
Message 1186 - Posted: 4 May 2013, 21:36:04 UTC

Awesome! Thanks!!!!
____________
Reno, NV
Team: SETI.USA

Profile STE\/E
           
Avatar
Send message
Joined: 28 Mar 10
Posts: 488
Credit: 3,503,378
RAC: 223
Total hours: 17,998,535
Message 1189 - Posted: 5 May 2013, 6:07:00 UTC

I still don't see where it is fixed, I ran 2 Renderfarm WU's on a Quad Core that ran for about 1.5 Hr's & all I have Credit for is 1.42 Hr's even though the Wu's were each using all 4 Core's while they were running ... Was using v3.50 WUProp Wu ...
____________

Profile STE\/E
           
Avatar
Send message
Joined: 28 Mar 10
Posts: 488
Credit: 3,503,378
RAC: 223
Total hours: 17,998,535
Message 1194 - Posted: 5 May 2013, 16:47:17 UTC

Looks like it's working now like it should ...
____________

zombie67 [MM]
           
Avatar
Send message
Joined: 30 Mar 10
Posts: 163
Credit: 7,440,049
RAC: 1,240
Total hours: 24,566,636
Message 1195 - Posted: 6 May 2013, 16:23:06 UTC

Does this fix also apply to MT GPU apps, like at Moo?
____________
Reno, NV
Team: SETI.USA

Profile [AF>WildWildWest]Sebastien
   
Dictator
Avatar
Send message
Joined: 28 Mar 10
Posts: 1806
Credit: 391,419
RAC: 95
Total hours: 816,106
Message 1201 - Posted: 7 May 2013, 10:34:31 UTC - in response to Message 1195.

For the moment, this fix is only for MT CPU apps.
I will search how to fix MT GPU apps next week.
____________

zombie67 [MM]
           
Avatar
Send message
Joined: 30 Mar 10
Posts: 163
Credit: 7,440,049
RAC: 1,240
Total hours: 24,566,636
Message 1512 - Posted: 10 Aug 2013, 2:09:28 UTC
Last modified: 10 Aug 2013, 2:10:50 UTC

Milkyway has an app called "MilkyWay@Home N-Body Simulation". It has a single-thread version and a multi-thread version. Some tasks are issued with ST and others with MT, seems to be random, or maybe based on the size of each task. So a machine is issued tasks, for BOTH kinds of applications with the same name.

http://milkyway.cs.rpi.edu/milkyway/apps.php

Does the wuprop app see the difference for each task, and count hours appropriately?
____________
Reno, NV
Team: SETI.USA

Profile BilBg
Avatar
Send message
Joined: 20 Jun 12
Posts: 63
Credit: 94,685
RAC: 0
Total hours: 108,788
Message 1513 - Posted: 11 Aug 2013, 13:33:00 UTC - in response to Message 1512.
Last modified: 11 Aug 2013, 13:56:02 UTC

So a machine is issued tasks, for BOTH kinds of applications with the same name.

(mind I don't run MilkyWay@Home)

Are you sure they have the same name?

The <user_friendly_name> maybe is different:
MilkyWay@Home N-Body Simulation v1.36
MilkyWay@Home N-Body Simulation v1.36 (mt)
http://milkyway.cs.rpi.edu/milkyway/results.php?hostid=504818&offset=0&show_names=0&state=0&appid=7

Search in your client_state.xml for:
<user_friendly_name>MilkyWay@Home

... and see what is in <name> tag above it.
(e.g. in both cases <name>milkyway_nbody</name> or different)

I think you can help the wuprop admin/developer if you post what you found (i.e. is there difference in <name> for MT vs no-MT app), e.g.:

<app>
<name>milkyway_nbody</name>
<user_friendly_name>MilkyWay@Home nbody Simulation</user_friendly_name>
</app>


You can also search for the .exe filenames (there have to be 2 .exe files in the project directory for MT and no-MT app)

e.g. this is from my client_state.xml if I search for setiathome_7.00_windows_intelx86.exe (the important part is <app_name>)
<app_version> <app_name>setiathome_v7</app_name> <version_num>700</version_num> <platform>windows_intelx86</platform> <avg_ncpus>1.000000</avg_ncpus> <max_ncpus>1.000000</max_ncpus> <flops>481695518.833688</flops> <file_ref> <file_name>setiathome_7.00_windows_intelx86.exe</file_name> <main_program/> </file_ref> .........



If you search for the name of the WU/task file (in my case 26fe09ab.9197.7434.5.12.126.vlar)
you'll also find the <app_name> associated with this task:
<workunit> <name>26fe09ab.9197.7434.5.12.126.vlar</name> <app_name>setiathome_v7</app_name> <version_num>700</version_num> <rsc_fpops_est>183887307748840.000000</rsc_fpops_est> <rsc_fpops_bound>1838873077488400.000000</rsc_fpops_bound> <rsc_memory_bound>33554432.000000</rsc_memory_bound> <rsc_disk_bound>33554432.000000</rsc_disk_bound> <file_ref> <file_name>26fe09ab.9197.7434.5.12.126.vlar</file_name> <open_name>work_unit.sah</open_name> </file_ref> </workunit>



You have to search for names like:
de_nbody_07_23_dark_2_1372784655_882526
ps_nbody_07_30_orphan_real_1372784655_882468

... found in your tasks (remove the last _0 _1 _2 ...):
http://milkyway.cs.rpi.edu/milkyway/results.php?hostid=504818&offset=0&show_names=1&state=0&appid=7


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

zombie67 [MM]
           
Avatar
Send message
Joined: 30 Mar 10
Posts: 163
Credit: 7,440,049
RAC: 1,240
Total hours: 24,566,636
Message 1514 - Posted: 11 Aug 2013, 18:29:22 UTC - in response to Message 1513.

So a machine is issued tasks, for BOTH kinds of applications with the same name.

(mind I don't run MilkyWay@Home)

Are you sure they have the same name?


Yes. Or at least wuprop records them as the same name. It is listed in the account page as only "MilkyWay@Home N-Body Simulation". There is no separate entry for ST vs MT.

You can see the same thing here: http://wuprop.boinc-af.org/active_projects.py

No version number, which is correct, as that is not part of the name*. And apparently, neither are the suffixes such as "(mt)" or "(ncni)".

*It is annoying when projects put the version number in the app name. The name should NOT change when changing the version number. I'm talking to you Mind Modeling!

<app>
<name>milkyway_nbody</name>
<user_friendly_name>MilkyWay@Home N-Body Simulation</user_friendly_name>
<non_cpu_intensive>0</non_cpu_intensive>
</app>

<app_version>
<app_name>milkyway_nbody</app_name>
<version_num>136</version_num>
<platform>x86_64-apple-darwin</platform>
<avg_ncpus>8.000000</avg_ncpus>
<max_ncpus>8.000000</max_ncpus>
<flops>4331244037.649016</flops>
<plan_class>mt</plan_class>
<api_version>6.13.0</api_version>
<cmdline>--nthreads 8</cmdline>
<file_ref>
<file_name>milkyway_nbody_1.36_x86_64-apple-darwin__mt</file_name>
<main_program/>
</file_ref>
</app_version>

<app_version>
<app_name>milkyway_nbody</app_name>
<version_num>136</version_num>
<platform>x86_64-apple-darwin</platform>
<avg_ncpus>1.000000</avg_ncpus>
<max_ncpus>1.000000</max_ncpus>
<flops>1134575980.270448</flops>
<api_version>6.13.0</api_version>
<file_ref>
<file_name>milkyway_nbody_1.36_x86_64-apple-darwin</file_name>
<main_program/>
</file_ref>
</app_version>

____________
Reno, NV
Team: SETI.USA

1 · 2 · Next
Post to thread

Message boards : Number crunching : MT tasks count only one thread time


Home | My Account | Message Boards | Results


Copyright © 2020 Sebastien