MT tasks count only one thread time

Message boards : Number crunching : MT tasks count only one thread time
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
zombie67 [MM]
             
Avatar

Send message
Joined: 30 Mar 10
Posts: 227
Credit: 9,069,389
RAC: 1,042
Message 1042 - Posted: 12 Mar 2013, 4:24:43 UTC
Last modified: 12 Mar 2013, 4:25:19 UTC

This was bought up in another thread. But since we haven't heard anything back from the admin, I am hoping a new, separate thread would get better results.

MT (multi-threaded) tasks are becoming more common. They are currently being sent by yafu, renderfarm, BURP and T4T. Also, I have heard that MW is working on one. Currently, the WUprop app counts either the task wall time, or the CPU time of only a single thread per task. Not sure which. But it needs to be counting the MT apps as "time x thread count". Other wise, the more threads are used, the less WUPROP badge credit you receive. It is an anti-incentive as it is today, to run MT tasks.

Is anything being done, or planned to be done, to address this issue?
Reno, NV
Team: SETI.USA
ID: 1042 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile STE\/E
             
Avatar

Send message
Joined: 28 Mar 10
Posts: 671
Credit: 3,987,303
RAC: 690
Message 1043 - Posted: 12 Mar 2013, 11:14:40 UTC

It is an anti-incentive as it is today, to run MT tasks


Right Z, so far I haven't run any of the MT Projects & I won't as long as they only count 1 Tread when your using 2-12 Treads depending on the CPU ...
https://signature.statseb.fr/sig-1323.png
https://stats.free-dc.org/badgesbanner.php?cpid=13a87c3a303bcdca4ba0ed600daebb6b
ID: 1043 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
zombie67 [MM]
             
Avatar

Send message
Joined: 30 Mar 10
Posts: 227
Credit: 9,069,389
RAC: 1,042
Message 1078 - Posted: 20 Mar 2013, 4:25:53 UTC

What does it take to get the admin to respond to problems? Seriously. I see he responds to some problem threads, but not others.

This is a simple problem statement: The hours for MT tasks hours are counted as if they used only a single thread (run time), when they should be counted by total thread time (CPU time).
Here is a simple example from yafu, with an 8 thread machine: http://yafu.dyndns.org/yafu/result.php?resultid=1003666

Run time 174.58
CPU time 873.23 <-- This is the field that should be counted for MT tasks. Not Run time.

Is the admin aware of the problem? And if so, what is his plan to address? Fix the app now, or in a week, or in a year? Even if the plan is "do nothing", we should to be told. Please, just let us know something. Thanks!
Reno, NV
Team: SETI.USA
ID: 1078 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Neil Polson
   
Avatar

Send message
Joined: 20 Apr 10
Posts: 20
Credit: 81,989
RAC: 0
Message 1105 - Posted: 26 Mar 2013, 17:48:57 UTC

You can add Edges@Home Autodock VINA app to the list too.
ID: 1105 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile STE\/E
             
Avatar

Send message
Joined: 28 Mar 10
Posts: 671
Credit: 3,987,303
RAC: 690
Message 1106 - Posted: 26 Mar 2013, 19:46:47 UTC - in response to Message 1105.  

You can add Edges@Home Autodock VINA app to the list too.


??? The Edges VINA only use 1 Thread Per CPU & isn't a MT Wu ...

https://signature.statseb.fr/sig-1323.png
https://stats.free-dc.org/badgesbanner.php?cpid=13a87c3a303bcdca4ba0ed600daebb6b
ID: 1106 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Neil Polson
   
Avatar

Send message
Joined: 20 Apr 10
Posts: 20
Credit: 81,989
RAC: 0
Message 1109 - Posted: 27 Mar 2013, 7:26:36 UTC - in response to Message 1106.  

You can add Edges@Home Autodock VINA app to the list too.


??? The Edges VINA only use 1 Thread Per CPU & isn't a MT Wu ...

That's what i thought until I ran it on my Winx64 dual core host. It immediately used both cores and the run time was 2x cpu time! Maybe it's a bug? I will post on their forum.
ID: 1109 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile STE\/E
             
Avatar

Send message
Joined: 28 Mar 10
Posts: 671
Credit: 3,987,303
RAC: 690
Message 1110 - Posted: 27 Mar 2013, 8:01:47 UTC
Last modified: 27 Mar 2013, 8:10:33 UTC

I only run Edges on my Linux Box's, the Wu's only take 1 Thread Per Wu on Linux or at least my Linux Box's anyway, will try some on a Windows Box & see what happens ...

EDIT: Looks like you were right, the Edges VINA Wu was taking between 2 & 3 Threads to run, it kept jumping back & forth between 2 or 3 Threads, I aborted it, I won't run the MT Wu' s as long as we're only getting Credit for 1 Thread here at WUProp ... :(
https://signature.statseb.fr/sig-1323.png
https://stats.free-dc.org/badgesbanner.php?cpid=13a87c3a303bcdca4ba0ed600daebb6b
ID: 1110 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
zombie67 [MM]
             
Avatar

Send message
Joined: 30 Mar 10
Posts: 227
Credit: 9,069,389
RAC: 1,042
Message 1115 - Posted: 28 Mar 2013, 3:11:47 UTC - in response to Message 1110.  

I only run Edges on my Linux Box's, the Wu's only take 1 Thread Per Wu on Linux or at least my Linux Box's anyway, will try some on a Windows Box & see what happens ...

EDIT: Looks like you were right, the Edges VINA Wu was taking between 2 & 3 Threads to run, it kept jumping back & forth between 2 or 3 Threads, I aborted it, I won't run the MT Wu' s as long as we're only getting Credit for 1 Thread here at WUProp ... :(


Yeah, I noticed this about a week ago too. I was looking into why a GPU was underperforming so much. So I restricted a thread....and still at 100% CPU....so a second thread...still 100% CPU. WTF?! After some more digging, I saw vina at like 30% on an 8 thread CPU. But I didn't have time to really look into it at the time, and forgot about it.

Now here are the parts that needs further investigation:

- does the EDGeS@Home project realize it? They really need to identify the app as MT in their app page.
- I see they *also* have an app called "Generic BOINC Application Client" which is a vbox app. And like T4T, it may be MT.
- fightmalaria@home also uses vina. Does that have the same "feature"?
Reno, NV
Team: SETI.USA
ID: 1115 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
       
Avatar

Send message
Joined: 7 Sep 10
Posts: 453
Credit: 945,109
RAC: 0
Message 1116 - Posted: 28 Mar 2013, 15:11:06 UTC - in response to Message 1115.  
Last modified: 28 Mar 2013, 15:11:46 UTC

I haven't seen a fightmalaria@home tasks use more than one thread. They are generally very short (~100s), so there is no need for MT.

On the Main Edges web page it says "also hosts several other applications at beta (experimental) level". Fine, but we need to know what we are crunching; I don't want my GPU to sit idle because of an erroneous/or unannounced Beta MT CPU app. I also noted that while you can select apps to run, the Edges server_status page doesn't reveal which tasks are available.

Boinc-wide there has been far too much live alpha/Beta testing recently, often ill-thought-out experiments by inexperienced researchers. The ultimate consequence of this is that crunchers are deterred from crunching (all projects).
ID: 1116 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Steve Hawker*
       

Send message
Joined: 31 Oct 12
Posts: 125
Credit: 655,634
RAC: 0
Message 1154 - Posted: 8 Apr 2013, 12:38:58 UTC - in response to Message 1116.  

Bump!

This really needs an answer....
ID: 1154 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile KPX
       

Send message
Joined: 6 Apr 10
Posts: 45
Credit: 872,006
RAC: 0
Message 1179 - Posted: 29 Apr 2013, 14:23:32 UTC

Admin, please, this needs a solution. MT apps can't be counted as if they ran on one thread only.
ID: 1179 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile [AF>WildWildWest] Sébastie...
     
Project administrator
Avatar

Send message
Joined: 28 Mar 10
Posts: 2869
Credit: 538,373
RAC: 136
Message 1185 - Posted: 4 May 2013, 19:03:52 UTC

The new version corrects this problem.
ID: 1185 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
zombie67 [MM]
             
Avatar

Send message
Joined: 30 Mar 10
Posts: 227
Credit: 9,069,389
RAC: 1,042
Message 1186 - Posted: 4 May 2013, 21:36:04 UTC

Awesome! Thanks!!!!
Reno, NV
Team: SETI.USA
ID: 1186 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile STE\/E
             
Avatar

Send message
Joined: 28 Mar 10
Posts: 671
Credit: 3,987,303
RAC: 690
Message 1189 - Posted: 5 May 2013, 6:07:00 UTC

I still don't see where it is fixed, I ran 2 Renderfarm WU's on a Quad Core that ran for about 1.5 Hr's & all I have Credit for is 1.42 Hr's even though the Wu's were each using all 4 Core's while they were running ... Was using v3.50 WUProp Wu ...
https://signature.statseb.fr/sig-1323.png
https://stats.free-dc.org/badgesbanner.php?cpid=13a87c3a303bcdca4ba0ed600daebb6b
ID: 1189 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile STE\/E
             
Avatar

Send message
Joined: 28 Mar 10
Posts: 671
Credit: 3,987,303
RAC: 690
Message 1194 - Posted: 5 May 2013, 16:47:17 UTC

ID: 1194 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
zombie67 [MM]
             
Avatar

Send message
Joined: 30 Mar 10
Posts: 227
Credit: 9,069,389
RAC: 1,042
Message 1195 - Posted: 6 May 2013, 16:23:06 UTC

Does this fix also apply to MT GPU apps, like at Moo?
Reno, NV
Team: SETI.USA
ID: 1195 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile [AF>WildWildWest] Sébastie...
     
Project administrator
Avatar

Send message
Joined: 28 Mar 10
Posts: 2869
Credit: 538,373
RAC: 136
Message 1201 - Posted: 7 May 2013, 10:34:31 UTC - in response to Message 1195.  

For the moment, this fix is only for MT CPU apps.
I will search how to fix MT GPU apps next week.
ID: 1201 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
zombie67 [MM]
             
Avatar

Send message
Joined: 30 Mar 10
Posts: 227
Credit: 9,069,389
RAC: 1,042
Message 1512 - Posted: 10 Aug 2013, 2:09:28 UTC
Last modified: 10 Aug 2013, 2:10:50 UTC

Milkyway has an app called "MilkyWay@Home N-Body Simulation". It has a single-thread version and a multi-thread version. Some tasks are issued with ST and others with MT, seems to be random, or maybe based on the size of each task. So a machine is issued tasks, for BOTH kinds of applications with the same name.

http://milkyway.cs.rpi.edu/milkyway/apps.php

Does the wuprop app see the difference for each task, and count hours appropriately?
Reno, NV
Team: SETI.USA
ID: 1512 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile BilBg
Avatar

Send message
Joined: 20 Jun 12
Posts: 63
Credit: 94,685
RAC: 0
Message 1513 - Posted: 11 Aug 2013, 13:33:00 UTC - in response to Message 1512.  
Last modified: 11 Aug 2013, 13:56:02 UTC

So a machine is issued tasks, for BOTH kinds of applications with the same name.

(mind I don't run MilkyWay@Home)

Are you sure they have the same name?

The <user_friendly_name> maybe is different:
MilkyWay@Home N-Body Simulation v1.36
MilkyWay@Home N-Body Simulation v1.36 (mt)
http://milkyway.cs.rpi.edu/milkyway/results.php?hostid=504818&offset=0&show_names=0&state=0&appid=7

Search in your client_state.xml for:
<user_friendly_name>MilkyWay@Home

... and see what is in <name> tag above it.
(e.g. in both cases <name>milkyway_nbody</name> or different)

I think you can help the wuprop admin/developer if you post what you found (i.e. is there difference in <name> for MT vs no-MT app), e.g.:

<app>
<name>milkyway_nbody</name>
<user_friendly_name>MilkyWay@Home nbody Simulation</user_friendly_name>
</app>


You can also search for the .exe filenames (there have to be 2 .exe files in the project directory for MT and no-MT app)

e.g. this is from my client_state.xml if I search for setiathome_7.00_windows_intelx86.exe (the important part is <app_name>)
<app_version>
    <app_name>setiathome_v7</app_name>
    <version_num>700</version_num>
    <platform>windows_intelx86</platform>
    <avg_ncpus>1.000000</avg_ncpus>
    <max_ncpus>1.000000</max_ncpus>
    <flops>481695518.833688</flops>
    <file_ref>
        <file_name>setiathome_7.00_windows_intelx86.exe</file_name>
        <main_program/>
    </file_ref>
    .........



If you search for the name of the WU/task file (in my case 26fe09ab.9197.7434.5.12.126.vlar)
you'll also find the <app_name> associated with this task:
<workunit>
    <name>26fe09ab.9197.7434.5.12.126.vlar</name>
    <app_name>setiathome_v7</app_name>
    <version_num>700</version_num>
    <rsc_fpops_est>183887307748840.000000</rsc_fpops_est>
    <rsc_fpops_bound>1838873077488400.000000</rsc_fpops_bound>
    <rsc_memory_bound>33554432.000000</rsc_memory_bound>
    <rsc_disk_bound>33554432.000000</rsc_disk_bound>
    <file_ref>
        <file_name>26fe09ab.9197.7434.5.12.126.vlar</file_name>
        <open_name>work_unit.sah</open_name>
    </file_ref>
</workunit>



You have to search for names like:
de_nbody_07_23_dark_2_1372784655_882526
ps_nbody_07_30_orphan_real_1372784655_882468

... found in your tasks (remove the last _0 _1 _2 ...):
http://milkyway.cs.rpi.edu/milkyway/results.php?hostid=504818&offset=0&show_names=1&state=0&appid=7





- ALF - "Find out what you don't do well ..... then don't do it!" :)
ID: 1513 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
zombie67 [MM]
             
Avatar

Send message
Joined: 30 Mar 10
Posts: 227
Credit: 9,069,389
RAC: 1,042
Message 1514 - Posted: 11 Aug 2013, 18:29:22 UTC - in response to Message 1513.  

So a machine is issued tasks, for BOTH kinds of applications with the same name.

(mind I don't run MilkyWay@Home)

Are you sure they have the same name?


Yes. Or at least wuprop records them as the same name. It is listed in the account page as only "MilkyWay@Home N-Body Simulation". There is no separate entry for ST vs MT.

You can see the same thing here: http://wuprop.boinc-af.org/active_projects.py

No version number, which is correct, as that is not part of the name*. And apparently, neither are the suffixes such as "(mt)" or "(ncni)".

*It is annoying when projects put the version number in the app name. The name should NOT change when changing the version number. I'm talking to you Mind Modeling!

<app>
<name>milkyway_nbody</name>
<user_friendly_name>MilkyWay@Home N-Body Simulation</user_friendly_name>
<non_cpu_intensive>0</non_cpu_intensive>
</app>

<app_version>
<app_name>milkyway_nbody</app_name>
<version_num>136</version_num>
<platform>x86_64-apple-darwin</platform>
<avg_ncpus>8.000000</avg_ncpus>
<max_ncpus>8.000000</max_ncpus>
<flops>4331244037.649016</flops>
<plan_class>mt</plan_class>
<api_version>6.13.0</api_version>
<cmdline>--nthreads 8</cmdline>
<file_ref>
<file_name>milkyway_nbody_1.36_x86_64-apple-darwin__mt</file_name>
<main_program/>
</file_ref>
</app_version>

<app_version>
<app_name>milkyway_nbody</app_name>
<version_num>136</version_num>
<platform>x86_64-apple-darwin</platform>
<avg_ncpus>1.000000</avg_ncpus>
<max_ncpus>1.000000</max_ncpus>
<flops>1134575980.270448</flops>
<api_version>6.13.0</api_version>
<file_ref>
<file_name>milkyway_nbody_1.36_x86_64-apple-darwin</file_name>
<main_program/>
</file_ref>
</app_version>

Reno, NV
Team: SETI.USA
ID: 1514 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : MT tasks count only one thread time

©2024 Sébastien