Run time hours lost

log in

Advanced search

Message boards : Number crunching : Run time hours lost

Author Message
nanoprobe
   
Avatar
Send message
Joined: 20 Feb 13
Posts: 34
Credit: 653,713
RAC: 0
Total hours: 3,329,673
Message 4273 - Posted: 4 Oct 2016, 12:32:36 UTC
Last modified: 4 Oct 2016, 12:32:55 UTC

I have been noticing lately that the time I run for WCG is not being accurately reported here. Today shows I only logged a little over 3k hours here while the my contribution chart on the WCG site shows more than 4k hours of run time which is what it should be. Why are the same amount of hours not being recorded here?

Profile Alan E. Monroe
   
Send message
Joined: 29 Aug 14
Posts: 1
Credit: 263,463
RAC: 95
Total hours: 807,699
Message 4274 - Posted: 4 Oct 2016, 14:04:38 UTC

I have not seen my hours update for over 24 hours.

Profile [SG-FC] dingdong
     
Send message
Joined: 14 Apr 10
Posts: 16
Credit: 2,023,965
RAC: 509
Total hours: 4,817,970
Message 4276 - Posted: 4 Oct 2016, 16:27:56 UTC - in response to Message 4274.
Last modified: 4 Oct 2016, 16:29:38 UTC

Yes, it doesn't count in "running time", but in "last day" there is a minus.
Oh, seems to be ok at this time...

nanoprobe
   
Avatar
Send message
Joined: 20 Feb 13
Posts: 34
Credit: 653,713
RAC: 0
Total hours: 3,329,673
Message 4285 - Posted: 6 Oct 2016, 15:24:33 UTC

It's still not showing the correct amount of run time. I have 200+ cores running @ WCG. That should give me 4800+ hours of daily run time. It's always 300-400 hours short on that here. Am I missing something?

Profile Steve Hawker*
       
Send message
Joined: 31 Oct 12
Posts: 125
Credit: 655,634
RAC: 0
Total hours: 1,576,278
Message 4287 - Posted: 6 Oct 2016, 17:40:37 UTC - in response to Message 4285.

It's still not showing the correct amount of run time. I have 200+ cores running @ WCG. That should give me 4800+ hours of daily run time. It's always 300-400 hours short on that here. Am I missing something?


Probably not.

First I find that OSX and Linux hardly ever log 24 hrs/core. On some apps I might get as low as 20hrs/day which I'm not losing sleep over.

You might have some time lost between returning your WUProp and starting the next one. Look at your computer with the highest loss and see if there's a delay in there.

In all, I'd say a 10% loss, while not perfect, isn't a massive loss.

If I'm running multiple GPU tasks to increase efficiency, I rarely get 100% of the hours. I have no idea why. Some projects cough up almost 100% some as low as 75%. But as WCG has no GPU work, this can't be it.

I don't know if you're running VMs to get 200+ cores or if you have 200+ bare metal cores.

nanoprobe
   
Avatar
Send message
Joined: 20 Feb 13
Posts: 34
Credit: 653,713
RAC: 0
Total hours: 3,329,673
Message 4288 - Posted: 6 Oct 2016, 19:48:25 UTC - in response to Message 4287.
Last modified: 6 Oct 2016, 19:52:36 UTC

It's still not showing the correct amount of run time. I have 200+ cores running @ WCG. That should give me 4800+ hours of daily run time. It's always 300-400 hours short on that here. Am I missing something?


Probably not.

First I find that OSX and Linux hardly ever log 24 hrs/core. On some apps I might get as low as 20hrs/day which I'm not losing sleep over.

You might have some time lost between returning your WUProp and starting the next one. Look at your computer with the highest loss and see if there's a delay in there.

In all, I'd say a 10% loss, while not perfect, isn't a massive loss.

If I'm running multiple GPU tasks to increase efficiency, I rarely get 100% of the hours. I have no idea why. Some projects cough up almost 100% some as low as 75%. But as WCG has no GPU work, this can't be it.

I don't know if you're running VMs to get 200+ cores or if you have 200+ bare metal cores.

10% loss seems to be about right but that sure adds up over the years. It's really no biggy. Just a curiosity. Only the 2 4 core android devices I run seem to show the full run time. (192 hours per day)
I'm not running VMs. I guess it would be more accurate to say I have 200+ threads (106 Intel Xeon cores)running Linux.
Thanks for the explanation.

Sergey Kovalchuk
         
Send message
Joined: 27 Oct 15
Posts: 201
Credit: 1,648,328
RAC: 6
Total hours: 2,703,087
Message 4289 - Posted: 7 Oct 2016, 6:12:50 UTC - in response to Message 4285.

1. Check hosts -> activity -> each info block from-to
you can see <6 hours of activity, sometimes 5.5h

2. Add <report_results_immediately>1</report_results_immediately> to cc_config
it will reduce the delay between tasks

3. stop or delete a project with a large network lag, for example QCN or CAS
they can "kill" other tasks and block results reporting

Profile skgiven
       
Avatar
Send message
Joined: 7 Sep 10
Posts: 453
Credit: 945,109
RAC: 0
Total hours: 2,101,570
Message 4291 - Posted: 7 Oct 2016, 16:37:37 UTC - in response to Message 4288.
Last modified: 7 Oct 2016, 16:42:30 UTC

When running some ATLAS tasks recently (which run in their own VM and used up lots/all the resources [RAM & Disk]) some of my times were out a lot.
Also, when I ran some Van Der Waerden Numbers tasks on a system with limited memory things didn't work out too well. In both cases LAIM off helped, somewhat, but not running them helped more :)
If you're running an nci project 24/7 and it's not touching 24h/day, you know something is wrong. Occasionally I get some network issues which are not helped by the Boinc scheduler, which following a brief outage/reset will sometimes not try the WUPorp project again for hours. Then there's that project not priority crap...

2. Add <report_results_immediately>1</report_results_immediately> to cc_config
it will reduce the delay between tasks

This can be added to the project (by admin), to report tasks on completion immediately. It's used by GPUGrid.
cc_config may only function on some BM versions.

nanoprobe
   
Avatar
Send message
Joined: 20 Feb 13
Posts: 34
Credit: 653,713
RAC: 0
Total hours: 3,329,673
Message 4316 - Posted: 19 Oct 2016, 12:14:53 UTC

It's getting worse at WCG. 18-20% lost run time.

nanoprobe
   
Avatar
Send message
Joined: 20 Feb 13
Posts: 34
Credit: 653,713
RAC: 0
Total hours: 3,329,673
Message 4536 - Posted: 31 Dec 2016, 13:29:34 UTC
Last modified: 31 Dec 2016, 13:32:40 UTC

Last update for WCG. I have been monitoring this for the past month.
I have 124 threads currently running at WCG. 124 threads running 24 hours a day should net me 2976 hours per day. I have not been credited more than 2400 hours on any day for the last month. Where is the other 500+ hours per day going? The my stats page at WCG shows all the hours.


Post to thread

Message boards : Number crunching : Run time hours lost


Home | My Account | Message Boards | Results


Copyright © 2024 Sebastien