Project server code update

The project will be taken down in about an hour to perform an update of the BOINC server code. Ideally you shouldn't notice anything, but usually the world isn't ideal. See you again on the other side.

Comments

Trotador
Trotador
Joined: 15 May 13
Posts: 7
Credit: 26,130,548
RAC: 0

Project server code update

Scheduler request failed: HTTP internal server error

is what I get

Trotador

nenym
nenym
Joined: 13 Jun 11
Posts: 15
Credit: 10,001,988
RAC: 0

RE: Scheduler request

Message 79898 in response to message 79897

Quote:

Scheduler request failed: HTTP internal server error

is what I get

The same here

Bernd Machenschalk
Bernd Machenschalk
Administrator
Joined: 15 Oct 04
Posts: 155
Credit: 6,218,130
RAC: 0

Does the problem persist? We

Does the problem persist?

We are testing the behavior of "CreditNew" on this project and will try to fix it if necessary. Be prepared for the unexpected!

BM

BM

Claggy
Claggy
Joined: 29 Dec 06
Posts: 122
Credit: 4,040,969
RAC: 0

Intel GPUs are now being

Intel GPUs are now being shown by the project in the computer details pages:

Computer 9008

But 'Use Intel GPU' isn't being shown on the Albert project preferences page in spite of there being intel GPU apps available, perhaps those apps need their settings adjusted?

Claggy

nenym
nenym
Joined: 13 Jun 11
Posts: 15
Credit: 10,001,988
RAC: 0

RE: Does the problem

Message 79901 in response to message 79899

Quote:

Does the problem persist?

We are testing the behavior of "CreditNew" on this project and will try to fix it if necessary. Be prepared for the unexpected!

BM

04/06/2014 10:53:32 | Albert@Home | Sending scheduler request: Requested by user.
04/06/2014 10:53:32 | Albert@Home | Requesting new tasks for CPU and NVIDIA GPU
04/06/2014 10:53:36 | Albert@Home | Scheduler request failed: HTTP internal server error
The machine has been restarted.

Note: Local time UTC+2 (Prag).

If your are going to use Dave's random number generator, I leave the project. Some CPU projects have fixed it to number generator of expected and acceptable range, but no GPU project has been successful in that deal. Good luck.

EDIT: Before leaving I'll try my favorite joke - using app_info to get BPR4 CPU task to be crunched by intel_gpu. I expect credit 0.5 instead of 62.5. Can be seen as wu 590960.

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

RE: If your are going to

Message 79902 in response to message 79901

Quote:
If your are going to use Dave's random number generator, I leave the project. Some CPU projects have fixed it to number generator of expected and acceptable range, but no GPU project has been successful in that deal. Good luck.


We know all that. The purpose of this test is, very specifically, to test and try out some fixes to CreditNew that some volunteers have spent the last nine months developing.

It would be most helpful if you would remain attached to the project, to generate some baseline data from a good range of hosts.

Albert has been chosen for this task specifically because it's a test project where nothing is expected to work anyway!

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Eyrie
Eyrie
Joined: 20 Feb 14
Posts: 48
Credit: 2,410
RAC: 0

RE: If your are going to

Message 79903 in response to message 79901

Quote:

If your are going to use Dave's random number generator, I leave the project. Some CPU projects have fixed it to number generator of expected and acceptable range, but no GPU project has been successful in that deal. Good luck.

EDIT: Before leaving I'll try my favorite joke - using app_info to get BPR4 CPU task to be crunched by intel_gpu. I expect credit 0.5 instead of 62.5.

hold your horses please!

What we are specifically trying to do is to make something that actually WORKS out of David's RNG!

But for that we need to verify first that we see on Albert what we know from SETI main, before we can go on to stick a proper algorithm into it.

So, PLEASE, bear with us while we establish the system works as expected (i.e. is crap) and then apply the patches into the critical areas.

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

RE: Does the problem

Message 79904 in response to message 79899

Quote:

Does the problem persist?

We are testing the behavior of "CreditNew" on this project and will try to fix it if necessary. Be prepared for the unexpected!

BM


I'm getting work OK on 'CPU only' requests.

But I've attached some extra hosts, which are requesting NVidia work as part of project initialisation - that's returning 'internal server error' (see email). I'll have to find some way of blocking that initial request - seems to be fine after that's out of the way, using a 'CPU only' venue.

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

nenym
nenym
Joined: 13 Jun 11
Posts: 15
Credit: 10,001,988
RAC: 0

OK, if Albert is not for

Message 79905 in response to message 79902

OK, if Albert is not for testing of applications only, but also for the credit system (as SetiBeta and ralph), I have no problem to help to generate baseline. It is important to know it. It that case I have no problem with low and random credit.

Eyrie
Eyrie
Joined: 20 Feb 14
Posts: 48
Credit: 2,410
RAC: 0

RE: RE: Does the problem

Message 79906 in response to message 79904

Quote:
Quote:

Does the problem persist?

We are testing the behavior of "CreditNew" on this project and will try to fix it if necessary. Be prepared for the unexpected!

BM


I'm getting work OK on 'CPU only' requests.

But I've attached some extra hosts, which are requesting NVidia work as part of project initialisation - that's returning 'internal server error' (see email). I'll have to find some way of blocking that initial request - seems to be fine after that's out of the way, using a 'CPU only' venue.


only worked after moving the host to a _fresh_ venue that only has CPU ticked.

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

Bernd Machenschalk
Bernd Machenschalk
Administrator
Joined: 15 Oct 04
Posts: 155
Credit: 6,218,130
RAC: 0

Found and fixed a bug in the

Found and fixed a bug in the scheduler.

Please try again.

BM

nenym
nenym
Joined: 13 Jun 11
Posts: 15
Credit: 10,001,988
RAC: 0

Seems to be OK. 04/06/2014

Message 79908 in response to message 79907

Seems to be OK.

04/06/2014 12:37:42 | Albert@Home | Sending scheduler request: Requested by user.
04/06/2014 12:37:42 | Albert@Home | Requesting new tasks for CPU and NVIDIA GPU
04/06/2014 12:37:45 | Albert@Home | Scheduler request completed: got 0 new tasks
04/06/2014 12:37:45 | Albert@Home | No tasks sent
04/06/2014 12:37:45 | Albert@Home | Tasks for CPU are available, but your preferences are set to not accept them


Claggy
Claggy
Joined: 29 Dec 06
Posts: 122
Credit: 4,040,969
RAC: 0

My i7-2600K got ATI/AMD work,

My i7-2600K got ATI/AMD work, But when I suspend Seti (where it's crunching OpenCL Seti v7 work), nothing happens, the ATI/AMD Wu isn't started.

https://albertathome.org/host/8143

Edit: Finally I manage to get it to error:

Activated exception handling...
[12:06:40][13760][INFO ] Starting data processing...
GPU type not found in init_data.xml
[12:06:40][13760][ERROR] Failed to get OpenCL platform/device info from BOINC (error: -161)!
[12:06:40][13760][ERROR] Demodulation failed (error: -161)!
12:06:40 (13760): called boinc_finish

https://albertathome.org/task/1453772

Claggy

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

My host 11361 got some CUDA

My host 11361 got some CUDA work (like result 1448685), which failed with 'No suitable CUDA device available!' - although there's a fully functional "NVIDIA GeForce GTX 750 Ti (2047MB) driver: 335.28", which crunches CUDA at other projects.

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Bernd Machenschalk
Bernd Machenschalk
Administrator
Joined: 15 Oct 04
Posts: 155
Credit: 6,218,130
RAC: 0

Our plan class specs that

Our plan class specs that were (semi-)automatically converted for the new server code were somewhat broken, causing probably all kinds of oddities for GPU tasks. Should be fixed now.

BM

BM

nenym
nenym
Joined: 13 Jun 11
Posts: 15
Credit: 10,001,988
RAC: 0

BRP4G cuda task is running OK

BRP4G cuda task is running OK at 9600GT/XP 32bit, driver 335.28.

zombie67 [MM]
zombie67 [MM]
Joined: 10 Oct 06
Posts: 73
Credit: 30,924,459
RAC: 0

RE: Intel GPUs are now

Message 79913 in response to message 79900

Quote:

Intel GPUs are now being shown by the project in the computer details pages:

Computer 9008

But 'Use Intel GPU' isn't being shown on the Albert project preferences page in spite of there being intel GPU apps available, perhaps those apps need their settings adjusted?

Claggy


+1 Still no setting in preferences to select intel GPU, like you can for AMD or nVidia. Other GPU projects have this, even Einstein.

Dublin, California
Team: SETI.USA

Claggy
Claggy
Joined: 29 Dec 06
Posts: 122
Credit: 4,040,969
RAC: 0

RE: Our plan class specs

Message 79914 in response to message 79911

Quote:

Our plan class specs that were (semi-)automatically converted for the new server code were somewhat broken, causing probably all kinds of oddities for GPU tasks. Should be fixed now.

BM


All my AtI/AMD tasks are predicted to take six seconds, when they get to 2 minutes 6 seconds they error:

https://albertathome.org/task/1455248

Quote:

7.2.42

Maximum elapsed time exceeded

Activated exception handling...
[20:37:27][14488][INFO ] Starting data processing...
[20:37:27][14488][INFO ] Using OpenCL platform provided by: Advanced Micro Devices, Inc.
[20:37:27][14488][INFO ] Using OpenCL device "Capeverde" by: Advanced Micro Devices, Inc.
[20:37:27][14488][INFO ] Checkpoint file unavailable: status.cpt (No such file or directory).
------> Starting from scratch...
[20:37:27][14488][INFO ] Header contents:
------> Original WAPP file: ./p2030.20131124.G176.16-01.04.S.b4s0g0.00000_DM336.00
------> Sample time in microseconds: 65.4762
------> Observation time in seconds: 274.62705
------> Time stamp (MJD): 56620.250187503654
------> Number of samples/record: 0
------> Center freq in MHz: 1214.289551
------> Channel band in MHz: 0.336182022
------> Number of channels/record: 960
------> Nifs: 1
------> RA (J2000): 53157.0385017
------> DEC (J2000): 314116.710699
------> Galactic l: 0
------> Galactic b: 0
------> Name: G176.16-01.04.S
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 4194304
------> Trial dispersion measure: 336 cm^-3 pc
------> Scale factor: 7.48281e-005
[20:37:29][14488][INFO ] Seed for random number generator is 1203156450.
[20:37:31][14488][INFO ] Derived global search parameters:
------> f_A probability = 0.08
------> single bin prob(P_noise > P_thr) = 1.32531e-008
------> thr1 = 18.139
------> thr2 = 21.241
------> thr4 = 26.2686
------> thr8 = 34.6478
------> thr16 = 48.9581
[20:38:27][14488][INFO ] Checkpoint committed!
[20:39:27][14488][INFO ] Checkpoint committed!
[20:39:44][14488][INFO ] OpenCL shutdown complete!
[20:39:44][14488][WARN ] BOINC wants us to quit prematurely or we lost contact! Exiting...

]]>

Claggy

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

Holmis reported the same for

Holmis reported the same for BRP4G-cuda32-nv301 in the problems area, except he inocculated his against "Exit status 197 EXIT_TIME_LIMIT_EXCEEDED" with a big boost to rsc_fpops_bound.

I guess one of us (and that probably means me) should fire up a GPU fetch and compare the calculations in the server log with what actually ends up in client_state.xml

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Claggy
Claggy
Joined: 29 Dec 06
Posts: 122
Credit: 4,040,969
RAC: 0

RE: Holmis reported the

Message 79916 in response to message 79915

Quote:

Holmis reported the same for BRP4G-cuda32-nv301 in the problems area, except he inocculated his against "Exit status 197 EXIT_TIME_LIMIT_EXCEEDED" with a big boost to rsc_fpops_bound.

I guess one of us (and that probably means me) should fire up a GPU fetch and compare the calculations in the server log with what actually ends up in client_state.xml


I saw his post after I posted mine, I'm letting them all error, as I want the fix to come from the project/the Boinc Devs, rather than a work around,

From my client_state.xml, BRP4G has an extra three digits compared to the BRP5 app (of which I haven't received any work yet, so shouldn't have been updated yet):

Quote:


einsteinbinary_BRP5
139
windows_x86_64
0.929041
1.000000
38787392469.934639
BRP5-opencl-ati
7.2.2

einsteinbinary_BRP5_1.39_windows_x86_64__BRP5-opencl-ati.exe



einsteinbinary_BRP4_1.00_graphics_windows_intelx86.exe
graphics_app


ATI
1.000000

377487360.000000

einsteinbinary_BRP4G
134
windows_x86_64
0.989277
0.989277
41492924173738.344000
BRP4G-opencl-ati
7.1.0

einsteinbinary_BRP4G_1.34_windows_x86_64__BRP4G-opencl-ati.exe



einsteinbinary_BRP4_1.00_graphics_windows_intelx86.exe
graphics_app


ATI
1.000000

377487360.000000

Claggy

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

And from my

And from my lappy:

Quote:
2014-06-04 20:28:09.8459 [PID=26529] [version] [AV#720] app_plan() returned false
2014-06-04 20:28:09.8459 [PID=26529] [version] [AV#716] (BRP4G-cuda32-nv301) adjusting projected flops based on PFC avg: 2124.60G
2014-06-04 20:28:09.8459 [PID=26529] [version] Best version of app einsteinbinary_BRP4G is [AV#716] (2124.60 GFLOPS)
2014-06-04 20:28:09.8459 [PID=26529] [send] est delay 0, skipping deadline check
2014-06-04 20:28:09.8460 [PID=26529] [version] get_app_version(): getting app version for WU#599227 (p2030.20131124.G175.87-01.48.S.b1s0g0.00000_1328) appid:29
2014-06-04 20:28:09.8460 [PID=26529] [version] returning cached version: [AV#716]
2014-06-04 20:28:09.8460 [PID=26529] [send] est delay 0, skipping deadline check
2014-06-04 20:28:09.8509 [PID=26529] [send] Sending app_version einsteinbinary_BRP4G 2 133 BRP4G-cuda32-nv301; projected 2124.60 GFLOPS
2014-06-04 20:28:09.8510 [PID=26529] [send] est. duration for WU 599227: unscaled 131.79 scaled 131.85
2014-06-04 20:28:09.8510 [PID=26529] [send] [HOST#11359] sending [RESULT#1455845 p2030.20131124.G175.87-01.48.S.b1s0g0.00000_1328_4] (est. dur. 131.85s (0h02m11s85)) (max time 2635.79s (0h43m55s79))


Both server and client are estimating 131 seconds. But a laptop NV GT 420M with 192 GFLOPS peak, running at 2.1 TeraFlop? Well, we wanted to check the PFC averages.....

Edit: the real problem in client_state is

Quote:

einsteinbinary_BRP4G
133
windows_intelx86
0.895024
0.895024
2124597383074.081300
BRP4G-cuda32-nv301

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

RE: I saw his post after I

Message 79918 in response to message 79916

Quote:
I saw his post after I posted mine, I'm letting them all error, as I want the fix to come from the project/the Boinc Devs, rather than a work around,


Understood. I'm going to try and run mine, to establish a real APR to contrast with that stupid 'PFC avg' initial estimate. Hopefully that'll generate some more ammunition to throw at David. Thank goodness the 32 tasks per day limit worked properly.....

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

So far, every single one of

So far, every single one of the CasA tasks I've run since this test started has ended in 'validate error'. That's across several machines, but the worst example is host 9130.

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

I see the CasA WUs (which

I see the CasA WUs (which were very old, generated in January, and incompatible with the current validator) have now been cancelled.

I'll abort all unstarted examples: should we abort jobs in progress too?

Edit - hold that thought. There are newly generated tasks in the database too, don't abort those.

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Bernd Machenschalk
Bernd Machenschalk
Administrator
Joined: 15 Oct 04
Posts: 155
Credit: 6,218,130
RAC: 0

RE: I see the CasA WUs

Message 79921 in response to message 79920

Quote:

I see the CasA WUs (which were very old, generated in January, and incompatible with the current validator) have now been cancelled.

I'll abort all unstarted examples: should we abort jobs in progress too?

Yes, please.

BM

BM

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

All the suspect CasA (GW)

Message 79922 in response to message 79921

All the suspect CasA (GW) tasks in the database have been cancelled and unconditionally aborted by the project. Any you still have running on your computers (after doing a project update) should run OK, as should any new ones you get issued.

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Claggy
Claggy
Joined: 29 Dec 06
Posts: 122
Credit: 4,040,969
RAC: 0

RE: All the suspect CasA

Message 79923 in response to message 79922

Quote:
All the suspect CasA (GW) tasks in the database have been cancelled and unconditionally aborted by the project. Any you still have running on your computers (after doing a project update) should run OK, as should any new ones you get issued.


I've got some fresh Casa tasks on my last contact, But I also got a single BRP task, But BRP is deselected on the work venue for that host
(I do have 'Run beta/test application versions?' and 'Run CPU versions of applications for which GPU versions are available' selected though):

https://albert.phys.uwm.edu/host_sched_logs/8/8143

https://albertathome.org/task/1431784

In progress tasks for computer 8143

Edit: added the log so we don't loose it:

Quote:
2014-06-05 09:04:29.2602 [PID=4416] Request: [USER#xxxxx] [HOST#8143] [IP xxx.xxx.xxx.103] client 7.2.42
2014-06-05 09:04:29.2613 [PID=4416 ] [send] [HOST#8143] app version 588 is reliable
2014-06-05 09:04:29.2613 [PID=4416 ] [send] set_trust: random choice for cons valid 43: yes
2014-06-05 09:04:29.2613 [PID=4416 ] [send] [AV#649] not reliable; cons valid 1 < 10
2014-06-05 09:04:29.2613 [PID=4416 ] [send] set_trust: cons valid 1 < 10, don't use single replication
2014-06-05 09:04:29.2613 [PID=4416 ] [send] [AV#707] not reliable; cons valid 0 < 10
2014-06-05 09:04:29.2613 [PID=4416 ] [send] set_trust: cons valid 0 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [AV#710] not reliable; cons valid 1 < 10
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: cons valid 1 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [AV#711] not reliable; cons valid 1 < 10
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: cons valid 1 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [AV#712] not reliable; cons valid 4 < 10
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: cons valid 4 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [AV#713] not reliable; cons valid 2 < 10
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: cons valid 2 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [AV#716] not reliable; cons valid 7 < 10
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: cons valid 7 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [AV#721] not reliable; cons valid 0 < 10
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: cons valid 0 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [AV#728] not reliable; cons valid 6 < 10
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: cons valid 6 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [AV#729] not reliable; cons valid 5 < 10
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: cons valid 5 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [AV#737] not reliable; cons valid 0 < 10
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: cons valid 0 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [AV#766] not reliable; cons valid 1 < 10
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: cons valid 1 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [AV#768] not reliable; cons valid 3 < 10
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: cons valid 3 < 10, don't use single replication
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [HOST#8143] app version 842 is reliable
2014-06-05 09:04:29.2614 [PID=4416 ] [send] set_trust: random choice for cons valid 13: yes
2014-06-05 09:04:29.2614 [PID=4416 ] [send] [HOST#8143] app version 843 is reliable
2014-06-05 09:04:29.2615 [PID=4416 ] [send] set_trust: random choice for cons valid 12: yes
2014-06-05 09:04:29.2615 [PID=4416 ] [send] Not using matchmaker scheduling; Not using EDF sim
2014-06-05 09:04:29.2615 [PID=4416 ] [send] CPU: req 259200.00 sec, 3.00 instances; est delay 0.00
2014-06-05 09:04:29.2615 [PID=4416 ] [send] AMD/ATI GPU: req 0.00 sec, 0.00 instances; est delay 0.00
2014-06-05 09:04:29.2616 [PID=4416 ] [send] work_req_seconds: 259200.00 secs
2014-06-05 09:04:29.2616 [PID=4416 ] [send] available disk 96.01 GB, work_buf_min 64800
2014-06-05 09:04:29.2616 [PID=4416 ] [send] on_frac 0.741574 active_frac 0.983919 gpu_active_frac 0.983914
2014-06-05 09:04:29.2616 [PID=4416 ] [send] CPU features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 popcnt aes syscall nx lm vmx tm2 pbe
2014-06-05 09:04:29.2622 [PID=4416 ] [mixed] sending locality work first
2014-06-05 09:04:29.2622 [PID=4416 ] [locality] [HOST#8143] removing file rand_PAS.bank.v3 from file_infos list
2014-06-05 09:04:29.2623 [PID=4416 ] [locality] [HOST#8143] removing file JPLEPH.405 from file_infos list
2014-06-05 09:04:29.2624 [PID=4416 ] [locality] [HOST#8143] removing file stochastic_full.bank from file_infos list
2014-06-05 09:04:29.2624 [PID=4416 ] [locality] [HOST#8143] removing file earth_09_11 from file_infos list
2014-06-05 09:04:29.2624 [PID=4416 ] [locality] [HOST#8143] removing file sun_09_11 from file_infos list
2014-06-05 09:04:29.2625 [PID=4416 ] [locality] [HOST#8143] removing file l1_0071.20_S6Direct from file_infos list
2014-06-05 09:04:29.2625 [PID=4416 ] [locality] [HOST#8143] removing file l1_0071.25_S6Direct from file_infos list
2014-06-05 09:04:29.2625 [PID=4416 ] [locality] [HOST#8143] removing file l1_0071.30_S6Direct from file_infos list
2014-06-05 09:04:29.2625 [PID=4416 ] [locality] [HOST#8143] removing file l1_0071.35_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0071.40_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0071.45_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0996.80_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0996.85_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0996.90_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0996.95_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.00_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.05_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.10_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.15_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.20_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.25_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.30_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.35_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.40_S6Direct from file_infos list
2014-06-05 09:04:29.2626 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.45_S6Direct from file_infos list
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.50_S6Direct from file_infos list
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.55_S6Direct from file_infos list
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.60_S6Direct from file_infos list
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.65_S6Direct from file_infos list
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.70_S6Direct from file_infos list
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.75_S6Direct from file_infos list
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.80_S6Direct from file_infos list
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] removing file l1_0997.85_S6Direct from file_infos list
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] has file h1_0071.20_S6Direct
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] has file h1_0071.25_S6Direct
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] has file h1_0071.30_S6Direct
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] has file h1_0071.35_S6Direct
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] has file h1_0071.40_S6Direct
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] has file h1_0071.45_S6Direct
2014-06-05 09:04:29.2627 [PID=4416 ] [locality] [HOST#8143] has file h1_0996.80_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0996.85_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0996.90_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0996.95_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.00_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.05_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.10_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.15_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.20_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.25_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.30_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.35_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.40_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.45_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.50_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.55_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.60_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.65_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.70_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.75_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.80_S6Direct
2014-06-05 09:04:29.2628 [PID=4416 ] [locality] [HOST#8143] has file h1_0997.85_S6Direct
2014-06-05 09:04:29.2657 [PID=4416 ] [version] get_app_version(): getting app version for WU#595851 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1943) appid:21
2014-06-05 09:04:29.2657 [PID=4416 ] [version] looking for version of einsteinbinary_BRP4
2014-06-05 09:04:29.2658 [PID=4416 ] [version] Checking plan class 'BRP4X64'
2014-06-05 09:04:29.2667 [PID=4416 ] [version] reading plan classes from file '/BOINC/projects/AlbertAtHome/plan_class_spec.xml'
2014-06-05 09:04:29.2667 [PID=4416 ] [version] [AV#588] (BRP4X64) setting projected flops based on host elapsed time avg: 4.93G
2014-06-05 09:04:29.2667 [PID=4416 ] [version] [AV#588] (BRP4X64) comparison pfc: 5.17G et: 4.93G
2014-06-05 09:04:29.2667 [PID=4416 ] [version] Best app version is now AV588 (5.05 GFLOP)
2014-06-05 09:04:29.2668 [PID=4416 ] [version] [AV#588] (BRP4X64) setting projected flops based on host elapsed time avg: 4.93G
2014-06-05 09:04:29.2668 [PID=4416 ] [version] [AV#588] (BRP4X64) comparison pfc: 5.17G et: 4.93G
2014-06-05 09:04:29.2668 [PID=4416 ] [version] Best version of app einsteinbinary_BRP4 is [AV#588] (4.93 GFLOPS)
2014-06-05 09:04:29.2668 [PID=4416 ] [send] est delay 0, skipping deadline check
2014-06-05 09:04:29.2688 [PID=4416 ] [send] Sending app_version einsteinbinary_BRP4 7 133 BRP4X64; projected 4.93 GFLOPS
2014-06-05 09:04:29.2689 [PID=4416 ] [CRITICAL] No filename found in [WU#595851 p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1943]
2014-06-05 09:04:29.2689 [PID=4416 ] [send] est. duration for WU 595851: unscaled 3548.71 scaled 4863.59
2014-06-05 09:04:29.2689 [PID=4416 ] [send] [HOST#8143] sending [RESULT#1431784 p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1943_0] (est. dur. 4863.59s (1h21m03s58)) (max time 70974.20s (19h42m54s19))
2014-06-05 09:04:29.2714 [PID=4416 ] [locality] send_old_work(p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1943_0) sent result created 463.2 hours ago [RESULT#1431784]
2014-06-05 09:04:29.2714 [PID=4416 ] [locality] Note: sent NON-LOCALITY result p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1943_0
2014-06-05 09:04:29.2714 [PID=4416 ] [locality] send_results_for_file(h1_0071.20_S6Direct)
2014-06-05 09:04:29.2734 [PID=4416 ] [locality] in_send_results_for_file(h1_0071.20_S6Direct, 0) prev_result.id=1454321
2014-06-05 09:04:29.2747 [PID=4416 ] [version] get_app_version(): getting app version for WU#606152 (h1_0071.20_S6Direct__S6CasAf40_71.35Hz_1) appid:28
2014-06-05 09:04:29.2747 [PID=4416 ] [version] looking for version of einstein_S6CasA
2014-06-05 09:04:29.2747 [PID=4416 ] [version] Checking plan class 'SSE2'
2014-06-05 09:04:29.2748 [PID=4416 ] [version] [AV#707] (SSE2) adjusting projected flops based on PFC avg: 2.45G
2014-06-05 09:04:29.2748 [PID=4416 ] [version] Best app version is now AV707 (8.22 GFLOP)
2014-06-05 09:04:29.2748 [PID=4416 ] [version] [AV#707] (SSE2) adjusting projected flops based on PFC avg: 2.45G
2014-06-05 09:04:29.2748 [PID=4416 ] [version] Best version of app einstein_S6CasA is [AV#707] (2.45 GFLOPS)
2014-06-05 09:04:29.2748 [PID=4416 ] [send] est. duration for WU 606152: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.2748 [PID=4416 ] [send] [WU#606152] meets deadline: 607.95 + 36212.11 < 604800
2014-06-05 09:04:29.2768 [PID=4416 ] [send] Sending app_version einstein_S6CasA 2 105 SSE2; projected 2.45 GFLOPS
2014-06-05 09:04:29.2770 [PID=4416 ] [locality] [HOST#8143] Already has file h1_0071.20_S6Direct
2014-06-05 09:04:29.2770 [PID=4416 ] [locality] [HOST#8143] reducing disk needed for WU by 5387688 bytes (length of h1_0071.20_S6Direct)
2014-06-05 09:04:29.2770 [PID=4416 ] [send] est. duration for WU 606152: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.2770 [PID=4416 ] [send] [HOST#8143] sending [RESULT#1454335 h1_0071.20_S6Direct__S6CasAf40_71.35Hz_1_0] (est. dur. 36212.11s (10h03m32s10)) (max time 528442.38s (146h47m22s38))
2014-06-05 09:04:29.4721 [PID=4416 ] [locality] in_send_results_for_file(h1_0071.20_S6Direct, 1) prev_result.id=1454335
2014-06-05 09:04:29.4737 [PID=4416 ] [version] get_app_version(): getting app version for WU#606153 (h1_0071.20_S6Direct__S6CasAf40_71.35Hz_0) appid:28
2014-06-05 09:04:29.4737 [PID=4416 ] [version] returning cached version: [AV#707]
2014-06-05 09:04:29.4738 [PID=4416 ] [send] est. duration for WU 606153: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.4738 [PID=4416 ] [send] [WU#606153] meets deadline: 5134.46 + 36212.11 < 604800
2014-06-05 09:04:29.4748 [PID=4416 ] [send] Sending app_version einstein_S6CasA 2 105 SSE2; projected 2.45 GFLOPS
2014-06-05 09:04:29.4751 [PID=4416 ] [locality] [HOST#8143] Already has file h1_0071.20_S6Direct
2014-06-05 09:04:29.4752 [PID=4416 ] [locality] [HOST#8143] reducing disk needed for WU by 5387688 bytes (length of h1_0071.20_S6Direct)
2014-06-05 09:04:29.4752 [PID=4416 ] [send] est. duration for WU 606153: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.4752 [PID=4416 ] [send] [HOST#8143] sending [RESULT#1454337 h1_0071.20_S6Direct__S6CasAf40_71.35Hz_0_0] (est. dur. 36212.11s (10h03m32s10)) (max time 528442.38s (146h47m22s38))
2014-06-05 09:04:29.4766 [PID=4416 ] [locality] in_send_results_for_file(h1_0071.20_S6Direct, 2) prev_result.id=1454337
2014-06-05 09:04:29.4779 [PID=4416 ] [debug] [locality] trigger h1_0071.20_S6Direct state after retrieval: nw=0 wa=1 nwa=1 wsr=0
2014-06-05 09:04:29.4780 [PID=4416 ] [locality] work generator says no work remaining for trigger h1_0071.20_S6Direct
2014-06-05 09:04:29.4780 [PID=4416 ] [locality] make_more_work_for_file(h1_0071.20_S6Direct, 2)=-1
2014-06-05 09:04:29.4785 [PID=4416 ] [locality] send_results_for_file(h1_0071.25_S6Direct)
2014-06-05 09:04:29.4792 [PID=4416 ] [locality] in_send_results_for_file(h1_0071.25_S6Direct, 0) prev_result.id=1281036
2014-06-05 09:04:29.4804 [PID=4416 ] [version] get_app_version(): getting app version for WU#606150 (h1_0071.25_S6Direct__S6CasAf40_71.35Hz_4) appid:28
2014-06-05 09:04:29.4804 [PID=4416 ] [version] returning cached version: [AV#707]
2014-06-05 09:04:29.4805 [PID=4416 ] [send] est. duration for WU 606150: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.4805 [PID=4416 ] [send] [WU#606150] meets deadline: 9660.98 + 36212.11 < 604800
2014-06-05 09:04:29.4812 [PID=4416 ] [send] Sending app_version einstein_S6CasA 2 105 SSE2; projected 2.45 GFLOPS
2014-06-05 09:04:29.4814 [PID=4416 ] [locality] [HOST#8143] Already has file h1_0071.25_S6Direct
2014-06-05 09:04:29.4814 [PID=4416 ] [locality] [HOST#8143] reducing disk needed for WU by 5387688 bytes (length of h1_0071.25_S6Direct)
2014-06-05 09:04:29.4814 [PID=4416 ] [send] est. duration for WU 606150: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.4814 [PID=4416 ] [send] [HOST#8143] sending [RESULT#1454328 h1_0071.25_S6Direct__S6CasAf40_71.35Hz_4_1] (est. dur. 36212.11s (10h03m32s10)) (max time 528442.38s (146h47m22s38))
2014-06-05 09:04:29.4824 [PID=4416 ] [locality] in_send_results_for_file(h1_0071.25_S6Direct, 1) prev_result.id=1454328
2014-06-05 09:04:29.4834 [PID=4416 ] [version] get_app_version(): getting app version for WU#606151 (h1_0071.25_S6Direct__S6CasAf40_71.35Hz_3) appid:28
2014-06-05 09:04:29.4834 [PID=4416 ] [version] returning cached version: [AV#707]
2014-06-05 09:04:29.4834 [PID=4416 ] [send] est. duration for WU 606151: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.4835 [PID=4416 ] [send] [WU#606151] meets deadline: 14187.49 + 36212.11 < 604800
2014-06-05 09:04:29.4841 [PID=4416 ] [send] Sending app_version einstein_S6CasA 2 105 SSE2; projected 2.45 GFLOPS
2014-06-05 09:04:29.4845 [PID=4416 ] [locality] [HOST#8143] Already has file h1_0071.25_S6Direct
2014-06-05 09:04:29.4846 [PID=4416 ] [locality] [HOST#8143] reducing disk needed for WU by 5387688 bytes (length of h1_0071.25_S6Direct)
2014-06-05 09:04:29.4846 [PID=4416 ] [send] est. duration for WU 606151: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.4846 [PID=4416 ] [send] [HOST#8143] sending [RESULT#1454330 h1_0071.25_S6Direct__S6CasAf40_71.35Hz_3_1] (est. dur. 36212.11s (10h03m32s10)) (max time 528442.38s (146h47m22s38))
2014-06-05 09:04:29.5814 [PID=4416 ] [locality] in_send_results_for_file(h1_0071.25_S6Direct, 2) prev_result.id=1454330
2014-06-05 09:04:29.5829 [PID=4416 ] [version] get_app_version(): getting app version for WU#606727 (h1_0071.25_S6Direct__S6CasAf40_71.35Hz_2) appid:28
2014-06-05 09:04:29.5829 [PID=4416 ] [version] returning cached version: [AV#707]
2014-06-05 09:04:29.5829 [PID=4416 ] [send] est. duration for WU 606727: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.5829 [PID=4416 ] [send] [WU#606727] meets deadline: 18714.00 + 36212.11 < 604800
2014-06-05 09:04:29.5838 [PID=4416 ] [send] Sending app_version einstein_S6CasA 2 105 SSE2; projected 2.45 GFLOPS
2014-06-05 09:04:29.5840 [PID=4416 ] [locality] [HOST#8143] Already has file h1_0071.25_S6Direct
2014-06-05 09:04:29.5840 [PID=4416 ] [locality] [HOST#8143] reducing disk needed for WU by 5387688 bytes (length of h1_0071.25_S6Direct)
2014-06-05 09:04:29.5840 [PID=4416 ] [send] est. duration for WU 606727: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.5840 [PID=4416 ] [send] [HOST#8143] sending [RESULT#1455921 h1_0071.25_S6Direct__S6CasAf40_71.35Hz_2_0] (est. dur. 36212.11s (10h03m32s10)) (max time 528442.38s (146h47m22s38))
2014-06-05 09:04:29.5851 [PID=4416 ] [locality] in_send_results_for_file(h1_0071.25_S6Direct, 3) prev_result.id=1455921
2014-06-05 09:04:29.5863 [PID=4416 ] [version] get_app_version(): getting app version for WU#606728 (h1_0071.25_S6Direct__S6CasAf40_71.4Hz_1) appid:28
2014-06-05 09:04:29.5863 [PID=4416 ] [version] returning cached version: [AV#707]
2014-06-05 09:04:29.5863 [PID=4416 ] [send] est. duration for WU 606728: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.5863 [PID=4416 ] [send] [WU#606728] meets deadline: 23240.52 + 36212.11 < 604800
2014-06-05 09:04:29.5871 [PID=4416 ] [send] Sending app_version einstein_S6CasA 2 105 SSE2; projected 2.45 GFLOPS
2014-06-05 09:04:29.5873 [PID=4416 ] [locality] [HOST#8143] Already has file h1_0071.25_S6Direct
2014-06-05 09:04:29.5873 [PID=4416 ] [locality] [HOST#8143] reducing disk needed for WU by 5387688 bytes (length of h1_0071.25_S6Direct)
2014-06-05 09:04:29.5873 [PID=4416 ] [send] est. duration for WU 606728: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:29.5873 [PID=4416 ] [send] [HOST#8143] sending [RESULT#1455923 h1_0071.25_S6Direct__S6CasAf40_71.4Hz_1_0] (est. dur. 36212.11s (10h03m32s10)) (max time 528442.38s (146h47m22s38))
2014-06-05 09:04:29.5884 [PID=4416 ] [locality] in_send_results_for_file(h1_0071.25_S6Direct, 4) prev_result.id=1455923
2014-06-05 09:04:29.5893 [PID=4416 ] [debug] [locality] trigger h1_0071.25_S6Direct state after retrieval: nw=0 wa=1 nwa=0 wsr=0
2014-06-05 09:04:29.5902 [PID=4416 ] [locality] make_more_work_for_file(h1_0071.25_S6Direct, 4)=0
2014-06-05 09:04:31.5905 [PID=4416 ] [locality] in_send_results_for_file(h1_0071.25_S6Direct, 5) prev_result.id=1455923
2014-06-05 09:04:31.5915 [PID=4416 ] [debug] [locality] trigger h1_0071.25_S6Direct state after retrieval: nw=0 wa=1 nwa=1 wsr=0
2014-06-05 09:04:31.5915 [PID=4416 ] [locality] work generator says no work remaining for trigger h1_0071.25_S6Direct
2014-06-05 09:04:31.5915 [PID=4416 ] [locality] make_more_work_for_file(h1_0071.25_S6Direct, 5)=-1
2014-06-05 09:04:31.5920 [PID=4416 ] [locality] send_results_for_file(h1_0071.30_S6Direct)
2014-06-05 09:04:31.5926 [PID=4416 ] [locality] in_send_results_for_file(h1_0071.30_S6Direct, 0) prev_result.id=0
2014-06-05 09:04:31.5935 [PID=4416 ] [version] get_app_version(): getting app version for WU#606767 (h1_0071.30_S6Direct__S6CasAf40_71.4Hz_5) appid:28
2014-06-05 09:04:31.5935 [PID=4416 ] [version] returning cached version: [AV#707]
2014-06-05 09:04:31.5935 [PID=4416 ] [send] est. duration for WU 606767: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:31.5935 [PID=4416 ] [send] [WU#606767] meets deadline: 27767.03 + 36212.11 < 604800
2014-06-05 09:04:31.5944 [PID=4416 ] [send] Sending app_version einstein_S6CasA 2 105 SSE2; projected 2.45 GFLOPS
2014-06-05 09:04:31.5945 [PID=4416 ] [locality] [HOST#8143] Already has file h1_0071.30_S6Direct
2014-06-05 09:04:31.5946 [PID=4416 ] [locality] [HOST#8143] reducing disk needed for WU by 5387688 bytes (length of h1_0071.30_S6Direct)
2014-06-05 09:04:31.5946 [PID=4416 ] [send] est. duration for WU 606767: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:31.5946 [PID=4416 ] [send] [HOST#8143] sending [RESULT#1456528 h1_0071.30_S6Direct__S6CasAf40_71.4Hz_5_0] (est. dur. 36212.11s (10h03m32s10)) (max time 528442.38s (146h47m22s38))
2014-06-05 09:04:31.5960 [PID=4416 ] [locality] in_send_results_for_file(h1_0071.30_S6Direct, 1) prev_result.id=1456528
2014-06-05 09:04:31.5971 [PID=4416 ] [version] get_app_version(): getting app version for WU#606768 (h1_0071.30_S6Direct__S6CasAf40_71.4Hz_4) appid:28
2014-06-05 09:04:31.5971 [PID=4416 ] [version] returning cached version: [AV#707]
2014-06-05 09:04:31.5971 [PID=4416 ] [send] est. duration for WU 606768: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:31.5971 [PID=4416 ] [send] [WU#606768] meets deadline: 32293.54 + 36212.11 < 604800
2014-06-05 09:04:31.5980 [PID=4416 ] [send] Sending app_version einstein_S6CasA 2 105 SSE2; projected 2.45 GFLOPS
2014-06-05 09:04:31.5987 [PID=4416 ] [locality] [HOST#8143] Already has file h1_0071.30_S6Direct
2014-06-05 09:04:31.5988 [PID=4416 ] [locality] [HOST#8143] reducing disk needed for WU by 5387688 bytes (length of h1_0071.30_S6Direct)
2014-06-05 09:04:31.5988 [PID=4416 ] [send] est. duration for WU 606768: unscaled 26422.12 scaled 36212.11
2014-06-05 09:04:31.5988 [PID=4416 ] [send] [HOST#8143] sending [RESULT#1456530 h1_0071.30_S6Direct__S6CasAf40_71.4Hz_4_0] (est. dur. 36212.11s (10h03m32s10)) (max time 528442.38s (146h47m22s38))
2014-06-05 09:04:31.6014 [PID=4416 ] [send] don't need more work
2014-06-05 09:04:31.6015 [PID=4416 ] [send] don't need more work
2014-06-05 09:04:31.6015 [PID=4416 ] [send] don't need more work
2014-06-05 09:04:31.6015 [PID=4416 ] [mixed] sending non-locality work second
2014-06-05 09:04:31.6074 [PID=4416 ] [version] get_app_version(): getting app version for WU#595892 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1984) appid:21
2014-06-05 09:04:31.6075 [PID=4416 ] [version] looking for version of einsteinbinary_BRP4
2014-06-05 09:04:31.6075 [PID=4416 ] [version] Checking plan class 'BRP4X64'
2014-06-05 09:04:31.6075 [PID=4416 ] [version] [AV#588] Don't need CPU jobs, skipping
2014-06-05 09:04:31.6075 [PID=4416 ] [version] Checking plan class 'BRP4SSE'
2014-06-05 09:04:31.6075 [PID=4416 ] [version] [AV#598] Don't need CPU jobs, skipping
2014-06-05 09:04:31.6075 [PID=4416 ] [version] returning NULL; platforms:
2014-06-05 09:04:31.6075 [PID=4416 ] [version] windows_x86_64
2014-06-05 09:04:31.6075 [PID=4416 ] [version] windows_intelx86
2014-06-05 09:04:31.6075 [PID=4416 ] [version] get_app_version(): getting app version for WU#606386 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3280) appid:29
2014-06-05 09:04:31.6075 [PID=4416 ] [version] looking for version of einsteinbinary_BRP4G
2014-06-05 09:04:31.6076 [PID=4416 ] [version] Checking plan class 'BRP4G-opencl-ati'
2014-06-05 09:04:31.6076 [PID=4416 ] [version] plan_class_spec: parsed project prefs setting 'gpu_util_brp' : true : 1.000000
2014-06-05 09:04:31.6076 [PID=4416 ] [version] [AV#721] Skipping AMD/ATI GPU version - user prefs say no AMD/ATI GPU
2014-06-05 09:04:31.6076 [PID=4416 ] [version] Checking plan class 'BRP4G-cuda32'
2014-06-05 09:04:31.6076 [PID=4416 ] [version] plan_class_spec: parsed project prefs setting 'gpu_util_brp' : true : 1.000000
2014-06-05 09:04:31.6076 [PID=4416 ] [version] plan_class_spec: No NVIDIA GPUs found
2014-06-05 09:04:31.6076 [PID=4416 ] [version] [AV#723] app_plan() returned false
2014-06-05 09:04:31.6076 [PID=4416 ] [version] Checking plan class 'BRP4G-cuda32-nv301'
2014-06-05 09:04:31.6076 [PID=4416 ] [version] plan_class_spec: parsed project prefs setting 'gpu_util_brp' : true : 1.000000
2014-06-05 09:04:31.6076 [PID=4416 ] [version] plan_class_spec: No NVIDIA GPUs found
2014-06-05 09:04:31.6076 [PID=4416 ] [version] [AV#716] app_plan() returned false
2014-06-05 09:04:31.6077 [PID=4416 ] [version] Checking plan class 'BRP4G-opencl-ati'
2014-06-05 09:04:31.6077 [PID=4416 ] [version] plan_class_spec: parsed project prefs setting 'gpu_util_brp' : true : 1.000000
2014-06-05 09:04:31.6077 [PID=4416 ] [version] [AV#720] Skipping AMD/ATI GPU version - user prefs say no AMD/ATI GPU
2014-06-05 09:04:31.6077 [PID=4416 ] [version] returning NULL; platforms:
2014-06-05 09:04:31.6077 [PID=4416 ] [version] windows_x86_64
2014-06-05 09:04:31.6077 [PID=4416 ] [version] windows_intelx86
2014-06-05 09:04:31.6077 [PID=4416 ] [version] get_app_version(): getting app version for WU#595880 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1972) appid:21
2014-06-05 09:04:31.6077 [PID=4416 ] [version] get_app_version(): getting app version for WU#595895 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1987) appid:21
2014-06-05 09:04:31.6077 [PID=4416 ] [version] get_app_version(): getting app version for WU#603298 (p2030.20131124.G176.16-01.04.S.b4s0g0.00000_1728) appid:29
2014-06-05 09:04:31.6078 [PID=4416 ] [version] get_app_version(): getting app version for WU#595893 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1985) appid:21
2014-06-05 09:04:31.6078 [PID=4416 ] [version] get_app_version(): getting app version for WU#595894 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1986) appid:21
2014-06-05 09:04:31.6078 [PID=4416 ] [version] get_app_version(): getting app version for WU#606383 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3232) appid:29
2014-06-05 09:04:31.6079 [PID=4416 ] [version] get_app_version(): getting app version for WU#595881 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1973) appid:21
2014-06-05 09:04:31.6079 [PID=4416 ] [version] get_app_version(): getting app version for WU#595882 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1974) appid:21
2014-06-05 09:04:31.6079 [PID=4416 ] [version] get_app_version(): getting app version for WU#606396 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3440) appid:29
2014-06-05 09:04:31.6079 [PID=4416 ] [version] get_app_version(): getting app version for WU#595882 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1974) appid:21
2014-06-05 09:04:31.6079 [PID=4416 ] [version] get_app_version(): getting app version for WU#595883 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1975) appid:21
2014-06-05 09:04:31.6080 [PID=4416 ] [version] get_app_version(): getting app version for WU#606384 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3248) appid:29
2014-06-05 09:04:31.6080 [PID=4416 ] [version] get_app_version(): getting app version for WU#595843 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1935) appid:21
2014-06-05 09:04:31.6080 [PID=4416 ] [version] get_app_version(): getting app version for WU#595883 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1975) appid:21
2014-06-05 09:04:31.6080 [PID=4416 ] [version] get_app_version(): getting app version for WU#606397 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3456) appid:29
2014-06-05 09:04:31.6080 [PID=4416 ] [version] get_app_version(): getting app version for WU#595844 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1936) appid:21
2014-06-05 09:04:31.6081 [PID=4416 ] [version] get_app_version(): getting app version for WU#595884 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1976) appid:21
2014-06-05 09:04:31.6081 [PID=4416 ] [version] get_app_version(): getting app version for WU#606397 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3456) appid:29
2014-06-05 09:04:31.6081 [PID=4416 ] [version] get_app_version(): getting app version for WU#595884 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1976) appid:21
2014-06-05 09:04:31.6081 [PID=4416 ] [version] get_app_version(): getting app version for WU#595885 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1977) appid:21
2014-06-05 09:04:31.6081 [PID=4416 ] [version] get_app_version(): getting app version for WU#606398 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3472) appid:29
2014-06-05 09:04:31.6082 [PID=4416 ] [version] get_app_version(): getting app version for WU#595885 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1977) appid:21
2014-06-05 09:04:31.6082 [PID=4416 ] [version] get_app_version(): getting app version for WU#595886 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1978) appid:21
2014-06-05 09:04:31.6082 [PID=4416 ] [version] get_app_version(): getting app version for WU#603293 (p2030.20131124.G176.16-01.04.S.b4s0g0.00000_1648) appid:29
2014-06-05 09:04:31.6082 [PID=4416 ] [version] get_app_version(): getting app version for WU#595886 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1978) appid:21
2014-06-05 09:04:31.6082 [PID=4416 ] [version] get_app_version(): getting app version for WU#595887 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1979) appid:21
2014-06-05 09:04:31.6083 [PID=4416 ] [version] get_app_version(): getting app version for WU#606398 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3472) appid:29
2014-06-05 09:04:31.6083 [PID=4416 ] [version] get_app_version(): getting app version for WU#595837 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1929) appid:21
2014-06-05 09:04:31.6083 [PID=4416 ] [version] get_app_version(): getting app version for WU#595887 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1979) appid:21
2014-06-05 09:04:31.6083 [PID=4416 ] [version] get_app_version(): getting app version for WU#603295 (p2030.20131124.G176.16-01.04.S.b4s0g0.00000_1680) appid:29
2014-06-05 09:04:31.6084 [PID=4416 ] [version] get_app_version(): getting app version for WU#595888 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1980) appid:21
2014-06-05 09:04:31.6084 [PID=4416 ] [version] get_app_version(): getting app version for WU#595888 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1980) appid:21
2014-06-05 09:04:31.6084 [PID=4416 ] [version] get_app_version(): getting app version for WU#603300 (p2030.20131124.G176.16-01.04.S.b4s0g0.00000_1760) appid:29
2014-06-05 09:04:31.6084 [PID=4416 ] [version] get_app_version(): getting app version for WU#595839 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1931) appid:21
2014-06-05 09:04:31.6084 [PID=4416 ] [version] get_app_version(): getting app version for WU#595889 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1981) appid:21
2014-06-05 09:04:31.6085 [PID=4416 ] [version] get_app_version(): getting app version for WU#603317 (p2030.20131124.G176.16-01.04.S.b4s0g0.00000_2032) appid:29
2014-06-05 09:04:31.6085 [PID=4416 ] [version] get_app_version(): getting app version for WU#595864 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1956) appid:21
2014-06-05 09:04:31.6085 [PID=4416 ] [version] get_app_version(): getting app version for WU#595889 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1981) appid:21
2014-06-05 09:04:31.6085 [PID=4416 ] [version] get_app_version(): getting app version for WU#603297 (p2030.20131124.G176.16-01.04.S.b4s0g0.00000_1712) appid:29
2014-06-05 09:04:31.6085 [PID=4416 ] [version] get_app_version(): getting app version for WU#595890 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1982) appid:21
2014-06-05 09:04:31.6086 [PID=4416 ] [version] get_app_version(): getting app version for WU#595865 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1957) appid:21
2014-06-05 09:04:31.6086 [PID=4416 ] [version] get_app_version(): getting app version for WU#595867 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1959) appid:21
2014-06-05 09:04:31.6086 [PID=4416 ] [version] get_app_version(): getting app version for WU#603296 (p2030.20131124.G176.16-01.04.S.b4s0g0.00000_1696) appid:29
2014-06-05 09:04:31.6086 [PID=4416 ] [version] get_app_version(): getting app version for WU#595868 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1960) appid:21
2014-06-05 09:04:31.6086 [PID=4416 ] [version] get_app_version(): getting app version for WU#595862 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1954) appid:21
2014-06-05 09:04:31.6087 [PID=4416 ] [version] get_app_version(): getting app version for WU#606388 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3312) appid:29
2014-06-05 09:04:31.6087 [PID=4416 ] [version] get_app_version(): getting app version for WU#595894 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1986) appid:21
2014-06-05 09:04:31.6087 [PID=4416 ] [version] get_app_version(): getting app version for WU#595869 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1961) appid:21
2014-06-05 09:04:31.6087 [PID=4416 ] [version] get_app_version(): getting app version for WU#606388 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3312) appid:29
2014-06-05 09:04:31.6087 [PID=4416 ] [version] get_app_version(): getting app version for WU#595863 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1955) appid:21
2014-06-05 09:04:31.6088 [PID=4416 ] [version] get_app_version(): getting app version for WU#595869 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1961) appid:21
2014-06-05 09:04:31.6088 [PID=4416 ] [version] get_app_version(): getting app version for WU#606389 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3328) appid:29
2014-06-05 09:04:31.6088 [PID=4416 ] [version] get_app_version(): getting app version for WU#595851 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1943) appid:21
2014-06-05 09:04:31.6088 [PID=4416 ] [version] get_app_version(): getting app version for WU#595870 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1962) appid:21
2014-06-05 09:04:31.6088 [PID=4416 ] [version] get_app_version(): getting app version for WU#606389 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3328) appid:29
2014-06-05 09:04:31.6089 [PID=4416 ] [version] get_app_version(): getting app version for WU#595870 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1962) appid:21
2014-06-05 09:04:31.6089 [PID=4416 ] [version] get_app_version(): getting app version for WU#595859 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1951) appid:21
2014-06-05 09:04:31.6089 [PID=4416 ] [version] get_app_version(): getting app version for WU#606390 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3344) appid:29
2014-06-05 09:04:31.6089 [PID=4416 ] [version] get_app_version(): getting app version for WU#595871 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1963) appid:21
2014-06-05 09:04:31.6090 [PID=4416 ] [version] get_app_version(): getting app version for WU#595860 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1952) appid:21
2014-06-05 09:04:31.6090 [PID=4416 ] [version] get_app_version(): getting app version for WU#606390 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3344) appid:29
2014-06-05 09:04:31.6090 [PID=4416 ] [version] get_app_version(): getting app version for WU#595852 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1944) appid:21
2014-06-05 09:04:31.6090 [PID=4416 ] [version] get_app_version(): getting app version for WU#595871 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1963) appid:21
2014-06-05 09:04:31.6090 [PID=4416 ] [version] get_app_version(): getting app version for WU#606391 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3360) appid:29
2014-06-05 09:04:31.6091 [PID=4416 ] [version] get_app_version(): getting app version for WU#595872 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1964) appid:21
2014-06-05 09:04:31.6091 [PID=4416 ] [version] get_app_version(): getting app version for WU#595872 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1964) appid:21
2014-06-05 09:04:31.6091 [PID=4416 ] [version] get_app_version(): getting app version for WU#606391 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3360) appid:29
2014-06-05 09:04:31.6091 [PID=4416 ] [version] get_app_version(): getting app version for WU#595853 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1945) appid:21
2014-06-05 09:04:31.6092 [PID=4416 ] [version] get_app_version(): getting app version for WU#595873 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1965) appid:21
2014-06-05 09:04:31.6092 [PID=4416 ] [version] get_app_version(): getting app version for WU#606392 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3376) appid:29
2014-06-05 09:04:31.6092 [PID=4416 ] [version] get_app_version(): getting app version for WU#595873 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1965) appid:21
2014-06-05 09:04:31.6092 [PID=4416 ] [version] get_app_version(): getting app version for WU#595831 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1923) appid:21
2014-06-05 09:04:31.6092 [PID=4416 ] [version] get_app_version(): getting app version for WU#606392 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3376) appid:29
2014-06-05 09:04:31.6093 [PID=4416 ] [version] get_app_version(): getting app version for WU#595874 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1966) appid:21
2014-06-05 09:04:31.6093 [PID=4416 ] [version] get_app_version(): getting app version for WU#595874 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1966) appid:21
2014-06-05 09:04:31.6093 [PID=4416 ] [version] get_app_version(): getting app version for WU#606393 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3392) appid:29
2014-06-05 09:04:31.6093 [PID=4416 ] [version] get_app_version(): getting app version for WU#595861 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1953) appid:21
2014-06-05 09:04:31.6094 [PID=4416 ] [version] get_app_version(): getting app version for WU#595875 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1967) appid:21
2014-06-05 09:04:31.6094 [PID=4416 ] [version] get_app_version(): getting app version for WU#606393 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3392) appid:29
2014-06-05 09:04:31.6094 [PID=4416 ] [version] get_app_version(): getting app version for WU#595866 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1958) appid:21
2014-06-05 09:04:31.6094 [PID=4416 ] [version] get_app_version(): getting app version for WU#595875 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1967) appid:21
2014-06-05 09:04:31.6095 [PID=4416 ] [version] get_app_version(): getting app version for WU#606394 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3408) appid:29
2014-06-05 09:04:31.6095 [PID=4416 ] [version] get_app_version(): getting app version for WU#595876 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1968) appid:21
2014-06-05 09:04:31.6095 [PID=4416 ] [version] get_app_version(): getting app version for WU#595876 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1968) appid:21
2014-06-05 09:04:31.6095 [PID=4416 ] [version] get_app_version(): getting app version for WU#606394 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3408) appid:29
2014-06-05 09:04:31.6096 [PID=4416 ] [version] get_app_version(): getting app version for WU#595877 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1969) appid:21
2014-06-05 09:04:31.6096 [PID=4416 ] [version] get_app_version(): getting app version for WU#595890 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1982) appid:21
2014-06-05 09:04:31.6096 [PID=4416 ] [version] get_app_version(): getting app version for WU#606395 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3424) appid:29
2014-06-05 09:04:31.6096 [PID=4416 ] [version] get_app_version(): getting app version for WU#595891 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1983) appid:21
2014-06-05 09:04:31.6097 [PID=4416 ] [version] get_app_version(): getting app version for WU#595891 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1983) appid:21
2014-06-05 09:04:31.6097 [PID=4416 ] [version] get_app_version(): getting app version for WU#603324 (p2030.20131124.G176.16-01.04.S.b4s0g0.00000_2144) appid:29
2014-06-05 09:04:31.6097 [PID=4416 ] [version] get_app_version(): getting app version for WU#595878 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1970) appid:21
2014-06-05 09:04:31.6097 [PID=4416 ] [version] get_app_version(): getting app version for WU#595892 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1984) appid:21
2014-06-05 09:04:31.6097 [PID=4416 ] [version] get_app_version(): getting app version for WU#606310 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_2064) appid:29
2014-06-05 09:04:31.6098 [PID=4416 ] [version] get_app_version(): getting app version for WU#595879 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1971) appid:21
2014-06-05 09:04:31.6834 [PID=4416 ] Sending reply to [HOST#8143]: 9 results, delay req 60.00
2014-06-05 09:04:31.6840 [PID=4416 ] Scheduler ran 2.429 seconds

Claggy

Bernd Machenschalk
Bernd Machenschalk
Administrator
Joined: 15 Oct 04
Posts: 155
Credit: 6,218,130
RAC: 0

Thanks for reporting. This

Message 79924 in response to message 79923

Thanks for reporting. This looks like a bug to me in current server (scheduler) code. May take a bit of time to investigate, though.

BM

BM

Claggy
Claggy
Joined: 29 Dec 06
Posts: 122
Credit: 4,040,969
RAC: 0

My this morning, ATI BRP4G

Message 79925 in response to message 79924

My this morning, ATI BRP4G tasks report the same wacky speeds (and short estimated durations) as last night,

Edit: got them all physically removed from my client_state.xml so they can be resent when the scheduler is fixed.

https://albert.phys.uwm.edu/host_sched_logs/8/8143

Quote:
2014-06-05 09:56:29.7913 [PID=7201 ] [version] looking for version of einsteinbinary_BRP4G
2014-06-05 09:56:29.7913 [PID=7201 ] [version] Checking plan class 'BRP4G-opencl-ati'
2014-06-05 09:56:29.7913 [PID=7201 ] [version] plan_class_spec: parsed project prefs setting 'gpu_util_brp' : true : 1.000000
2014-06-05 09:56:29.7913 [PID=7201 ] [version] [AV#721] (BRP4G-opencl-ati) adjusting projected flops based on PFC avg: 34968.78G
2014-06-05 09:56:29.7913 [PID=7201 ] [version] Best app version is now AV721 (18620.28 GFLOP)
2014-06-05 09:56:29.7913 [PID=7201 ] [version] [AV#721] (BRP4G-opencl-ati) adjusting projected flops based on PFC avg: 34968.78G
2014-06-05 09:56:29.7914 [PID=7201 ] [version] Best version of app einsteinbinary_BRP4G is [AV#721] (34968.78 GFLOPS)
2014-06-05 09:56:29.7914 [PID=7201 ] [send] est delay 0, skipping deadline check
2014-06-05 09:56:29.7914 [PID=7201 ] [version] get_app_version(): getting app version for WU#606395 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3424) appid:29
2014-06-05 09:56:29.7914 [PID=7201 ] [version] returning cached version: [AV#721]
2014-06-05 09:56:29.7914 [PID=7201 ] [send] est delay 0, skipping deadline check
2014-06-05 09:56:29.7923 [PID=7201 ] [RESULT#1454918] expected to be unsent; instead, state is 4
2014-06-05 09:56:29.7923 [PID=7201 ] [version] get_app_version(): getting app version for WU#595902 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1994) appid:21
2014-06-05 09:56:29.7923 [PID=7201 ] [version] returning cached version: [AV#588]
2014-06-05 09:56:29.7923 [PID=7201 ] [version] get_app_version(): getting app version for WU#595873 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1965) appid:21
2014-06-05 09:56:29.7923 [PID=7201 ] [version] returning cached version: [AV#588]
2014-06-05 09:56:29.7923 [PID=7201 ] [version] get_app_version(): getting app version for WU#606395 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3424) appid:29
2014-06-05 09:56:29.7923 [PID=7201 ] [version] returning cached version: [AV#721]
2014-06-05 09:56:29.7923 [PID=7201 ] [send] est delay 0, skipping deadline check
2014-06-05 09:56:29.7924 [PID=7201 ] [version] get_app_version(): getting app version for WU#606395 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3424) appid:29
2014-06-05 09:56:29.7924 [PID=7201 ] [version] returning cached version: [AV#721]
2014-06-05 09:56:29.7924 [PID=7201 ] [send] est delay 0, skipping deadline check
2014-06-05 09:56:29.7928 [PID=7201 ] [RESULT#1454919] expected to be unsent; instead, state is 4
2014-06-05 09:56:29.7928 [PID=7201 ] [version] get_app_version(): getting app version for WU#595902 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_1994) appid:21
2014-06-05 09:56:29.7928 [PID=7201 ] [version] returning cached version: [AV#588]
2014-06-05 09:56:29.7928 [PID=7201 ] [version] get_app_version(): getting app version for WU#595915 (p2030.20131124.G175.86-01.90.N.b2s0g0.00000_2007) appid:21
2014-06-05 09:56:29.7928 [PID=7201 ] [version] returning cached version: [AV#588]
2014-06-05 09:56:29.7928 [PID=7201 ] [version] get_app_version(): getting app version for WU#606407 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3616) appid:29
2014-06-05 09:56:29.7929 [PID=7201 ] [version] returning cached version: [AV#721]
2014-06-05 09:56:29.7929 [PID=7201 ] [send] est delay 0, skipping deadline check
2014-06-05 09:56:29.7929 [PID=7201 ] [version] get_app_version(): getting app version for WU#606407 (p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3616) appid:29
2014-06-05 09:56:29.7929 [PID=7201 ] [version] returning cached version: [AV#721]
2014-06-05 09:56:29.7929 [PID=7201 ] [send] est delay 0, skipping deadline check
2014-06-05 09:56:29.7974 [PID=7201 ] [send] Sending app_version einsteinbinary_BRP4G 7 134 BRP4G-opencl-ati; projected 34968.78 GFLOPS
2014-06-05 09:56:29.7976 [PID=7201 ] [send] est. duration for WU 606407: unscaled 8.01 scaled 10.96
2014-06-05 09:56:29.7976 [PID=7201 ] [send] [HOST#8143] sending [RESULT#1454943 p2030.20131124.G176.16-01.04.S.b2s0g0.00000_3616_1] (est. dur. 10.96s (0h00m10s95)) (max time 160.14s (0h02m40s14))

Claggy

Holmis
Holmis
Joined: 4 Jan 05
Posts: 89
Credit: 2,104,736
RAC: 0

I tried asking for more tasks

I tried asking for more tasks to my Nvidia GPU and got the following in Boinc's Event log:

[pre]05/06/2014 12:17:53 | Albert@Home | Requesting new tasks for NVIDIA
05/06/2014 12:17:53 | Albert@Home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
05/06/2014 12:17:53 | Albert@Home | [sched_op] NVIDIA work request: 102560.41 seconds; 0.00 devices
05/06/2014 12:17:53 | Albert@Home | [sched_op] intel_gpu work request: 0.00 seconds; 0.00 devices
05/06/2014 12:17:55 | Albert@Home | Scheduler request completed: got 0 new tasks
05/06/2014 12:17:55 | Albert@Home | [sched_op] Server version 703
05/06/2014 12:17:55 | Albert@Home | Project requested delay of 60 seconds
05/06/2014 12:17:55 | Albert@Home | [sched_op] Deferring communication for 00:01:00
05/06/2014 12:17:55 | Albert@Home | [sched_op] Reason: requested by project[/pre]
As you can see there was no reason given for why I didn't receive any tasks.
Next step was checking the server contact log and I found this:

[pre]2014-06-05 10:17:54.8969 [PID=8307 ] [version] Checking plan class 'BRP4G-cuda32-nv301'
2014-06-05 10:17:54.8969 [PID=8307 ] [version] plan_class_spec: parsed project prefs setting 'gpu_util_brp' : true : 0.500000
2014-06-05 10:17:54.8969 [PID=8307 ] [version] [AV#716] daily quota exceeded[/pre]
So the reason was that I've already had my fill for the day.
Checking the Application details for my host gives:

[pre]Binary Radio Pulsar Search (Arecibo, GPU) 1.33 windows_intelx86 (BRP4G-cuda32-nv301)
Number of tasks completed 13
Max tasks per day 45
Number of tasks today 54
Consecutive valid tasks 13
Average processing rate 56.59266205016
Average turnaround time 0.29 days[/pre]
So I'm over the daily quota, but why didn't the scheduler tell me so in the reply to Boinc?

Bernd Machenschalk
Bernd Machenschalk
Administrator
Joined: 15 Oct 04
Posts: 155
Credit: 6,218,130
RAC: 0

I enabled another debug flag

I enabled another debug flag (debug_array) to possibly get a grip on the app selection issue.

This means that the scheduler log excerpts that you see published for your hosts will get even longer. Please don't post these here in all gory detail, these are kept for ~200d on the server for the devs & admins anyway.

BM

BM

Claggy
Claggy
Joined: 29 Dec 06
Posts: 122
Credit: 4,040,969
RAC: 0

I got some of those tasks

Message 79928 in response to message 79927

I got some of those tasks resent:

https://albert.phys.uwm.edu/host_sched_logs/8/8143

Claggy

Holmis
Holmis
Joined: 4 Jan 05
Posts: 89
Credit: 2,104,736
RAC: 0

RE: I enabled another debug

Message 79929 in response to message 79927

Quote:

I enabled another debug flag (debug_array) to possibly get a grip on the app selection issue.

This means that the scheduler log excerpts that you see published for your hosts will get even longer. Please don't post these here in all gory detail, these are kept for ~200d on the server for the devs & admins anyway.

BM


I just made a work request for CPU work and was granted 10 S6CasA tasks and one BRP4 task.
In my Einstein@home prefs the BRP4 search is not selected but Beta-apps are.

Unfortunately Boinc contacted the scheduler again before I could check the server log so I missed it, just wanted to point out that there should be 2 logs at around 15:46 today.

This is the first line from the second contact, the first contact that assigned the CPU tasks should have occurred a few minutes before this one.
2014-06-05 15:46:56.9050 [PID=16227] Request: [USER#xxxxx] [HOST#2267] [IP xxx.xxx.xxx.226] client 7.2.42

Claggy
Claggy
Joined: 29 Dec 06
Posts: 122
Credit: 4,040,969
RAC: 0

Got some of tasks resent

Message 79930 in response to message 79927

Got some of tasks resent again, still the same, tasks are predicted to take 16 seconds, this host hasn't completed it's 11 validations of that app_version yet, so it's using the initial estimate, and not it's app_version APR yet:

Quote:
Binary Radio Pulsar Search (Arecibo, GPU) 1.34 windows_x86_64 (BRP4G-opencl-ati)
Number of tasks completed 7
Max tasks per day 1
Number of tasks today 0
Consecutive valid tasks 0
Average processing rate 61.916362373902
Average turnaround time 0.82 days

Claggy

zombie67 [MM]
zombie67 [MM]
Joined: 10 Oct 06
Posts: 73
Credit: 30,924,459
RAC: 0

RE: RE: If your are going

Message 79931 in response to message 79902

Quote:
Quote:
If your are going to use Dave's random number generator, I leave the project. Some CPU projects have fixed it to number generator of expected and acceptable range, but no GPU project has been successful in that deal. Good luck.

We know all that. The purpose of this test is, very specifically, to test and try out some fixes to CreditNew that some volunteers have spent the last nine months developing.

It would be most helpful if you would remain attached to the project, to generate some baseline data from a good range of hosts.

Albert has been chosen for this task specifically because it's a test project where nothing is expected to work anyway!

At first, I thought this test must be going on with other apps that I am not running, because my Binary Radio Pulsar Search (Arecibo, GPU) tasks were still getting a flat 1000 per. I guess it took a while to kick in. This morning, I can see all of the validated tasks with differing credits awarded. There are a couple with ~500. A couple ~300-400. All the rest range from 90-120 credits.

So, what do we need to do to get this CreditNew problem fixed?

Dublin, California
Team: SETI.USA

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

RE: RE: RE: If your are

Message 79932 in response to message 79931

Quote:
Quote:
Quote:
If your are going to use Dave's random number generator, I leave the project. Some CPU projects have fixed it to number generator of expected and acceptable range, but no GPU project has been successful in that deal. Good luck.

We know all that. The purpose of this test is, very specifically, to test and try out some fixes to CreditNew that some volunteers have spent the last nine months developing.

It would be most helpful if you would remain attached to the project, to generate some baseline data from a good range of hosts.

Albert has been chosen for this task specifically because it's a test project where nothing is expected to work anyway!

At first, I thought this test must be going on with other apps that I am not running, because my Binary Radio Pulsar Search (Arecibo, GPU) tasks were still getting a flat 1000 per. I guess it took a while to kick in. This morning, I can see all of the validated tasks with differing credits awarded. There are a couple with ~500. A couple ~300-400. All the rest range from 90-120 credits.

So, what do we need to do to get this CreditNew problem fixed?


We're still generating the baseline - as you noticed, it took a few attempts to disable the previous fixed credits: now we can see and quantify the scale of the problem. There was another glitch with the CasA (GW) tasks this morning, so they still haven't properly started.

But rest assured, there are people editing away in the background even as I type.

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Holmis
Holmis
Joined: 4 Jan 05
Posts: 89
Credit: 2,104,736
RAC: 0

RE: We're still generating

Message 79933 in response to message 79932

Quote:

We're still generating the baseline - as you noticed, it took a few attempts to disable the previous fixed credits: now we can see and quantify the scale of the problem. There was another glitch with the CasA (GW) tasks this morning, so they still haven't properly started.

But rest assured, there are people editing away in the background even as I type.


So a few of questions about this test of the credit system:

Do we mere mortals need to do anything special or do we just run task and let the wizards take care of things in the background?

Is there something I or any other regular user can do to help and/or speed things up?

Should I/we focus on a special search or run them all?

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

RE: RE: We're still

Message 79934 in response to message 79933

Quote:
Quote:

We're still generating the baseline - as you noticed, it took a few attempts to disable the previous fixed credits: now we can see and quantify the scale of the problem. There was another glitch with the CasA (GW) tasks this morning, so they still haven't properly started.

But rest assured, there are people editing away in the background even as I type.


So a few of questions about this test of the credit system:

Do we mere mortals need to do anything special or do we just run task and let the wizards take care of things in the background?

Is there something I or any other regular user can do to help and/or speed things up?

Should I/we focus on a special search or run them all?


Well, I can only answer as the Sorcerer's Apprentice - I'm following what the wizards are doing, and trying to interpret their Delphic utterances.

The whole CreditNew structure - if you can dignify it as a structure - is basically built round CPU applications, with coprocessors tacked on as an afterthought. So it would perhaps be a good idea - most helpful - to fire through some extra CasA/GW tasks, so the baseline for those catches up after the slow start. But we're just going into a long (3-day) weekend in Germany, so there's no rush. Just keep taking the tablets as usual, and see how dirty the laundry gets. Can anybody beat Zombie for variability? I've seen him get from below 100, to above 10,000, for the par-1000 BRP4G tasks.

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Holmis
Holmis
Joined: 4 Jan 05
Posts: 89
Credit: 2,104,736
RAC: 0

RE: So it would perhaps be

Message 79935 in response to message 79934

Quote:
So it would perhaps be a good idea - most helpful - to fire through some extra CasA/GW tasks, so the baseline for those catches up after the slow start. But we're just going into a long (3-day) weekend in Germany, so there's no rush. Just keep taking the tablets as usual, and see how dirty the laundry gets.


Roger that, will run some extra CasA tasks and then let the server decide.

Quote:
Can anybody beat Zombie for variability?


Well, the server seems to think I've had to much and are now issuing credit between 88.84 - 127.05 per BRP4G task. Wish I could get 10,000+ for a task, would be good for my RAC! =)

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

Having urged everyone to run

Having urged everyone to run GW/CasA over the weekend, I found I couldn't download any myself - and it looks like we're running low on BRP4 too.

But I managed to get a good run of BRP4 validated (and continuing), so I dug out the trusty old graphing tool.

http://i1148.photobucket.com/albums/o562/R_Haselgrove/AlbertCreditNewBRP4.png

That shows the effect and variability of CreditNew in what is - supposedly - the best-case scenario: single-threaded CPU applications.

I can only easily plot my own reporting time on the X-axis, which is far from ideal for spotting trends (many validations come much later) - but it gives a visual indication of what we're up against.

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Bernd Machenschalk
Bernd Machenschalk
Administrator
Joined: 15 Oct 04
Posts: 155
Credit: 6,218,130
RAC: 0

RE: Having urged everyone

Message 79937 in response to message 79936

Quote:
Having urged everyone to run GW/CasA over the weekend, I found I couldn't download any myself - and it looks like we're running low on BRP4 too.

Sorry, S6CasA was still throttled due to the validator issues. Has been fixed. The BRP4 WUG should get new data every 12h, currently there are >3000 BRP4 tasks unsent.

BM

BM

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

I've been asked to dig out

I've been asked to dig out some statistics from that graph I posted earlier.

These are the same six hosts, with a few extra validations since this morning, and the 62.50 flat-rate credit results stripped out.

[pre]Host: 5367 9130 11359 11360 11361 11362
i7-3770 i5-4570 i5 M Q6600 Q6600 Xeon E5320

Credit for BRP4

Maximum 91.08 54.74 54.43 52.32 57.67 52.09
Minimum 40.66 34.40 44.37 40.19 42.77 38.25
Average 50.80 47.13 47.74 46.63 47.38 45.50
Median 49.58 47.29 47.58 46.75 47.41 45.58
Std Dev 6.18 3.23 1.70 1.93 2.28 2.85

Completed 369 169 118 74 136 59
APR 3.55845 3.19964 1.61535 2.33970 2.05929 1.20705[/pre]
I'm not sure I believe that Std Dev, but Excel is insistent.

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

Holmis
Holmis
Joined: 4 Jan 05
Posts: 89
Credit: 2,104,736
RAC: 0

Following Richard's example

Following Richard's example I've put together plots of the credit awarded to host 2267 since the server upgrade.

Plot of credit for BRP4X64, ARP=4.13

Plot of credit for BRP4G, ARP=58.02

Plot of credit for S6CasA, ARP=3.14

And finally if anyones interested here's the Excel document with both the data and plots.

To summaries:
BRP4X64 is all over the place but "always" lower than the fix credit before the upgrade.
BRP4G took a nose dive and is slowly recovering, at least it appears to be going in the right direction.
S6CasA only has 9 validated tasks so cant really tell but seems to be like BRP4X64.

jason_gee
jason_gee
Joined: 4 Jun 14
Posts: 111
Credit: 1,043,639
RAC: 0

Hi All, Thanks very much

Hi All,
Thanks very much guys, That'll he very helpful. Even though we've characterised a lot of the instabilities (and isolated some sources) I'm still surprised, by the CPU applications in particular, to see 3 standard deviations covering from tightest example ~+/-10% (unstable but not all that bad) right up through some ~+/- 37% in the worse examples.

The engineering aspects are there in code, but it makes a big difference to see them in actual numbers,

Thanks again,
Jason

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

Surprised in which direction

Message 79941 in response to message 79940

Surprised in which direction - wider spread than you expected, or narrower?

On the one hand, I think there's evidence in the graphs that the initial spread was wide, but quite quickly narrowed significantly. My guess is that would be repeated every time a project released a new app_version.

On the other hand, I certainly ran my hosts quite intensively on BRP4 over the weekend, and I run them 24/7 with a stable mix of tasks. My results probably insert relatively little instability into the database, and here on a test project with few active participants, I suspect the same applies to other users too. In the rough-and-tumble of a production project, would the variability be greater?

I've switched to concentrate on GW/CasA today, and I'll grab some data from the users who have started on GPU work already.

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

jason_gee
jason_gee
Joined: 4 Jun 14
Posts: 111
Credit: 1,043,639
RAC: 0

RE: Surprised in which

Message 79942 in response to message 79941

Quote:

Surprised in which direction - wider spread than you expected, or narrower?

On the one hand, I think there's evidence in the graphs that the initial spread was wide, but quite quickly narrowed significantly. My guess is that would be repeated every time a project released a new app_version.

On the other hand, I certainly ran my hosts quite intensively on BRP4 over the weekend, and I run them 24/7 with a stable mix of tasks. My results probably insert relatively little instability into the database, and here on a test project with few active participants, I suspect the same applies to other users too. In the rough-and-tumble of a production project, would the variability be greater?

I've switched to concentrate on GW/CasA today, and I'll grab some data from the users who have started on GPU work already.

Surprised in how well the (lackadaisical) response of the system matches predictions of established engineering control systems theory... and that anyone would sample stochastic ('natural world' noisy) data, hook it up to hard logic, and expect a stable system to result.... apparently without consulting a single control systems engineer.

As of now *to me*, the purposes of credit and RAC are for quantifying work, and for comparing against other hosts. In those contexts you might see convergence for your individual machine... but for comparative purposes those numbers, say against my host, are not meaningful on a human level ... They are quite useful, however, in estimating the entropy resulting from bad design, and so what will be required to fix it.

( even though we can probably redefine the terms entropy and chaos to include mashing noise together to make some guesses, I suspect my interpretations of the purpose of credit as a measure of work are long obliterated by the current system, and require extensive correction)

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

I didn't understand a word of

Message 79943 in response to message 79942

I didn't understand a word of that :P (well the words, maybe, but not the sentences)

Preliminary stats now I've got the GW framework set up (very preliminary - only 26 results validated):

[pre]Host: 5367 9130 11359 11360 11361 11362
i7-3770 i5-4570 i5 M Q6600 Q6600 Xeon E5320

Credit for GW-CasA

Maximum 262.35 290.46 0.00 294.04 223.03 330.82
Minimum 214.65 200.24 0.00 259.35 196.81 255.91
Average 237.06 241.14 #DIV/0! 277.81 209.92 295.37
Median 238.77 228.51 #NUM! 280.04 209.92 299.38
Std Dev 15.45 30.16 #DIV/0! 17.45 18.54 37.62[/pre]

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg

jason_gee
jason_gee
Joined: 4 Jun 14
Posts: 111
Credit: 1,043,639
RAC: 0

RE: RE: Surprised in

Message 79944 in response to message 79942

Quote:
Quote:

Surprised in which direction - wider spread than you expected, or narrower?

On the one hand, I think there's evidence in the graphs that the initial spread was wide, but quite quickly narrowed significantly. My guess is that would be repeated every time a project released a new app_version.

On the other hand, I certainly ran my hosts quite intensively on BRP4 over the weekend, and I run them 24/7 with a stable mix of tasks. My results probably insert relatively little instability into the database, and here on a test project with few active participants, I suspect the same applies to other users too. In the rough-and-tumble of a production project, would the variability be greater?

I've switched to concentrate on GW/CasA today, and I'll grab some data from the users who have started on GPU work already.

Surprised in how well the (lackadaisical) response of the system matches predictions of established engineering control systems theory... and that anyone would sample stochastic ('natural world' noisy) data, hook it up to hard logic, and expect a stable system to result.... apparently without consulting a single control systems engineer.

As of now *to me*, the purposes of credit and RAC are for quantifying work, and for comparing against other hosts. In those contexts you might see convergence for your individual machine... but for comparative purposes those numbers, say against my host, are not meaningful on a human level ... They are quite useful, however, in estimating the entropy resulting from bad design, and so what will be required to fix it.

( even though we can probably redefine the terms entropy and chaos to include mashing noise together to make some guesses, I suspect my interpretations of the purpose of credit as a measure of work are long obliterated by the current system, and require extensive correction)

I'll add, after private communications, that apparently there was some level of consultation during design &/or implementation of the system.... but that apparently that credit and work estimates would be intimately connected was not raised.

I accept that the interpretations I have that both Credit and Time estimates are abstract 'human tools' separate from the computation, is probably a relatively novel contribution to that overall picture, and so that stability in perceived usefulness of those figures may not have been considered a big issue until well after the chaos had arisen (on other projects)

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

jason_gee
jason_gee
Joined: 4 Jun 14
Posts: 111
Credit: 1,043,639
RAC: 0

RE: I didn't understand a

Message 79945 in response to message 79943

Quote:

I didn't understand a word of that :P (well the words, maybe, but not the sentences)

Preliminary stats now I've got the GW framework set up (very preliminary - only 26 results validated):

[pre]Host: 5367 9130 11359 11360 11361 11362
i7-3770 i5-4570 i5 M Q6600 Q6600 Xeon E5320

Credit for GW-CasA

Maximum 262.35 290.46 0.00 294.04 223.03 330.82
Minimum 214.65 200.24 0.00 259.35 196.81 255.91
Average 237.06 241.14 #DIV/0! 277.81 209.92 295.37
Median 238.77 228.51 #NUM! 280.04 209.92 299.38
Std Dev 15.45 30.16 #DIV/0! 17.45 18.54 37.62[/pre]

Yep, there's that +/- 30% ( std dev x 3 )... let's see how she settles :)

Quote:
Quote:
I didn't understand...
... and that anyone would sample stochastic ('natural world' noisy) data, hook it up to hard logic, and expect a stable system to result....

i.e. Garbage in -> Garbage out.

http://en.wikipedia.org/wiki/Garbage_in,_garbage_out

[Edit:] choice new sig:

Quote:
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
—Charles Babbage, Passages from the Life of a Philosopher

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 143
Credit: 5,409,572
RAC: 0

Oh, you're going to love this

Oh, you're going to love this one

[pre] Jason Holmis Claggy Zombie Zombie (Mac)
Host: 11363 2267 9008 6490 6109
GTX 780 GTX 660 GT 650M TITAN GTX 680MX

Credit for BRP4G (GPU)

Maximum 1170.48 1036.86 10239.0 1654.85 11847.50
Minimum 115.82 88.84 153.90 25.79 94.88
Average 548.33 463.98 3875.88 874.96 2256.70
Median 468.80 390.21 2977.38 865.33 1591.80
Std Dev 431.90 268.52 2873.26 362.30 2395.61[/pre]
I'll upload a graph after lunch, when my monitor has cooled down and I've stopped laughing.

I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.

http://www.boincsynergy.com/images/stats/comb-3475.jpg