Project server code update
Submitted on 2 Jun 2014 8:59:18 UTC
The project will be taken down in about an hour to perform an update of the BOINC server code. Ideally you shouldn't notice anything, but usually the world isn't ideal. See you again on the other side.
Language
Copyright © 2024 Einstein@Home. All rights reserved.
Comments
RE: Ok, so it is
)
I agree, though 'true' averages can be fine and established quickly. 10% of the marketing flops should be near enough ballpark for a new host to get it going... which scaling or combination of scalings, is breaking the initial GPU estimate is a mystery to me at the moment, though I have no doubt it'll be much easier to spot with new hostIds in phase 2 when all the averages get replaced with actively controlled dampers.
Pass1 (starting point)
CPU coarse scaling correction
-- look for unexpected effects (e.g. are the GPU apps completely unconnected as expected here)
Pass2 (replace sampled averages with controllers, actively damped)
-- look for GPU scaling errors, particularly new hostids / apps
Pass3
-- GPU scaling logic refinement if needed (probably is)
Got enough to draw up something for passes one and two, will get a coffee & a break, then get to some documenting and coding
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: RE: Ok, so it is
)
Didn't I just extensively calculate that 1% is more like it?!
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
RE: RE: RE: Ok, so it
)
Yes, I'm talking from the intent written in code and comments at this point, not what it's actually achieving. If I were to comment on what it's actually achieving, I would have to invent some more words
[Edit:] something like "Bandaids on top of fudge factors applied to magic numbers" comes to mind, though doesn't quite capture it.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
um, no. It's achieving
)
um, no.
It's achieving chaos. :D
Chaos theory tells us that that means that at least 3 coupled differential equations are in play :) 'three is chaos'.
To get the system into a steady-state, means either uncoupling or stabilising sub-equations.
From a mathematical pov this is quite fascinating.
I doubt you'd as easily produce a chaotic system if you were actually trying to get one. :D
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
...
)
...
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: um, no. It's achieving
)
Yes, reminds me of a tongue in cheek comment I made suggesting the climate people might be interested in this... oh well
Yes we can, after poking the CPU app scale in pass 1, in pass 2 place the two scaling equations (scheduler & validation) into separate time domains so they stop interacting in weird ways, and damp the third, which is stochastic non-linear non-deterministic ( elapsed time based samples), then look for more logic issues.
I'm pretty convinced that there is a logic breakage there for new GPU hosts, but can't put my finger on it yet. It'll fall out during the first 2 passes I reckon.
[Edit:] I see the boinc messageboard echo in here works fine :)
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: [Edit:] I see the boinc
)
Beg your pardon?
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
RE: RE: [Edit:] I see the
)
Double posts seem to happen a lot (to me anyway) [not this time]
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: RE: RE: [Edit:] I
)
Your resident moderator(s) will probbaly be pleased if you red-x them for hiding. That's tongue in cheek. For once it's not me getting those reports :D
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
RE: RE: RE: RE: [Edit
)
Done!
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
OK, so it seems that my
)
OK, so it seems that my 2-hour dinner break was largely taken up by a discussion of initial for new hosts. That's fine: I think it's one of the most overdue refinements of CreditNew, so let's run with it for a while.
In the CPU case, do we agree that the Whetstone benchmark understates the scientific performance by some ratio R, where R is greater than one, and rises to somewhere around 3 or 4 in the AVX case?
And in the GPU case, do we agree that marketing 'peak FLOPS' overstates the scientific performance by some ratio r, where r is less than one, and may take a value of 0.1 or 0.01?
The question becomes, how do we choose values of R and r which are "good enough" to get a new cruncher started, without frightening him/her with infeasible runtime estimates, or risking his/her tasks being aborted for 'TIME_LIMIT_EXCEEDED'?
Jason has proposed a function MIN(SIMD_capability(CPU), SIMD_optimisation(app_ver)) for the CPU case. We need a curve for that...
Nobody seems to have studied the GPU case, but surely r is knowable? (A 'known unknown', on the Rumsfeld grid).
So, is r the same for AMD, NVIDIA and Intel? Is r the same for each major hardware version from each of the manufacturers? Is r the same for all project applications? Do we need to scale r by GPU utilization factor? We need a curve for that, too...
And once we have R and r "good enough" to get us started, it becomes a non-problem: provided they let us complete 11 tasks, APR takes over and seems adequate. Jason tells us that properly damped averages will take over from administrator responsibility for outlier management.
And now we need to talk about credit...
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg
Yes, for initial scaling, new
)
Yes, for initial scaling, new CPU host and/or app, we're talking a fairly simple matter of providing a coarse 'improvement'. The objectives for that as an initial test will be just to use existing information where available, such as Android's SIMD aware bench that was snuck in without the corresponding server side awareness. x86 (from FPU through AVX) will actually be slightly simpler since at that point the mechanism only needs relatively coarse numbers (explained later), and we have no SIMD aware bench to confuse the picture right now.
Picturing the Android case in the context of our recent SIMAP adventure makes things a bit simpler. For that Android case, a flag 'client_bench_is_simd_aware' would be set (by platform and cpu feature information readily available in the scheduler request. The missing part of that is a corresponding plan class spec setting defining the max supported simd vector length by the app. Correcting that gives the coarse (pre efficiency scaling) information necessary to put the raw pfc correction (think DCF) nack in the ballpark where averages can cope, the other key component being returned elapsed. Fine correction (for efficiency & utilisation) is for the averaging portion, and then adjusts the host_scale, then subsequent normalisation.
A take away from this point is that precise original project estimates are not required, any more than within some reasonable theoretically based 'minimal' number of operations estimate. If those were 'perfect', utilisation was 100% and users never used their hosts except for crunching, host_scale (think DCF again) would be 1 and not vary. That's obviously not a practical design goal from any standpoint. ... However these scales are registering as less than 1, which simply means the wrong numbers are being used ( and what's wrong is not , so far, the project supplied number of operations)
For x86 Since we (the scheduler) know the client version and platform, as well as the CPU features and application capabilities, again there's enough information to set flags draft a coarse correction. (until such time as a SIMD aware bench is crafted for x86, an easy server side switch to throw when appropriate)
That quite coarse correction is a major stabilising one simply by stopping claims of less than the minimum possible to do the tasks getting in. Those are the key claims that mess up several stages. Since such low claims do get in at the moment, and break the minimumal # of operations, fixing that in coarse fashion is the first step.
WHere SIMD bench is not available, *reasonable* SIMD correction to keep things in the right order of magnitude need only be as rough as existing Boinc Whetstone plus 50% per power of two of the vector length (x1.5^(logbase2(used_simd_vector_length). That gives coarse correction for a common example SSE-SSE4.2 being on average about 2.25x faster than a pure FPU implementation.
At this point the 'ideal' (not quite as implemented) flow of estimate, and where domain knowledge comes into play becomes the key. Localising Domain specific knowledge is one software engineering & control systems concept Boinc software has problems with throughout:
- Original task estimates: Theoretically derived, very coarse, never changes (unless the work does). has no knowledge of applications or hardware (and should never have). They only know about theoretical number of operations required to do some work (This bit *appears* fine at this stage, pending thorough checks for obvious omission of key computing steps
- Global scale (a bit less coarse, should change very slowly, on the order of 1000's of tasks): Only knows about a bunch of applications, and 'normalises' the performance scales (currently against the 'most efficient' app available for given tasks.
- Host Scale: (Think Server Side DCF) Has substantial application and host domain specific knowledge available that is currently misused. SHould be responsive to hardware, application and utilisation change without falling over.
(change on the order of 10's of reports
- Client side: Should be most responsive, is currently disabled &/or broken, probably on the misguided idea that this was upsetting the earlier steps.
What currently upsets the system, making stage 1 look worse than it is, is that steps 2 through 4 are seriously under-engineered and spaghettified. Control domains in stages 2 & 3 in particular overlap and interact, and process irrelevant noise that could be easily conditioned.
GPU has indeed been studied as well, but as we come from seti-land, step 2 cross application version scaling clouds the issue. The extent to which this cross-scaling is intertwined here is not yet clear, but GPU will require the reverse process to CPU correction.. stabilise first then coarse correct second. That's because the hardware is more diverse and ultimately (in the seti case at least) relies on the CPU end being stable.
Note that at no stage do any of these ammendments/refinements explicitly adjust the credit. They absorb design limitations inherent in the underdesigned - pverimplemented control system.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
right, move some info over to
)
right, move some info over to th Wiki then...
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
RE: right, move some info
)
A lot of that's being documented right now, in an experimentation format. Copy/paste for the sakes of verifying/discussing might help. The general concepts there give way to pretty specific adjustments from here on.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: RE: right, move some
)
If you write it there is a tendency for the faces of the general public to go blank, while their collective eyes glaze over.
You can check out mine, it's short.
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
RE: RE: RE: right, move
)
I don't see it on the wiki yet ?
Mine's for the purpose of ensuring we don't all go off on Random directions just now. We established enough details to make pretty rapid progress, and stop and adjust at key points.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: I don't see it on the
)
You need to follow the 'Day zero observations' Link from the 'Observations from live run at Albert' section
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
RE: RE: I don't see it
)
Thanks, perusing with a short break from my dictophone.
[Edit:] Yep should be able to tie objectives on the passes in nicely to that. Yeah I was aware of the logarithmic scales hiding a multitude of the sins, so it's great to see that awareness reflected in the non-log representation. Nose will be back to the grindstone a bit later.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: [Edit:] Yep should be
)
Did I mention faces going blank and eyes glazing over?
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
RE: RE: [Edit:] Yep
)
At this stage the documentation is for our benefit. For many others the proof is in the pudding, and how we get there of little importance to them...
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: RE: RE: [Edit:] Yep
)
Yes, but "our benefit" includes the wider benefit of getting code changes accepted into the master BOINC codebase. Hand-waving demonstrations of "but look - it works" are simply descending to the level of the lowest common denominator. Recording and documenting the steps which led to the conclusions being implemented in code will help - even if only by giving us the warm fuzzies that come from occupying the moral high ground.
I've started documenting the wingmates who co-validate my 'high outlier' credit scores, but no pattern has emerged yet.
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg
RE: RE: RE: [Edit:] Yep
)
I don;t queston the importance of documentation.
It's just that you usuallt manage to talk at a level everybody else (including me) finds hard to understand. You ain't talkinf to fellow engeneers.
Try the 'explain to an intelligent 16 year old' approach. Else you end up explaining everything twice.
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
RE: Yes, but "our benefit"
)
Only what you can explain you have understood properly.
Patterns in chaos? Yes there are some, but you could as easily try to fimd to identical snowflakes.
I'm intrigued by the high outliers too.
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
RE: RE: RE: RE: [Edit
)
How is it you're managing to criticise a document you've never seen ? FWIW my elderly mother understands it so far, even having to ask 'what's a controller?'
I've got absolutely no intentions of condescending to you or anyone else on basic control systems theory within the experiment, though I can certainly point out some background reading.
If I must, I'll draw an analogy. When you drive your car you're a 'controller'. If you drive your car like Boinc drives estimates and credit, then according to that data you have no business being behind the wheel of a car at all.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: RE: RE: RE: [Edit
)
Thanks :) yes it could be simply artefacts of the instabilities (and so the repeated mentions of chaos). It'll be a good marker to see if we squished all the gremlins out.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: I've started
)
Validated with different app versions, like x86 on one and x64 on another?
Been running a number of CPU hosts on and off for months, mostly Arm, before the upgrade the best app, ie Neon app was only sent to my Arm hosts unless I aborted tasks to drive the Max tasks per day down low enough,
(My 2012 HTC One S and the 1.43 Neon app only produced validate errors, and the scheduler wouldn't send the 1.43 VFP app unless I did that, it completed 5 of those O.K),
the 1.44 Neon app is good through, and has completed over 200 hundred now with hardly a problem, no more VFP tasks have been sent.
The two Parallellas were only doing Neon tasks before hand, afterwards they started picking up non Neon tasks, they are at 11 and 10 validations so far for non Neon, and 21 and 23 for Neon,
The 2012 Nexus 7 had done only Neon tasks beforehand, afterwards it's picked up VFP tasks, done 8 of those against 37 of Neon, the VFP app is about half the speed of the Neon app,
The C2D T8100 Linux x64 host before hand only picked up x64 BRP tasks, it's completed 271 so far, SSE2 x86 tasks have never been sent,
On the HD7770 it picked up 1.34 windows_x86_64 (BRP4G-opencl-ati) and 1.34 windows_intelx86 (BRP4G-opencl-ati) work, these are different apps, with different file sizes, the x86_x64 app had some validations from beforehand,
afterwards they were failing with max time exceeded errors for a few days, the x86 work got sent when the x64 max tasks per day got too low,
since the x64 tasks got a reasonable speed estimate their tasks complete O.K, x86 tasks haven't been sent again, I have no idea which app is fastest,
looks as if there are scheduler differences between sending CPU and GPU apps, I would have expected some x86 work to be sent.
It's similar with the Perseus Arm Survey, I've had work from 1.39 windows_x86_64 (BRP5-opencl-ati) and 1.39 windows_intelx86 (BRP5-opencl-ati), the x64 app has some validations, the x86 none,
while you can't tell the difference from the tasks page, Boinc Manager shows different duration estimates for the two, 33secs for x64, 20secs for x86, stderr.txt doesn't seem to tell them apart.
CreditNew seems to use different calculations depending whether an app is above or below a sample level, could it be that one app version is above the sample level and the other isn't?
Claggy
RE: ... CreditNew seems to
)
Yes there are two levels above initial, being numbers of tasks validated. For host_scale (host app version) it's >10 validations, which should then switch from whetstone*elapsed related to PFC 'averages' (scaling kicks in). For the global scaling of app versions, it's >100 validations total from all hosts, which is the cross scaling point between all app versions. Both would make steps.
He's put some weights on those averages (to attempt) to reduce their (noisy) effects. That's the bit that's the same as a chip in a 5$ CD Player & doesn't work so well stability wise.
Late coming hosts (there's been >100 by others, <=10 locally) to the app *could* be getting their whetstone*elapsed (peak_flops*elapsed for the GPU case) value global scaled on top, that'd where it gets all spaghetified and the noise buries a lot.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: Validated with
)
Nice thought, and worth checking. But 11 of my 15 'high scores' have been for Windows / NVidia, and every one of them has been for the same (BRP4G-cuda32-nv301) application that I'm running.
Mind you, five of those 11 Windows 'high scores' have been against host 8733: scanning that machine's task list by eyeball only, one does seem to have a significantly higher chance of hitting the jackpot if paired with that host. Can anybody spot anything special in the published metrics for that host? I can't.
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg
RE: RE: Validated with
)
Having a squizz now.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
While you do that, here's the
)
While you do that, here's the full list of pairings against my host 5367 which have given me a credit score above 2,500: 15 out of 86 validations so far.
HostID owner platform device
11455 mikey Win7/64 AMD ATI Radeon HD 5800/5900 series (Cypress/Hemlock) (1024MB) driver: 1.4.1848 OpenCL: 1.2
10100 AMG11041963 Win7/64 "GeForce GTS 450" (192 CUDA cores / 606.34 GFLOPS)
8733 Anonymous Win7/32 "GeForce 8800 GTX" (128 CUDA cores / 518.40 GFLOPS)
8733 Anonymous Win7/32 "GeForce 8800 GTX" (128 CUDA cores / 518.40 GFLOPS)
6517 Cazamarcianos Win7/32 "GeForce G210" (16 CUDA cores / 67.20 GFLOPS)
7157 Andrew Dicker Darwin "GeForce GTX 680MX" (0 CUDA cores / 0.00 GFLOPS) (Darwin)
3719 doug Darwin AMD ATI Radeon HD 5770 (1024MB) OpenCL: 1.1 (Darwin: shows errors in stderr for task 1495019)
11418 Lion Win Server 2008 AMD Bonaire (2048MB) OpenCL: 1.2
3637 B Johansson Win7/64 "GeForce GTX 760" (0 CUDA cores / 0.00 GFLOPS)
11189 Jean-Emmanuel BUISSON Win7/64 "GeForce GTX 580" (512 CUDA cores / 1581.06 GFLOPS)
8733 Anonymous Win7/32 "GeForce 8800 GTX" (128 CUDA cores / 518.40 GFLOPS)
8733 Anonymous Win7/32 "GeForce 8800 GTX" (128 CUDA cores / 518.40 GFLOPS)
1463 Anonymous WinXP/32 "GeForce 9600 GT" (64 CUDA cores / 336.00 GFLOPS)
11189 Jean-Emmanuel BUISSON Win7/64 "GeForce GTX 580" (512 CUDA cores / 1581.06 GFLOPS)
8733 Anonymous Win7/32 "GeForce 8800 GTX" (128 CUDA cores / 518.40 GFLOPS)
Whenever possible, I've taken the "device" string from the stderr report by the BRP4G app: where that is too sparse to be meaningful, I've taken it from the host details instead.
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg
RE: Whenever possible, I've
)
Thanks. It's looking like a fairly complex (chaotic) interaction.
Do you still have one of those 8800/9800 GPUs knocking about to do a test ? EVen better if you can drop its clock rate without the client picking up a change in GFLOPS (marketing peak_flops).
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: RE: Whenever
)
I've still got them, but they're all dismounted and lying on the bench now. I've also got a dead host that didn't survive the transplant, but I don't think I have a realistic chance of reviving that. Pity - that was a nice Q9300 (but the GTX 470 is running safely in another host).
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg
RE: RE: RE: Whenever
)
That's alright, that would be desperate measures if a small mind experiment fails to shed light [I could potential downclock my GPU, or dig out old 9600GSOs if they have enough VRAM). working on the mind experiment first.
Initial state (WU issue):
-task is issued to some 'converged' (i.e. well past 10 validations) host with an older GPU (8800GTX) running one task at a time
- task is issued to Richard's host with 670's (also converged), 2x tasks at a time
- tasks process
...
8800GTX gets about 33 GFLOPs with the task ( Pfc ~0.05 )
Richard's gets about 93 GFlops ( Pfc ~ ? Richard what's Boinc say your marketing peak flops is ?)
...
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: CreditNew seems to use
)
The crucial code seems to start by calling 'assign_credit_set' at
http://boinc.berkeley.edu/gitweb/?p=boinc-v2.git;a=blob;f=sched/credit.cpp;h=93c554ebd3ffb0b9d3e505a214aae8bc85cef787;hb=HEAD#l877
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg
RE: RE: CreditNew seems
)
yeah, that's the back end part after [report, during] validation and fits in the mind experiment we're working on as the last stage --> averages the claims between the two hosts
[Edit:]
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: RE: RE: CreditNew
)
I'm beginning to wonder whether it's going to be worth asking Bernd (when he's back from holiday, next week) to do a snapshot database dump of the two vectors
881 vector& app_versions,
882 vector& host_app_versions,
(database tables)
for us to pore over.
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg
RE: RE: RE: RE: Credi
)
Could be. I think this experiment will shed some light though. What's those Boinc marketing peak_flops for the 670's ?
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: [Edit:] RE: ...Richar
)
05-Jun-2014 19:23:54 [---] Starting BOINC client version 7.3.19 for windows_x86_64
05-Jun-2014 19:23:54 [---] log flags: file_xfer, sched_ops, task, coproc_debug, sched_op_debug, work_fetch_debug
05-Jun-2014 19:23:54 [---] Libraries: libcurl/7.33.0 OpenSSL/1.0.1g zlib/1.2.8
05-Jun-2014 19:23:54 [---] Data directory: D:\BOINCdata
05-Jun-2014 19:23:54 [---] [coproc] launching child process at D:\BOINC\boinc.exe
05-Jun-2014 19:23:54 [---] [coproc] relative to directory D:\BOINCdata
05-Jun-2014 19:23:54 [---] [coproc] with data directory "D:\BOINCdata"
05-Jun-2014 19:23:54 [---] CUDA: NVIDIA GPU 0: GeForce GTX 670 (driver version 337.88, CUDA version 6.0, compute capability 3.0, 2048MB, 1885MB available, 2915 GFLOPS peak)
05-Jun-2014 19:23:54 [---] CUDA: NVIDIA GPU 1: GeForce GTX 670 (driver version 337.88, CUDA version 6.0, compute capability 3.0, 2048MB, 1958MB available, 2915 GFLOPS peak)
05-Jun-2014 19:23:54 [---] OpenCL: NVIDIA GPU 0: GeForce GTX 670 (driver version 337.88, device version OpenCL 1.1 CUDA, 2048MB, 1885MB available, 2915 GFLOPS peak)
05-Jun-2014 19:23:54 [---] OpenCL: NVIDIA GPU 1: GeForce GTX 670 (driver version 337.88, device version OpenCL 1.1 CUDA, 2048MB, 1958MB available, 2915 GFLOPS peak)
05-Jun-2014 19:23:54 [---] OpenCL: Intel GPU 0: Intel(R) HD Graphics 4000 (driver version 9.18.10.3257, device version OpenCL 1.2, 1240MB, 1240MB available, 45 GFLOPS peak)
05-Jun-2014 19:23:54 [---] OpenCL CPU: Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz (OpenCL driver vendor: Intel(R) Corporation, driver version 1.2, device version OpenCL 1.2 (Build 66956))
These Alberts are running exclusively - two-up - on device 0: device 1 is permanently assigned to GPUGrid.
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg
Thanks: RE: Initial state
)
Thanks:
8800GTX gets about 33 GFLOPs (from APR) give or take with the task ( Pfc ~0.06 )
Richard's 670 gets about 93 GFlops, give or take (APR presumably after ~2x slowdown) , ( Pfc ~0.03)
'claim' gets averaged across the wingmen (Eric's change IIRC, it used to select the minimum) so the ratio 0.06/0.3 (=2x , average .045/.03= 1.5x) would appear indicate a course credit scale entering. Then add the known +/- 30%+ variation in both directions. [sheer chance]
If along the right lines, then a few people with high end GPUs should pack as many tasks as possible into VRAM, run as slow as possible. Whenever an old GPU wingman that could only run 1 task at a time is encountered, then there should be a high likelihood of jackpot.... Unless everyone does it and overall credit gets scaled down, in which case the antique cruncher's credit would drop.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: Richard's 670 gets
)
You're looking at a very old app_version.
The current value is 68.88 GFLOPS, from the very last entry in the application details list (Number of tasks today 20) ;)
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg
RE: ... Then add the known
)
Award from that case should then be anyway from about 1.5x=30%=1x , through to 1.5x1.3 = 1.95x, on average, ignoring noise.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: RE: Richard's 670
)
That would be worse, Jackpot :)
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: RE: RE: Richard's
)
So you're saying that a host which has a very low actual throughput, relative to its marketing rating, will 'claim high' for credit?
Given that cuda32 will run slow-ish on Kepler hardware, does that imply we should start up a cottage industry for de-optimising GPU apps, so they earn more gollum points per task?
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg
RE: RE: RE: RE: Richa
)
Absolutely on both counts. while the jackpot scales are well above 1x on average for multi-tasking at least. That correlates with Eric's observations that the system appears to penalise everyone for optimisation on a holistic level. You have al lthe noise and natural usage variation clouding the issues, along with the improper scaling on CPU apps dragging it all down where connected, but in essence it all says that Boinc just gets confused when it sees computers that can do multitasking and/or have situations too far outside the 'average'. That's the key characteristic that it's a control system issue, as opposed to a statistical one.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: ...That's the key
)
I'd like to clarify that in light of Carola's request for explaining things in more layman's terms.
It's been proven as you catch a ball, your brain doesn't do statistics for the control of your arms. It does a number of differential (calculus) equations to compute the trajectories, in parallel, in real time.
i.e. statistics is a modelling tool, not a control system. Control systems deal on a case-by-case basis, statistics via probabilities over a large population.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: So you're saying that a
)
My HD7770 against another HD7770 (3,215):
https://albertathome.org/workunit/620885
My HD7770 against another HD7770 (4,555):
https://albertathome.org/workunit/618068
against a HD 7500/7600/8500/8600 series (2,927):
https://albertathome.org/workunit/620828
against a HD 5800/5900 series (2,897):
https://albertathome.org/workunit/620875
against a HD 6900 series (3,409):
https://albertathome.org/workunit/619539
against a GeForce G210 (3,218):
https://albertathome.org/workunit/620250
against a 8800GTX (3,013):
https://albertathome.org/workunit/619497
against a 8800GTX (4,890):
https://albertathome.org/workunit/617804
against a 9600 GT (3,525):
https://albertathome.org/workunit/618083
against a 9600 GT (3,258):
https://albertathome.org/workunit/618072
against a 9600 GT (3,374):
https://albertathome.org/workunit/618075
against a 9600 GT (3,441):
https://albertathome.org/workunit/618080
against a NVS 4200M (4,598):
https://albertathome.org/workunit/606864
against a GT 555M (4,229):
https://albertathome.org/workunit/612309
against a GTX 670M (3,388):
https://albertathome.org/workunit/617797
against a GTX 680 (3,363)
https://albertathome.org/workunit/617769
Against Richard's GTX670 (all around 2400):
https://albertathome.org/workunit/620884
https://albertathome.org/workunit/620851
https://albertathome.org/workunit/620495
https://albertathome.org/workunit/620346
I guess AMD's, legacy NV's, and modern mobile NV's have a relative high flops to their actual throughput.
Claggy
RE: I guess AMD's, legacy
)
Absolutely, you got it. With GeForce line then the faster the GPU then fixed system latencies dominate to a greater degree ( As I understand both AMD and NV are affected, probably Intel too without past history). Proper latency hiding techniques are most likely a disruptive technology in that sense. That'll mess with the stats some more as the developer world comes to grips with it. It's been a long time that similar CPU multithreading's been around though, and a lot of devs can't deal with that today.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
OK, we have a working theory
)
OK, we have a working theory for the pseudo-random outliers, subject to wider checking tomorrow (after shut-eye) across my whole gamut of wingmates.
Next up: the gradual upward drift in credit across the population as a whole?
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg
RE: OK, we have a working
)
Will have to stew on that one for sure :). Perhaps look at the deadlines assigned to older CPUs (I presume far from expired ?). Since their Boinc Whetstones would be closer to their actual throughput, they should gradually roll in late to the party and cause a levelling. Fast, early, hosts by contrast drive things down by virtue of saturating the early statistics. The efefcts of that may be tempered here, by the app being resitricted to SSE2 (? like seti's astropulse), so the extreme latecomers would be SSE amd FPU (if available, e.g. Pentium 1/II. android with no vfp or NEON)
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
RE: RE: OK, we have a
)
I've started on the process of retrieving 'time of validation' (later of [self, wingmate]), and wingmate data for non-lottery-winners. Will take a while, to guard against RSI.
I didn't want to spam the boards with my stats - just milestone theads - but apparently signatures are no longer optional. Follow the link if you're interested.
http://www.boincsynergy.com/images/stats/comb-3475.jpg