[mythtv] [mythtv-commits] Ticket #2782: automatic simultaneous jobs scaling

[Michael T. Dean]

On 12/07/2006 02:25 PM, osma ahvenlampi wrote:
> On 12/7/06, Chris Pinkham <cpinkham at bc2va.org> wrote:
>   
>> cards now, this is more of a possibility, but there are other things that
>> we need to consider than just CPU.   Just because my system can run X jobs
>> before the CPU usage gets above Y% doesn't mean I want to run X jobs because
>> that will affect things like storage, potentially network throughput, etc..
>>     
> The way I wrote the patch, it will only start a job if 25% of CPU is
> "idle". That 25% is hardcoded because not only did I want to make this
> automatic, I also really wanted not to introduce any more config
> options (believing that software should "do the right thing right"
> without configuring). 25% should also be 1) easily enough for
> recording and playback, especially since the jobs will be run niced
> anyway, and 2) actually relatively rare, especially on a 1-2 core
> machine since most candidate jobs would consume all resources
> available.
>
> It also considers ONLY time reported truly "idle", ie. not used by
> sys, user, nice OR iowait,
>   

Guess I'll be disabling this if it goes in.

My master backend:
$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- 
----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy 
id wa
 4  0  14788   5864   1816 252752    0    0   100   112   82   53 99  1  
0  0

My slave backend:
$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- 
----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy 
id wa
 1  0     48  50296  42792 339832    0    0    49   107   77   10 99  1  
0  0

My dedicated frontend (with mythjobqueue):
(not powered on and I'm 2000 miles away from the power button, but its 
idle value will say "0")

The idle value /never/ changes on any of the hosts, thanks to a little 
program I call AthWarm (most call SETI/BOINC).  :)

No problem, though, I wouldn't mind disabling job scaling.  (Now, 
disabling BOINC, OTOH, will not happen :).

>> Users will probably always prefer a late commercial flag or transcode rather
>> than a corrupted recording.
>>     
> This has never been an issue for me, but I use DVB receivers instead
> of framegrabbers.
>   

This will happen even without encoding video (i.e. even with DVB or 
HDTV) if I/O wait becomes "critical".

Mike


My /proc/stat from my master backend:

cpu  838779 36201503 208958 10015 1537 16826 73892 0
cpu0 838779 36201503 208958 10015 1537 16826 73892 0
intr 202552568 93372945 8 0 1 1 1 0 1 0 0 1 1 1 0 1220376 3550776 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 366394 0 0 0 0 0 0 0 99468785 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 4573277 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0
ctxt 148767913
btime 1165165447
processes 8462
procs_running 2
procs_blocked 0

(only the I/O wait time is lower than idle time, showing that I don't 
let my idle process get its share of the CPU--I'm pretty sure it will 
never "finish" on my system :)