[nylug-talk] x86-64 for 1GiB+ programs (especially in VMs) -- WAS: LOWMEM vs. HIGHMEM performance advantage?

[Peter C. Norton]

On Tue, Apr 22, 2008 at 04:28:19PM -0400, jh wrote:
> But... the application does shell out to do certain graphic-related 
> operations. It is hard to say how much time is spent in the python app, 
> and how much is spent in something it shells out to.

Get the developer to help you profile it with the python hotshot
module (compiled in c, less impactful when the app starts
thrashing). That'll help identify for you where you're spending time
in the code.

> >>> d) I guarantee you it has nothing to do with page tables. But original
> >>> poster can't figure out how to diagnose and troubleshoot performance
> >>> (not even vmstat data, much less using oprofile) - so we end up
> >>> conjecturing about page tables and other crap.
> >> What, exactly, would either oprofile or vmstat reveal? I'm not sure that
> >> this comment was entirely helpful.
> > vmstat would tell if you are cpu-pegged or disk-pegged. it'd also show 
> > whether (possibly) on a smaller dataset, entire dataset fits in your ram 
> > buffer cache, whereas on a larger dataset it requires disk reads/writes.
> > 
> > ie do vmstat during the run on "small" dataset and "large" dataset. post 
> > it.
> 
> Here's vmstat running now (large test set) - will run it again with 
> smaller tomorrow.
> 
> procs -----------memory----------
>   r  b   swpd   free   buff  cache
>   1  0    156  68056     20 1121232
>
> ---swap-- -----io---- -system-- ----cpu----
>   si   so    bi    bo   in   cs us sy id wa
>     0    0   893    65  155  231 13 27 54  5

Vmstat is better used to provide a time series sample of what's going
on. Eg using "vmstat -a 1" would give you more more info because you
can see what is being done second-to-second when the process is busy.

-Peter

-- 
The 5 year plan:
In five years we'll make up another plan.
Or just re-use this one.