[nylug-talk] x86-64 for 1GiB+ programs (especially in VMs) -- WAS: LOWMEM vs. HIGHMEM performance advantage?

[Alex Pilosov]

On Tue, 22 Apr 2008, Bryan J. Smith wrote:

> I think these statements sum up the "failure to communicate" ...  ;)
> 
> 
> Alex Pilosov wrote:  
> > And I've debunked this.
> > a) If you have a program that uses >3GB you have no choice but to do that.
> > b) If you have a program that uses <3GB, you shouldn't.
> 
> And ...
> 
> > the original poster's issue is either
> > a) motherboard that doesn't set up MTTR right
> > b) some userland application stupidity (O(N^2) algorithm or somesuch)
> > c) some swap-related stupidity
> > d) I guarantee you it has nothing to do with page tables. But original
> > poster can't figure out how to diagnose and troubleshoot performance
> > (not even vmstat data, much less using oprofile) - so we end up
> > conjecturing about page tables and other crap.
> 
> Plus ...
> 
> > there is nothing. nothing. nothing. "far" about 1GB. Nothing.
> > Nothing.  Argh. There is no need to attend to page tables.
> 
> It's obvious you have no comprehension of how the i686 kernel works,
> especially when the kernel starts using the processor page tables
> excessively for unnecessary memory management for the limited i686
> memory models.  There is no "magic barrier" for a single program or
> multiple programs at 1GiB or 3GiB at all, that is just a farce, when it
> comes to performance -- one overly proliferated by people who do not
> understand how the i686 kernel, and its various memory models, work.
You make assumptions that other people don't know what they are talking
about. While that may be true when you are posting to nylug generally,
occasionally you step on people who shouldn't be stepped on. ;)

> Page tables _are_always_ in use!  It's not a 4KiB v. 2MiB or other,
> related issue.  The problem is when you start killing the memory model
> with excessive ones, because of the limitations of the memory model.
> There is no "magic amount of memory/memory usage," and I have
> re-emphasized my statement was 'in my experience' and, furthermore,
> stated a 'benchmark' was required for the user's application.
@*(#*&(@#$&*#$&*

Show me something, anything, that shows that HIGHMEM hurts performance. IN
ANY WAY. ANYHOW.  (after 2.4.22 kernels). 

IT DOES NOT. END OF STORY. *(@#*(@#. No need for specific benchmarks for 
his application. It is ir-#$*(@#$-relevant.

> If you understood this you'd realize that LOMEM/HIMEM has _everything_
> to do with page tables.  Until then, you keep looking at
> "application-level" considerations.  I honestly would have hoped that by
> the original poster talking about LOMEM/HIMEM, you would have stopped to
> understand that it was part of the issue.  A different i686 kernel and
> its memory model _may_ help.  But if he has a program crossing 1GiB,
> then I'd say it's ripe for a consideration for x86-64.
THERE IS NOTHING SPECIAL ABOUT 1GB. SHOW BENCHMARKS OR ANYTHING TO
DEMONSTRATE HIGHMEM HURTS PERFORMANCE. PUT UP OR SHUT UP.

Specifically, benchmark showing that 3/1 split (aka HIGHMEM) is any slower 
than 1/3 split when you have <=4GB of ram total. (with very very large 
amounts of ram, you might run into issue of page tables filling up most of 
1GB of "kernel" ram)

> And of course there is ...
> 
> Alex Pilosov wrote:
> > What the heck does it have to do about java and python?
> 
> From:  http://nylug.org/pipermail/nylug-talk/2008-April/037679.html
> 
>   "Custom app for document processing. Written in Python."
> 
> And, finally, there's my absolute favorite ... ;)
> 
> Alex Pilosov wrote:
> > fwiw, it is actually *possible* for java/python or other bytecoded
> > languages to do user-space >4GB addressing via segmentation. (I don't
> > know if there are VMs that *do* that but its theoretically possible.
> > It is also very possible that such VM might outperform "native" 64-bit
> > VM due to lack of 64bitness bloat)
> 
> Did I not already talk about "far" pointers?  ;)
> 
> i686 uses 48-bit pointers (you call it "segmentation", which occurs
> _before_ 3GiB, x86-64 uses 48-bit pointers.  I can't make this stuff up.
> I wish I could, and I know you application-developer "only" types think
> I'm lying, but it's the reality when it comes to the OS/system-level.  
> ;)
...so the heck what? What does this have to do with anything?

ps: i've been hacking axp kernel back when you were repackaging public 
domain software. pfft.

> You keep talking about "64-bitness bloat" as if it is a certainty, all
> from the application-level (and not exactly correct either ;).  I'm
> trying to inform you that i686 kernels have their own performance issues
> when you're using great amount of memory, let alone for a single
> program.  Sooner or later, you'll dive into the internals of the i686
> and x86-64 kernels, and then you'll have the epiphany I've been trying
> to get you to see.  ;)
Benchmarks. Put up or shut up. Just one.

> Until then, "you're right" and "I'm wrong."  You're trying to debunk
> something I'm not even talking about, and you have yet to comprehend
> (despite my attempts).  I can't even attempt to discuss things that you
> have had no exposure to.  You will keep assuming it's all about the
> application, and you're fine on a single application up to 3GiB.
Switching from mumbojumbo to 'too complicated for you' mode.

I perfectly understand the issue of extra level of page tables to address
memory. I perfectly understand the additional step of having to map memory 
into kernel in order to access it. 

However, that *does not hurt performance in any significant way*.
(#define significant 5%)

-alex