[nylug-talk] Benefits/drawbacks of building Linux as a package [was: Looking for recommendations on Linux Distro]

[Chris Knadle]

Combined replies within.

On Sunday 16 March 2008, Ruben Safir wrote:
> On Sat, Mar 15, 2008 at 08:48:34PM -0400, Ron Guerin wrote:
> > Ruben Safir wrote:
> > > Package managers by and large, actually all of them, suck in the first
> > > place Your checklist is wishful thinking.  And I'm not talking out of
> > > my hat.  I'm talking from a SUSE 5.3 distro running on a P2 right now
> > > which has been continually patched by hand for a LOT of years now.
> >
> > If by suck, you mean works extremely well and makes millions of people
> > happy, then yeah, most of them suck.  Our package managers suck.
>
> No SUCK I mean SUCK ... like creating a d at mn hell of inconstitancies
> by different packagers even WITHIN single distributions that cause
> a nightmare of incongruent requirments that lead you to chase packages
> into the night to make something work to only find out that 5 other things
> are now broken or the latest package that was used has to be UNINSTALLED to
> satisfy the stupidy of the new package you want to install.
>
> This is package hell and it results, first and formost, from packaging
> systems that can't even FIND what the heck is on your hard drive let alone
> understand what they NEED in their databases.

   YES, I've had lots of these problems with RPM-based distributions over the 
years.  So if you're feeling vehement flaming fire+brimstone hatred for 
RPM "dependency hell", frankly count me in -- I've likewise been there and 
had similar feelings.  :-/  Things have gotten somewhat better with newer 
RPM-based distributions that have incorporated 'yum' which can download 
dependencies for you -- but I still agree that the RPM format doesn't [or at 
the very least didn't at one time] have enough intelligence built into it, at 
least to my current knowledge.  [Admittedly I'm somewhat ignorant of RPM 
package internals, as it hasn't been my focus.]
   Knowing where you are I know this is going to sound hard to believe, but I 
mostly feel joy when I think about DEB packages.  97% of the time it's 
everything you'd likely wish from a package management system, and the 3% 
that's left are mainly the rare cases where there are package conflicts 
during upgrades [which are handle-able via putting them on 'hold'] -- and I 
really only see these happen in the "unstable" branch.  On rare occasion a 
package is uploaded that is broken or has the wrong control information, but 
again only in "unstable", so that's to be expected somewhat.  Not only do DEB 
packages know about the dependencies, but they also know the *reverse* 
dependencies -- I.E. "this" package knows which other packages depend on it.  
Nothing is perfect, but I have to admit that I think it's pretty slick.
   So like Ron said -- as crazy as it sounds -- the package management system 
actually makes me happy.  [Though natually it did take some time to get used 
to before that happened.]


On Sunday 16 March 2008, Ruben Safir wrote:
> On Sat, Mar 15, 2008 at 09:49:38PM -0400, Chris Knadle wrote:
> > On Saturday 15 March 2008, Ruben Safir wrote:
> > > Reinstalling the kernal from a packagemanager doesn't allow for both
> > > kernels to be exected through grub incase something goes wrong.
> >
> >    I'm assuming you're talking about the 'savedefault' option.
...
> The configuration issue is with having the both images remain in the boot
> director, and for both images configured in grub or lilo.

   Unfortunately I'm not sure I know what you mean.  I'm probably 
misinterpreting but I'll make an attempt anyway...
   Do you mean the softlink at /vmlinuz ?  If that's what you meant then 
depending on the system it may not be a problem anymore -- 'update-grub' now 
removes the /vmlinuz and /initrd.img softlinks when there is more than one 
kernel image installed.

> Then there is the issues with mkinitrd or whatever program you have to
> do the same step as well as consistency in the modules.
>
> All of these issues have been vastly improved over the last 5 years, and
> yet all of them still remain so NEVER be suprised if when loading a new
> kernel with a package manager that you can't actually reboot.  The few
> times a successful kernel swap has worked for me I've been in near shock. 
> I've got to be feeling pretty lazy to even attempt it and it has to be a
> hugely non-criticle device.
>
> Frankly, when I need to upgrade a kernel that is a huge sign that it is
> time to upgrade the whole distro.

   Hmm.  I vaguely remember having some problems along these lines, but it was 
back when I was installing kernels without package management.

   I've been rebuilding the kernel on my desktop about once a month -- not 
because I really *have* to, mind you, but mostly for fun I guess...  and I've 
been doing it for the past...  uh...   *sigh*  I honestly I can't remember 
how long I've been doing it.  6 years?  Maybe more?  And I've been building 
my kernels directly to a .deb package for at least the past 3 years.  And in 
every case, if the kernel didn't boot it's because I misconfigured it, rather 
than any problem with the package management system.

   I'm not saying that you're wrong, mind you -- I guess I'm just saying that 
I don't fully understand what you've run into because I haven't run into it 
myself, or if I have it's been too long for me to remember any of the 
details.

> >    Just to make sure we're on the same page: when I speak about
> > installing kernels via package management, I mean *configuring and
> > compiling your own* and having the build system automatically make it
> > into a package for you. I.E. *NOT* using PRE-packaged kernels.
>
> If you care to go that route, by all means, go for it.  It seems just as
> easy at that point to just compile the thing and be done with it unless you
> need to do a rollout of a thosand other machines.  Then I agree with you
> 100%.  You'd had better make package, preferably one that works like
> kickstart on a bearwolf cluster.

   Well believe it or not even for just one system I still build the kernel to 
a package, ironically because it saves me work.  I know that's backwards from 
common sense, so I'll illustrate.

   Making + installing with Debian package management:
      make menuconfig
      fakeroot make-kpkg clean modules_clean
      time nice fakeroot make-kpkg --revision=2.6.24.3+686+initrd+crk1 \
         --initrd kernel_image modules_image
      cd ..
      su
      dpkg -i *.deb

   During the package installation 'update-grub' is run automatically, so I 
don't have to touch the boot menu.  The external modules that I've been 
making, all of which are made at the same time by the "modules_image" 
parameter: nvidia-kernel-source, squashfs-source, kqemu-source, cloop-src, 
lzma-source (used by squashfs).  These are drivers I actually use -- cloop 
for making custom Knoppix LiveCDs, squashfs for OpenSuSE LiveCDs, qemu for 
running LiveCDs without rebooting, and of course the 3D Nvidia driver.
   And on a humorous note -- I'm also running the Debian 'popularity contest' 
on my desktop, so the custom kernel packages I make above show up in the 
public popcon statistics.  :-P

   Removing the kernel + all of the associated packages from above is even 
easier, because I just remove them using one of the many package managers (I 
use 'aptitude', which shows all of the above packages in the 'Obsolete and 
Locally Created Packages' section).


   Making + compiling without package management:
      make menuconfig
      make clean
      time nice make bzimage modules
      su
      make modules_install install
      cp arch/i386/boot/bzImage /boot/vmlinuz-2.6.24.3-686-initrd-crk1
      cp System.map /boot/System.map-2.6.24.3-686-initrd-crk1
      cp .config /boot/config-2.6.24.3-686-initrd-crk1
      mkinitramfs 2.6.24.3-686-initrd-crk1    [or something like this]
      update-grub

   ... and then I have to figure out how to make each one of the external 
modules.

   Removing the kernel + drivers:
      su
      rm -rf /lib/modules/2.6.24.3-686-initrd-crk1
      rm -f /boot/*2.6.24.3-686-initrd-crk1
      update-grub


   So oddly enough -- compiling directly to a .deb is just...  easier.  ?  
Hard to believe, but that's what I've found.

> > > The single biggest mistake someone can make aside from a dread aweful
> > > rm command in jest is to install the Kernel from anything but an
> > > authenticed source from kernel.org.
> >
> >    And the .deb kernel packages I make are compiled from the git kernel
> > source at kernel.org -- and if that isn't authenticated, I don't know
> > what is. You *DO NOT* have to use the packaged sources from the distro if
> > you don't want to.  Personally I don't.
>
> If your compiling from source, does it matter?

   Well, yes, because if you patch the source the patch and the source have to 
match.  I do like the fact that the git repos have SHA1 hashes for all of the 
contents, but likewise the kernel source in the Debian repository has 
checksums also, so either one can be verified.
   The Debian kernel source is usually a bit stripped-down for various 
reasons, and I'm, well, I'm a "crazy nut" that does a lot of experimentation, 
so that's why I use the kernel.org source.  ;-)

> I've not run into a kernel that wouldn't compile.

   I had it happen a good bit in the 2.4 days, but not lately with 2.6; 
although I can't get the kernel to compile with gcc-4.3 -- the compile bombs 
around the point of making the bzImage.

> I will, however, repeat that the task is getting so arduous at this point
> that using preconfigured packages is looking more and more inviting with
> every passing month.

   Yeah.  :-(  That's what pushed me away from Slackware.
   I still don't custom compile _applications_ much, so I have more to learn 
about the 'debuild' system -- I'm slowly getting there.


...
> But at the next NYLXS installfest, I'd be glad to sit down with you and
> compile a kernel with you and we can both likely shake our heads together
> trying to figure out what all that stuff is at this point.

   Hahaha!  Given.  I don't pretend to understand every single option that the 
kernel has to offer.

> The kernel has become like NYC government.  Its so bloated by special
> interests that you need a degree in electrical engineering and 20+ years of
> experience to know what the heck its asking.  There was a time I was
> familiar with every single hardware that was being discussed in the config,
> and that wasn't that long ago.

   I don't doubt you.  I'm not sure I ever understood *every* hardware option.  
And of course these days, well, forget it.  Heck there are even multiple 
drivers for the same IDE device, and depending on which driver is used the 
device could show up as 'hde' or 'sda'.

> BTW - there is a work around in the config gui force the external modules
> to show up, but I'd need to research it to find it again.  I believe I
> stumbled on it when I downloaded the latest ALSA drivers from the ALSA site
> and patched it into the new kernel.

   Huh!  I've never heard of that.  I'll have a look also, but if you happen 
to find that again that would be interesting to know.

   Thanks.


On Sunday 16 March 2008, Ruben Safir wrote:
> On Sat, Mar 15, 2008 at 09:49:38PM -0400, Chris Knadle wrote:
...
> >     # savedefault=true
> >
> > then when a new kernel is added to the list by update-grub, *all* of the
> > kernel entries will have 'savedefault' after them, which is I believe
> > what you're looking for.
>
> BTW - grub is another issue itself.
>
> There seems to have been a change in grubs syntax at some point because it
> tells me my file has bad options in it and defaults to a cursors like
> interface.
>
> It still boots, but its anoying.

   Yeah, IIRC grub has gone through a number of changes.  I really wish there 
were a detailed man page for 'menu.lst'.  :-/  But documentation aside, I 
love what it's capable of doing.


   Thanks for writing back again.  :-)
   Cheers

   -- Chris

-- 

Chris Knadle
Chris.Knadle at coredump.us