[nylug-talk] Big filesystem recommendation - which one?

[H. G.]

On 8/21/07, Steven Lembark <lembark at wrkhors.com> wrote:
>
> David Rosenstrauch wrote:
> > jh wrote:
> >> I'm getting ready to build a new fileserver. Hardware raid, about a
> >> dozen HDs, about 5TB of storage. NFS and Samba file serving duties.
> >>
> >> My peculiar needs are that my work entails huge populations of
> >> tiny-to-small files - for example, 20 million TIFF images in a
> >> collection, each between 5 and 500k in size.
> >>
> >> The last file server I built used XFS on a 1.5TB filesystem (this was a
> >> few years ago, when 1.5TB was still slightly impressive), and it worked
> >> pretty well - hell, it still works great. Deletions are a bit pokey
> with
> >> XFS it seems, but that's my only complaint.
> >>
> >> What would you use today?
> >>
> >> jh
> >
> > I've read that ReiserFS works well with large numbers of files.
>
> RFS is nice for huge nubmers of small files (think
> imap4 server); XFS handles larger files more
> gracefully with less CPU overhead. There are a few
> good benchmark articles floating around.
>
> I've used XFS without any problems on high-volume
> filesystems and ones with really large files. The
> realtime volume can be a big performance help also.
>
> Second vote for XFS.   Its pretty flexible, and its ability to resize/grow
at runtime (although now thats no longer a feature exclusive to XFS)
makes it an excellent choice for running on top of things such as LVM,
where the underlying volumes may be arbitrarily resized.