[nylug-talk] The best way to mirror 6M+ files?

[N.J. Thomas]

* David Rosenstrauch <darose at darose.net> [2006-05-08 11:03:24 -0400]:
> > It is a very bad idea to have >10k files in same directory
> 
> Well, that's not necessarily true.  It really depends on the file
> system being used.  Some file systems (reiserfs comes to mind) can
> handle large numbers of files - i.e., much larger than 10K - without
> any problem.

It's still good practice to avoid populating a directory with an
obscenely large number of files since some tools make certain size
assumptions.

In this case, the OP would still run out of memory on the directory if
he was using reiserfs -- rsync consumes some amount of space (~100
bytes, more if you use certain switches) per file in a given directory.

(I know the correct answer is to rewrite the tools, but we all know
that's not gonna happen anytime soon.)

Thomas

-- 
N.J. Thomas
njt at ayvali.org
Etiamsi occiderit me, in ipso sperabo