[nylug-talk] dig weird behavior

[Michael Bacarella]

On Wed, Sep 27, 2006 at 11:44:25PM -0400, Frank D. Greco wrote:
> I wrote a small bash script (on RH AS3) that looped through a set of 
> hostnames and used dig to find aliases (etc), i.e.,
> 
> 	for i in $HOSTS
> 	do
> 		dig ... | grep ...
> 	done
> 
> Worked fine as expected.  But since my $HOSTS has been getting quite 
> large, I attempted to parallelize the guts of the loop:
> 
> 	for i in $HOSTS
> 	do
> 		dig ... | grep ... > $TMPDIR/$i   &
> 	done
> 
> 	wait
> 	cat $TMPDIR/* | sort
> 
> Surprisingly, this hung.  When I checked my $TMPDIR, all the files 
> were created, but some were 0 bytes.  When I hit CTRL-C on this script,
> I got some odd error about sockcount == 0 from dig.c
> 
> Anyone know what's going on here?

The shell creates the $TMPDIR/$i files before it even executes the
processes on the command line; it's possible you're just not waiting
long enough for them all to finish.

If they're not finishing, you might be hitting some resource limit
(ulimit -a) that causes dig to hang.  Or maybe there's a concurrency
issue with having that many digs running.  How big's the host list?
You could be running out of ephemeral ports.

Don't think this stuff is robust just because it runs important
internet infrastructure. ;)