blog darkness...

words, music, food

Entries for date "October 2018"

tools.DistCp: Exception encountered java.lang.IllegalArgumentException: n must be positive

When using distcp on hadoop, I sometimes get the following error:

tools.DistCp: Exception encountered java.lang.IllegalArgumentException: n must be positive

Which is slightly misleading.  Internet searches (Stack Overflow) make it seem like this can be transient or when you have an empty directory (n = 0).  In my case, the error was because the number of nested directories in my recursive copy was too large for distcp to handle ...

Continue reading »