This might be the reason for some of the problems I've been having lately getting a previously working remote X to work from work to my Gentoo Linux system with the $DISPLAY variable just not working. Anyway, the mailing list came to the rescue again with these two links telling you to comment out the DISPLAY variable declaration in /etc/security/pam_env.conf to fix it.