classloading isolation improvements for stricter dependencies

Related to HADOOP-11656, we would like to make a number of improvements in terms of classloading isolation so that user-code can run safely without worrying about dependency collisions with the Hadoop dependencies.

By the same token, it should raised the quality of the user code and its specified classpath so that users get clear signals if they specify incorrect classpaths.

This will contain a proposal that will include several improvements some of which may not be backward compatible. As such, it should be targeted to the next major revision of Hadoop.