Error:
18/06/13 1:04:11 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:379)
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:394)
at org.apache.hadoop.util.Shell.
(Shell.java:387)
at org.apache.hadoop.hive.conf.HiveConf$ConfVars.findHadoopBinary(HiveConf.java:2327)
at org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:365)
at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:105)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:228)
at org.apache.spark.sql.SparkSession$.hiveClassesArePresent(SparkSession.scala:963)
at org.apache.spark.repl.Main$.createSparkSession(Main.scala:91)
Solution:
(Note:You Should have the rights of Administrator.)
Step1:
Download winutils.exe binary from https://github.com/steveloughran/winutils or http://public-repo-1.hortonworks.com/hdp-win-alpha/winutils.exe repository.
Save winutils.exe binary to a directory of your choice, e.g. c:\winutils\bin.
Step2:
Using command prompt to set the Environment variable As mentioned below.
set HADOOP_HOME=c:\winutils
Set PATH environment variable to include %HADOOP_HOME%\bin as follows:
set PATH=%HADOOP_HOME%\bin;%PATH%
Or
same can be done by right click on PC->properties ->Advance System Setting -> Environment Variable.
Step3:
Restart the Eclipse or Intellij IDE
No comments:
Post a Comment