BIG DATA is getting Bigger and Bigger
BIG DATA Getting Started with HADOOP
BIG DATA Cloudera and Oracle
BIG DATA CDH Single Node Setup
BIG DATA HADOOP Services Startup and Shutdown
BIG DATA Moving a file to HDFS
BIG DATA HADOOP Testing with MapReduce Examples Part 1
BIG DATA HADOOP Testing with MapReduce Examples Part 2
BIG DATA HADOOP Testing with MapReduce Examples Part 3
When using start-all.sh , I got the message "This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh"
So it is clear that start-all.sh should not be user and the same hold true with stop-all.sh
Using start-dfs.sh it gives me JAVA_HOME is not set
hadoop@bigdataserver1:~/hadoop/sbin> sh start-dfs.sh
which: no start-dfs.sh in (/home/hadoop/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/games:/opt/gnome/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/home/hadoop/hadoop/bin)
13/03/13 11:44:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [localhost]
localhost: Error: JAVA_HOME is not set and could not be found.
localhost: Error: JAVA_HOME is not set and could not be found.
Starting secondary namenodes [0.0.0.0]
0.0.0.0: Error: JAVA_HOME is not set and could not be found.
13/03/13 11:45:00 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
hadoop@bigdataserver1:~/hadoop/sbin> echo $JAVA_HOME
/home/hadoop/jdk1.6.0_18
hadoop@bigdataserver1:~/hadoop/sbin>
But you can see that the environment is sources properly.
Solution is to update hadoop-env.sh with the JAVA_HOME
hadoop@fravm097023:~/hadoop/etc/hadoop> grep -i JAVA_HOME hadoop-env.sh
# The only required environment variable is JAVA_HOME. All others are
# set JAVA_HOME in this file, so that it is correctly defined on
# export JAVA_HOME=${JAVA_HOME}
export JAVA_HOME=/home/hadoop/jdk1.6.0_18
hadoop@fravm097023:~/hadoop/etc/hadoop>
BIG DATA Getting Started with HADOOP
BIG DATA Cloudera and Oracle
BIG DATA CDH Single Node Setup
BIG DATA HADOOP Services Startup and Shutdown
BIG DATA Moving a file to HDFS
BIG DATA HADOOP Testing with MapReduce Examples Part 1
BIG DATA HADOOP Testing with MapReduce Examples Part 2
BIG DATA HADOOP Testing with MapReduce Examples Part 3
When using start-all.sh , I got the message "This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh"
So it is clear that start-all.sh should not be user and the same hold true with stop-all.sh
Using start-dfs.sh it gives me JAVA_HOME is not set
hadoop@bigdataserver1:~/hadoop/sbin> sh start-dfs.sh
which: no start-dfs.sh in (/home/hadoop/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/games:/opt/gnome/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/home/hadoop/hadoop/bin)
13/03/13 11:44:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [localhost]
localhost: Error: JAVA_HOME is not set and could not be found.
localhost: Error: JAVA_HOME is not set and could not be found.
Starting secondary namenodes [0.0.0.0]
0.0.0.0: Error: JAVA_HOME is not set and could not be found.
13/03/13 11:45:00 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
hadoop@bigdataserver1:~/hadoop/sbin> echo $JAVA_HOME
/home/hadoop/jdk1.6.0_18
hadoop@bigdataserver1:~/hadoop/sbin>
But you can see that the environment is sources properly.
Solution is to update hadoop-env.sh with the JAVA_HOME
hadoop@fravm097023:~/hadoop/etc/hadoop> grep -i JAVA_HOME hadoop-env.sh
# The only required environment variable is JAVA_HOME. All others are
# set JAVA_HOME in this file, so that it is correctly defined on
# export JAVA_HOME=${JAVA_HOME}
export JAVA_HOME=/home/hadoop/jdk1.6.0_18
hadoop@fravm097023:~/hadoop/etc/hadoop>
After that DFS and YARN services will come up
hadoop@bigdataserver1:~/hadoop/sbin> sh start-dfs.sh
which: no start-dfs.sh in (/home/hadoop/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/games:/opt/gnome/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/home/hadoop/hadoop/bin)
13/03/13 11:51:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [localhost]
localhost: starting namenode, logging to /home/hadoop/hadoop/logs/hadoop-hadoop-namenode-bigdataserver1.out
localhost: starting datanode, logging to /home/hadoop/hadoop/logs/hadoop-hadoop-datanode-bigdataserver1.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /home/hadoop/hadoop/logs/hadoop-hadoop-secondarynamenode-bigdataserver1.out
13/03/13 11:51:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
hadoop@bigdataserver1:~/hadoop/sbin>
hadoop@bigdataserver1:~/hadoop/sbin> sh start-yarn.sh
starting yarn daemons
which: no start-yarn.sh in (/home/hadoop/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/games:/opt/gnome/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/home/hadoop/hadoop/bin)
starting resourcemanager, logging to /home/hadoop/hadoop/logs/yarn-hadoop-resourcemanager-bigdataserver1.out
localhost: starting nodemanager, logging to /home/hadoop/hadoop/logs/yarn-hadoop-nodemanager-bigdataserver1.out
hadoop@bigdataserver1:~/hadoop/sbin>
