Monday, July 15, 2013

Hadoop single node cluster in ease.......

***Cent OS 6.4 ( 32bit/64bit) configuring CDH4***
Single Node Cluster:---

* Install centos iso image in vm workstation.

* Download java1.6 based on Cent OS 32 bit or 64 bit

eg : 32 bit

* Give executable permission to jdk-6u43-linux-i586-rpm.bin

$chmod 755 jdk-6u43-linux-i586-rpm.bin

* Install java from root user(where java is there)


* Export java home
#export JAVA_HOME=/usr/java/jdk1.6.0_43

* Download required hadoop version

* Switch to other user   [ To add user:
                                               Switch to root user and
                                               #su adduser <username>  (default group of user is root)]

#su <username>

* Unzip hadoop tar file           
$tar -zxvf hadoop-1.o.3.tar.gz

* Making hadoop recognize java
 cd /hadoop/conf
and add following line
export JAVA_HOME=/usr/java/jdk1.6.0_43     //your java will be installed here
save and quit

* Configuring HADOOP_HOME directory to hadoop instalation directory

* Goto your home dir
$cd ~

open .bashrc file in vi editor and add following lines
export HADOOP_HOME=<hadoop installed location>

* Add user to sudoers file
goto root user and open /etc/sudoers file and add following line
<username> ALL=(ALL)    NOPASSWD:ALL

*Making update-alternatives working
add following line to .bashrc file in u r home dir

*export PATH=$PATH:/sbin:/usr/sbin:/usr/local/sbin

*Make jps running

goto home $cd ~

open .bashrc file and add following line

export PATH=$PATH:/usr/java/jdk1.6.0_43/bin

*Set all configurations

#vi /home/training/hadoop/hadoop-1.0.3/conf/core-site.xml

#vi /home/training/hadoop/hadoop-1.0.3/conf/mapred-site.xml


#vi /home/training/hadoop/hadoop-1.0.3/conf/hdfs-site.xml


Format the namenode [Format your name-node when you set up your cluster for first time]

$hadoop namenode -format

*Start all the services

    You can directly start services from your home as

*Open browser and check weather the services start or not

        http://localhost:50070         or                  http://localhost:50030

*Installing eclipse in node

change to root user and do the following

download eclipse
   eg.: eclipse-java-europa-winter-linux-gtk.tar.gz

create a dir eclipse under /home/training

copy the downloaded file to eclipse folder and untar the file

tar -zxvf eclipse-java-europa-winter-linux-gtk.tar.gz

*Change permissions for eclipse dir

chmod -R +r /opt/eclipse

*Create Eclipse executable on /usr/bin path

touch /usr/bin/eclipse

chmod 755 /usr/bin/eclipse

## Open eclipse file with your favourite editor ##
nano -w /usr/bin/eclipse

## Paste following content to file ##
export ECLIPSE_HOME="/home/training/eclipse"

$ECLIPSE_HOME/eclipse $*

*Bring eclipse icon on desktop

## Create following file, with our favourite editor ##

## Add following content to file and save ##
[Desktop Entry]
Comment=Eclipse SDK 4.2.1

after successful installation goto Applications->programming->eclipse->right click and ->addthis launcher to desktop

launch eclipse by double clicking eclipse icon on desktop

click on new project-->select mapreduce project-->click on configure Hadoop Install dir adnd give <hadoop install location>

No comments:

Post a Comment