Monthly Archives: March 2013

BIG DATA + ORACLE + HADOOP

Posted on by Standard Reply

This is my Idea and explanation to one of my colleague who asked me HOW TO USE ORACLE TECHNOLOGY WITH BIG DATA.

As you can see below, we can combine the open source technology HADOOP with Oracle Technologies.  I hope you can easily understand it 🙂

bigdata

Map Reduce Program Error in Ubuntu

Posted on by Standard 2

Some times while Running a Map Reduce Program may result in this kind of Error ,the reason is output directory exits already ,which means you have already run it some time before .

hduser@ubuntu:/home/hemanth/hadoop$ bin/hadoop jar hadoop-examples-1.0.4.jar pi 10 100
Warning: $HADOOP_HOME is deprecated.
Number of Maps = 10
Samples per Map = 100
java.io.IOException: Tmp directory
hdfs://localhost:54310/user/hduser/PiEstimator_TMP_3_141592654 already exists.
Please remove it first.
at org.apache.hadoop.examples.PiEstimator.estimate(PiEstimator.java:270)
at org.apache.hadoop.examples.PiEstimator.run(PiEstimator.java:342)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.examples.PiEstimator.main(PiEstimator.java:351)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
5)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:6
8)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
5)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

I resolved this issue like this :

hduser@ubuntu:/home/hemanth/hadoop$ cd bin
hduser@ubuntu:/home/hemanth/hadoop/bin$ fs -rmr
hdfs://localhost:54310/user/hduser/PiEstimator_TMP_3_141592654
Warning: $HADOOP_HOME is deprecated.
Deleted hdfs://localhost:54310/user/hduser/PiEstimator_TMP_3_141592654

Now you run this Program ,you will get the out put