Cluster v1

Most components have already arrived and this is how my cluster looks like now:


Hadoop Cluster

I am still waiting for additional storage to come. So far there is about 4TB of space on HDFS:

$ hadoop fs -df -h 
Filesystem            Size    Used  Available  Use%
hdfs://h1.home:8020  3.8 T  39.5 G      3.6 T    1%

Out of those 4 nodes one has RAM upgraded to 10GB and it also acts as name node and task tracker / resource manager. Later I will probably add separate host for those services.

I have also executed some MR v1 jobs. Terasort test for 100 000 000 records (~10GB) takes 25 mintes to finish, but I will include more detailed benchmarks when the cluster becomes fully operational.

Leave a Reply