Last summer I decided to build a simple Hadoop cluster, just to have a bit of fun. Since then it has become the default processing platform for almost all Wikipedia related articles on this blog. I started with three nodes, now it has the following configuration:

  • 6 nodes (1 name node + resource manager, 4 node managers + data nodes, 1 client)
  • 18 cores (4 x 2 CPUs x 2 cores + 2 CPUs)
  • 8TB of storage (4x 2TB hard disks)
  • 54 GB of RAM

These nodes are HP xw6400 and xw6200 workstations.

Hadoop cluster

Power usage is:

  • 20W in stand-by,
  • 780W when started, but idle,
  • 1kW under load.

Power usage under load