Category Archives: Tech 技术

Hadoop MultipleInputs sample usage

MultipleInputs is a feature that supports different input formats in the MapReduce.

For example, we have two files with different formats:

(1) First file format:

VALUE

(2) Second file format:

VALUE ADDITIONAL

In order to read the custom format, we need to write Record Class, RecordReader, InputFormat for each one.

InputFormat is needed by MultipleInputs, an InputFormat use RecordReader to read the file and return value, the value is a Record Class instance

Here is the implementation:
Continue reading

Hadoop GenericWritable sample usage

GenericWritable is another Hadoop feature that let you pass values with different types to the reducer, it’s a wrapper for Writable instances.

Suppose we have different input formats (see MultipleInput), one is FirstClass and another one is SecondClass.(note: you can have multiple, not just 2). And you want to include both of them in your reducer based on the same key value, here is what you can do:

We use the same code used in MultipleInput.

Continue reading

Build Local Single Node Hadoop Cluster on Linux

This post shows how to build a local single node Hadoop cluster on Linux.

Prerequisite:

(1) Install JDK , Download Link

(2) install ANT, Download Link

Use the bin version, and add the following lines in your
~/.bash_profile

export ANT_HOME=YOUR_ANT_PATH
export JAVA_HOME=YOUR_JDK_PATH
export PATH=${PATH}:${ANT_HOME}/bin

 

Install Hadoop:

(1) Download Hadoop, Download Link
Continue reading

root后的HTC amaze 4G 升级Android 4.0 Ice Cream Sandwich

早听说android适合折腾,这回官方Ice Cream Sandwich出来后,还真好好体验的一回。

首先介绍一下本人的android:

(0) Tmobile 签约机

(1) HTC Amaze 4G,android version 2.3.4, software number 1.43.531.3

(2) unlocked bootloader

(3) custom recovery

(4) rooted

(5) S-on ( 没有进行S-off, 因为我怕不安全…)

(6) 官方rom,没有刷custom roms

(7) 没有 sd card, 事实证明,这是最要命的!

Continue reading