Hadoop Map Reduce程序
|
当我尝试基于Hadoop 0.20 API的Hadoop in Action书中的Map Reduce编程示例时,出现了错误
java.io.IOException:类型与地图中的值不匹配:预期的org.apache.hadoop.io.IntWritable,收到的org.apache.hadoop.io.Text
但据我检查,我通过的一切都正确。如果有人可以帮助我,那将真的很有帮助。
这是代码。它与书中的代码相同。
@SuppressWarnings(\"unused\")
public class CountPatents extends Configured implements Tool {
@SuppressWarnings(\"deprecation\")
public static class MapClass extends MapReduceBase implements Mapper<Text, Text, Text, Text> {
public void map(Text key, Text value,OutputCollector<Text, Text> output,Reporter reporter) throws IOException {
output.collect(value, key);
}
}
public static class Reduce extends MapReduceBase implements Reducer<Text, Text, Text, IntWritable> {
public void reduce(Text key, Iterator<Text> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
int count=0;
while(values.hasNext()){
count=count+1;
values.next();
}
output.collect(key, new IntWritable(count));
}
}
public int run(String[] args) throws Exception {
Configuration conf = getConf();
JobConf job = new JobConf(conf, CountPatents.class);
Path in = new Path(args[0]);
Path out = new Path(args[1]);
FileInputFormat.setInputPaths(job, in);
FileOutputFormat.setOutputPath(job, out);
job.setJobName(\"MyJob\");
job.setMapperClass(MapClass.class);
job.setReducerClass(Reduce.class);
job.setInputFormat(KeyValueTextInputFormat.class);
job.setOutputFormat(TextOutputFormat.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
job.set(\"key.value.separator.in.input.line\", \",\");
JobClient.runJob(job);
return 0;
}
public static void main(String[] args) throws Exception {
int res = ToolRunner.run(new Configuration(), new CountPatents(), args);
System.exit(res);
}
}
没有找到相关结果
已邀请:
5 个回复
队辅坟阮阶
时将作业的输出设置为文本类型,但是化简器的输出类型设置为IntWritable。这很可能是错误。
悍蕾驮苇袜
断跑胺弄萎
郸身
setMapOutputKeyClass setMapOutputValueClass
凄挡