Wie implementiere ich die sichere Massenladung von hbase?
Ich habe bereits eine Massenladung in hbase in einem Kerberos-Cluster mit einer ähnlichen Treiberklasse erstellt (funktioniert):
public static void main(String[] args) {
try {
int response = ToolRunner.run(HBaseConfiguration.create(), new HBaseBulkLoadDriver(), args);
if(response == 0) {
System.out.println("Job is successfully completed...");
} else {
System.out.println("Job failed...");
}
} catch(Exception exception) {
exception.printStackTrace();
}
}
@Override
public int run(String[] args) throws Exception {
int result=0;
final String inputPath = args[0];
final String outputPath = args[1];
final String keytab = args[2];
Configuration configuration = getConf();
configuration.set("data.seperator", DATA_SEPERATOR);
configuration.set("hbase.table.name",TABLE_NAME);
// configuration.set("INTRO",COLUMN_FAMILY_INTRO);
configuration.set("hbase.zookeeper.quorum","zk_quorum");
configuration.set("hbase.zookeeper.property.clientPort","2181");
configuration.set("hbase.master","master:port");
configuration.set("hadoop.security.authentication", "Kerberos");
configuration.set("hbase.security.authentication", "kerberos");
//configuration.set("COLUMN_FAMILY_2",COLUMN_FAMILY_2);
Job job = new Job(configuration);
// job configuration
job.setJarByClass(HBaseBulkLoadDriver.class);
job.setJobName("Bulk Loading HBase Table:"+TABLE_NAME);
job.setInputFormatClass(TextInputFormat.class);
job.setMapOutputKeyClass(ImmutableBytesWritable.class);
//mapper class
job.setMapperClass(HBaseBulkLoadMapper.class);
FileInputFormat.addInputPaths(job,inputPath);
FileSystem.getLocal(getConf()).delete(new Path(outputPath), true);
FileOutputFormat.setOutputPath(job, new Path(outputPath));
job.setMapOutputValueClass(Put.class);
HFileOutputFormat.configureIncrementalLoad(job, new HTable(configuration,TABLE_NAME));
job.waitForCompletion(true);
System.out.println("Output written to folder :" + outputPath);
System.out.println("To proceed loading files user: hbase:hbase must own recursivly the folder!");
System.out.println("Is hbase user owing the folder?press Y to load the data , press N and job will fail");
String IsHbaseOwnerOftheFolder = System.console().readLine();
if (job.isSuccessful() && IsHbaseOwnerOftheFolder.equals("Y")) {
HBaseBulkLoad.doBulkLoad(outputPath, keytab, TABLE_NAME);
} else {
result = -1;
}
return result;
}
Nun möchte ich die sichere Massenauslastung implementieren, aber es scheint, dass dies mithilfe des Coprozessor-Frameworks (hbase 1.0.0) implementiert werden muss. Kann mir jemand ein vollständiges Beispiel für die Verwendung der Methode securebulkloadHFiles geben? Danke für die Hilf