envelope icon indicating copy to clipboard operation
envelope copied to clipboard

pass Config to custom HBaseSerde

Open andvit opened this issue 6 years ago • 2 comments

Hello

com.cloudera.labs.envelope.utils.hbase.HBaseUtils // HBaseSerde util public static HBaseSerde getSerde(Config config) { HBaseSerde serde; if (config.hasPath(SERDE_PROPERTY)) { String serdeImpl = config.getString(SERDE_PROPERTY); if (serdeImpl.equals(DEFAULT_SERDE_PROPERTY)) { return new DefaultHBaseSerde(); } else { try { Class> clazz = Class.forName(serdeImpl); Constructor> constructor = clazz.getConstructor(); return (HBaseSerde) constructor.newInstance(); } catch (Exception e) { LOG.error("Could not construct custom HBaseSerde instance [" + serdeImpl + "]: " + e); throw new RuntimeException(e); } } } else { serde = new DefaultHBaseSerde(); } serde.configure(config);

return serde;

}

in case of custom serde, serde.configure(config) is never called, there is no way to access config in provided serde

according to the comment below configure(config) should be called com.cloudera.labs.envelope.utils.hbase.HBaseSerde /**

Configure the Serde. This will be passed the contents of an "input" or "output" configuration section. @param config */ void configure(Config config);

andvit avatar Jun 15 '18 20:06 andvit

simple change from return (HBaseSerde) constructor.newInstance(); to serde = (HBaseSerde) constructor.newInstance(); resolves the issue

andvit avatar Jun 15 '18 20:06 andvit

Thanks @andvit, you're absolutely right. Will ensure this is fixed and rolled into the next release. We're also missing a unit test for custom serdes. To re-iterate, the fix is as described:

Line 174 to serde = (HBaseSerde) constructor.newInstance();

Line 179 to serde = new DefaultHBaseSerde();

ianbuss avatar Jun 18 '18 01:06 ianbuss