storm-contrib icon indicating copy to clipboard operation
storm-contrib copied to clipboard

[storm-rdbms] java.lang.NullPointerException when using storm on cluster mode

Open jamesweb1 opened this issue 11 years ago • 3 comments

It can run correctly in the local mode when I write to my mysql database. But when I use in storm cluster mode, it fails.

I submit my topology by StormSubmitter.submitTopology(mTopologyName,mTopologyConfig,mBuilder.createTopology());

It can create table successfully, but it can't write anything to my mysql in RDBMSDumperBolt.

java.lang.RuntimeException: java.lang.NullPointerException at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:77) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] at backtype.storm.daemon.executor$eval3918$fn__3919$fn__3931$fn__3978.invoke(executor.clj:745) ~[na:na] at backtype.storm.util$async_loop$fn__384.invoke(util.clj:433) ~[na:na] at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55] Caused by: java.lang.NullPointerException: null at storm.contrib.rdbms.RDBMSDumperBolt.execute(RDBMSDumperBolt.java:95) ~[stormjar.jar:na] at backtype.storm.topology.BasicBoltExecutor.execute(BasicBoltExecutor.java:50) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] at backtype.storm.daemon.executor$eval3918$fn__3919$tuple_action_fn__3921.invoke(executor.clj:630) ~[na:na] at backtype.storm.daemon.executor$mk_task_receiver$fn__3839.invoke(executor.clj:398) ~[na:na] at backtype.storm.disruptor$clojure_handler$reify__1560.onEvent(disruptor.clj:58) ~[na:na] at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:104) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] ... 6 common frames omitted 2014-05-29 06:11:15 b.s.d.executor [ERROR]

Can anyone help me to solve this tough problem? Thank a lots:)))

jamesweb1 avatar May 29 '14 06:05 jamesweb1

Hi @jamesweb1

I am also getting the same error when i am trying to read data from RDBMS in my spout code, could you please let me know whether it is resolvable?

Thanks...

SanthoshPrasad avatar Dec 01 '14 07:12 SanthoshPrasad

I remember it causes from you didn't organize your code well. You almost need to put everything in prepare function including RDBMSCommunicator. For example, you may call 'RDBMSConnector connector = new RDBMSConnector();'. You need to construct them in prepare function instead in the class member. If any question, welcome to raise your question here.

jamesweb1 avatar Dec 01 '14 08:12 jamesweb1

Thank you @jamesweb1 , it's working fine now the problem is with the storm class path...

SanthoshPrasad avatar Dec 01 '14 13:12 SanthoshPrasad