streamparse icon indicating copy to clipboard operation
streamparse copied to clipboard

feature request: to set config path for sparse commands

Open mbande opened this issue 8 years ago • 3 comments

in my project, I need to call some sparse commands inside bolts, e.g. get some topology stats via sparse stats. but they all should run within project dir to find config.json all this commands can take an exact config path to be able to run everywhere

mbande avatar Dec 13 '16 14:12 mbande

Could you elaborate on what your bolt is trying to do? I feel like there may be a cleaner way to get what you want.

dan-blanchard avatar Dec 13 '16 14:12 dan-blanchard

i have a process topology that based on user needs, it should act on different set of inputs. e.g. user 1 sends input set 1 to be processed, user 2 needs input set 2 to be processed, etc. and at any time, just one input set could be processed. and after processing each input set is done, some stats should be reported like how many tuples is processed, how many failed, how long it taked to run, etc. what i have done maybe seems strange, but it's just simple: when request from user 1 come, i submit the topology with new name process_1, reading from redis db_1, then when there is no more input and there is no any processing tuple (all emitted tuples count are equal to acked+failed), the topology processes next user request, submitting that topology, and killing himself! there is some other scenarios like when a super user pauses a user request, which application deactivates topology. in most of this situations, the topology itself calls a sparse command to submit/kill/deactivate

mbande avatar Dec 13 '16 15:12 mbande

@mbande In 3.12.0 we released a change that allowed users to pass the path to the "config.json" file. Have you tried that out to see if it does what you were looking to do?

codywilbourn avatar Jan 08 '18 21:01 codywilbourn