Documentation for Search and Profiler

[surajnayak]

How to use search and profiler ? I just get blank page? Can someone provide brief example ?

[danielcweeks]

@surajnayak, In order to search or view anything, you first need to go through the process of setting it up (described in the readme). Once you have data from your hadoop cluster indexed, you should be able to search and profile jobs.

You can read more about it here: http://techblog.netflix.com/2014/09/inviso-visualizing-hadoop-performance.html

[surajnayak]

@danielcweeks, I tried following the documentation, but got below error :

$ python jes.py 
INFO:inviso-monitor:Publishing event: (cluster_1_dev) job_1458228381455_0111 2016-03-17T18:24:31+00:00
INFO:inviso-handler:Processing 1 events
ERROR:snakebite.client:RpcBufferedReader only managed to read 0 out of 4 bytes
ERROR:snakebite.client:RpcBufferedReader only managed to read 0 out of 4 bytes
ERROR:snakebite.client:RpcBufferedReader only managed to read 0 out of 4 bytes
ERROR:inviso-handler:Failure to read block 1131916133
Traceback (most recent call last):
  File "/apps/inviso/jes/inviso/handler.py", line 135, in process_event
    self._process_config(doc, event)
  File "/apps/inviso/jes/inviso/handler.py", line 180, in _process_config
    content = self.fetch_content(event['config.uri'])
  File "/apps/inviso/jes/inviso/handler.py", line 168, in fetch_content
    for line in c.text([p.path]):
  File "/apps/venv/base/lib/python2.6/site-packages/snakebite/client.py", line 853, in text
    include_children=False, recurse=False):
  File "/apps/venv/base/lib/python2.6/site-packages/snakebite/client.py", line 1083, in _find_items
    entry = processor(full_path, fileinfo.fs)
  File "/apps/venv/base/lib/python2.6/site-packages/snakebite/client.py", line 851, in <lambda>
    processor = lambda path, node, check_crc=check_crc: self._handle_text(path, node, check_crc)
  File "/apps/venv/base/lib/python2.6/site-packages/snakebite/client.py", line 862, in _handle_text
    for load in self._read_file(path, node, False, check_crc):
  File "/apps/venv/base/lib/python2.6/site-packages/snakebite/client.py", line 1033, in _read_file
    raise Exception("Failure to read block %s" % block.b.blockId)
Exception: Failure to read block 1131916133
INFO:inviso-handler:Indexing 0 documents
INFO:inviso-handler:Events complete.
INFO:inviso-monitor:Publishing event: (cluster_1_dev) job_1458228381455_0124 2016-03-17T18:23:51+00:00
INFO:inviso-handler:Processing 1 events
ERROR:snakebite.client:RpcBufferedReader only managed to read 0 out of 4 bytes
ERROR:snakebite.client:RpcBufferedReader only managed to read 0 out of 4 bytes
ERROR:snakebite.client:RpcBufferedReader only managed to read 0 out of 4 bytes
ERROR:inviso-handler:Failure to read block 1131915997
Traceback (most recent call last):

[danielcweeks]

Genie is not required for this to work. It appears that you're having trouble getting access to hdfs to read the history files.

What version of hadoop are you using? You might want to take a look at the closed issues as well because others may have hit this same thing. On Mar 19, 2016 1:08 AM, "Suraj Nayak" [email protected] wrote:

Is Genie required for elasticsearch to work ?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Netflix/inviso/issues/11#issuecomment-198664954

[othman-essabir]

Applying this patch should work https://github.com/spotify/snakebite/commit/68db2f2eb39ca7525f28506a0c420dda9d852768

[surajnayak]

Thank you so much @othman-essabir. The patch worked!