Grimoirelab Github (Enterprise): Error feeding raw from github 404 Client Error: Not Found for url

[dai-mk]

Hi,

when setting GrimoireLab to a private instance of Github Enterprise it produces a 404 error.

Can help me find out what is causing this error?

Error feeding raw from github (https://<snip>.com/<orgname>/): 404 Client Error: Not Found for url

Is Github Enterprise really supported? Running Enterprise version 2.21.3

setup.cfg:

[github]
api-token =<snip>
enterprise-url = https://<snip>.com

[sduenas]

Not sure how you are running it. Would you mind to try it using perceval? Perceval is the tool that retrieves information from github and other datasources: https://github.com/chaoss/grimoirelab-perceval

Use --enterprise-url and --api-token parameters when you run it.

[dai-mk]

@sduenas I managed to avoid this error by changing

[github]
api-token =<snip>
enterprise-url = https://<snip>.com

to

[github:issue]
raw_index = github_raw
enterprise-url = <snip>
enriched_index = github_enriched
api-token =  <snip>
category = issue
sleep-for-rate = true
no-archive = true
studies = [enrich_onion:github]

[enrich_onion:github]
in_index_iss = github_issues_onion-src
in_index_prs = github_prs_onion-src
out_index_iss = github-issues-onion_enriched
out_index_prs = github-prs-onion_enriched

However it shows no data in the dashboard, but when I run perceval github --enterprise-url=https://<snip>.com --api-token=<snip>myorg myrepo I get data. Strange. Is my setup,cfg incorrect? I want to display issues for a repo.

[sduenas]

What are you using to run the platform? A docker image or modred?

[dai-mk]

I'm using the docker image "grimoirelab/full, tag latest, image id de57756b355b, created 10 months ago, size 2,13GB

Don't have better information to identify it.

[sduenas]

You should check the logs to see if there's an error or something wrong.

When you run the container you can see the logs running the next command:

docker logs -f -t --tail=all <container_id>

You can get the container id running docker ps.

If you don't see anything there, you can open a shell in the container and check the logs file there:

docker exec -it <container_id> env TERM=xterm /bin/bash

Logs should be stored under the path file you set in logs_dir in [general] section on the configuration file.

[dai-mk]

Logs don't look suspicious.

user@lab:~/grimoirelab$ sudo docker logs -f -t --tail=all grimoirelab
2020-09-02T12:52:47.851330131Z Starting container: d62dfa62e0e5
2020-09-02T12:52:47.851377025Z Starting Elasticsearch
[ ok 09-02T12:52:53.370337892Z [....] Starting Elasticsearch Server:.
2020-09-02T12:52:53.371853484Z Waiting for Elasticsearch to start...
2020-09-02T12:53:00.418524243Z tcp        0      0 0.0.0.0:9200            0.0.0.0:*               LISTEN      -
2020-09-02T12:53:01.420867016Z Elasticsearch started
2020-09-02T12:53:01.420898589Z Starting MariaDB
[ ok 09-02T12:53:02.599714302Z [....] Starting MariaDB database server: mysqld.
2020-09-02T12:53:02.605075229Z Waiting for MariaDB to start...
2020-09-02T12:53:02.632048832Z tcp6       0      0 :::3306                 :::*                    LISTEN      -
2020-09-02T12:53:03.640219640Z MariaDB started
2020-09-02T12:53:03.640409830Z Starting Kibiter
2020-09-02T12:53:03.646089072Z Waiting for Kibiter to start...
2020-09-02T12:53:09.828687066Z ...Kibiter started
2020-09-02T12:53:10.831964282Z Starting SirMordred to build a GrimoireLab dashboard
2020-09-02T12:53:10.831997199Z This will usually take a while...
2020-09-02T12:53:25.605226740Z Collection for jira: starting...
2020-09-02T12:53:25.625356340Z Collection for dockerhub: starting...
2020-09-02T12:53:25.631492029Z Collection for git: starting...
2020-09-02T12:53:25.646770237Z Collection for jenkins: starting...
2020-09-02T12:53:25.664431091Z Collection for github:issue: starting...
2020-09-02T12:53:27.847091090Z Collection for jenkins: finished after 00:00:02 hours
2020-09-02T12:53:28.489751440Z Loading blacklist...
2020-09-02T12:53:28.489779740Z 0/0 blacklist entries loaded
2020-09-02T12:53:28.489787078Z Loading unique identities...
2020-09-02T12:53:30.057502839Z Collection for dockerhub: finished after 00:00:04 hours
2020-09-02T12:53:34.630781053Z Collection for github:issue: finished after 00:00:08 hours
2020-09-02T12:53:35.901168896Z Collection for jira: finished after 00:00:10 hours
2020-09-02T12:53:39.103866526Z 53/53 unique identities loaded
2020-09-02T12:55:00.477687507Z Collection for git: finished after 00:01:34 hours

The file log is very large, but the only negative line I could find was:

2020-09-02 13:14:20,725 - grimoire_elk.enriched.enrich - DEBUG - Enriched index does not include SH ids for merged_by_data_id. Can not refresh it.

[sduenas]

You have other data sources configured. Do you get any data on the dashboard related to those datasources?

You can use Kibana or a browser to check the contents of the indices.

https://www.elastic.co/guide/en/kibana/current/console-kibana.html

You can also use curl or wget to do the same:

curl http://localhost:9200/_cat/indices

[sduenas]

Closing this due to no activity.