Deploy on Heroku

[alexisrosuel]

Auto-reviewers: @NiharikaRay @matthewwardrop @earthmancash @danfrankj

I am trying to host my knowledge repo on Heroku, and I have the same issue than issue 314 . This issue has been closed because the author found another solution (using Docker), but I would like to reopen this issue in order to fix it directly.

As a reminder, here is the Procfile to deploy on Heroku : web: knowledge_repo --repo /app runserver --port $PORT

and here is the error log :

2017-08-04T18:53:26.093397+00:00 app[web.1]:   File "/app/.heroku/python/bin/knowledge_repo", line 228, in <module>
2017-08-04T18:53:26.093587+00:00 app[web.1]:     repo = knowledge_repo.KnowledgeRepository.for_uri(args.repo)
2017-08-04T18:53:26.093590+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/knowledge_repo/repository.py", line 46, in for_uri
2017-08-04T18:53:26.093687+00:00 app[web.1]:     return cls._get_subclass_for(scheme)(uri, *args, **kwargs)
2017-08-04T18:53:26.093688+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/knowledge_repo/repository.py", line 76, in __init__
2017-08-04T18:53:26.093788+00:00 app[web.1]:     self.init(**kwargs)
2017-08-04T18:53:26.093790+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/knowledge_repo/repositories/gitrepository.py", line 66, in init
2017-08-04T18:53:26.093912+00:00 app[web.1]:     self.path = self.uri.replace('git://', '')
2017-08-04T18:53:26.093914+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/knowledge_repo/repositories/gitrepository.py", line 88, in path
2017-08-04T18:53:26.093998+00:00 app[web.1]:     path), "Provided path '{}' is not a valid repository.".format(path)
2017-08-04T18:53:26.094002+00:00 app[web.1]: AssertionError: Provided path '/app' is not a valid repository.

Thanks

[danfrankj]

Hey @alexisrosuel! Thank you for your interest in the knowledge repo.

It looks like the error you're running into here is that /app is not a repository, meaning that it is not a repository that has been initialized by the knowledge repo tooling.

Specifically, you'd want to first initialize a repository

knowledge_repo --repo <repo_path> init

and then you'll be able to start the webserver

knowledge_repo --repo <repo_path> runserver --port $PORT

(thought it won't have any knowledge posts yet)

Hope that helps and feel free to re-open if you're still running into this error.

[danfrankj]

Ah - just read the related issue and I see that somehow Heroku is deleting the .git file. I'll look into this

[alexisrosuel]

Hi, thanks for your answer! Indeed the first solution does not work, here are some details :

if I run the heroku bash knowledge_repo --repo <repo_path> init, and then restart with heroku restart, I still have the same error

2017-08-04T18:53:26.093397+00:00 app[web.1]:   File "/app/.heroku/python/bin/knowledge_repo", line 228, in <module>
2017-08-04T18:53:26.093587+00:00 app[web.1]:     repo = knowledge_repo.KnowledgeRepository.for_uri(args.repo)
2017-08-04T18:53:26.093590+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/knowledge_repo/repository.py", line 46, in for_uri
2017-08-04T18:53:26.093687+00:00 app[web.1]:     return cls._get_subclass_for(scheme)(uri, *args, **kwargs)
2017-08-04T18:53:26.093688+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/knowledge_repo/repository.py", line 76, in __init__
2017-08-04T18:53:26.093788+00:00 app[web.1]:     self.init(**kwargs)
2017-08-04T18:53:26.093790+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/knowledge_repo/repositories/gitrepository.py", line 66, in init
2017-08-04T18:53:26.093912+00:00 app[web.1]:     self.path = self.uri.replace('git://', '')
2017-08-04T18:53:26.093914+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/knowledge_repo/repositories/gitrepository.py", line 88, in path
2017-08-04T18:53:26.093998+00:00 app[web.1]:     path), "Provided path '{}' is not a valid repository.".format(path)
2017-08-04T18:53:26.094002+00:00 app[web.1]: AssertionError: Provided path '/app' is not a valid repository.

But if I launch the server within the heroku bash, it seems to work :

~ $ knowledge_repo --repo . init
Repository already exists. Do you want to convert it to a knowledge data repository? Note that this will override any existing `README.md` and `.knowledge_repo_config.py` files, and replace any submodule at `.resources`. (y/n) y
Knowledge repository created for uri `/app`.
~ $ knowledge_repo --repo . runserver --port $PORT
INFO:alembic.runtime.migration:Context impl SQLiteImpl.
INFO:alembic.runtime.migration:Will assume non-transactional DDL.

and I then get an application error when trying to open the url :

2017-10-12T07:39:40.515362+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=knowledge-repository.herokuapp.com request_id=5496d080-dbff-4c26-8289-99505f8251df fwd="37.71.214.130" dyno= connect= service= status=503 bytes= protocol=https
2017-10-12T07:39:40.901949+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/favicon.ico" host=knowledge-repository.herokuapp.com request_id=6c409e48-6c7c-4602-bf4b-11489357f941 fwd="37.71.214.130" dyno= connect= service= status=503 bytes= protocol=https

[danfrankj]

Hey @alexisrosuel, your experience matches mine exactly. The main issue is, as you mentioned that heroku doesn't copy the .git files and therefore KR doesn't recognize the directory that you specified as a repository. You can init a repo in heroku bash as you mention, but I don't think the changes you make there persist.

I'm not too familiar with Heroku myself but will play around with this to see if there's a way to do this.

[alexisrosuel]

Hey @danfrankj !

Thanks for looking this, don't hesitate to ask me more details if needed :)

[matthewwardrop]

Hi All!

A while back I started work on improving the tooling related to the knowledge repo in https://github.com/airbnb/knowledge-repo/pull/308, and as part of that I added support for "FolderKnowledgeRepositories", which are basically a collection of knowledge posts in zip or folder form inside a folder. That should have no trouble working in a heroku context. Can you confirm? [No configuration changes should be necessary for you, since the FolderKnowledgeRepository takes over the unprefixed knowledge repository path, and to get at git you will have to add a 'git://' prefix].

That work lapsed due to some paid time off, and my wife's being now quite heavily pregnant... but I'll finish it off soon. If that branch does work for you, then I'll merge the FolderKnowledgeRepository support in even if the the actual new scripts are not ready.

[alexisrosuel]

Hi all !

The branch update folder repo works fine for me on Heroku :)

Thanks a lot !

[matthewwardrop]

@alexisrosuel Glad to hear it! Thanks for testing!

We'll leave this issue open until this code makes it into master in case others come by with the same issues :).

[ethen8181]

Hi, was curious if the Heroku setting made it in master yet and if it hasn't is there an expected date. Thanks!

[superfuji57]

Is there another solution or workaround as the folder.py solution is still pending?