speedup development by using skaffold file sync

[jlewi]

trafficstars

PR #77 contains skaffold.yaml files to continuously build and deploy the microservices.

Skaffold is building the containers on the cluster using Kaniko. The builds look like they are taking O(minutes) to rebuild.

We should be able to speed that up by taking advantage of skaffold's file sync feature to directly synchronize files to the containers and skipping the container rebuilds.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label kind/feature to this issue, with a confidence of 0.81. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[jlewi]

I gave this a quick try and skaffold gives me the warning

WARN[0772] error adding dirty artifact to changeset: inferring syncmap for image gcr.io/issue-label-bot-dev/bot-worker: SyncMap is not supported by this builder 

Opend GoogleContainerTools/skaffold#3448

[jlewi]

Using manual mode for skaffold file sync seems to work; Here's my sample stanza

    sync:
        manual:
        - src: 'py/code_intelligence/*.py'
          dest: '/'
        - src: 'py/label_microservice/*.py'
          dest: '/'        

I was relying on flask's reloader to reload things when the code changed. If instead of using flask we just have a binary running pubsub; we might need to run it in a subprocess and restart it ourselves.

Then again maybe we should continue to use flask to do things like a health check.

[jlewi]

Also hit GoogleContainerTools/skaffold#3454

[jlewi]

I created a simple file to run a program in a subprocess and auto restart it whenever some files change. https://github.com/kubeflow/code-intelligence/blob/06125b0cd2e25e01b1ec67cf444987747f071600/py/code_intelligence/run_with_auto_restart.py

So far this appears to be working as expected with skaffold's file sync.

[psykidellic]

I hit the same issue today using the custom builder. If i set infer, it gives me the synmacp error. But doing manual worked. This is a simple Flask based app. Let me know if you need more info to work on this or some other issue, i can respond to.