[WIP] IPFS Cluster Integration

[sanderpick]

Adds the ability to sidecar IPFS Cluster on Textile.

Changelog

coming

Closes

• Closes #734

Fixes

• The --repo flag was no longer parsing ~ (missed in the kingpin refactor)

TODO

• [x] Implement IPFSConnector (ipfs/cluster.go)

[balupton]

• The --repo flag was no longer parsing ~ (missed in the kingpin refactor)

In the code diff, where is the code for this change? If we are doing --repo "~/repo" that is incorrect behaviour to expand the ~, as the expansion of ~ is a shell script feature, rather than something we should handle. For instance, the valid way to do this would be --repo ~/repo, or if the path has special characters inside of it, then these work --repo ~/"repo" as well as --repo "$HOME/repo".

E.g.

# this will output ~/repo as valid intentional behaviour
# as otherwise we can't do filenames that have ~ in them
echo "~/repo"

# these will output the home directory: /Users/balupton/repo
echo ~/repo
echo ~/"repo"
echo "$HOME/repo"

[sanderpick]

Right, but I'm okay being "incorrect" here in support of a better UX. I'll stand corrected the day somebody complains because they actually had a directory called "~" :)

[sanderpick]

Ok, @hsanjuan - I've got all the pieces in place here. However, running into an authorization error when calling SyncAll (https://github.com/textileio/go-textile/blob/sander/cluster/cluster/main_test.go#L188):

17:31:31.842 ERROR  p2p-gorpc: error handling RPC: client does not have permissions to this method, service name: Cluster, method name: SyncAllLocal server.go:176
17:31:31.859 DEBUG    cluster: rpc auth error client does not have permissions to this method, service name: Cluster, method name: SyncAllLocal cluster.go:1597

If I understand correctly, the RPC auth is based solely on the cluster secret. They are matching in the test-generated config files.

I'm putting this down for now but let me know if you have any quick ideas...

[sanderpick]

Something occurred to me at dinner... I think the intention here was to not use a secret since the host needs to be on the main IPFS network. Instead, use the new trusted peers mechanism. That seems to work, but so far I haven't been able to get the pin state of the local daemon to reflect in the cluster consensus. More later...

[hsanjuan]

Something occurred to me at dinner... I think the intention here was to not use a secret since the host needs to be on the main IPFS network. Instead, use the new trusted peers mechanism. That seems to work, but so far I haven't been able to get the pin state of the local daemon to reflect in the cluster consensus. More later...

Yes, correct. You will need to set TrustedPeers to the peer IDs of your other cafes (suggestions to improve this for your usecase accepted). Cluster secret is not used at all since you already have a host without a network protector.

Can I get write access here, I will have a look to the code and it may be useful now or in the future to commit some changes to the branch (simpler than a PR to the PR :P)

[sanderpick]

Thanks for the comments @hsanjuan! You should have write access now.