XOA Quick Deploy with Proxy

[cocoon]

The "XOA Quick Deploy" fails for me as I needs to use a proxy.

It would be great if the modals would offer to configure two optional proxies:

1. XCP-NG Server Proxy with the option to use it "temporary (only for the XOA deployment)" or "permanent"

2. XOA Proxy with the option to use the same as configured before or a different one, and this will auto configure the XOA appliance with the needed proxy information

I tried to configure the XCP-NG Server proxy manually in /etc/profile but that did not work for the deployment. Still trying to find the right place ...

Update: It seems like xe isn't using a proxy at all, that might need changes for the api to be able to use the proxy?

Just found your request here, where they call it "vm-import from a URL is unsupported": https://bugs.xenserver.org/browse/XSO-740

But maybe you can work it out somehow.

Just thinking, maybe it would be possible to extend the api with a new command to fetch the file from an url using a proxy, something like "xe proxy-get sr-uuid=... url=... proxy=http://my.proxy:8080"

[julien-f]

What happens exactly? Did you get an error message?

Todo:

1. figure out how to configure the proxy used by the XAPI
   • from the XAPI itself?
   • from the environment? does it need to be restarted?
2. if not possible, can we proxy the import by the web page itself?

[cocoon]

Yes I get an error regarding a timeout.

As far as I researched, the starting call comes from here: https://github.com/xcp-ng/xcp-ng-release/blob/2b1cc963bed972f02d5b9b9e9dcdb2b9271980b5/src/xenserver/opt/xensource/www/asset/deploy.js#L144

'VM.import', 'http://xoa.io:8888/

I can see it is resolving the IP using my DNS but the proxy is unused. I did the same from the commandline using putty to the xcp-ng server:

xe vm-import sr-uuid=9... url=http://xoa.io:8888/

And this is the same picture, DNS yes, proxy no.

If I do a simple "wget http://google.de" from the commandline, DNS and Proxy is used automatically.

The same just for fun I trried this: xe vm-import sr-uuid=9... url=http://myproxy:8080/ and I see the activity on the proxy.

[olivierlambert]

Can you reach the port 8888 in question? Do you have a firewall or something blocking it maybe?

[cocoon]

I can reach it without a problem, but I can see that there is no activity at the proxy at all, so no firewall issue here.

[olivierlambert]

Does wget http://xoa.io:8888/ works?

[julien-f]

@olivierlambert The issue is that vm.import of the XAPI does not take the http_proxy env variable into account (and does not appear to support proxies at all).

[olivierlambert]

So it won't work behind a proxy then :/ The only way would be to modify XAPI to handle it. @johnelse any hint?

[johnelse]

I agree that xapi would need modifying to support importing via proxy.

xapi actually already looks up the http_proxy environment variable in a couple of places - see Xapi_support.do_upload and Xapi_host.bugreport_upload.

[erSitzt]

I dont understand why this does not use one of the default ports that work most of the time (80/443), even with most defaults firewall policies and transparent proxies ?? And not beeing able to use the proxy env vars makes this even worse...

Is there a good reason for only using port 8888 ??

And sorry @olivierlambert i posted this on the forum before i found this issue

[olivierlambert]

Yes, there's a good reason, as you can imagine: with (now) old version of XenServer, we needed to use HTTP/1.0

So for that, we had to write a dedicated web server and listen to a port outside our already used classical web servers (443/80).

Now, since XS 7.1 (IIRC, we should check) we can import directly form an URL with HTTP/1.1

I'm AFK for 2 weeks, but I suppose @julien-f can take a look a provide a script with an URL directly accessible in :443.

[erSitzt]

Thanks @olivierlambert for the info. A little hint in the docs would help, because the error in the cli or using the web interface is not very helpful. And maybe this turns off people, looking for an VMware alternative like me, when the first more or less cryptic error show up almost immedeately after installing the first node.

After i found this issue i just added the required firewall policy and it worked fine, thanks !

[julien-f]

@olivierlambert HTTPS appears to still be unsupported in XCP-ng 8.2.1:

> xe vm-import url=https://xoa.io/xoa.xva
The server failed to handle your request, due to an internal error. The given message may give details useful for debugging the problem.
message: (Failure "Unsupported URI scheme: https")

[olivierlambert]

Hmm that's weird. We did test it in HTTPS last time. Are we using an HTTPS 1.1 server?

[julien-f]

AFAICT, the error is triggered before attempting to connect to the server.

And yes, Caddy supports both HTTP 1.1 and 2.

[olivierlambert]

Okay so after some checking:

• that's HTTP 1.1 which supported since 7.1 (we should probably remove our custom web server for older XenServer, and serve it normally now)
• HTTPS for import isn't working yet. We are raising the priority internally to get VM import in HTTPS, expect some news ideally before the end of the year.