Use relative URLs in PROTO files

[omichel]

I ran this script in the WEBOTS_HOME/projects folder to perform the conversion automatically:

import glob

for file in glob.glob('**/*.proto', recursive=True):
    unix_file = file.replace('\\', '/')
    search = '"webots://projects/' + unix_file[:unix_file.rfind('/') + 1]
    with open(file, 'r') as f:
        data = f.read()
    new_data = data.replace(search, '"')
    if new_data != data:
        print(unix_file)
        with open(file, 'w', newline='\n') as f:
            f.write(new_data)

[omichel]

I am fixing this: reverting to webots:// URLs in the header part of the PROTO files.

[omichel]

I used this script to revert the webots://... URLs in the header of PROTO files and fix the problem reported:

import glob
import os

for file in glob.glob('**/*.proto', recursive=True):
    unix_file = file.replace('\\', '/')
    url = '"webots://projects/' + unix_file[:unix_file.rfind('/') + 1]
    fd = open(file, 'r')
    output = open('tmp.proto', 'w', newline='\n')
    for line in fd:
        if 'field MFString' in line and '"webots://' not in line and '[]' not in line and ('.jpg"' in line or '.png"' in line):
            print(unix_file)
            print(line)
            new_line = line.replace('"', url, 1)
            print(new_line)
        else:
            new_line = line
        output.write(new_line)
    fd.close()
    output.close()
    os.remove(file)
    os.rename('tmp.proto', file)

[stefaniapedrazzi]

When remote PROTO models are edited and copied to the local project, relative URLs are no longer working and the assets are not found. How do you handle it? What is the advantage of using relative URLs?

[omichel]

When remote PROTO models are edited and copied to the local project, relative URLs are no longer working and the assets are not found. How do you handle it?

We should probably download the resources corresponding to the relative URL, in addition of the PROTO...

What is the advantage of using relative URLs?

It is simpler for the user to set them when creating a PROTO file.

[stefaniapedrazzi]

It is simpler for the user to set them when creating a PROTO file.

Yes, I agree. For custom PROTOs is easier to use relative URLs. But I think that it is easier for users if the default PROTOs we distribute use remote URLs. Otherwise users will have lot issues when copying and moving files between projects. I think that the remote paths are simplifying a lot assets dependencies, so I'm not sure that the changes in this PR are really good for users.

[stefaniapedrazzi]

We should probably download the resources corresponding to the relative URL, in addition of the PROTO...

If we still have to copy the assets to local projects, we loose all the advantages to use remote URLs.

[omichel]

We should probably download the resources corresponding to the relative URL, in addition of the PROTO...

If we still have to copy the assets to local projects, we loose all the advantages to use remote URLs.

You are right, downloading the assets is not a good idea. Instead we should convert the relative URLs to https:// URLs (as we do for the webots:// URLs).

[omichel]

However the mechanism to convert relative urls to https when editing the proto is not implemented and thus editing protos is broken.

I will be taking care of this in a new PR.

[BenjaminDeleze]

I will be taking care of this in a new PR.

Is it a good practice as it will introduce a bug in master?

[omichel]

Is it a good practice as it will introduce a bug in master?

I mean we won't merge this PR until the other PR fixing the issue is merged, so that the master branch won't be broken.

[BenjaminDeleze]

Is it possible to fix the conflicts such that I can test with the fix of #5051 ?

[omichel]

I just fixed the problem.

However, it remains a little problem with the controller path which is not found. I believe one way to fix it would be to compute the controller path to find the controller inside the Webots installation folder. However, this solution is not nice because it would either set the controller as a long relative path that would break if the project is moved or as a an absolute path which is not portable. An alternative to fixing the controller path would be to copy the controller from the Webots installation directory to the local project, but this seems a bit overkill as the user initially just wants to use the PROTO, not necessarily copy its default controller.

So, I see two options:

• change the controller to "".
• ask the user if she want the default robot controller to be copied in its project or if she prefers the controller to set to "".
• Do nothing and let the user fix the controller path.

I would tend to prefer the first one as it simplifies the user experience and does what it is supposed to do (e.g., download a PROTO in the local project, not a controller). The default controller is anyhow used when running the provided demo worlds.

[omichel]

We also have the same problem for the robot window and the controllerArgs parameters. I propose to set the window to <generic> and controllerArgs to [].

[BenjaminDeleze]

I think that even in the previous version the controller path was broken (so solution 3).

But I would say that solution 1 is nicer.

Solution 2 is nice too because it offers more choice but I agree that I might be overkill.

So I would go with solution 1 (and put the robot window to generic and so on).

But I would put a message to inform the user that controller/robot window will not be copy. Otherwise I am afraid that they will download a robot, change its color and not understand why the robot is not moving anymore.

[ygoumaz]

I believe this would fix issue #4905.

[omichel]

I am done with the implementation: the controller and window fields are set to <generic>. However, I didn't change the controllerArgs for several reasons:

• It doesn't harm to leave it to the original values, which is in 99% of the cases []. The only exceptions are:
  • Thymio2.proto: "port=33333"
  • InvertedPendulumBenchmark: [ "seed=2" ]
  • VisualTrackingBenchmark.proto: [ "hit-error=0.1" ] Even for these rare cases, it's not a problem to leave them as is.
• It is difficult to do it in a generic way and handle multi-line values if any.

[omichel]

I added a message for robots to warn that the controller/window fields will be set to generic. I believe the remote control problem is not critical. We could possibly add a proto parameter to enable it and leave it disabled by default, so that we don't get this warning.

[omichel]

I found a more elegant fix for the E-puck and Darwin-op robots: if the window is set to <generic> or <none>, the remote control library is not initialized, thus no warning is raised. However, it will raise as soon as the user sets a specific robot window (which should be pretty rare and only for advanced users).