Custom neural nets.

[Andrealberti]

Hi, I am working on a project and I need to define a custom model of the neural network, in particular I need to set a conv1d configuration for the inputs. I didn't find so much about this, so I have been posting here hoping you could help me, is it possible to do that? How?. Thanks in advance.

[mmattar]

Hi @Andrealberti - I'm not sure I understand your question. Can you clarify what you're looking for? Also, if this isn't a Bug or Feature Request - you're much better off posting your expanded question on the Forum (https://forum.unity.com/forums/ml-agents.453/).

[Andrealberti]

Hi @mmattar, I have been posting here because I didn't receive any answer to my post in the forum until yesterday morning. This is my request, I am working on MLAgents trying to train a policy in a parking car environment, which I have built. I am using a raycast sensor as a lidar and, because I haven't achieved any results using the ppo implemented policy, I basically want to build a custom trainer with a custom nn (adding a conv1d layer), is it possible to customize a model of neural nets? If the answer is yes, how can I do it? I tryed to work with the Python API following this topic: https://forum.unity.com/threads/q-learning-implementation.915008/ , but without success, in fact, the environment doesn't seem to comunicate with my python script. Regards, Andrea

[mmattar]

Hi @Andrealberti - unfortunately, we don't provide a mechanism to easily customize our trainers, but it is something we're currently exploring. Part of the challenge here is that we wouldn't be able to guarantee that the model you've trained will be supported by Barracuda (the inference library we use). One option which I think you're already exploring based on another GitHub Issue is to implement your own trainer directly using our ml-agents-env or gym-unity wrappers. Does this make sense?

[Andrealberti]

Hi @mmattar, Thanks for your reply, now it is all super clear, I just wanted to have a confirm of what I have seen both in forum and here in the issues. I know the problems of barracuda, but if I am not mistaken, the conv1d layers, which is the one I need, is supported by the library, isn't it? Then, talking about ml-agents-env or gym-unity wrappers, yes, at the moment I was exploring these options, but I have found the implementation very hard, I have communication problems between my UnityEnv and the PythonAPI, here is the error:

mlagents_envs.exception.unitytimeoutexception: the unity environment took too long to respond. make sure that : the environment does not need user interaction to launch the agents' behavior parameters > behavior type is set to "default" the environment and the python interface have compatible versions.

Anyway, now the challenge is trying add a class 1Dencoder here: https://github.com/Unity-Technologies/ml-agents/blob/003e8aed3be28c56ea6b9b7d784cc142d41e7c19/ml-agents/mlagents/trainers/torch/encoders.py adding in it Conv1d layers, the aim is to process the output vector of the lidar as an audio signal, in case of good results I will let you know. Feel free to give me your feedback on my last idea, I will appreciate it. Thanks a lot. Regards, Andrea

[miguelalonsojr]

Hey @Andrealberti , 1d Convs are supported in Barracuda, however, they would have to use the Conv2D operator path, which might make them a bit slow.

Re: the gym wrapper and python API, it seems like there might be an issue the environment itself. If communication to and from Unity hangs, then the python API will timeout. Are you seeing this when training with the editor open or with a build of the environment?

[Andrealberti]

Hi again @miguelalonsojr, I solved the communication problems changing the build settings in Player Settings>Player>Other Settings>Configuration>Scripting Backend from IL2CPP to Mono, so I'm able to start a training using the API. Now I'm struggling with 1d conv, I didn't understand what have you said about that, can you please repeat it? Do you think it makes sense using 1d convs to manage the RayPerceptionSensor outputs signal? Or it would simply be enough to use a bigger dense net? I'm training a vehicle to park and I'm testing the generalization limit of RL, but I haven't get any good result so far with dense nets, so I thought that 1d conv could have been suitable.