Please take a look into the article and improve the system with additional options
Here is an article: http://robotinvader.com/blog/?p=460 Edit: here is an even more up to date article from the same author http://robotinvader.com/blog/?p=554
What it is is basically instead of the raycasting it uses camera panoramic view (can be used the same camera as you character instead for ease of use), and marks each mesh with a unique color. Based on the unique color you create a dictionary of color and mesh accordingly. Based on the color and compute shader you can get the occluded data (a color which will not be visible will signal the mesh that will not be visible).
In theory, you can achieve greater performance this way (no raycasts) but it does require to use compute shader at runtime.
I'm very curious, @Kink3d what do you think about it?
Also, here is another approach https://developer.oculus.com/blog/occlusion-culling-for-mobile-vr-developing-a-custom-solution/