videokit icon indicating copy to clipboard operation
videokit copied to clipboard

White shadows issue in JPEG image sequence in iOS AR app having linear color space

Open thesanketkale opened this issue 1 year ago • 0 comments

I am seeing the shadows coming white in the JPEG image sequences on iOS in our AR app having Linear color space built with Unity 2022.3.16f1. On Android, the shadows look fine but on iOS all shadows appear white in the JPEG image sequences. I am using VideoKit version 0.0.18alpha2 and Function version 0.0.4.

Also, just for your reference, the shadows are coming fine in the MP4 video recorded with the same asset on the same device with the same app.

I have attached the shader I am using for casting shadows on detected planes along with a demo image of the JPEG sequence.

White_Shadow_in_AR_on_iOS

Here is the shader that I am using for generating shadows in AR:

Shader "InvisibleShadowCaster"
{
	Properties
	{
		_ShadowIntensity("Shadow Intensity", Range(0, 1)) = 0.6
	}
	SubShader
	{
		Tags{ "Queue" = "AlphaTest" }
		Pass
		{
			Tags{ "LightMode" = "ForwardBase" }
			Cull Back
			ZWrite Off
			Blend SrcAlpha OneMinusSrcAlpha
			CGPROGRAM
			#pragma vertex vert
			#pragma fragment frag
			#pragma multi_compile_fwdbase
			#include "UnityCG.cginc"
			#include "AutoLight.cginc"
			uniform float _ShadowIntensity;
			struct v2f
			{
				float4 pos : SV_POSITION;
				LIGHTING_COORDS(0,1)
			};
			v2f vert(appdata_base v)
			{
				v2f o;
				o.pos = UnityObjectToClipPos(v.vertex);
				TRANSFER_VERTEX_TO_FRAGMENT(o);

				return o;
			}
			fixed4 frag(v2f i) : COLOR
			{
				float attenuation = LIGHT_ATTENUATION(i);
				return fixed4(0,0,0,(1 - attenuation)*_ShadowIntensity);
			}
			ENDCG
		}
	}
	Fallback "VertexLit"
}

P.S. - I don't remember exactly when but I think I had raised a similar issue back in the day with white shadows coming in the mp4 videos recorded using NatCorder on iOS and you had solved it with some color correction for linear color space. Maybe you need to apply the fix to JPEG image sequences as well.

thesanketkale avatar Feb 12 '24 13:02 thesanketkale