[NativeDemo][iOS] when the peer disconnect by other or hangup, and call again the remote video never show up

[xelven]

Hi, I has build up NativeDemo and work all fine at first time, that mean when the App first time start the video audio stream all fine, but if one of hangup the connect and try to call again, the remoteView show nothing. I try to reset, like alloc new or reset still doesn't work:

nativeHandler = [[OpenWebRTCNativeHandler alloc] initWithDelegate:self]; [nativeHandler setSelfView:self.selfView]; [nativeHandler setRemoteView:self.remoteView];

/////////////////////////////////

• (IBAction)handupCallAction:(id)sender { [nativeHandler terminateCall]; }

is there any suggestion?

[xelven]

and I found the [OpenWebRTC initialize] in delegate

• (void)initialize { if (self == [AppDelegate class]) { [OpenWebRTC initialize]; } }

should I reset that in the ViewController?

[stefanalund]

I don't have a good answer right now. You should not have to do [OpenWebRTC initialize]; a second time at least.

[xelven]

BTW, I got this WARNING: ** (:1094): WARNING **: Tag 'self-view' has already been registered for another window handle

** (:1094): WARNING **: Tag 'remote-view' has already been registered for another window handle

is there any way can let me reset those?

[xelven]

Thanks for your response @stefanalund I saw there have the method is our_window_registry void owr_window_registry_unregister(OwrWindowRegistry *registry, const gchar *tag);

Can I call it with parameter like this? owr_window_registry_unregister(owr_window_registry_get(), REMOTE_VIEW_TAG);

[stefanalund]

There are C API's in OpenWebRTC, such as:

owr_window_registry_unregister(owr_window_registry_get(), SELF_VIEW_TAG);

But those have not be exposed up to the iOS SDK yet. Perhaps you can try to add an API in the iOS SDK for unregister the video views, e.g.

[nativeHandler removeSelfView:self.selfView];

[xelven]

Thanks @stefanalund I add two method in OpenWebRTCNativeHandler.h & .m below:

in OpenWebRTCNativeHandler.h

- (void)removeSelfView;
- (void)removeRemoteView;

in OpenWebRTCNativeHandler.m

- (void)removeSelfView
{
    owr_window_registry_unregister(owr_window_registry_get(),SELF_VIEW_TAG);
}
- (void)removeRemoteView
{
    owr_window_registry_unregister(owr_window_registry_get(),REMOTE_VIEW_TAG);
}

and add to hangup every time, but also got any error in second time, and the remote view are black the error log below:

[video/capture] Front Camera

** (<unknown>:1101): CRITICAL **: _owr_window_registry_register_renderer: assertion '!data->renderer' failed
[video/capture] Back Camera
[audio/capture] Default audio input

[xelven]

seems the remote video view are black has other issue relate: after I test few time sometimes after hangup the second time are works sometimes are not, and in Second time will always show the error

[video/capture] Front Camera

** (<unknown>:1101): CRITICAL **: _owr_window_registry_register_renderer: assertion '!data->renderer' failed
[video/capture] Back Camera
[audio/capture] Default audio input

But when the black remote view show up the error is below:

2015-10-23 15:37:37.047 testWebRTC[1117:406209] [OpenWebRTCNativeHandler] Handled remote candidate

(<unknown>:1117): GStreamer-CRITICAL **: Element rtpbin already has a pad named send_rtp_sink_1, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Element dtls_srtp_rtp_encoder_1 already has a pad named rtp_sink_1, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Element srtp-encoder already has a pad named rtp_sink_1, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_1 is not unique in element srtp-encoder, not adding

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_src_1 is not unique in element srtp-encoder, not adding

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_1 is not unique in element dtls_srtp_rtp_encoder_1, not adding

** (<unknown>:1117): WARNING **: (gstdtlssrtpenc.c:315):add_ghost_pad: runtime check failed: (ret)

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_1 is not unique in element send-output-bin-1, not adding

** (<unknown>:1117): WARNING **: (owr_transport_agent.c:1170):link_rtpbin_to_send_output_bin: runtime check failed: (linked_ok)

(<unknown>:1117): GStreamer-CRITICAL **: Element rtpbin already has a pad named send_rtp_sink_2, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Element dtls_srtp_rtp_encoder_2 already has a pad named rtp_sink_2, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Element srtp-encoder already has a pad named rtp_sink_2, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_2 is not unique in element srtp-encoder, not adding

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_src_2 is not unique in element srtp-encoder, not adding

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_2 is not unique in element dtls_srtp_rtp_encoder_2, not adding

** (<unknown>:1117): WARNING **: (gstdtlssrtpenc.c:315):add_ghost_pad: runtime check failed: (ret)

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_2 is not unique in element send-output-bin-2, not adding

** (<unknown>:1117): WARNING **: (owr_transport_agent.c:1170):link_rtpbin_to_send_output_bin: runtime check failed: (linked_ok)

(<unknown>:1117): GStreamer-CRITICAL **: Element rtpbin already has a pad named send_rtp_sink_1, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Element dtls_srtp_rtp_encoder_1 already has a pad named rtp_sink_1, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Element srtp-encoder already has a pad named rtp_sink_1, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_1 is not unique in element srtp-encoder, not adding

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_src_1 is not unique in element srtp-encoder, not adding

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_1 is not unique in element dtls_srtp_rtp_encoder_1, not adding

** (<unknown>:1117): WARNING **: (gstdtlssrtpenc.c:315):add_ghost_pad: runtime check failed: (ret)

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_1 is not unique in element send-output-bin-1, not adding

** (<unknown>:1117): WARNING **: (owr_transport_agent.c:1170):link_rtpbin_to_send_output_bin: runtime check failed: (linked_ok)

(<unknown>:1117): GStreamer-CRITICAL **: Element rtpbin already has a pad named send_rtp_sink_2, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Element dtls_srtp_rtp_encoder_2 already has a pad named rtp_sink_2, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Element srtp-encoder already has a pad named rtp_sink_2, the behaviour of  gst_element_get_request_pad() for existing pads is undefined!

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_2 is not unique in element srtp-encoder, not adding

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_src_2 is not unique in element srtp-encoder, not adding

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_2 is not unique in element dtls_srtp_rtp_encoder_2, not adding

** (<unknown>:1117): WARNING **: (gstdtlssrtpenc.c:315):add_ghost_pad: runtime check failed: (ret)

(<unknown>:1117): GStreamer-CRITICAL **: Padname rtp_sink_2 is not unique in element send-output-bin-2, not adding

** (<unknown>:1117): WARNING **: (owr_transport_agent.c:1170):link_rtpbin_to_send_output_bin: runtime check failed: (linked_ok)
==== Error message start ====
Error in element srtp-encoder.
Error: GStreamer encountered a general supporting library error.
Debugging info: gstsrtpenc.c(1094): gst_srtp_enc_process_buffer (): /GstPipeline:transport-agent-3/GstBin:transport_bin_3/GstBin:send-output-bin-2/GstDtlsSrtpEnc:dtls_srtp_rtp_encoder_2/GstSrtpEnc:srtp-encoder:
Unable to protect buffer (protect failed) code 10
==== Error message stop ====
==== Error message start ====
Error in element srtp-encoder.
Error: GStreamer encountered a general supporting library error.
Debugging info: gstsrtpenc.c(1094): gst_srtp_enc_process_buffer (): /GstPipeline:transport-agent-3/GstBin:transport_bin_3/GstBin:send-output-bin-2/GstDtlsSrtpEnc:dtls_srtp_rtp_encoder_2/GstSrtpEnc:srtp-encoder:
Unable to protect buffer (protect failed) code 10
==== Error message stop ====
==== Error message start ====
Error in element source-18.
Error: Internal data flow error.
Debugging info: gstbasesrc.c(2943): void gst_base_src_loop(GstPad *) (): /GstPipeline:transport-agent-3/GstBin:source-bin-18/GstInterAudioSrc:source-18:
streaming task paused, reason error (-5)
==== Error message stop ====
==== Error message start ====
Error in element source-queue-18.
Error: Internal data flow error.
Debugging info: gstqueue.c(970): gboolean gst_queue_handle_sink_event(GstPad *, GstObject *, GstEvent *) (): /GstPipeline:transport-agent-3/GstBin:source-bin-18/GstQueue:source-queue-18:
streaming task paused, reason error (-5)
==== Error message stop ====
** Message: Got remote source: Remote video stream
2015-10-23 15:37:38.328 testWebRTC[1117:406277] gotRemoteSourceWithName: Remote video stream
** Message: Got remote source: Remote audio stream
2015-10-23 15:37:38.334 testWebRTC[1117:406277] gotRemoteSourceWithName: Remote audio stream

is there any suggestion?

[xelven]

the openwebrtc I install the pakage from your website openwebrtc-devel-0.3.0-ios-universal.pkg and my Podfile:

platform :ios, '8.0'

target ‘testWebRTC’ do
    pod 'InAppSettingsKit', '~> 2.4.4'
    pod 'OpenWebRTC-SDK',  :git => 'https://github.com/EricssonResearch/openwebrtc-ios-sdk.git'
end

target ‘testWebRTCTests' do
    pod 'OpenWebRTC-SDK',  :git => 'https://github.com/EricssonResearch/openwebrtc-ios-sdk.git'
end

[stefanalund]

@adam-be @pererikb restarting a session seem to work in Bowser but not with the native SDK. Could you please provide some pointers on what needs to be done to handle this?

[xelven]

@stefanalund @adam-be @pererikb please help,,,,, please give me some point to handle that, I can coding C/C++. thanks a lot : )

[xelven]

I think I find a way for my test project work,

what I done is just remove the 2 OpenWebRTCVideoView from storyboard, add it by code alloc, insert subview, etc.. and then every after terminateCall, remove2view, remove subview. re-alloc OpenWebRTCVideoView ,insert subview, then will works.

and I will keep looking why this happen

[stefanalund]

This may be needed to properly clean out old references. Is it working for you now?

[xelven]

Yes, remove everything after the hangup even views, then before next call init everything then will works. I not sure what I am done is best way, but however is works for me.

[xelven]

May I ask a question I got error after test few time call & hangup

2015-11-23 17:45:44.874 testWebRTC[8887:1864070] 17:45:44.874 WARNING:  [owr_main_loop] 1251: AURemoteIO::Stop: error 0x10000004 calling TerminateOwnIOThread
2015-11-23 17:45:44.877 testWebRTC[8887:1864439] 17:45:44.877 ERROR:    [AURemoteIO::IOThread] >aurioc> 1503: AURemoteIO@0x1729a020: IOThread exiting with error 0x10004002
[video/capture] Front Camera
[video/capture] Back Camera
[audio/capture] Default audio input
2015-11-23 17:45:44.913 testWebRTC[8887:1864070] gotLocalSourcesWithNames: (
    "Front Camera",
    "Back Camera",
    "Default audio input"
)
[video/capture] Front Camera

** (<unknown>:8887): CRITICAL **: _owr_window_registry_register_renderer: assertion '!data->renderer' failed
[video/capture] Back Camera
[audio/capture] Default audio input
2015-11-23 17:45:56.416 testWebRTC[8887:1864070] gotLocalSourcesWithNames: (
    "Front Camera",
    "Back Camera",
    "Default audio input"
)
** (<unknown>:8887): CRITICAL **: _owr_media_source_release_source: assertion 'GST_IS_ELEMENT(source)' failed

(<unknown>:8887): GStreamer-CRITICAL **: gst_element_set_state: assertion 'GST_IS_ELEMENT (element)' failed

(<unknown>:8887): GStreamer-CRITICAL **: gst_bin_remove: assertion 'GST_IS_ELEMENT (element)' failed

what's it meaning?

2015-11-23 17:45:44.874 testWebRTC[8887:1864070] 17:45:44.874 WARNING:  [owr_main_loop] 1251:  [owr_main_loop] 1251: AURemoteIO::Stop: error 0x10000004 calling TerminateOwnIOThread
2015-11-23 17:45:44.877 testWebRTC[8887:1864439] 17:45:44.877 ERROR:    [AURemoteIO::IOThread] >aurioc> 1503: AURemoteIO@0x1729a020: IOThread exiting with error 0x10004002

[xelven]

@stefanalund @superdump Hi guys, I think I figure out why this happened. since I checkout the latest version of openwebrtc and this still happened. I think I fixed in my test project.

** (<unknown>:1101): CRITICAL **: _owr_window_registry_register_renderer: assertion '!data->renderer' failed

don't know why, when the hangup the reset called, and the renderer should be unref then should be call owr_video_renderer_finalize to clean the renderer reference. But GLib didn't call the finalize function pointer.

after I test all fine, now I looking into the other crash issue in latest of openwebrtc.

ERROR:owr_transport_agent.c:917:remove_existing_send_source_and_payload: assertion failed: (sinkpad)

[virorum]

Is there an update on this? I am getting the same issue with hanging up a call and preparing to start a new one.

[xelven]

@virorum I fixed in my test project, but not sure it is best way. Just call the owr_video_renderer_finalize when you hangup in the reset method.

[virorum]

Do you have a code snippet that shows the fixed reset method that you can share?

[xelven]

@virorum sure about that bro, my way like this in reset() for added the owr_video_renderer_finalize

static void reset()
{
    GList *media_sessions, *item;
    OwrMediaRenderer *renderer;
    OwrMediaSession *media_session;

    is_offering = FALSE;
    is_answering = FALSE;

    stopRtcTimer();

    if (renderers) {
        for (item = renderers; item; item = item->next) {
            renderer = OWR_MEDIA_RENDERER(item->data);
            owr_media_renderer_set_source(renderer, NULL);
            if(OWR_IS_VIDEO_RENDERER(renderer))
                owr_video_renderer_finalize(renderer);
        }
        g_list_free_full(renderers, g_object_unref);
        renderers = NULL;
    }

then you copy the owr_video_renderer_finalize this method from openwebrtc,

[virorum]

Thanks @xelven - copying the owr_video_renderer_finalize method doesn't seem straight forward :(

[virorum]

Just copying that method in doesn't work as there are references which are no longer valid, so it would be great to see how you adapted the finalize method to work. Can you share this also?

[xelven]

@virorum nice, then you just copy everything they references. for example:

typedef struct {
    GHashTable *table;
    GMutex mutex;
} OwrMessageOriginBusSet;

typedef struct {
    guintptr window_handle;
    gboolean window_handle_set;

    OwrVideoRenderer *renderer; /* weak reference */
} WindowHandleData;

struct _OwrWindowRegistryPrivate {
    GHashTable *registry_hash_table;
    OwrMessageOriginBusSet *message_origin_bus_set;
};

struct _OwrMediaRendererPrivate {
    GMutex media_renderer_lock;
    OwrMediaType media_type;
    OwrMediaSource *source;
    gboolean disabled;

    GstElement *pipeline;
    GstElement *src, *sink;
    OwrMessageOriginBusSet *message_origin_bus_set;
};

they just need the struct memory size for that pointer : ) any problem let me know~

[xelven]

@virorum did u got it, bro?

[virorum]

No :( I must be missing some vital piece. I couldn't find an implementation that uses this in any of the SDK forks either.

[xelven]

@virorum in OpenWebRTCNativeHandler.m please make sure all struct has copy.

void _owr_window_registry_unregister_renderer(OwrWindowRegistry *window_registry,
                                              const gchar *tag, OwrVideoRenderer *video_renderer)
{
    OwrWindowRegistryPrivate *priv;
    WindowHandleData *data;

    g_return_if_fail(OWR_IS_WINDOW_REGISTRY(window_registry));
    g_return_if_fail(OWR_IS_VIDEO_RENDERER(video_renderer));

    priv = window_registry->priv;

    data = g_hash_table_lookup(priv->registry_hash_table, tag);
    g_return_if_fail(data);
    g_return_if_fail(data->renderer == video_renderer);

    data->renderer = NULL;
    if (!data->window_handle_set)
        g_hash_table_remove(priv->registry_hash_table, tag);
}

static void owr_video_renderer_finalize(GObject *object)
{
    OwrVideoRenderer *renderer = OWR_VIDEO_RENDERER(object);
    OwrVideoRendererPrivate *priv = renderer->priv;

    if (priv->tag) {
        _owr_window_registry_unregister_renderer(owr_window_registry_get(), priv->tag, renderer);
        g_free(priv->tag);
        priv->tag = NULL;
    }

//    G_OBJECT_CLASS(owr_video_renderer_parent_class)->finalize(object);
}

static void reset()
{
    GList *media_sessions, *item;
    OwrMediaRenderer *renderer;
    OwrMediaSession *media_session;

    is_offering = FALSE;
    is_answering = FALSE;

    if (renderers) {
        for (item = renderers; item; item = item->next) {
            renderer = OWR_MEDIA_RENDERER(item->data);
            owr_media_renderer_set_source(renderer, NULL);
//            g_object_unref(renderer);
            if(OWR_IS_VIDEO_RENDERER(renderer))
                owr_video_renderer_finalize(renderer);
//            g_free(renderer);
            renderer = NULL;
        }
//        g_list_free_full(renderers, g_object_unref);
        g_list_free(renderers);
        renderers = NULL;
    }

    if (transport_agent) {
        media_sessions = g_object_steal_data(G_OBJECT(transport_agent), "media-sessions");
        for (item = media_sessions; item; item = item->next) {
            media_session = OWR_MEDIA_SESSION(item->data);
            owr_media_session_set_send_source(media_session, NULL);
        }
        g_list_free(media_sessions);
        g_object_unref(transport_agent);
        transport_agent = NULL;
    }

    g_list_free(local_sources);
    local_sources = NULL;
    local_video_renderer = NULL;
    remote_video_renderer = NULL;
    staticSelf.localSourceArray = [NSMutableArray array];
}

[luiyen]

aw.I meet the problem when hangup call too ERROR:owr_transport_agent.c:917:remove_existing_send_source_and_payload: assertion failed: (sinkpad)

[xelven]

I found few way for avoid this crash, but still have other question. I create a issue in openwebrtc

[yingzhuo1314]

@pubbus Do you have solve this problem?? I meet the same problem when hangup call too.

[luiyen]

@yingzhuo1314 @xelven created a issue in https://github.com/EricssonResearch/openwebrtc/issues/574

[cwack-letsdev]

my solution was like that. First copy some methods from OpenWebrtc Project and overwrite them. Then check if video renderer and do a finalize before.

//BEG OVERWRITTEN STUFF

typedef struct { GHashTable *table; GMutex mutex; } OwrMessageOriginBusSet;

typedef struct { guintptr window_handle; gboolean window_handle_set;

OwrVideoRenderer *renderer; /* weak reference */

} WindowHandleData;

struct _OwrVideoRendererPrivate { guint width; guint height; gdouble max_framerate; gint rotation; gboolean mirror; gchar *tag; };

struct _OwrWindowRegistryPrivate { GHashTable *registry_hash_table; OwrMessageOriginBusSet *message_origin_bus_set; };

void _owr_window_registry_unregister_renderer_OVERWRITTEN(OwrWindowRegistry *window_registry, const gchar *tag, OwrVideoRenderer *video_renderer) { OwrWindowRegistryPrivate *priv; WindowHandleData *data;

g_return_if_fail(OWR_IS_WINDOW_REGISTRY(window_registry));
g_return_if_fail(OWR_IS_VIDEO_RENDERER(video_renderer));

priv = window_registry->priv;

data = g_hash_table_lookup(priv->registry_hash_table, tag);
g_return_if_fail(data);
g_return_if_fail(data->renderer == video_renderer);

data->renderer = NULL;
if (!data->window_handle_set)
    g_hash_table_remove(priv->registry_hash_table, tag);

}

static void owr_video_renderer_finalize_OVERWRITTEN(GObject *object) { OwrVideoRenderer *renderer = OWR_VIDEO_RENDERER(object); OwrVideoRendererPrivate *priv = renderer->priv;

if (priv->tag) {
    if(OWR_IS_VIDEO_RENDERER(renderer)){
        _owr_window_registry_unregister_renderer_OVERWRITTEN(owr_window_registry_get(), priv->tag, renderer);
    }
    g_free(priv->tag);
    priv->tag = NULL;
}

//G_OBJECT_CLASS(owr_video_renderer_parent_class)->finalize(object);

}

//END OVERWRITTEN STUFF

static void reset() { GList *media_sessions, *item; OwrMediaRenderer *renderer; OwrMediaSession *media_session;

is_offering = FALSE;
is_answering = FALSE;

if (renderers) {
    for (item = renderers; item; item = item->next) {
        renderer = OWR_MEDIA_RENDERER(item->data);
        if(OWR_IS_VIDEO_RENDERER(renderer)){
           owr_video_renderer_finalize_OVERWRITTEN(renderer);
            owr_media_renderer_set_source(renderer, NULL);
        }
        else {

            if(OWR_IS_AUDIO_RENDERER(renderer)){
                owr_media_renderer_set_source(renderer, NULL);
            }
        }
    }
    g_list_free_full(renderers, g_object_unref);
    renderers = NULL;
}

etc....