CESNET
Netopeer2 is not passing the filtering XPath to sysrepo for an operational callback
root@tsx-100:~# netopeer2-server -V
netopeer2-server 2.0.30
compile time: Dec 7 2021, 12:59:41
root@tsx-100:~#
I start it netopeer2-server this way
root@tsx-100:~# netopeer2-server -d -c MSG,SYSREPO -t 10
And issue a request that looks like
netconf-console -r 20000 --host 192.168.254.2 --port 830 --user root --password canoga --ns xxx=urn:canogaperkins:tsx:time-distribution --get /xxx:switch-timing-details/oper-timing-source
The same request can be done with netopeer2-cli as well with no difference. What I see in the debug output is
[INF]: LN: Accepted a connection on 0.0.0.0:830 from 192.168.254.1:55854.
[INF]: SR: Session 447 (user "root", CID 34) created.
[INF]: LN: Received an SSH message "request-service" of subtype "ssh-userauth".
[INF]: LN: Received an SSH message "request-auth" of subtype "password".
[INF]: LN: User "root" authenticated.
[INF]: LN: Received an SSH message "request-channel-open" of subtype "session".
[INF]: LN: Received an SSH message "request-channel" of subtype "subsystem".
[DBG]: LN: Session 1: Sending message:
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><capabilities><capability>urn:ietf:params:netconf:base:1.0</capability><capability>urn:ietf:params:netconf:base:1.1</capability><capability>urn:ietf:params:netconf:capability:writable-running:1.0</capability><capability>urn:ietf:params:netconf:capability:candidate:1.0</capability><capability>urn:ietf:params:netconf:capability:confirmed-commit:1.1</capability><capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</capability><capability>urn:ietf:params:netconf:capability:validate:1.1</capability><capability>urn:ietf:params:netconf:capability:startup:1.0</capability><capability>urn:ietf:params:netconf:capability:xpath:1.0</capability><capability>urn:ietf:params:netconf:capability:with-defaults:1.0?basic-mode=explicit&also-supported=report-all,report-all-tagged,trim,explicit</capability><capability>urn:ietf:params:netconf:capability:notification:1.0</capability><capability>urn:ietf:params:netconf:capability:interleave:1.0</capability>
[DBG]: LN: Session 1: Sending message:
<capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,https,ftp,ftps,file</capability><capability>urn:ietf:params:xml:ns:yang:ietf-yang-metadata?module=ietf-yang-metadata&revision=2016-08-05</capability><capability>urn:ietf:params:xml:ns:yang:1?module=yang&revision=2021-04-07</capability><capability>urn:ietf:params:xml:ns:yang:ietf-inet-types?module=ietf-inet-types&revision=2013-07-15</capability><capability>urn:ietf:params:xml:ns:yang:ietf-yang-types?module=ietf-yang-types&revision=2013-07-15</capability><capability>urn:ietf:params:netconf:capability:yang-library:1.1?revision=2019-01-04&content-id=32</capability><capability>urn:sysrepo:plugind?module=sysrepo-plugind&revision=2020-12-10</capability><capability>urn:ietf:params:xml:ns:netconf:base:1.0?module=ietf-netconf&revision=2013-09-29&features=writable-running,candidate,confirmed-commit,rollback-on-error,validate,startup,url,xpath</capability><capability>
[DBG]: LN: Session 1: Sending message:
urn:ietf:params:xml:ns:yang:ietf-netconf-acm?module=ietf-netconf-acm&revision=2018-02-14</capability><capability>urn:ietf:params:xml:ns:yang:ietf-netconf-with-defaults?module=ietf-netconf-with-defaults&revision=2011-06-01</capability><capability>urn:ietf:params:xml:ns:yang:ietf-netconf-notifications?module=ietf-netconf-notifications&revision=2012-02-06</capability><capability>urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring?module=ietf-netconf-monitoring&revision=2010-10-04</capability><capability>urn:ietf:params:xml:ns:netmod:notification?module=nc-notifications&revision=2008-07-14</capability><capability>urn:ietf:params:xml:ns:netconf:notification:1.0?module=notifications&revision=2008-07-14</capability><capability>urn:ietf:params:xml:ns:yang:ietf-x509-cert-to-name?module=ietf-x509-cert-to-name&revision=2014-12-10</capability><capability>urn:ietf:params:xml:ns:yang:iana-crypt-hash?module=iana-crypt-hash&revision=2014-08-06</capability><capability>
[DBG]: LN: Session 1: Sending message:
urn:canogaperkins:tsx:types?module=canoga-tsx-types&revision=2020-05-29</capability><capability>urn:ietf:params:xml:ns:yang:iana-if-type?module=iana-if-type&revision=2014-05-08</capability><capability>urn:mef:yang:mef-types?module=mef-types&revision=2018-07-11</capability></capabilities><session-id>1</session-id></hello>
[DBG]: LN: Session 1: Sending message:
]]>]]>
[DBG]: LN: Session 1: Received message:
<?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:params:netconf:base:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:base:1.1</nc:capability><nc:capability>urn:ietf:params:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:candidate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:startup:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp</nc:capability><nc:capability>urn:ietf:params:netconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:xpath:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:notification:1.0</nc:capability><nc:capability>urn:liberouter:params:netconf:capability:power-control:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:interleave:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:with-defaults:1.0</nc:capability></nc:capabilities></nc:hello>
[INF]: SR: Session 448 (user "root", CID 34) created.
[INF]: SR: There are no subscribers for "ietf-netconf-notifications" notifications.
[INF]: NP: Generated new event (netconf-session-start).
[DBG]: LN: Session 1: Received message:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:07912124-2c2d-4e88-b786-257aafe1e475"><nc:get xmlns:xxx="urn:canogaperkins:tsx:time-distribution"><nc:filter type="xpath" select="/xxx:switch-timing-details/oper-timing-source"/></nc:get></nc:rpc>
[INF]: SR: Published event "rpc" "/ietf-netconf:get" with ID 1 priority 0 for 1 subscribers.
[INF]: SR: Processing "/ietf-netconf:get" "rpc" event with ID 1 priority 0 (remaining 1 subscribers).
[INF]: SR: Published event "operational" "/canoga-tsx-time-distribution:switch-timing-details" with ID 12.
[INF]: SR: Event "operational" with ID 12 succeeded.
[INF]: SR: Published event "operational" "/canoga-tsx-time-distribution:switch-timing-details/ptp-port-details" with ID 9.
[INF]: SR: Event "operational" with ID 9 succeeded.
[INF]: SR: Successful processing of "rpc" event with ID 1 priority 0 (remaining 0 subscribers).
[INF]: SR: Event "rpc" with ID 1 priority 0 succeeded.
[DBG]: LN: Session 1: Sending message:
#284
[DBG]: LN: Session 1: Sending message:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:07912124-2c2d-4e88-b786-257aafe1e475"><data><switch-timing-details xmlns="urn:canogaperkins:tsx:time-distribution"><oper-timing-source>local</oper-timing-source></switch-timing-details></data></rpc-reply>
[DBG]: LN: Session 1: Sending message:
##
[INF]: NP: Session 1: thread 4 event new RPC.
[DBG]: LN: Session 1: Received message:
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:adc95a05-83f2-415d-863e-de3e7d1b3e68"><nc:close-session/></nc:rpc>
[DBG]: LN: Session 1: Sending message:
#135
[DBG]: LN: Session 1: Sending message:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:adc95a05-83f2-415d-863e-de3e7d1b3e68"><ok/></rpc-reply>
[DBG]: LN: Session 1: Sending message:
##
[INF]: NP: Session 1: thread 0 event new RPC.
[INF]: NP: Session 1: thread 0 event session terminated.
[INF]: SR: There are no subscribers for "ietf-netconf-notifications" notifications.
[INF]: NP: Generated new event (netconf-session-end).
The incoming message received by netopeer2-server looks correct and has the filter
<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:07912124-2c2d-4e88-b786-257aafe1e475"><nc:get xmlns:xxx="urn:canogaperkins:tsx:time-distribution"><nc:filter type="xpath" select="/xxx:switch-timing-details/oper-timing-source"/></nc:get></nc:rpc>
However it looks like sysrepo is not being called with the filter
[INF]: SR: Published event "operational" "/canoga-tsx-time-distribution:switch-timing-details" with ID 12.
[INF]: SR: Event "operational" with ID 12 succeeded.
[INF]: SR: Published event "operational" "/canoga-tsx-time-distribution:switch-timing-details/ptp-port-details" with ID 9.
[INF]: SR: Event "operational" with ID 9 succeeded.
I have verified in the callback in my code that the path is simply / instead of the filter.
This is causing quite a performance hit as even though I am requesting one leaf, other subordinate callbacks are being called as well. The ultimate result is correct however as I assume the nodes to be returned are being filtered by netopeer2-server.
I have reproduced the same issue with locally built netopeer2-server and sysrepo and the oper_data_pull_example
The netopeer2-cli request
> get --filter-xpath /examples:stats/counter2
DATA
<data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<stats xmlns="urn:examples">
<counter2>1052</counter2>
</stats>
</data>
>
The netopeer2-server debug output for the request
[DBG]: LN: Session 1: Received message:
<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="43"><get xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><filter type="xpath" xmlns:e="urn:examples" select="/e:stats/e:counter2"/></get></rpc>
[INF]: SR: Published event "rpc" "/ietf-netconf:get" with ID 3 priority 0 for 1 subscribers.
[INF]: SR: Processing "/ietf-netconf:get" "rpc" event with ID 3 priority 0 (remaining 1 subscribers).
[INF]: SR: Published event "operational" "/examples:stats" with ID 3.
[INF]: SR: Event "operational" with ID 3 succeeded.
[INF]: SR: Successful processing of "rpc" event with ID 3 priority 0 (remaining 0 subscribers).
[INF]: SR: Event "rpc" with ID 3 priority 0 succeeded.
[DBG]: LN: Session 1: Sending message:
#161
[DBG]: LN: Session 1: Sending message:
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="43"><data><stats xmlns="urn:examples"><counter2>1052</counter2></stats></data></rpc-reply>
[DBG]: LN: Session 1: Sending message:
##
[INF]: NP: Session 1: thread 1 event new RPC.
and the operational callback output
brett@brett-Precision-7740:~/git/netconf/sysrepo/build/examples$ sudo ./oper_data_pull_example examples /examples:stats
Application will provide data "/examples:stats" of "examples".
========== LISTENING FOR REQUESTS ==========
========== DATA FOR "examples" "/examples:stats" REQUESTED ======="/examples:*" REQ ==========
Note that I added in the dumping of the request_path parameter being passed to the callback which is the REQ above
I think the issue is here in netconf.c
for (i = 0; i < filter->count; ++i) {
if (np2srv_get_first_ns(filter->filters[i].str, &start, &len)) {
/* not the simple format, use it as it is */
str = strdup(filter->filters[i].str);
selection = filter->filters[i].selection;
} else {
/* get all the data of a module */
if (asprintf(&str, "/%.*s:*", len, start) == -1) {
str = NULL;
}
selection = 1;
}
Maybe the np2srv_get_first_ns test is wrong and should be negated?
This is likely related to the change for this bug https://github.com/sysrepo/sysrepo/issues/2231?
I guess I don't understand the kind of filter expression that would end up calling this code
/* not the simple format, use it as it is */
str = strdup(filter->filters[i].str);
selection = filter->filters[i].selection;
I am in a dilemma. I need Netopeer2 2.x for the confirmed commit capability that was added but I cannot readily change the Netconf client to use two calls of get-data with running and operational datastores to simulate what get returns, yet at the same time, I cannot have the performance hit of the change that was added in https://github.com/sysrepo/sysrepo/issues/2231 where pretty much any normal xpath filter is not being passed to sysrepo causing all operational callbacks for a module to be called.
Frankly I don't understand why the xpath filter is not being passed to sysrepo when the xpath filter starts with a valid namespace. This same functionality is being used in the implementation of the get-data and the xpath filter is being passed to sysrepo.
Maybe some explanation of what the problem is if the xpath filter with a namespace is passed to sysrepo in the get implementation would make it clearer for me.
Sorry, I wanted to write something but I forgot. It is not that simple to understand why get needs to work the way it does but the change was made based on the use-case in the issue you have referenced, for it to be supported. The only efficient solution is to use get-data. There may be some optimization possible for get but the complexity of them would be quite high. I do not have the time to spend on that and do not even think it is worth it since the operation is basically deprecated.
