fetch command is not working for ndn

[MahdiBaghbani]

Description

ccn-lite-fetch command is not working for ndn as expected while you can get individual named objects using peek with -n flag, there is no way to get stream of objects with fetch

Steps to reproduce the issue

1. use ccn-lite-produce and create a named object stream out of a video file (or any file) with size 2048 (or any size. max 4096)
2. setup relay and set content store to the directory containing files created in last step.
3. use ccn-lite-peek -s ndn2013 -n1 -u a.b.c.d\port URI will successfully receive named object from relay.
4. use ccn-lite-fetch -s ndn2013 -u a.b.c.d\port URI will fail.

Expected results

receive NDO stream with ccn-lite-fetch

relay -v debug for successful ccn-lite-peek:

[D] 37.5000: ccn_iribu_core_RX ifndx=0, 32 bytes
[D] 37.5000:   face 1, peer=127.0.0.1/40826
[D] 37.5000: ccn_iribu_ndntlv_forwarder (32 bytes left)
[D] 37.5000: ccn_iribu_ndntlv_bytes2pkt len=30
[D] 37.5000:   check interest type
[I] 37.5000:   incoming interest=</ndn/test/movie/%00%01>ndn2013 nonce=551599065 from=127.0.0.1/40826
[D] 37.5000:   searching in CS
[D] 37.5000:   matching content for interest, content 0x55f991155f98
[D] 37.5000:   found matching content 0x55f991155f98
[D] 37.5000: udp sendto 127.0.0.1/40826 returned 4096

peek -v trace for successful ccn-lite-peek:

[T] 0.0000: using udp address 127.0.0.1/9998
[T] 0.0001: ccn_iribu_URItoPrefix(suite=ndn2013, uri=/ndn/test/movie)
[D] 0.0002: [T] 0.0002: result: 1, buf: /, avail size: 2048, buflen: 2048
[T] 0.0002: result: 1, buf: n, avail size: 2047, buflen: 2048
[T] 0.0002: result: 1, buf: d, avail size: 2046, buflen: 2048
[T] 0.0002: result: 1, buf: n, avail size: 2045, buflen: 2048
[T] 0.0003: result: 1, buf: /, avail size: 2044, buflen: 2048
[T] 0.0003: result: 1, buf: t, avail size: 2043, buflen: 2048
[T] 0.0003: result: 1, buf: e, avail size: 2042, buflen: 2048
[T] 0.0003: result: 1, buf: s, avail size: 2041, buflen: 2048
[T] 0.0003: result: 1, buf: t, avail size: 2040, buflen: 2048
[T] 0.0003: result: 1, buf: /, avail size: 2039, buflen: 2048
[T] 0.0003: result: 1, buf: m, avail size: 2038, buflen: 2048
[T] 0.0004: result: 1, buf: o, avail size: 2037, buflen: 2048
[T] 0.0004: result: 1, buf: v, avail size: 2036, buflen: 2048
[T] 0.0004: result: 1, buf: i, avail size: 2035, buflen: 2048
[T] 0.0004: result: 1, buf: e, avail size: 2034, buflen: 2048
prefix </ndn><(null)> became /ndn/test/movie
[T] 0.0004: sending request, iteration 0
[T] 0.0004: Packet length: 32
[D] 0.0004: interest has 32 bytes
[D] 0.0005: sendto returned 32
[T] 0.0005:   waiting for packet
[D] 0.0010: received 4096 bytes

Actual results

relay cant find NDO in CS and tries to find it in (possibly) other nodes by re-transmitting, will exist after default 7 re-transmits.

relay -v debug for unsuccessful ccn-lite-fetch:

[D] 139.3000: ccn_iribu_core_RX ifndx=0, 28 bytes
[D] 139.3000:   face 2, peer=127.0.0.1/44862
[D] 139.3000: ccn_iribu_ndntlv_forwarder (28 bytes left)
[D] 139.3000: ccn_iribu_ndntlv_bytes2pkt len=26
[D] 139.3000:   check interest type
[I] 139.3000:   incoming interest=</ndn/test/movie>ndn2013 nonce=1746424685 from=127.0.0.1/44862
[D] 139.3000:   searching in CS
[D] 139.3000:   created new interest entry 0x55f99055c508 (prefix=/ndn/test/movie)
[D] 139.3000:   appending interest entry 0x55f99055c508
[D] 139.3000:   appending a new pendint entry 0x55f99055c578 </ndn/test/movie>(0x55f99055bf28)
[D] 139.3000: ccn_iribu_interest_propagate
[D] 140.3000:  retransmit 0 </ndn/test/movie>
[D] 140.3000: ccn_iribu_interest_propagate
[D] 141.3000:  retransmit 1 </ndn/test/movie>
[D] 141.3000: ccn_iribu_interest_propagate
[D] 142.3000:  retransmit 2 </ndn/test/movie>
[D] 142.3000: ccn_iribu_interest_propagate
[D] 142.3000: ccn_iribu_core_RX ifndx=0, 28 bytes
[D] 142.3000:   face 2, peer=127.0.0.1/44862
[D] 142.3000: ccn_iribu_ndntlv_forwarder (28 bytes left)
[D] 142.3000: ccn_iribu_ndntlv_bytes2pkt len=26
[D] 142.3000:   check interest type
[I] 142.3000:   incoming interest=</ndn/test/movie>ndn2013 nonce=1224701301 from=127.0.0.1/44862
[D] 142.3000:   searching in CS
[D] 142.3000:   appending interest entry 0x55f99055c508
[D] 142.3000:   we found a matching interest, updating time
[D] 143.3000:  retransmit 3 </ndn/test/movie>
[D] 143.3000: ccn_iribu_interest_propagate
[D] 144.3000:  retransmit 4 </ndn/test/movie>
[D] 144.3000: ccn_iribu_interest_propagate
[D] 145.3000:  retransmit 5 </ndn/test/movie>
[D] 145.3000: ccn_iribu_interest_propagate
[D] 145.3000: ccn_iribu_core_RX ifndx=0, 28 bytes
[D] 145.3000:   face 2, peer=127.0.0.1/44862
[D] 145.3000: ccn_iribu_ndntlv_forwarder (28 bytes left)
[D] 145.3000: ccn_iribu_ndntlv_bytes2pkt len=26
[D] 145.3000:   check interest type
[I] 145.3000:   incoming interest=</ndn/test/movie>ndn2013 nonce=183843651 from=127.0.0.1/44862
[D] 145.3000:   searching in CS
[D] 145.3000:   appending interest entry 0x55f99055c508
[D] 145.3000:   we found a matching interest, updating time
[D] 146.3000:  retransmit 6 </ndn/test/movie>
[D] 146.3000: ccn_iribu_interest_propagate
[D] 147.3000:  timeout: remove interest 0x0x55f99055c508 </ndn/test/movie>
[D] 175.3000: face_remove relay=0x55f99053f2c8 face=0x55f99055bd88

fetch -v trace for unsuccessful ccn-lite-fetch:

[T] 0.0000: using suite 6:ndn2013
[T] 4.2e-0: using udp address 127.0.0.1/9998
[T] 7.4e-0: ccn_iribu_URItoPrefix(suite=ndn2013, uri=/ndn/test/movie)
[D] 0.0001: fetching first chunk...
[I] 0.0001: [T] 0.0001: result: 1, buf: /, avail size: 2048, buflen: 2048
[T] 0.0001: result: 1, buf: n, avail size: 2047, buflen: 2048
[T] 0.0001: result: 1, buf: d, avail size: 2046, buflen: 2048
[T] 0.0001: result: 1, buf: n, avail size: 2045, buflen: 2048
[T] 0.0002: result: 1, buf: /, avail size: 2044, buflen: 2048
[T] 0.0002: result: 1, buf: t, avail size: 2043, buflen: 2048
[T] 0.0002: result: 1, buf: e, avail size: 2042, buflen: 2048
[T] 0.0002: result: 1, buf: s, avail size: 2041, buflen: 2048
[T] 0.0002: result: 1, buf: t, avail size: 2040, buflen: 2048
[T] 0.0002: result: 1, buf: /, avail size: 2039, buflen: 2048
[T] 0.0002: result: 1, buf: m, avail size: 2038, buflen: 2048
[T] 0.0002: result: 1, buf: o, avail size: 2037, buflen: 2048
[T] 0.0002: result: 1, buf: v, avail size: 2036, buflen: 2048
[T] 0.0002: result: 1, buf: i, avail size: 2035, buflen: 2048
[T] 0.0002: result: 1, buf: e, avail size: 2034, buflen: 2048
fetching first chunk for prefix '/ndn/test/movie'
[T] 0.0002: Packet length: 28
[W] 3.0020: timeout after block_on_read
[W] 3.0020: timeout
[I] 3.0020: Retry 1 of 3
[D] 3.0020: fetching first chunk...
[I] 3.0020: [T] 3.0020: result: 1, buf: /, avail size: 2048, buflen: 2048
[T] 3.0020: result: 1, buf: n, avail size: 2047, buflen: 2048
[T] 3.0020: result: 1, buf: d, avail size: 2046, buflen: 2048
[T] 3.0020: result: 1, buf: n, avail size: 2045, buflen: 2048
[T] 3.0020: result: 1, buf: /, avail size: 2044, buflen: 2048
[T] 3.0020: result: 1, buf: t, avail size: 2043, buflen: 2048
[T] 3.0020: result: 1, buf: e, avail size: 2042, buflen: 2048
[T] 3.0020: result: 1, buf: s, avail size: 2041, buflen: 2048
[T] 3.0020: result: 1, buf: t, avail size: 2040, buflen: 2048
[T] 3.0020: result: 1, buf: /, avail size: 2039, buflen: 2048
[T] 3.0020: result: 1, buf: m, avail size: 2038, buflen: 2048
[T] 3.0020: result: 1, buf: o, avail size: 2037, buflen: 2048
[T] 3.0020: result: 1, buf: v, avail size: 2036, buflen: 2048
[T] 3.0020: result: 1, buf: i, avail size: 2035, buflen: 2048
[T] 3.0020: result: 1, buf: e, avail size: 2034, buflen: 2048
fetching first chunk for prefix '/ndn/test/movie'
[T] 3.0020: Packet length: 28
[W] 6.0050: timeout after block_on_read
[W] 6.0050: timeout
[I] 6.0050: Retry 2 of 3
[D] 6.0050: fetching first chunk...
[I] 6.0060: [T] 6.0060: result: 1, buf: /, avail size: 2048, buflen: 2048
[T] 6.0060: result: 1, buf: n, avail size: 2047, buflen: 2048
[T] 6.0060: result: 1, buf: d, avail size: 2046, buflen: 2048
[T] 6.0060: result: 1, buf: n, avail size: 2045, buflen: 2048
[T] 6.0060: result: 1, buf: /, avail size: 2044, buflen: 2048
[T] 6.0060: result: 1, buf: t, avail size: 2043, buflen: 2048
[T] 6.0060: result: 1, buf: e, avail size: 2042, buflen: 2048
[T] 6.0060: result: 1, buf: s, avail size: 2041, buflen: 2048
[T] 6.0060: result: 1, buf: t, avail size: 2040, buflen: 2048
[T] 6.0060: result: 1, buf: /, avail size: 2039, buflen: 2048
[T] 6.0060: result: 1, buf: m, avail size: 2038, buflen: 2048
[T] 6.0060: result: 1, buf: o, avail size: 2037, buflen: 2048
[T] 6.0060: result: 1, buf: v, avail size: 2036, buflen: 2048
[T] 6.0060: result: 1, buf: i, avail size: 2035, buflen: 2048
[T] 6.0060: result: 1, buf: e, avail size: 2034, buflen: 2048
fetching first chunk for prefix '/ndn/test/movie'
[T] 6.0060: Packet length: 28
[W] 9.0090: timeout after block_on_read
[W] 9.0090: timeout
[I] 9.0090: Retry 3 of 3

Versions

Operating system: Linux Build environment: GCC 9

[MahdiBaghbani]

I think this problem is about how ccn-lite-fetch sends interests

this is peek command interest received at relay:

[I] 37.5000:   incoming interest=</ndn/test/movie/%00%01>ndn2013 nonce=551599065 from=127.0.0.1/40826
[D] 37.5000:   searching in CS
[D] 37.5000:   matching content for interest, content 0x55f991155f98

interest has chunk number and found in CS

and here is for fetch:

[I] 139.3000:   incoming interest=</ndn/test/movie>ndn2013 nonce=1746424685 from=127.0.0.1/44862
[D] 139.3000:   searching in CS
[D] 139.3000:   created new interest entry 0x55f99055c508 (prefix=/ndn/test/movie)

original URI without chunk number and cannot be found on CS

for some reason fetch is not sending interest with some chunk number prefix and decides to send the main URI.

I couldn't find what is the NDN specification for sending stream requests but I came up with 2 ways to handle this:

1. fetch sends first chunk number instead of the original URI
2. relay search should try to find chunked object files.

I think the first one should be the case since you have already done the same exception for CCNTLV: https://github.com/cn-uofbasel/ccn-lite/blob/da0d9de8d82349dff845acc62d37242dd09b3d3d/src/ccnl-utils/src/ccn-lite-fetch.c#L270-L281

the only change is to add NDN suite to the if statement but I don't know about the comment above it.