Smb file multiflow 4861 v4.1

[catenacyber]

Link to redmine ticket: https://redmine.openinfosecfoundation.org/issues/4861

Describe changes:

• smb : handle multi-stream file transfers

Continuation of #7863

This is a draft for feedback.

Questions :

• How good is this SMB2_FILE_ENDOFFILE_INFO assumption ? cf https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/341f4ead-ce46-46ca-8f6c-4f3e10caf204
• Should we handle reads as writes ?
• This draft is SMB2 only, would we want SMB1 ? Style questions :
• Should we move ContainerTHashTable ContainerSmbHt to app-layer-smb.c ?
• Should app-layer-htp-range.h be renamed util-file-range.h ?
• Should we make HTPFileCloseHandleRange generic ?

Blocker question Also, the current multi-stream logic will not log a file until it is complete. Because there may be a new flow coming that will complete the file. And because to log it, we have to move its ownership from the global hash table to the transaction which will end up freeing it... Thoughts about this @victorjulien ? I now think the ownership should not be moved back to a transaction and the logging happen on its own...

TODOs:

• There are other SMB tickets to create based on the comments
• I realize there is still doc to be written about app-layer.protocols.http.byterange.memcap

[catenacyber]

PS : if you compile with FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION your hash memcap is 16Mbytes unconfigurable...

[catenacyber]

More TODO: Fuzzing bugs to be fixed

[suricata-qa]

Information:

ERROR: QA failed on smb_files_sha256.

ERROR: QA failed on tlpw1_files_sha256.

field	baseline	test	%
	tlpw1_stats_chk
.app_layer.error.http.parser	64	47	73.44%
	tlpr1_stats_chk
.app_layer.error.http.parser	1548	1103	71.25%
	generic_stats_chk
.capture.kernel_drops	0	76906	-
.flow.end.tcp_state.syn_sent	0	116	-
.flow.end.tcp_state.syn_recv	0	1	-
.flow.end.tcp_state.fin_wait1	0	16	-
.flow.end.tcp_state.fin_wait2	0	4	-
.flow.end.tcp_state.time_wait	0	5	-
.flow.end.tcp_state.last_ack	0	3	-
.flow.end.tcp_state.close_wait	0	24	-
.tcp.reassembly_gap	80952	89850	110.99%
.app_layer.error.http.parser	0	13	-
.app_layer.error.ftp.gap	0	3	-
.app_layer.error.smtp.gap	0	20	-
.app_layer.error.dcerpc_tcp.parser	0	2	-

Pipeline 9209

[catenacyber]

Waits for CI fix cf https://github.com/OISF/suricata/pull/7871

But progress can be made meanwhile cd other TODOs and questions

[catenacyber]

Replaced by https://github.com/OISF/suricata/pull/7920