Netgoblin fork

[warsang]

The Netgoblin fork is part of a Conix project which aims to create a tool for assisting protocol inference. Netgoblin is intended to work with the TAPIRE CLI another part of the project which has not been open-sourced yet (as it is not yet fully developed). The Netgoblin fork features:

• Several "bug" corrections :

1. TypeEncodingFunctions + ASCII conversion would delete data (refer to https://github.com/warsang/netzob/commit/09ae2ddd97cabb65ef65ee2b67c2fd86566140b8).
2. Support for the Alt domain field in the Value class
3. Support for the Alt domain field in the InternetChecksum
4. Cells displayed in ISO-8859-1

• CRC32 domain field + doctests
• CRC32 Seeker and Field creation + doctests
• CRC32 symbol clustering (lacks doctests for now)
• IP seeker and Field creation + doctests
• Size seeker capable of finding inside field Size relations + automatic Field creation + doctests
• HeaderDetector + doctests
• Value persisten SVAS, allows operations such as incremental field specialisation (see doctests)
• RelationFinder now supports data and equality relations
• Session, Source IP and Destination IP in symbol display
• Session added to symbol in PCAP importer

[codecov-io]

Codecov Report

Merging #103 into next will increase coverage by 0.64%. The diff coverage is 60.43%.

@@            Coverage Diff             @@
##             next     #103      +/-   ##
==========================================
+ Coverage   68.54%   69.18%   +0.64%     
==========================================
  Files         157      159       +2     
  Lines        9484    10694    +1210     
==========================================
+ Hits         6501     7399     +898     
- Misses       2983     3295     +312

Impacted Files	Coverage Δ
netzob/src/netzob/Inference/Grammar/GenericMAT.py	0% <ø> (ø)	:arrow_up:
netzob/src/netzob/Model/Vocabulary/Types/Raw.py	85.13% <ø> (ø)	:arrow_up:
.../Vocabulary/Domain/Specializer/FieldSpecializer.py	94.28% <ø> (ø)	:arrow_up:
.../netzob/Model/Vocabulary/Domain/Specializer/all.py	100% <ø> (ø)	:arrow_up:
...ocabulary/Domain/Specializer/MessageSpecializer.py	76.69% <ø> (ø)	:arrow_up:
...unctions/EncodingFunctions/ZLibEncodingFunction.py	96.96% <ø> (ø)	:arrow_up:
...ob/Model/Grammar/Transitions/AbstractTransition.py	83.33% <ø> (-0.5%)	:arrow_down:
netzob/src/netzob/Common/Utils/SortableObject.py	75% <ø> (ø)	:arrow_up:
...odel/Grammar/Transitions/CloseChannelTransition.py	51.72% <ø> (ø)	:arrow_up:
.../Vocabulary/Domain/Specializer/SpecializingPath.py	95.23% <ø> (ø)	:arrow_up:
... and 125 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 5006b8d...6be0c5d. Read the comment docs.

[warsang]

Yes, I did notice this. I have been struggling to get rid of it, however I am afraid I missed quite a bit. Nevertheless I don't believe it affects netzob functionality so testing the features should be ok. The few issues however are that it probably reduces performance and overall code coverage.

Moreover I am not sure how to merge this PR as it is quite big and has features which Netzob might not need(ex: Hex also converted to Non Ascii printable characters by TypeEncodingFunction). Gbossert suggested on IRC to do several smaller PR'S. I have only had the time to do two as of today.

If you find some code duplicates I'd be happy to correct them!

[Sygus]

"Yes, I did notice this. I have been struggling to get rid of it"

I see at least two different ways to resolve this situation:

• Either by starting from a clean version of the netzob/next branch, and then applying each of your commits by git cherry-pick (http://mispdev.blogspot.fr/2013/02/github-cherry-picking-commits-from-pull.html). Then create a new PR.
• Or by rebasing (https://git-scm.com/book/fr/v1/Les-branches-avec-Git-Rebaser) your work on the current version of netzob/next. You will probably have a lot of conflicts to resolve.

I clearly prefer the first one, as you will have a better control of what you're doing (even though it will take some time).