Remove unwraps and other runtime panics that result from invalid input

[Gnurou]

An invalid stream should not make the program panic. This issue is to track the removal of all sources of panic from the code:

• unwraps as a result of invalid input,
• Explicit calls to panic and expect,
• Potentially out-of-bounds accesses in vectors or array (replace them with get).

[dwlsalmeida]

I assume we can still panic from logic errors and general sanity-checks from asserts and friends, right?