streamly icon indicating copy to clipboard operation
streamly copied to clipboard

Replacing `Continue` with `Partial` in the chunk parser does not work as expected

Open adithyaov opened this issue 2 years ago • 1 comments
trafficstars

See https://github.com/composewell/streamly/commit/9efd2b211cb926dfd2236cb1867b8d96d5b09ff0

Data.Parser.ParserK/o-n-heap/sequenceA in bench:Data.Parser.ParserK fails with Segmentation fault (core dumped)

adithyaov avatar Jul 14 '23 15:07 adithyaov

The commit might be lost, see the patch file below for more context:

From f6d6c78d966a3cb283fe578bae4187b9860ae1cc Mon Sep 17 00:00:00 2001
From: Adithya Kumar <[email protected]>
Date: Thu, 13 Jul 2023 23:47:05 +0530
Subject: [PATCH] Fix ParserK to respect the Partial and Continue state

---
 .../Internal/Data/Parser/ParserK/Chunked.hs    | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/core/src/Streamly/Internal/Data/Parser/ParserK/Chunked.hs b/core/src/Streamly/Internal/Data/Parser/ParserK/Chunked.hs
index 2643c6a2..67dbc5fd 100644
--- a/core/src/Streamly/Internal/Data/Parser/ParserK/Chunked.hs
+++ b/core/src/Streamly/Internal/Data/Parser/ParserK/Chunked.hs
@@ -420,15 +420,19 @@ parseDToK pstep initial extract cont !offset0 !usedCount !input = do
         onBack offset1 elemSize constr pst = do
             let pos = offset1 - start
              in if pos >= 0
-                then go SPEC offset1 pst
+                then goCheck offset1 pst constr
                 else constr (pos `div` elemSize) pst
 
+        {-# INLINE goCheck #-}
+        goCheck !cur !pst constr =
+            if cur >= end
+            then constr ((end - start) `div` SIZE_OF(a))  pst
+            else go SPEC cur pst
+
         -- Note: div may be expensive but the alternative is to maintain an element
         -- offset in addition to a byte offset or just the element offset and use
         -- multiplication to get the byte offset every time, both these options
         -- turned out to be more expensive than using div.
-        go !_ !cur !pst | cur >= end =
-            onContinue ((end - start) `div` SIZE_OF(a))  pst
         go !_ !cur !pst = do
             let !x = unsafeInlineIO $ peekWith contents cur
             pRes <- pstep pst x
@@ -447,15 +451,15 @@ parseDToK pstep initial extract cont !offset0 !usedCount !input = do
                 ParserD.Done n b ->
                     onDone ((back n - start) `div` elemSize) b
                 ParserD.Partial 0 pst1 ->
-                    go SPEC next pst1
+                    goCheck next pst1 onPartial
                 ParserD.Partial 1 pst1 ->
-                    go SPEC cur pst1
+                    goCheck cur pst1 onPartial
                 ParserD.Partial n pst1 ->
                     onBack (back n) elemSize onPartial pst1
                 ParserD.Continue 0 pst1 ->
-                    go SPEC next pst1
+                    goCheck next pst1 onContinue
                 ParserD.Continue 1 pst1 ->
-                    go SPEC cur pst1
+                    goCheck cur pst1 onContinue
                 ParserD.Continue n pst1 ->
                     onBack (back n) elemSize onContinue pst1
                 ParserD.Error err ->
-- 
2.17.1

adithyaov avatar Jul 25 '23 06:07 adithyaov