k6 icon indicating copy to clipboard operation
k6 copied to clipboard

Implement the `ReadableStream.pipeThrough` method

Open oleiade opened this issue 1 year ago • 0 comments

Feature Description

Description

As part of our ongoing efforts to improve k6’s streaming capabilities, we propose adding support for the ReadableStream.pipeThrough method. This method is a key feature of the Streams API, allowing ReadableStream instances to be easily piped through TransformStream objects, creating a powerful and flexible way to process data streams in k6.

Background and prerequisites

Currently, k6 supports ReadableStream but lacks support for TransformStream and the pipeThrough method, limiting the ability to build complex streaming data processing pipelines. pipeThrough provides a convenient and declarative way to connect ReadableStream to TransformStream, greatly simplifying the code needed for stream transformations.

Implementing pipeThrough is tied to supporting TransformStream, as the two features are designed to work together.

Usage example

Once TransformStream and pipeThrough are implemented, the following example would be possible:

// Step 1: Create a readable stream (e.g., from a file or network response)
const readableStream = getReadableStreamSomehow();

// Step 2: Define a simple transform stream to modify the data
const transformStream = new TransformStream({
  transform(chunk, controller) {
    // Example transformation: Convert all text to uppercase
    controller.enqueue(chunk.toUpperCase());
  },
});

// Step 3: Pipe the readable stream through the transform stream
readableStream
  .pipeThrough(transformStream)
  .getReader()
  .read()
  .then(({ done, value }) => {
    if (!done) {
      console.log('Transformed Value:', value);
    }
  });

Suggested Solution (optional)

No response

Already existing or connected issues / PRs (optional)

depends on #3919 #3920 #2974

oleiade avatar Aug 28 '24 09:08 oleiade