[WIP] Add auto-detection of secator pipes into workflows

[Copilot]

Plan: Auto-detect Pipes into Workflow

Analysis

• Secator already detects piped input via S_ISFIFO(os.fstat(0).st_mode) in cli.py
• Piped input/output flags are propagated through run_opts to runners
• Current behavior: piped commands run independently, each saving its own results

Implementation Plan

• [x] Explore codebase and understand current piping mechanism
• [x] Detect when secator is receiving piped input from another secator command
• [x] Create detection function for secator JSON output in pipes
• [x] Pass previous task results to next task in pipe chain
• [x] Disable reports for intermediate tasks in pipe chain
• [x] Enable reports only at the end of the pipe chain
• [x] Add tests for the new functionality
• [x] Address code review feedback
• [ ] Manually test the pipe chain behavior with real commands (requires tool installation)
• [ ] Update documentation if needed

Key Changes Made

1. Added detect_secator_piped_input() function in utils.py

   • Detects if piped input contains secator JSON output (checks for _type and _source fields)
   • Converts JSON lines to OutputType objects
   • Improved documentation to explain expected input format

2. Modified expand_input() function in utils.py

   • Now detects secator piped input and stores results in context
   • Sets is_secator_pipe and secator_piped_results in context object

3. Updated CLI helper in cli_helper.py

   • Extracts secator pipe detection from context
   • Passes previous task results to the current runner
   • Disables reports for intermediate tasks (when output is also piped)
   • Enables reports for the last task in chain (when output is not piped)
   • Prints informative message about detected pipe chain
   • Improved parameter formatting for better readability

4. Added comprehensive unit tests in tests/unit/test_pipe_detection.py

   • Tests detection of regular input (should not be detected as pipe)
   • Tests detection of URL, Port, and Subdomain secator outputs
   • Tests handling of mixed input, empty input, blank lines
   • Tests that non-secator JSON is not detected as a pipe

5. Code review improvements

   • Enhanced function docstring with detailed format specifications
   • Kept import inside function to avoid circular dependency
   • Fixed parameter formatting for consistency

Original prompt

---

This section details on the original issue you should resolve

<issue_title>feat: auto-detect pipes into workflow</issue_title> <issue_description>Auto-detect secator pipes and convert them into workflows (save results at the end of the last task, group task results in single json output, etc...)</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes freelabz/secator#600

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[coderabbitai[bot]]

[!IMPORTANT]

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[ocervell]

@copilot this is not at all what I was asking.

Let me clarify:

When running Secator pipes like: secator x subfinder | secator x naabu | secator x httpx, Secator should auto-understand that a workflow yaml needs to be built and replace the pipe by a workflow execution of the constructed yaml (put in ~/.secator/templates/) like:

type: workflow
name: temp_01
tasks:
  subfinder:
  naabu:
    targets_:
    - subdomain.host
  httpx:
    targets_:
    -type: port
     field: "{host}:{port}"