beats icon indicating copy to clipboard operation
beats copied to clipboard

[8.19](backport #47870) javascript processor: replace global paths with per-beat paths

Open mergify[bot] opened this issue 2 weeks ago • 2 comments

Proposed commit message

Changes:

  • Add SetPaths(path *paths.Path) method to jsProcessor that accepts a per-beat paths configuration
  • Defer file-based source initialization until SetPaths is called
  • For inline sources, initialization still happens immediately in NewFromConfig
  • loadSources now takes a *paths.Path parameter and uses pathConfig.Resolve() instead of the global paths.Resolve()

Fixes https://github.com/elastic/beats/issues/46988

Checklist

  • [x] My code follows the style guidelines of this project
  • [x] I have commented my code, particularly in hard-to-understand areas
  • [ ] ~~I have made corresponding changes to the documentation~~
  • [ ] ~~I have made corresponding change to the default configuration files~~
  • [x] I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
  • [ ] ~~I have added an entry in ./changelog/fragments using the changelog tool.~~

How to test this PR locally

Run filebeat with:

path.config: myconfig/

filebeat.inputs:
  - type: filestream
    id: input-a
    paths:
      - /tmp/logs/a.log

    processors:
      - script:
          lang: javascript
          file: test_processor.js
          tag: test-js-processor

output.console:
  enabled: true

myconfig/test_processor.js:

function process(event) {
    event.Put("js_processor.processed", true);
    event.Put("js_processor.timestamp", new Date().toISOString());

    var msg = event.Get("message");
    if (msg) {
        event.Put("message_upper", msg.toUpperCase());
    }

    return event;
}

write some logs:

yes 'some log' | head -n 10000 > /tmp/logs/a.log

See output:

{
  "@timestamp": "2025-12-02T18:09:36.488Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "9.3.0"
  },
  "ecs": {
    "version": "8.0.0"
  },
  "log": {
    "offset": 89586,
    "file": {
      "path": "/tmp/logs/a.log",
      "device_id": "38",
      "inode": "73159",
      "fingerprint": "83016ba24a8d31ccb16d2230eabcb1f043fa4c65914339eb954619b5c13fd55a"
    }
  },
  "message": "some log",
  "input": {
    "type": "filestream"
  },
  "js_processor": {
    "processed": true,
    "timestamp": "2025-12-02T18:09:36.488Z"
  },
  "message_upper": "SOME LOG",
  "host": {
    "name": "laptop"
  },
  "agent": {
    "version": "9.3.0",
    "ephemeral_id": "f6bb5c64-d2d1-4dad-a0e1-bbc553b617ec",
    "id": "40aef7d0-efcb-4613-a1df-d9bc42ff36b9",
    "name": "laptop",
    "type": "filebeat"
  }
}

Related issues

  • Closes #46988
  • Relates #47353
    This is an automatic backport of pull request #47870 done by Mergify.

mergify[bot] avatar Dec 11 '25 08:12 mergify[bot]

:robot: GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

github-actions[bot] avatar Dec 11 '25 08:12 github-actions[bot]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

elasticmachine avatar Dec 11 '25 08:12 elasticmachine

This pull request has not been merged yet. Could you please review and merge it @orestisfl? 🙏

mergify[bot] avatar Dec 15 '25 05:12 mergify[bot]