beats x-pack/filebeat/module/salesforce: Refactor a Salesforce module

Breaking change

NOTE: Please ignore salesforce input changes.

Description

This pull request aims to restructure the existing filebeat Salesforce module, transitioning it to utilize the new Salesforce input. The changes are subject to review once the new Salesforce input has been merged.

The current implementation of the Salesforce module employs httpjson and cometd inputs to capture historical and real-time data from Salesforce, respectively. Historical data is obtained from EventLogFiles, which stores data in files that can be retrieved from Salesforce using REST API and SOQL Queries. We use httpjson input to download and process this historical data. Real-time data is collected through real-time event monitoring, utilizing the pub-sub mechanism supported by Salesforce and cometd input can be used for the same.

During the collection of data from historical forms, we encountered challenges, with a significant issue being out-of-memory (OOM) issues. These errors could be reproduced with a large number of files from Salesforce, causing the httpjson input to utilize a significant amount of memory and eventually get killed by the kernel (oomk). To address this issue, we propose a solution by introducing a new Salesforce input. This new input is easily triggerable and capable of managing a large volume of data.

We also have plans to enhance the current Salesforce module to support more features as part of this improvement. Since most issues were closely tied to Salesforce, the decision was made to create a new input rather than repeatedly addressing problems with the existing httpjson input.

Checklist

[x] My code follows the style guidelines of this project
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have made corresponding change to the default configuration files
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Configuration


  fileset-name:
    enabled: true
    var.api_version: 56

    var.jwt_enabled: true

    var.client_id: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    var.client_username: "[email protected]"
    var.client_key_path: client_key.pem
    var.jwt_url: https://login.salesforce.com

    var.client_secret: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    var.token_url: "https://login.salesforce.com"
    var.user: "[email protected]"
    var.password: "P@$$W0₹D"

    var.url: "https://instance_id.my.salesforce.com"

    var.event_log_file: true
    var.elf_interval: 1h
    var.real_time: true
    var.real_time_interval: 5m

Salesforce Module Configuration Parameters:

enabled:
- Status: true
- Description: Enables the Salesforce module fileset.
var.api_version:
- Version: 56
- Description: Sets the Salesforce API version for interaction.
var.jwt_enabled:
- Enabled: true
- Parameters:
  - var.client_id: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  - var.client_username: "[email protected]"
  - var.client_key_path: client_key.pem
  - var.jwt_url: https://login.salesforce.com
- Description: Enables JSON Web Token (JWT) authentication and specifies related details.
Client Authentication Details:
- Parameters:
  - var.client_id: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  - var.client_secret: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  - var.token_url: "https://login.salesforce.com"
  - var.user: "[email protected]"
  - var.password: "P@$$W0₹D"
- Description: Provides client authentication for User Password Flow.
var.url:
- URL: "https://instance_id.my.salesforce.com"
- Description: Specifies the base URL for the Salesforce instance.
Event Log File Collection:
- Enabled: true
- Parameters:
  - var.elf_interval: 1 hour
- Description: Enables the collection of event log files with a specified interval.
Real-Time Event Monitoring:
- Enabled: true
- Parameters:
  - var.real_time_interval: 5 minutes
- Description: Enables real-time event monitoring with a specified interval.

Related issues

Relates https://github.com/elastic/obs-infraobs-team/issues/1248
Closes https://github.com/elastic/obs-infraobs-team/issues/1277

Dec 27 '23 05:12 kush-elastic

This pull request does not have a backport label. If this is a bug or security fix, could you label this PR @kush-elastic? 🙏. For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

Dec 27 '23 05:12 mergify[bot]

:broken_heart: Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-12-27T05:51:51.601+0000
Duration: 340 min 41 sec

Test stats :test_tube:

Test	Results
Failed	9
Passed	28827
Skipped	2015
Total	30851

Test errors

Expand to view the tests failures

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_183_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fDkD4xoOfMXKFl1cJOo_V', 'salesforce.apex.cpu_time': 79, 'salesforce.apex.login_key': 'MroEDyLplzWdOHPJ', 'salesforce.apex.run_time': 162, 'salesforce.apex.number_soql_queries': 0, 'salesforce.apex.db_total_time': 0, 'salesforce.apex.callout_time': 0, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexExecution', 'salesforce.apex.quiddity': 'V', 'salesforce.apex.is_long_running_request': 0, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.entry_point': 'VF- /apex/listApexClass.apexp', 'input.type': 'log', '@timestamp': '2021-11-02T09:20:46.003Z', 'log.offset': 0, 'service.type': 'salesforce', 'event.duration': 1, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-execution', 'event.dataset': 'salesforce.apex', 'fileset.name': 'apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_183_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_183_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-1-cf71744d-b117-4c1d-82dc-e986492957a0/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_execution.ndjson.log'
objects = [{'@timestamp': '2021-11-02T09:20:46.003Z', 'agent': {'ephemeral_id': 'a62115a4-e57e-480c-9d3c-d410b0a3585a', 'id': '0...apex-execution', 'dataset': 'salesforce.apex', 'duration': 1, 'ingested': '2023-12-27T10:32:55.251971698Z', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fDkD4xoOfMXKFl1cJOo_V', 'salesforce.apex.cpu_time': 79, 'salesforce.apex.login_key': 'MroEDyLplzWdOHPJ', 'salesforce.apex.run_time': 162, 'salesforce.apex.number_soql_queries': 0, 'salesforce.apex.db_total_time': 0, 'salesforce.apex.callout_time': 0, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexExecution', 'salesforce.apex.quiddity': 'V', 'salesforce.apex.is_long_running_request': 0, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.entry_point': 'VF- /apex/listApexClass.apexp', 'input.type': 'log', '@timestamp': '2021-11-02T09:20:46.003Z', 'log.offset': 0, 'service.type': 'salesforce', 'event.duration': 1, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-execution', 'event.dataset': 'salesforce.apex', 'fileset.name': 'apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_184_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fMTLXm8qV3kTVl1cJNxMV', 'salesforce.apex.request.status': 'S', 'salesforce.apex.cpu_time': 17, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexSoap', 'salesforce.apex.login_key': '1HQDaq9tglsI4Nlo', 'salesforce.apex.method_name': 'getContactIdAndNames', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.db_total_time': 14928037, 'input.type': 'log', '@timestamp': '2021-11-09T11:48:38.534Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'event.duration': 36, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-soap', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.url': 'APEXSOAP', 'user.roles': 'Standard', 'user.id': '0055j000000utlP'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_184_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_184_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-1-cf71744d-b117-4c1d-82dc-e986492957a0/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_soap.ndjson.log'
objects = [{'@timestamp': '2021-11-09T11:48:38.534Z', 'agent': {'ephemeral_id': '6dc76254-d4bf-4a0c-be62-69eb975e280f', 'id': '0....0'}, 'event': {'action': 'apex-soap', 'category': 'network', 'dataset': 'salesforce.apex', 'duration': 36, ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fMTLXm8qV3kTVl1cJNxMV', 'salesforce.apex.request.status': 'S', 'salesforce.apex.cpu_time': 17, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexSoap', 'salesforce.apex.login_key': '1HQDaq9tglsI4Nlo', 'salesforce.apex.method_name': 'getContactIdAndNames', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.db_total_time': 14928037, 'input.type': 'log', '@timestamp': '2021-11-09T11:48:38.534Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'event.duration': 36, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-soap', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.url': 'APEXSOAP', 'user.roles': 'Standard', 'user.id': '0055j000000utlP'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_185_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': 'TID:10799890000ce8142a', 'salesforce.apex.event_type': 'ExternalCustomApexCallout', 'salesforce.apex.select': '(ColumnSelection:[aggregation=NONE, columnName=DisplayUrl, tableName=HealthcareBlog], ColumnSelection:[aggregation=NONE, columnName=ExternalId, tableName=HealthcareBlog])', 'salesforce.apex.offset': 0, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.orderby': '(Order:[columnName=ExternalId, direction=ASCENDING, tableName=HealthcareBlog], Order:[columnName=ExternalId, direction=ASCENDING, tableName=HealthcareBlog])', 'input.type': 'log', '@timestamp': '2021-11-10T10:11:09.997Z', 'log.offset': 0, 'service.type': 'salesforce', 'event.duration': 13, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': ['apex-external-custom-callout'], 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.outcome': 'failure', 'fileset.name': 'apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_185_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_185_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-1-cf71744d-b117-4c1d-82dc-e986492957a0/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_external_callout.ndjson.log'
objects = [{'@timestamp': '2021-11-10T10:11:09.997Z', 'agent': {'ephemeral_id': 'b5f74117-6229-49d6-9924-50600fcfbaeb', 'id': '0...on': ['apex-external-custom-callout'], 'category': 'network', 'dataset': 'salesforce.apex', 'duration': 13, ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': 'TID:10799890000ce8142a', 'salesforce.apex.event_type': 'ExternalCustomApexCallout', 'salesforce.apex.select': '(ColumnSelection:[aggregation=NONE, columnName=DisplayUrl, tableName=HealthcareBlog], ColumnSelection:[aggregation=NONE, columnName=ExternalId, tableName=HealthcareBlog])', 'salesforce.apex.offset': 0, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.orderby': '(Order:[columnName=ExternalId, direction=ASCENDING, tableName=HealthcareBlog], Order:[columnName=ExternalId, direction=ASCENDING, tableName=HealthcareBlog])', 'input.type': 'log', '@timestamp': '2021-11-10T10:11:09.997Z', 'log.offset': 0, 'service.type': 'salesforce', 'event.duration': 13, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': ['apex-external-custom-callout'], 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.outcome': 'failure', 'fileset.name': 'apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_186_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fMAJzbahOQk2kl1cJ0uSV', 'salesforce.apex.request.status': 'S', 'salesforce.apex.cpu_time': 94, 'salesforce.apex.login_key': 'ieC2BVibz9GliuON', 'salesforce.apex.db_total_time': 232002551, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexRestApi', 'salesforce.apex.media_type': 'application/json;charset=UTF-8', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.db_blocks': 4401, 'salesforce.apex.db_cpu_time': 130, 'salesforce.apex.user_agent': 5236, 'input.type': 'log', '@timestamp': '2021-11-09T06:00:01.635Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'http.request.method': 'POST', 'http.response.status_code': 200, 'event.duration': 478, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.url': '/Account/', 'user.roles': 'Standard', 'user.id': '0055j000000utlP'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_186_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_186_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-1-cf71744d-b117-4c1d-82dc-e986492957a0/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_rest.ndjson.log'
objects = [{'@timestamp': '2021-11-09T06:00:01.635Z', 'agent': {'ephemeral_id': '51442fcb-0a17-434f-a8d8-a9f04d3c736f', 'id': '0...y': 'network', 'dataset': 'salesforce.apex', 'duration': 478, 'ingested': '2023-12-27T10:33:34.908564184Z', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fMAJzbahOQk2kl1cJ0uSV', 'salesforce.apex.request.status': 'S', 'salesforce.apex.cpu_time': 94, 'salesforce.apex.login_key': 'ieC2BVibz9GliuON', 'salesforce.apex.db_total_time': 232002551, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexRestApi', 'salesforce.apex.media_type': 'application/json;charset=UTF-8', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.db_blocks': 4401, 'salesforce.apex.db_cpu_time': 130, 'salesforce.apex.user_agent': 5236, 'input.type': 'log', '@timestamp': '2021-11-09T06:00:01.635Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'http.request.method': 'POST', 'http.response.status_code': 200, 'event.duration': 478, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.url': '/Account/', 'user.roles': 'Standard', 'user.id': '0055j000000utlP'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_187_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fDkS1iz2_p5i-l1cJOqAV', 'salesforce.apex.cpu_time': 3, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexTrigger', 'salesforce.apex.entity_name': 'Book__c', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.trigger.name': 'HelloWorldTrigger', 'salesforce.apex.trigger.id': '01q5j000000ClvF', 'salesforce.apex.trigger.type': 'BeforeInsert', 'input.type': 'log', '@timestamp': '2021-11-02T09:25:02.894Z', 'log.offset': 0, 'service.type': 'salesforce', 'fileset.name': 'apex', 'event.duration': 4, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-trigger', 'event.type': 'connection', 'event.dataset': 'salesforce.apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_187_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_187_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-1-cf71744d-b117-4c1d-82dc-e986492957a0/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_trigger.ndjson.log'
objects = [{'@timestamp': '2021-11-02T09:25:02.894Z', 'agent': {'ephemeral_id': 'cb7a0aed-ae03-4f70-ba5e-6b450a6e8ad6', 'id': '0... 'apex-trigger', 'dataset': 'salesforce.apex', 'duration': 4, 'ingested': '2023-12-27T10:33:48.111464942Z', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fDkS1iz2_p5i-l1cJOqAV', 'salesforce.apex.cpu_time': 3, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexTrigger', 'salesforce.apex.entity_name': 'Book__c', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.trigger.name': 'HelloWorldTrigger', 'salesforce.apex.trigger.id': '01q5j000000ClvF', 'salesforce.apex.trigger.type': 'BeforeInsert', 'input.type': 'log', '@timestamp': '2021-11-02T09:25:02.894Z', 'log.offset': 0, 'service.type': 'salesforce', 'fileset.name': 'apex', 'event.duration': 4, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-trigger', 'event.type': 'connection', 'event.dataset': 'salesforce.apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_188_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'log.offset': 0, 'source.ip': '10.0.1.123', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.cpu_time': -1, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexCallout', 'salesforce.apex.run_time': 1305, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.type': 'OData', 'salesforce.apex.uri': 'CALLOUT-LOG', 'input.type': 'log', '@timestamp': '2021-11-02T09:12:50.488Z', 'related.ip': ['10.0.1.123'], 'service.type': 'salesforce', 'http.request.method': 'GET', 'http.request.bytes': -1, 'http.response.bytes': 256, 'event.duration': 1293, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-callout', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.outcome': 'failure', 'event.url': 'https://api-prod.guidance.sfdc.sh/odata/Accounts?%24top=1&%24filter=%28startswith%28Organization_id_dw__c%2C%2700D5j000000VI3n%27%29%29&%24select=Aov_band__c%2CCombo_sector__c%2CCsg_coverage__c%2CCsg_portfolio__c%2CId%2CIndustry%2CL1_aov_cloud1__c%2CL1_aov_cloud2__c%2CLocked_industry__c%2CLocked_sector__c%2CSfdc_sector__c%2CSfid%2CSub_sector__c%2CSupportLevel__c%2CTotal_aov__c', 'user.id': '0055j000000utlP'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_188_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_188_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-1-cf71744d-b117-4c1d-82dc-e986492957a0/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_callout.ndjson.log'
objects = [{'@timestamp': '2021-11-02T09:12:50.488Z', 'agent': {'ephemeral_id': 'b0b35bf3-292a-47f4-ae49-92f76207f427', 'id': '0... 'event': {'action': 'apex-callout', 'category': 'network', 'dataset': 'salesforce.apex', 'duration': 1293, ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'log.offset': 0, 'source.ip': '10.0.1.123', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.cpu_time': -1, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexCallout', 'salesforce.apex.run_time': 1305, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.type': 'OData', 'salesforce.apex.uri': 'CALLOUT-LOG', 'input.type': 'log', '@timestamp': '2021-11-02T09:12:50.488Z', 'related.ip': ['10.0.1.123'], 'service.type': 'salesforce', 'http.request.method': 'GET', 'http.request.bytes': -1, 'http.response.bytes': 256, 'event.duration': 1293, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-callout', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.outcome': 'failure', 'event.url': 'https://api-prod.guidance.sfdc.sh/odata/Accounts?%24top=1&%24filter=%28startswith%28Organization_id_dw__c%2C%2700D5j000000VI3n%27%29%29&%24select=Aov_band__c%2CCombo_sector__c%2CCsg_coverage__c%2CCsg_portfolio__c%2CId%2CIndustry%2CL1_aov_cloud1__c%2CL1_aov_cloud2__c%2CLocked_industry__c%2CLocked_sector__c%2CSfdc_sector__c%2CSfid%2CSub_sector__c%2CSupportLevel__c%2CTotal_aov__c', 'user.id': '0055j000000utlP'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_189_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'setupaudittrail', 'old_value': 'setupaudittrail-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-setupaudittrail'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-setupaudittrail-rest'}}, full object: 
  {'log.offset': 0, 'fileset.name': 'setupaudittrail', 'tags': ['salesforce-setupaudittrail', 'forwarded'], 'salesforce.setup_audit_trail.event_type': 'SetupAuditTrail', 'salesforce.setup_audit_trail.display': 'For user [email protected], the User Verified Email status changed to verified', 'salesforce.setup_audit_trail.section': 'Manage Users', 'salesforce.setup_audit_trail.created_by_id': '1234j000000q9s7ABC', 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-06T07:13:07.000Z', 'service.type': 'salesforce', 'event.original': '{"attributes":{"type":"SetupAuditTrail","url":"/services/data/v52.0/sobjects/SetupAuditTrail/0Ym5j12345BJKvZCAX"},"Action":"changedUserEmailVerifiedStatusVerified","CreatedByContext":null,"CreatedById":"1234j000000q9s7ABC","CreatedByIssuer":null,"CreatedDate":"2021-10-06T07:13:07.000+0000","DelegateUser":null,"Display":"For user [email protected], the User Verified Email status changed to verified","Id":"0Ym5j12345BJKvABCD","ResponsibleNamespacePrefix":null,"Section":"Manage Users"}', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'changedUserEmailVerifiedStatusVerified', 'event.id': '0Ym5j12345BJKvABCD', 'event.type': 'admin', 'event.dataset': 'salesforce.setupaudittrail', 'event.url': '/services/data/v52.0/sobjects/SetupAuditTrail/0Ym5j12345BJKvZCAX', 'user.name': '[email protected]', 'user.id': '1234j000000q9s7ABC'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'setupaudittrail', 'old_value': 'setupaudittrail-rest'}}, 'i...'][0]": 'salesforce-setupaudittrail'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-setupaudittrail-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_189_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_189_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-1-cf71744d-b117-4c1d-82dc-e986492957a0/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/setupaudittrail/test/salesforce_setupaudittrail.ndjson.log'
objects = [{'@timestamp': '2021-10-06T07:13:07.000Z', 'agent': {'ephemeral_id': '341491b1-e031-4acc-ab23-bb5ffef51f28', 'id': '0...'created': '2021-10-06T07:13:07.000Z', 'dataset': 'salesforce.setupaudittrail', 'id': '0Ym5j12345BJKvABCD', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'setupaudittrail', 'old_value': 'setupaudittrail-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-setupaudittrail'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-setupaudittrail-rest'}}, full object: 
E             {'log.offset': 0, 'fileset.name': 'setupaudittrail', 'tags': ['salesforce-setupaudittrail', 'forwarded'], 'salesforce.setup_audit_trail.event_type': 'SetupAuditTrail', 'salesforce.setup_audit_trail.display': 'For user [email protected], the User Verified Email status changed to verified', 'salesforce.setup_audit_trail.section': 'Manage Users', 'salesforce.setup_audit_trail.created_by_id': '1234j000000q9s7ABC', 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-06T07:13:07.000Z', 'service.type': 'salesforce', 'event.original': '{"attributes":{"type":"SetupAuditTrail","url":"/services/data/v52.0/sobjects/SetupAuditTrail/0Ym5j12345BJKvZCAX"},"Action":"changedUserEmailVerifiedStatusVerified","CreatedByContext":null,"CreatedById":"1234j000000q9s7ABC","CreatedByIssuer":null,"CreatedDate":"2021-10-06T07:13:07.000+0000","DelegateUser":null,"Display":"For user [email protected], the User Verified Email status changed to verified","Id":"0Ym5j12345BJKvABCD","ResponsibleNamespacePrefix":null,"Section":"Manage Users"}', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'changedUserEmailVerifiedStatusVerified', 'event.id': '0Ym5j12345BJKvABCD', 'event.type': 'admin', 'event.dataset': 'salesforce.setupaudittrail', 'event.url': '/services/data/v52.0/sobjects/SetupAuditTrail/0Ym5j12345BJKvZCAX', 'user.name': '[email protected]', 'user.id': '1234j000000q9s7ABC'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'setupaudittrail', 'old_value': 'setupaudittrail-rest'}}, 'i...'][0]": 'salesforce-setupaudittrail'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-setupaudittrail-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_190_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'login', 'old_value': 'login-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-login'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-login-rest'}}, full object: 
  {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'login', 'tags': ['salesforce-login', 'forwarded'], 'salesforce.login.cpu_time': 54, 'salesforce.login.login_key': 'iQzw40l45YkJUNKj', 'salesforce.login.run_time': 94, 'salesforce.login.db_total_time': 37142518, 'salesforce.login.api_version': 9998, 'salesforce.login.user_id_derived': '0055j000000utlPAAQ', 'salesforce.login.event_type': 'Login', 'salesforce.login.organization_id': '00D5j000000VI3n', 'salesforce.login.client_ip': '43.224.11.237', 'salesforce.login.request_id': '4exJslfc21z80-l1cJOnxV', 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-19T04:42:04.256Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'tls.cipher': 'ECDHE-RSA-AES256-GCM-SHA384', 'tls.version': '1.2', 'tls.version_protocol': 'TLS', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'login-attempt', 'event.type': 'info', 'event.category': 'authentication', 'event.dataset': 'salesforce.login', 'event.url': '/index.jsp', 'event.outcome': 'success', 'user.roles': 'Standard', 'user.id': '0055j000000utlP', 'user.email': '[email protected]', 'user_agent.name': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'login', 'old_value': 'login-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-login'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-login-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_190_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_190_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-1-cf71744d-b117-4c1d-82dc-e986492957a0/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/login/test/salesforce_login.ndjson.log'
objects = [{'@timestamp': '2021-10-19T04:42:04.256Z', 'agent': {'ephemeral_id': 'd1d5ff98-82bf-4b53-93d4-05903f15d506', 'id': '0... 'category': 'authentication', 'dataset': 'salesforce.login', 'ingested': '2023-12-27T10:34:27.725895050Z', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'login', 'old_value': 'login-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-login'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-login-rest'}}, full object: 
E             {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'login', 'tags': ['salesforce-login', 'forwarded'], 'salesforce.login.cpu_time': 54, 'salesforce.login.login_key': 'iQzw40l45YkJUNKj', 'salesforce.login.run_time': 94, 'salesforce.login.db_total_time': 37142518, 'salesforce.login.api_version': 9998, 'salesforce.login.user_id_derived': '0055j000000utlPAAQ', 'salesforce.login.event_type': 'Login', 'salesforce.login.organization_id': '00D5j000000VI3n', 'salesforce.login.client_ip': '43.224.11.237', 'salesforce.login.request_id': '4exJslfc21z80-l1cJOnxV', 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-19T04:42:04.256Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'tls.cipher': 'ECDHE-RSA-AES256-GCM-SHA384', 'tls.version': '1.2', 'tls.version_protocol': 'TLS', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'login-attempt', 'event.type': 'info', 'event.category': 'authentication', 'event.dataset': 'salesforce.login', 'event.url': '/index.jsp', 'event.outcome': 'success', 'user.roles': 'Standard', 'user.id': '0055j000000utlP', 'user.email': '[email protected]', 'user_agent.name': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'login', 'old_value': 'login-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-login'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-login-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_191_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'logout', 'old_value': 'logout-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-logout'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-logout-rest'}}, full object: 
  {'log.offset': 0, 'source.ip': '123.201.231.42', 'fileset.name': 'logout', 'tags': ['salesforce-logout', 'forwarded'], 'salesforce.logout.organization_by_id': '00D5j000000VI3n', 'salesforce.logout.app_type': 1000, 'salesforce.logout.login_key': 'OKyrFuSUIZVr9YzF', 'salesforce.logout.browser_type': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36', 'salesforce.logout.session_level': 'STANDARD(db=1,api=STANDARD)', 'salesforce.logout.resolution_type': 9999, 'salesforce.logout.user_id_derived': '0055j000000utlPAAQ', 'salesforce.logout.event_type': 'Logout', 'salesforce.logout.platform_type': 1015, 'salesforce.logout.session_type': 'UI', 'salesforce.logout.client_version': 9998, 'salesforce.logout.user_initiated_logout': 1, 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-19T05:07:07.128Z', 'related.ip': ['123.201.231.42'], 'service.type': 'salesforce', 'event.code': '4exLFFQZNa5xxFl1cJNwOV', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'logout', 'event.type': 'info', 'event.category': 'authentication', 'event.dataset': 'salesforce.logout', 'user.roles': 'Standard(db=S,api=Standard)', 'user.id': '0055j000000utlP'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'logout', 'old_value': 'logout-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-logout'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-logout-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_191_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_191_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-1-cf71744d-b117-4c1d-82dc-e986492957a0/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/logout/test/salesforce_logout.ndjson.log'
objects = [{'@timestamp': '2021-10-19T05:07:07.128Z', 'agent': {'ephemeral_id': 'c2411d47-bbdf-4f08-94f5-caab97996834', 'id': '0...: 'logout', 'category': 'authentication', 'code': '4exLFFQZNa5xxFl1cJNwOV', 'dataset': 'salesforce.logout', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'logout', 'old_value': 'logout-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-logout'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-logout-rest'}}, full object: 
E             {'log.offset': 0, 'source.ip': '123.201.231.42', 'fileset.name': 'logout', 'tags': ['salesforce-logout', 'forwarded'], 'salesforce.logout.organization_by_id': '00D5j000000VI3n', 'salesforce.logout.app_type': 1000, 'salesforce.logout.login_key': 'OKyrFuSUIZVr9YzF', 'salesforce.logout.browser_type': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36', 'salesforce.logout.session_level': 'STANDARD(db=1,api=STANDARD)', 'salesforce.logout.resolution_type': 9999, 'salesforce.logout.user_id_derived': '0055j000000utlPAAQ', 'salesforce.logout.event_type': 'Logout', 'salesforce.logout.platform_type': 1015, 'salesforce.logout.session_type': 'UI', 'salesforce.logout.client_version': 9998, 'salesforce.logout.user_initiated_logout': 1, 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-19T05:07:07.128Z', 'related.ip': ['123.201.231.42'], 'service.type': 'salesforce', 'event.code': '4exLFFQZNa5xxFl1cJNwOV', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'logout', 'event.type': 'info', 'event.category': 'authentication', 'event.dataset': 'salesforce.logout', 'user.roles': 'Standard(db=S,api=Standard)', 'user.id': '0055j000000utlP'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'logout', 'old_value': 'logout-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-logout'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-logout-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

Steps errors

Expand to view the steps failures

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

Took 101 min 0 sec . View more details here
Description: mage pythonIntegTest

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

Took 99 min 44 sec . View more details here
Description: mage pythonIntegTest

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

Took 99 min 2 sec . View more details here
Description: mage pythonIntegTest

`Error signal`

Took 0 min 0 sec . View more details here
Description: Error 'hudson.AbortException: script returned exit code 1'

:bug: Flaky test report

:grey_exclamation: There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors

:broken_heart: There are test failures but not known flaky tests, most likely a genuine test failure.

Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_183_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_184_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_185_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_186_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_187_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_188_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_189_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_190_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_191_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

:robot: GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Dec 27 '23 11:12 elasticmachine

:broken_heart: Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Duration: 331 min 6 sec

Pipeline error

This error is likely related to the pipeline itself. Click here and then you will see the error (either incorrect syntax or an invalid configuration).

:grey_exclamation: Flaky test report

No test was executed to be analysed.

:robot: GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Jan 17 '24 09:01 elasticmachine

:broken_heart: Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Duration: 328 min 5 sec

Pipeline error

This error is likely related to the pipeline itself. Click here and then you will see the error (either incorrect syntax or an invalid configuration).

:grey_exclamation: Flaky test report

No test was executed to be analysed.

:robot: GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Jan 18 '24 12:01 elasticmachine

This pull request is now in conflicts. Could you fix it? 🙏 To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b refactor_salesforce_module upstream/refactor_salesforce_module
git merge upstream/main
git push upstream refactor_salesforce_module

Jan 25 '24 03:01 mergify[bot]

:grey_exclamation: Build Aborted

There is a new build on-going so the previous on-going builds have been aborted.

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2024-02-01T07:13:56.987+0000
Duration: 204 min 59 sec

Test stats :test_tube:

Test	Results
Failed	9
Passed	28877
Skipped	2014
Total	30900

Test errors

Expand to view the tests failures

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_250_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'login', 'old_value': 'login-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-login'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-login-rest'}}, full object: 
  {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'login', 'tags': ['salesforce-login', 'forwarded'], 'salesforce.login.cpu_time': 54, 'salesforce.login.login_key': 'iQzw40l45YkJUNKj', 'salesforce.login.run_time': 94, 'salesforce.login.db_total_time': 37142518, 'salesforce.login.api_version': 9998, 'salesforce.login.user_id_derived': '0055j000000utlPAAQ', 'salesforce.login.event_type': 'Login', 'salesforce.login.organization_id': '00D5j000000VI3n', 'salesforce.login.client_ip': '43.224.11.237', 'salesforce.login.request_id': '4exJslfc21z80-l1cJOnxV', 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-19T04:42:04.256Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'tls.cipher': 'ECDHE-RSA-AES256-GCM-SHA384', 'tls.version': '1.2', 'tls.version_protocol': 'TLS', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'login-attempt', 'event.type': 'info', 'event.category': 'authentication', 'event.dataset': 'salesforce.login', 'event.url': '/index.jsp', 'event.outcome': 'success', 'user.roles': 'Standard', 'user.id': '0055j000000utlP', 'user.email': '[email protected]', 'user_agent.name': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'login', 'old_value': 'login-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-login'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-login-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_250_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_250_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-10-df6931f9-971d-4d10-8fe1-15d8d9e79b4c/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/login/test/salesforce_login.ndjson.log'
objects = [{'@timestamp': '2021-10-19T04:42:04.256Z', 'agent': {'ephemeral_id': '1fbfa1ec-a03d-4f6a-985e-01248bc171fd', 'id': '9... 'category': 'authentication', 'dataset': 'salesforce.login', 'ingested': '2024-02-01T08:41:47.262527519Z', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'login', 'old_value': 'login-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-login'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-login-rest'}}, full object: 
E             {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'login', 'tags': ['salesforce-login', 'forwarded'], 'salesforce.login.cpu_time': 54, 'salesforce.login.login_key': 'iQzw40l45YkJUNKj', 'salesforce.login.run_time': 94, 'salesforce.login.db_total_time': 37142518, 'salesforce.login.api_version': 9998, 'salesforce.login.user_id_derived': '0055j000000utlPAAQ', 'salesforce.login.event_type': 'Login', 'salesforce.login.organization_id': '00D5j000000VI3n', 'salesforce.login.client_ip': '43.224.11.237', 'salesforce.login.request_id': '4exJslfc21z80-l1cJOnxV', 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-19T04:42:04.256Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'tls.cipher': 'ECDHE-RSA-AES256-GCM-SHA384', 'tls.version': '1.2', 'tls.version_protocol': 'TLS', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'login-attempt', 'event.type': 'info', 'event.category': 'authentication', 'event.dataset': 'salesforce.login', 'event.url': '/index.jsp', 'event.outcome': 'success', 'user.roles': 'Standard', 'user.id': '0055j000000utlP', 'user.email': '[email protected]', 'user_agent.name': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'login', 'old_value': 'login-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-login'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-login-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_251_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'setupaudittrail', 'old_value': 'setupaudittrail-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-setupaudittrail'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-setupaudittrail-rest'}}, full object: 
  {'log.offset': 0, 'fileset.name': 'setupaudittrail', 'tags': ['salesforce-setupaudittrail', 'forwarded'], 'salesforce.setup_audit_trail.event_type': 'SetupAuditTrail', 'salesforce.setup_audit_trail.display': 'For user [email protected], the User Verified Email status changed to verified', 'salesforce.setup_audit_trail.section': 'Manage Users', 'salesforce.setup_audit_trail.created_by_id': '1234j000000q9s7ABC', 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-06T07:13:07.000Z', 'service.type': 'salesforce', 'event.original': '{"attributes":{"type":"SetupAuditTrail","url":"/services/data/v52.0/sobjects/SetupAuditTrail/0Ym5j12345BJKvZCAX"},"Action":"changedUserEmailVerifiedStatusVerified","CreatedByContext":null,"CreatedById":"1234j000000q9s7ABC","CreatedByIssuer":null,"CreatedDate":"2021-10-06T07:13:07.000+0000","DelegateUser":null,"Display":"For user [email protected], the User Verified Email status changed to verified","Id":"0Ym5j12345BJKvABCD","ResponsibleNamespacePrefix":null,"Section":"Manage Users"}', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'changedUserEmailVerifiedStatusVerified', 'event.id': '0Ym5j12345BJKvABCD', 'event.type': 'admin', 'event.dataset': 'salesforce.setupaudittrail', 'event.url': '/services/data/v52.0/sobjects/SetupAuditTrail/0Ym5j12345BJKvZCAX', 'user.name': '[email protected]', 'user.id': '1234j000000q9s7ABC'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'setupaudittrail', 'old_value': 'setupaudittrail-rest'}}, 'i...'][0]": 'salesforce-setupaudittrail'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-setupaudittrail-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_251_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_251_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-10-df6931f9-971d-4d10-8fe1-15d8d9e79b4c/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/setupaudittrail/test/salesforce_setupaudittrail.ndjson.log'
objects = [{'@timestamp': '2021-10-06T07:13:07.000Z', 'agent': {'ephemeral_id': 'c1b4c437-89b1-4997-8e50-89a6ce550285', 'id': '9...'created': '2021-10-06T07:13:07.000Z', 'dataset': 'salesforce.setupaudittrail', 'id': '0Ym5j12345BJKvABCD', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'setupaudittrail', 'old_value': 'setupaudittrail-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-setupaudittrail'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-setupaudittrail-rest'}}, full object: 
E             {'log.offset': 0, 'fileset.name': 'setupaudittrail', 'tags': ['salesforce-setupaudittrail', 'forwarded'], 'salesforce.setup_audit_trail.event_type': 'SetupAuditTrail', 'salesforce.setup_audit_trail.display': 'For user [email protected], the User Verified Email status changed to verified', 'salesforce.setup_audit_trail.section': 'Manage Users', 'salesforce.setup_audit_trail.created_by_id': '1234j000000q9s7ABC', 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-06T07:13:07.000Z', 'service.type': 'salesforce', 'event.original': '{"attributes":{"type":"SetupAuditTrail","url":"/services/data/v52.0/sobjects/SetupAuditTrail/0Ym5j12345BJKvZCAX"},"Action":"changedUserEmailVerifiedStatusVerified","CreatedByContext":null,"CreatedById":"1234j000000q9s7ABC","CreatedByIssuer":null,"CreatedDate":"2021-10-06T07:13:07.000+0000","DelegateUser":null,"Display":"For user [email protected], the User Verified Email status changed to verified","Id":"0Ym5j12345BJKvABCD","ResponsibleNamespacePrefix":null,"Section":"Manage Users"}', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'changedUserEmailVerifiedStatusVerified', 'event.id': '0Ym5j12345BJKvABCD', 'event.type': 'admin', 'event.dataset': 'salesforce.setupaudittrail', 'event.url': '/services/data/v52.0/sobjects/SetupAuditTrail/0Ym5j12345BJKvZCAX', 'user.name': '[email protected]', 'user.id': '1234j000000q9s7ABC'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'setupaudittrail', 'old_value': 'setupaudittrail-rest'}}, 'i...'][0]": 'salesforce-setupaudittrail'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-setupaudittrail-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_252_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fDkS1iz2_p5i-l1cJOqAV', 'salesforce.apex.cpu_time': 3, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexTrigger', 'salesforce.apex.entity_name': 'Book__c', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.trigger.name': 'HelloWorldTrigger', 'salesforce.apex.trigger.id': '01q5j000000ClvF', 'salesforce.apex.trigger.type': 'BeforeInsert', 'input.type': 'log', '@timestamp': '2021-11-02T09:25:02.894Z', 'log.offset': 0, 'service.type': 'salesforce', 'fileset.name': 'apex', 'event.duration': 4, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-trigger', 'event.type': 'connection', 'event.dataset': 'salesforce.apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_252_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_252_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-10-df6931f9-971d-4d10-8fe1-15d8d9e79b4c/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_trigger.ndjson.log'
objects = [{'@timestamp': '2021-11-02T09:25:02.894Z', 'agent': {'ephemeral_id': '6ddd9416-ffbc-4d4b-bb09-9a986bab973c', 'id': '9... 'apex-trigger', 'dataset': 'salesforce.apex', 'duration': 4, 'ingested': '2024-02-01T08:42:13.917575163Z', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fDkS1iz2_p5i-l1cJOqAV', 'salesforce.apex.cpu_time': 3, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexTrigger', 'salesforce.apex.entity_name': 'Book__c', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.trigger.name': 'HelloWorldTrigger', 'salesforce.apex.trigger.id': '01q5j000000ClvF', 'salesforce.apex.trigger.type': 'BeforeInsert', 'input.type': 'log', '@timestamp': '2021-11-02T09:25:02.894Z', 'log.offset': 0, 'service.type': 'salesforce', 'fileset.name': 'apex', 'event.duration': 4, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-trigger', 'event.type': 'connection', 'event.dataset': 'salesforce.apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_253_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fMAJzbahOQk2kl1cJ0uSV', 'salesforce.apex.request.status': 'S', 'salesforce.apex.cpu_time': 94, 'salesforce.apex.login_key': 'ieC2BVibz9GliuON', 'salesforce.apex.db_total_time': 232002551, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexRestApi', 'salesforce.apex.media_type': 'application/json;charset=UTF-8', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.db_blocks': 4401, 'salesforce.apex.db_cpu_time': 130, 'salesforce.apex.user_agent': 5236, 'input.type': 'log', '@timestamp': '2021-11-09T06:00:01.635Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'http.request.method': 'POST', 'http.response.status_code': 200, 'event.duration': 478, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.url': '/Account/', 'user.roles': 'Standard', 'user.id': '0055j000000utlP'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_253_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_253_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-10-df6931f9-971d-4d10-8fe1-15d8d9e79b4c/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_rest.ndjson.log'
objects = [{'@timestamp': '2021-11-09T06:00:01.635Z', 'agent': {'ephemeral_id': 'cbdcc212-f3f4-4576-8f17-3016c4962887', 'id': '9...y': 'network', 'dataset': 'salesforce.apex', 'duration': 478, 'ingested': '2024-02-01T08:42:27.152383245Z', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fMAJzbahOQk2kl1cJ0uSV', 'salesforce.apex.request.status': 'S', 'salesforce.apex.cpu_time': 94, 'salesforce.apex.login_key': 'ieC2BVibz9GliuON', 'salesforce.apex.db_total_time': 232002551, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexRestApi', 'salesforce.apex.media_type': 'application/json;charset=UTF-8', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.db_blocks': 4401, 'salesforce.apex.db_cpu_time': 130, 'salesforce.apex.user_agent': 5236, 'input.type': 'log', '@timestamp': '2021-11-09T06:00:01.635Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'http.request.method': 'POST', 'http.response.status_code': 200, 'event.duration': 478, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.url': '/Account/', 'user.roles': 'Standard', 'user.id': '0055j000000utlP'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_254_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fDkD4xoOfMXKFl1cJOo_V', 'salesforce.apex.cpu_time': 79, 'salesforce.apex.login_key': 'MroEDyLplzWdOHPJ', 'salesforce.apex.run_time': 162, 'salesforce.apex.number_soql_queries': 0, 'salesforce.apex.db_total_time': 0, 'salesforce.apex.callout_time': 0, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexExecution', 'salesforce.apex.quiddity': 'V', 'salesforce.apex.is_long_running_request': 0, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.entry_point': 'VF- /apex/listApexClass.apexp', 'input.type': 'log', '@timestamp': '2021-11-02T09:20:46.003Z', 'log.offset': 0, 'service.type': 'salesforce', 'event.duration': 1, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-execution', 'event.dataset': 'salesforce.apex', 'fileset.name': 'apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_254_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_254_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-10-df6931f9-971d-4d10-8fe1-15d8d9e79b4c/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_execution.ndjson.log'
objects = [{'@timestamp': '2021-11-02T09:20:46.003Z', 'agent': {'ephemeral_id': 'c378e394-7835-4e52-aea3-6685b487a210', 'id': '9...apex-execution', 'dataset': 'salesforce.apex', 'duration': 1, 'ingested': '2024-02-01T08:42:40.425270714Z', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fDkD4xoOfMXKFl1cJOo_V', 'salesforce.apex.cpu_time': 79, 'salesforce.apex.login_key': 'MroEDyLplzWdOHPJ', 'salesforce.apex.run_time': 162, 'salesforce.apex.number_soql_queries': 0, 'salesforce.apex.db_total_time': 0, 'salesforce.apex.callout_time': 0, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexExecution', 'salesforce.apex.quiddity': 'V', 'salesforce.apex.is_long_running_request': 0, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.entry_point': 'VF- /apex/listApexClass.apexp', 'input.type': 'log', '@timestamp': '2021-11-02T09:20:46.003Z', 'log.offset': 0, 'service.type': 'salesforce', 'event.duration': 1, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-execution', 'event.dataset': 'salesforce.apex', 'fileset.name': 'apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_255_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': 'TID:10799890000ce8142a', 'salesforce.apex.event_type': 'ExternalCustomApexCallout', 'salesforce.apex.select': '(ColumnSelection:[aggregation=NONE, columnName=DisplayUrl, tableName=HealthcareBlog], ColumnSelection:[aggregation=NONE, columnName=ExternalId, tableName=HealthcareBlog])', 'salesforce.apex.offset': 0, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.orderby': '(Order:[columnName=ExternalId, direction=ASCENDING, tableName=HealthcareBlog], Order:[columnName=ExternalId, direction=ASCENDING, tableName=HealthcareBlog])', 'input.type': 'log', '@timestamp': '2021-11-10T10:11:09.997Z', 'log.offset': 0, 'service.type': 'salesforce', 'event.duration': 13, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': ['apex-external-custom-callout'], 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.outcome': 'failure', 'fileset.name': 'apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_255_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_255_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-10-df6931f9-971d-4d10-8fe1-15d8d9e79b4c/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_external_callout.ndjson.log'
objects = [{'@timestamp': '2021-11-10T10:11:09.997Z', 'agent': {'ephemeral_id': 'd773449a-6401-4ba0-b667-55da5d56144a', 'id': '9...on': ['apex-external-custom-callout'], 'category': 'network', 'dataset': 'salesforce.apex', 'duration': 13, ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': 'TID:10799890000ce8142a', 'salesforce.apex.event_type': 'ExternalCustomApexCallout', 'salesforce.apex.select': '(ColumnSelection:[aggregation=NONE, columnName=DisplayUrl, tableName=HealthcareBlog], ColumnSelection:[aggregation=NONE, columnName=ExternalId, tableName=HealthcareBlog])', 'salesforce.apex.offset': 0, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.orderby': '(Order:[columnName=ExternalId, direction=ASCENDING, tableName=HealthcareBlog], Order:[columnName=ExternalId, direction=ASCENDING, tableName=HealthcareBlog])', 'input.type': 'log', '@timestamp': '2021-11-10T10:11:09.997Z', 'log.offset': 0, 'service.type': 'salesforce', 'event.duration': 13, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': ['apex-external-custom-callout'], 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.outcome': 'failure', 'fileset.name': 'apex', 'user.id': '0055j000000utlP', 'tags': ['salesforce-apex', 'forwarded']}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_256_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fMTLXm8qV3kTVl1cJNxMV', 'salesforce.apex.request.status': 'S', 'salesforce.apex.cpu_time': 17, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexSoap', 'salesforce.apex.login_key': '1HQDaq9tglsI4Nlo', 'salesforce.apex.method_name': 'getContactIdAndNames', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.db_total_time': 14928037, 'input.type': 'log', '@timestamp': '2021-11-09T11:48:38.534Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'event.duration': 36, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-soap', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.url': 'APEXSOAP', 'user.roles': 'Standard', 'user.id': '0055j000000utlP'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_256_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_256_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-10-df6931f9-971d-4d10-8fe1-15d8d9e79b4c/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_soap.ndjson.log'
objects = [{'@timestamp': '2021-11-09T11:48:38.534Z', 'agent': {'ephemeral_id': '1636db9b-333d-433d-a398-9df200ae7ad8', 'id': '9....0'}, 'event': {'action': 'apex-soap', 'category': 'network', 'dataset': 'salesforce.apex', 'duration': 36, ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'log.offset': 0, 'source.ip': '43.224.11.237', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.request.id': '4fMTLXm8qV3kTVl1cJNxMV', 'salesforce.apex.request.status': 'S', 'salesforce.apex.cpu_time': 17, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexSoap', 'salesforce.apex.login_key': '1HQDaq9tglsI4Nlo', 'salesforce.apex.method_name': 'getContactIdAndNames', 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.db_total_time': 14928037, 'input.type': 'log', '@timestamp': '2021-11-09T11:48:38.534Z', 'related.ip': ['43.224.11.237'], 'service.type': 'salesforce', 'event.duration': 36, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-soap', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.url': 'APEXSOAP', 'user.roles': 'Standard', 'user.id': '0055j000000utlP'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_257_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
  {'log.offset': 0, 'source.ip': '10.0.1.123', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.cpu_time': -1, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexCallout', 'salesforce.apex.run_time': 1305, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.type': 'OData', 'salesforce.apex.uri': 'CALLOUT-LOG', 'input.type': 'log', '@timestamp': '2021-11-02T09:12:50.488Z', 'related.ip': ['10.0.1.123'], 'service.type': 'salesforce', 'http.request.method': 'GET', 'http.request.bytes': -1, 'http.response.bytes': 256, 'event.duration': 1293, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-callout', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.outcome': 'failure', 'event.url': 'https://api-prod.guidance.sfdc.sh/odata/Accounts?%24top=1&%24filter=%28startswith%28Organization_id_dw__c%2C%2700D5j000000VI3n%27%29%29&%24select=Aov_band__c%2CCombo_sector__c%2CCsg_coverage__c%2CCsg_portfolio__c%2CId%2CIndustry%2CL1_aov_cloud1__c%2CL1_aov_cloud2__c%2CLocked_industry__c%2CLocked_sector__c%2CSfdc_sector__c%2CSfid%2CSub_sector__c%2CSupportLevel__c%2CTotal_aov__c', 'user.id': '0055j000000utlP'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_257_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_257_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-10-df6931f9-971d-4d10-8fe1-15d8d9e79b4c/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/apex/test/salesforce_apex_callout.ndjson.log'
objects = [{'@timestamp': '2021-11-02T09:12:50.488Z', 'agent': {'ephemeral_id': '8fbb7fac-0ada-43e0-9e98-7be8840280c5', 'id': '9... 'event': {'action': 'apex-callout', 'category': 'network', 'dataset': 'salesforce.apex', 'duration': 1293, ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}}, full object: 
E             {'log.offset': 0, 'source.ip': '10.0.1.123', 'fileset.name': 'apex', 'tags': ['salesforce-apex', 'forwarded'], 'salesforce.access_mode': 'rest', 'salesforce.apex.cpu_time': -1, 'salesforce.apex.user_id_derived': '0055j000000utlPAAQ', 'salesforce.apex.event_type': 'ApexCallout', 'salesforce.apex.run_time': 1305, 'salesforce.apex.organization_id': '00D5j000000VI3n', 'salesforce.apex.type': 'OData', 'salesforce.apex.uri': 'CALLOUT-LOG', 'input.type': 'log', '@timestamp': '2021-11-02T09:12:50.488Z', 'related.ip': ['10.0.1.123'], 'service.type': 'salesforce', 'http.request.method': 'GET', 'http.request.bytes': -1, 'http.response.bytes': 256, 'event.duration': 1293, 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'apex-callout', 'event.type': 'connection', 'event.category': 'network', 'event.dataset': 'salesforce.apex', 'event.outcome': 'failure', 'event.url': 'https://api-prod.guidance.sfdc.sh/odata/Accounts?%24top=1&%24filter=%28startswith%28Organization_id_dw__c%2C%2700D5j000000VI3n%27%29%29&%24select=Aov_band__c%2CCombo_sector__c%2CCsg_coverage__c%2CCsg_portfolio__c%2CId%2CIndustry%2CL1_aov_cloud1__c%2CL1_aov_cloud2__c%2CLocked_industry__c%2CLocked_sector__c%2CSfdc_sector__c%2CSfid%2CSub_sector__c%2CSupportLevel__c%2CTotal_aov__c', 'user.id': '0055j000000utlP'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'apex', 'old_value': 'apex-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-apex'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-apex-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_258_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['fileset.name']": {'new_value': 'logout', 'old_value': 'logout-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-logout'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-logout-rest'}}, full object: 
  {'log.offset': 0, 'source.ip': '123.201.231.42', 'fileset.name': 'logout', 'tags': ['salesforce-logout', 'forwarded'], 'salesforce.logout.organization_by_id': '00D5j000000VI3n', 'salesforce.logout.app_type': 1000, 'salesforce.logout.login_key': 'OKyrFuSUIZVr9YzF', 'salesforce.logout.browser_type': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36', 'salesforce.logout.session_level': 'STANDARD(db=1,api=STANDARD)', 'salesforce.logout.resolution_type': 9999, 'salesforce.logout.user_id_derived': '0055j000000utlPAAQ', 'salesforce.logout.event_type': 'Logout', 'salesforce.logout.platform_type': 1015, 'salesforce.logout.session_type': 'UI', 'salesforce.logout.client_version': 9998, 'salesforce.logout.user_initiated_logout': 1, 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-19T05:07:07.128Z', 'related.ip': ['123.201.231.42'], 'service.type': 'salesforce', 'event.code': '4exLFFQZNa5xxFl1cJNwOV', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'logout', 'event.type': 'info', 'event.category': 'authentication', 'event.dataset': 'salesforce.logout', 'user.roles': 'Standard(db=S,api=Standard)', 'user.id': '0055j000000utlP'}
assert 3 == 0
 +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'logout', 'old_value': 'logout-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-logout'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-logout-rest'}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_258_salesforce>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_258_salesforce>
test_file = '/var/lib/jenkins/workspace/PR-37509-10-df6931f9-971d-4d10-8fe1-15d8d9e79b4c/src/github.com/elastic/beats/x-pack/filebeat/module/salesforce/logout/test/salesforce_logout.ndjson.log'
objects = [{'@timestamp': '2021-10-19T05:07:07.128Z', 'agent': {'ephemeral_id': 'd748f328-0fe2-4c40-a094-c99becac1559', 'id': '9...: 'logout', 'category': 'authentication', 'code': '4exLFFQZNa5xxFl1cJNwOV', 'dataset': 'salesforce.logout', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['fileset.name']": {'new_value': 'logout', 'old_value': 'logout-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-logout'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-logout-rest'}}, full object: 
E             {'log.offset': 0, 'source.ip': '123.201.231.42', 'fileset.name': 'logout', 'tags': ['salesforce-logout', 'forwarded'], 'salesforce.logout.organization_by_id': '00D5j000000VI3n', 'salesforce.logout.app_type': 1000, 'salesforce.logout.login_key': 'OKyrFuSUIZVr9YzF', 'salesforce.logout.browser_type': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36', 'salesforce.logout.session_level': 'STANDARD(db=1,api=STANDARD)', 'salesforce.logout.resolution_type': 9999, 'salesforce.logout.user_id_derived': '0055j000000utlPAAQ', 'salesforce.logout.event_type': 'Logout', 'salesforce.logout.platform_type': 1015, 'salesforce.logout.session_type': 'UI', 'salesforce.logout.client_version': 9998, 'salesforce.logout.user_initiated_logout': 1, 'salesforce.access_mode': 'rest', 'input.type': 'log', '@timestamp': '2021-10-19T05:07:07.128Z', 'related.ip': ['123.201.231.42'], 'service.type': 'salesforce', 'event.code': '4exLFFQZNa5xxFl1cJNwOV', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'salesforce', 'event.action': 'logout', 'event.type': 'info', 'event.category': 'authentication', 'event.dataset': 'salesforce.logout', 'user.roles': 'Standard(db=S,api=Standard)', 'user.id': '0055j000000utlP'}
E           assert 3 == 0
E            +  where 3 = len({'values_changed': {"root['fileset.name']": {'new_value': 'logout', 'old_value': 'logout-rest'}}, 'iterable_item_added': {"root['tags'][0]": 'salesforce-logout'}, 'iterable_item_removed': {"root['tags'][1]": 'salesforce-logout-rest'}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

Steps errors

Expand to view the steps failures

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

Took 102 min 59 sec . View more details here
Description: mage pythonIntegTest

`x-pack/filebeat-windows-2016-windows-2016 - mage build unitTest`

Took 7 min 41 sec . View more details here
Description: mage build unitTest

`x-pack/filebeat-windows-2016-windows-2016 - mage build unitTest`

Took 2 min 21 sec . View more details here
Description: mage build unitTest

`Error signal`

Took 0 min 0 sec . View more details here
Description: Error 'org.jenkinsci.plugins.workflow.steps.FlowInterruptedException'

:robot: GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Feb 01 '24 10:02 elasticmachine

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Duration: 150 min 59 sec

:grey_exclamation: Flaky test report

No test was executed to be analysed.

:robot: GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Feb 01 '24 15:02 elasticmachine

This pull request is now in conflicts. Could you fix it? 🙏 To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b refactor_salesforce_module upstream/refactor_salesforce_module
git merge upstream/main
git push upstream refactor_salesforce_module

Feb 05 '24 11:02 mergify[bot]

This pull request does not have a backport label. If this is a bug or security fix, could you label this PR @kush-elastic? 🙏. For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

Feb 05 '24 11:02 mergify[bot]

:broken_heart: Build Failed

Buildkite Build
Commit: 7c2a973bef9bc2111d2b64d534fc0e408591f224

Failed CI Steps

:pipeline::arrow_up: Upload Pipeline: .buildkite/winlogbeat/pipeline.winlogbeat.yml

History

:broken_heart: Build #1104 failed cf7719aabee2917372e4724fbb00bad2127974f2
:broken_heart: Build #989 failed 13471e1509d8165cd946475b7be35d32a28f8ba5

cc @kush-elastic

Mar 13 '24 12:03 elasticmachine

:green_heart: Build Succeeded

Buildkite Build
Commit: 7c2a973bef9bc2111d2b64d534fc0e408591f224

History

:green_heart: Build #1941 succeeded cf7719aabee2917372e4724fbb00bad2127974f2
:green_heart: Build #1826 succeeded 13471e1509d8165cd946475b7be35d32a28f8ba5

cc @kush-elastic

Mar 13 '24 12:03 elasticmachine

:green_heart: Build Succeeded

Buildkite Build
Commit: 7c2a973bef9bc2111d2b64d534fc0e408591f224

History

:green_heart: Build #3155 succeeded cf7719aabee2917372e4724fbb00bad2127974f2
:green_heart: Build #3040 succeeded 13471e1509d8165cd946475b7be35d32a28f8ba5

cc @kush-elastic

Mar 13 '24 12:03 elasticmachine

:green_heart: Build Succeeded

Buildkite Build
Commit: 7c2a973bef9bc2111d2b64d534fc0e408591f224

History

:green_heart: Build #1948 succeeded cf7719aabee2917372e4724fbb00bad2127974f2
:green_heart: Build #1833 succeeded 13471e1509d8165cd946475b7be35d32a28f8ba5

cc @kush-elastic

Mar 13 '24 12:03 elasticmachine

:green_heart: Build Succeeded

Buildkite Build
Commit: 7c2a973bef9bc2111d2b64d534fc0e408591f224

History

:green_heart: Build #3579 succeeded cf7719aabee2917372e4724fbb00bad2127974f2
:green_heart: Build #3462 succeeded 13471e1509d8165cd946475b7be35d32a28f8ba5
:green_heart: Build #2452 succeeded ae075acede95e12e4c82f51c302fed8bd6dd8681
:green_heart: Build #2451 succeeded 2303896688aced2d7940adeb9df32b7e386d9bd2
:green_heart: Build #2448 succeeded eb5e1383546ae8d4e4937b7f81e32a7603af95fa
:green_heart: Build #2446 succeeded 85c9bfe9f0ac07fc025d46e0b65d4046337cb994

cc @kush-elastic

Mar 13 '24 12:03 elasticmachine

:green_heart: Build Succeeded

Buildkite Build
Commit: 7c2a973bef9bc2111d2b64d534fc0e408591f224

History

:green_heart: Build #2238 succeeded cf7719aabee2917372e4724fbb00bad2127974f2
:green_heart: Build #2122 succeeded 13471e1509d8165cd946475b7be35d32a28f8ba5

cc @kush-elastic

Mar 13 '24 12:03 elasticmachine

For "setupaudittrail/ingest/pipeline.yml":

diff --git a/x-pack/filebeat/module/salesforce/setupaudittrail/ingest/pipeline.yml b/x-pack/filebeat/module/salesforce/setupaudittrail/ingest/pipeline.yml
index f7776e8e3b..5995fab56a 100644
--- a/x-pack/filebeat/module/salesforce/setupaudittrail/ingest/pipeline.yml
+++ b/x-pack/filebeat/module/salesforce/setupaudittrail/ingest/pipeline.yml
@@ -1,160 +1,170 @@
 ---
 description: Pipeline for parsing Salesforce SetupAuditTrail logs
 processors:
-- json:
-    field: message
-    target_field: json
-    ignore_failure: true
-- set:
-    field: event.ingested
-    value: '{{_ingest.timestamp}}'
-- rename:
-    field: message
-    target_field: event.original
-    ignore_missing: true
-    if: 'ctx.event?.original == null'
-- set:
-    field: salesforce.setup_audit_trail.document_id
-    copy_from: _id
-    ignore_empty_value: true
+    - json:
+          field: message
+          target_field: json
+          on_failure:
+              - drop: {}
 
-- date:
-    field: json.CreatedDate
-    target_field: "@timestamp"
-    formats:
-    - ISO8601
-    ignore_failure: true
+    - set:
+          field: event.ingested
+          value: "{{@timestamp}}"
+    - rename:
+          field: message
+          target_field: event.original
+          ignore_missing: true
+          if: "ctx.event?.original == null"
+    - set:
+          field: salesforce.setup_audit_trail.document_id
+          copy_from: _id
+          ignore_empty_value: true
 
-- rename:
-    field: json.attributes.type
-    target_field: salesforce.setup_audit_trail.event_type
-    ignore_missing: true
-- rename:
-    field: json.CreatedByContext
-    target_field: salesforce.setup_audit_trail.created_by_context
-    ignore_missing: true
-- rename:
-    field: json.CreatedById
-    target_field: salesforce.setup_audit_trail.created_by_id
-    ignore_missing: true
-- rename:
-    field: json.CreatedByIssuer
-    target_field: salesforce.setup_audit_trail.created_by_issuer
-    ignore_missing: true
-- rename:
-    field: json.DelegateUser
-    target_field: salesforce.setup_audit_trail.delegate_user
-    ignore_missing: true
-- rename:
-    field: json.Display
-    target_field: salesforce.setup_audit_trail.display
-    ignore_missing: true
-- rename:
-    field: json.ResponsibleNamespacePrefix
-    target_field: salesforce.setup_audit_trail.responsible_namespace_prefix
-    ignore_missing: true
-- rename:
-    field: json.Section
-    target_field: salesforce.setup_audit_trail.section
-    ignore_missing: true
+    - date:
+          field: json.CreatedDate
+          target_field: "@timestamp"
+          formats:
+              - ISO8601
+          on_failure:
+              - set:
+                    field: "@timestamp"
+                    value: "{{_ingest.timestamp}}"
+              - set:
+                    field: error.message
+                    value: "Failed to parse CreatedDate: {{json.CreatedDate}}"
 
-#######################
-## ECS Event Mapping ##
-#######################
+    # Normalize field names
+    - rename:
+          field: json.attributes.type
+          target_field: salesforce.setup_audit_trail.event_type
+          ignore_missing: true
+    - rename:
+          field: json.CreatedByContext
+          target_field: salesforce.setup_audit_trail.created_by_context
+          ignore_missing: true
+    - rename:
+          field: json.CreatedById
+          target_field: salesforce.setup_audit_trail.created_by_id
+          ignore_missing: true
+    - rename:
+          field: json.CreatedByIssuer
+          target_field: salesforce.setup_audit_trail.created_by_issuer
+          ignore_missing: true
+    - rename:
+          field: json.DelegateUser
+          target_field: salesforce.setup_audit_trail.delegate_user
+          ignore_missing: true
+    - rename:
+          field: json.Display
+          target_field: salesforce.setup_audit_trail.display
+          ignore_missing: true
+    - rename:
+          field: json.ResponsibleNamespacePrefix
+          target_field: salesforce.setup_audit_trail.responsible_namespace_prefix
+          ignore_missing: true
+    - rename:
+          field: json.Section
+          target_field: salesforce.setup_audit_trail.section
+          ignore_missing: true
 
-- set:
-    field: ecs.version
-    value: "8.5.0"
-    ignore_failure: true
-    ignore_empty_value: true
-- rename:
-    field: json.Id
-    target_field: event.id
-    ignore_missing: true
-    ignore_failure: true
-- rename:
-    field: json.Action
-    target_field: event.action
-    ignore_missing: true
-    ignore_failure: true
-- rename:
-    field: json.attributes.url
-    target_field: event.url
-    ignore_missing: true
-    ignore_failure: true
-- date:
-    field: json.CreatedDate
-    target_field: event.created
-    formats: 
-    - ISO8601
-    ignore_failure: true
-- set:
-    field: event.type
-    value: ["admin"]
-    ignore_failure: true
-    ignore_empty_value: true
-- set:
-    field: event.kind
-    value: "event"
-    ignore_failure: true
-    ignore_empty_value: true
-- set:
-    field: event.dataset
-    value: "salesforce.setupaudittrail"
-    ignore_failure: true
-    ignore_empty_value: true
-- set:
-    field: event.module
-    value: "salesforce"
-    ignore_failure: true
-    ignore_empty_value: true
+    #######################
+    ## ECS Event Mapping ##
+    #######################
 
-######################
-## ECS User Mapping ##
-######################
+    - set:
+          field: ecs.version
+          value: "8.5.0"
+          ignore_failure: true
+          ignore_empty_value: true
+    - rename:
+          field: json.Id
+          target_field: event.id
+          ignore_missing: true
+          ignore_failure: true
+    - rename:
+          field: json.Action
+          target_field: event.action
+          ignore_missing: true
+          ignore_failure: true
+    - rename:
+          field: json.attributes.url
+          target_field: event.url
+          ignore_missing: true
+          ignore_failure: true
+    - date:
+          field: json.CreatedDate
+          target_field: event.created
+          formats:
+              - ISO8601
+          ignore_failure: true
+    - set:
+          field: event.type
+          value: ["admin"]
+          ignore_failure: true
+          ignore_empty_value: true
+    - set:
+          field: event.kind
+          value: "event"
+          ignore_failure: true
+          ignore_empty_value: true
+    - set:
+          field: event.dataset
+          value: "salesforce.setupaudittrail"
+          ignore_failure: true
+          ignore_empty_value: true
+    - set:
+          field: event.module
+          value: "salesforce"
+          ignore_failure: true
+          ignore_empty_value: true
 
-- set:
-    field: user.id
-    copy_from: salesforce.setup_audit_trail.created_by_id
-    ignore_failure: true
-    ignore_empty_value: true
-- dissect:
-    field: salesforce.setup_audit_trail.display
-    pattern: "For user %{user.name}, %{?}"
-    ignore_failure: true
+    ######################
+    ## ECS User Mapping ##
+    ######################
 
-#############
-## Cleanup ##
-#############
+    - set:
+          field: user.id
+          copy_from: salesforce.setup_audit_trail.created_by_id
+          ignore_failure: true
+          ignore_empty_value: true
+    - dissect:
+          field: salesforce.setup_audit_trail.display
+          pattern: "For user %{user.name}"
+          ignore_failure: true
 
-- script:
-    description: Drops null/empty values recursively
-    lang: painless
-    source: |
-        boolean dropEmptyFields(Object object) {
-            if (object == null || object == "") {
-                return true;
-            } else if (object instanceof Map) {
-                ((Map) object).values().removeIf(value -> dropEmptyFields(value));
-                return (((Map) object).size() == 0);
-            } else if (object instanceof List) {
-                ((List) object).removeIf(value -> dropEmptyFields(value));
-                return (((List) object).length == 0);
-            }
-            return false;
-        }
-        dropEmptyFields(ctx);
-- remove:
-    field:
-      - json
-      - message
-    ignore_missing: true
-- remove:
-    field: event.original
-    if: "ctx.tags == null || !(ctx.tags.contains('preserve_original_event'))"
-    ignore_failure: true
-    ignore_missing: true
+    #############
+    ## Cleanup ##
+    #############
+
+    # Drop empty fields
+    - script:
+          lang: painless
+          source: |
+              boolean dropEmptyFields(Object object) {
+                if (object == null || object == "") {
+                  return true;
+                } else if (object instanceof Map) {
+                  ((Map) object).values().removeIf(value -> dropEmptyFields(value));
+                  return (((Map) object).size() == 0);
+                } else if (object instanceof List) {
+                  ((List) object).removeIf(value -> dropEmptyFields(value));
+                  return (((List) object).size() == 0);
+                }
+                return false;
+              }
+              dropEmptyFields(ctx);
+
+    - remove:
+          field:
+              - json
+              - message
+          ignore_missing: true
+    - remove:
+          field: event.original
+          if: "ctx.tags == null || !(ctx.tags.contains('preserve_original_event'))"
+          ignore_failure: true
+          ignore_missing: true
 on_failure:
-- set:
-    field: error.message
-    value: '{{_ingest.on_failure_message}}'
+    - set:
+          field: error.message
+          value: "{{_ingest.on_failure_message}}"

Apr 03 '24 09:04 shmsr

For "logout/ingest/default.yml"

diff --git a/x-pack/filebeat/module/salesforce/logout/ingest/default.yml b/x-pack/filebeat/module/salesforce/logout/ingest/default.yml
index 212ad3b245..0539f1df64 100644
--- a/x-pack/filebeat/module/salesforce/logout/ingest/default.yml
+++ b/x-pack/filebeat/module/salesforce/logout/ingest/default.yml
@@ -4,10 +4,13 @@ processors:
 - json:
     field: message
     target_field: json
-    ignore_failure: true
+    on_failure:
+    - append:
+        field: error.message
+        value: "Failed to parse JSON: {{{_ingest.on_failure_message}}}"
 - set:
     field: event.original
-    value: '{{{message}}}'
+    copy_from: message
     ignore_empty_value: true
     if: 'ctx.event?.original == null'
 - set:
@@ -21,6 +24,7 @@ processors:
     - json.USER_ID
     target_field: _id
     ignore_missing: true
+    ignore_failure: true
 - set:
     field: ecs.version
     value: "8.5.0"
@@ -30,14 +34,12 @@ processors:
     field: salesforce.login.document_id
     copy_from: _id
     ignore_empty_value: true
-
 - pipeline:
-    if: ctx.event.provider == "Object"
     name: '{< IngestPipeline "object" >}'
+    if: ctx.event?.provider == "Object"
 - pipeline:
-    if: ctx.event.provider == "EventLogFile"
     name: '{< IngestPipeline "eventlogfile" >}'
-
+    if: ctx.event?.provider == "EventLogFile"
 - set:
     field: event.type
     value: ["info"]
@@ -78,33 +80,32 @@ processors:
     if: ctx?.source?.ip != null
     allow_duplicates: false
     ignore_failure: true
-
 - remove:
     field: event.original
-    if: "ctx.tags == null || !(ctx.tags.contains('preserve_original_event'))"
+    if: "ctx.tags == null || !ctx.tags.contains('preserve_original_event')"
     ignore_failure: true
     ignore_missing: true
 - script:
     description: Drops null/empty values recursively.
     lang: painless
     source: |
-        boolean dropEmptyFields(Object object) {
-            if (object == null || object == "") {
-                return true;
-            } else if (object instanceof Map) {
-                ((Map) object).values().removeIf(value -> dropEmptyFields(value));
-                return (((Map) object).size() == 0);
-            } else if (object instanceof List) {
-                ((List) object).removeIf(value -> dropEmptyFields(value));
-                return (((List) object).length == 0);
-            }
-            return false;
+      boolean dropEmptyFields(Object object) {
+        if (object == null || object == "") {
+          return true;
+        } else if (object instanceof Map) {
+          ((Map) object).values().removeIf(value -> dropEmptyFields(value));
+          return ((Map) object).isEmpty();
+        } else if (object instanceof List) {
+          ((List) object).removeIf(value -> dropEmptyFields(value));
+          return ((List) object).isEmpty();
         }
-        dropEmptyFields(ctx);
+        return false;
+      }
+      dropEmptyFields(ctx);
 - remove:
     field:
-      - json
-      - message
+    - json
+    - message
     ignore_failure: true
     ignore_missing: true
 
@@ -112,6 +113,9 @@ on_failure:
 - set:
     field: event.kind
     value: pipeline_error
-- append:
+- set:
     field: error.message
-    value: '{{{ _ingest.on_failure_message }}}'
+    value: '{{{_ingest.on_failure_message}}}'
+- set:
+    field: error.type
+    value: '{{{_ingest.on_failure_processor_type}}}'
\ No newline at end of file

Apr 03 '24 09:04 shmsr

For "logout/ingest/eventlogfile.yml":

diff --git a/x-pack/filebeat/module/salesforce/logout/ingest/eventlogfile.yml b/x-pack/filebeat/module/salesforce/logout/ingest/eventlogfile.yml
index 8abd4f3bc6..c829f85d01 100644
--- a/x-pack/filebeat/module/salesforce/logout/ingest/eventlogfile.yml
+++ b/x-pack/filebeat/module/salesforce/logout/ingest/eventlogfile.yml
@@ -6,221 +6,228 @@ processors:
     target_field: "@timestamp"
     formats:
     - ISO8601
-    ignore_failure: true
-- rename: 
+    on_failure:
+    - append:
+        field: error.message
+        value: "Failed to parse TIMESTAMP_DERIVED field: {{{_ingest.on_failure_message}}}"
+- rename:
     field: json.SESSION_TYPE
     target_field: salesforce.logout.session.type
-    ignore_failure: true
     ignore_missing: true
 - script:
     description: Set session type value from session type character.
     lang: painless
-    ignore_failure: true
     source: |
-        Map map = new HashMap();
-        map.put("A", "API");
-        map.put("I", "APIOnlyUser");
-        map.put("N", "ChatterNetworks");
-        map.put("Z", "ChatterNetworksAPIOnly");
-        map.put("C", "Content");
-        map.put("P", "OauthApprovalUI");
-        map.put("O", "Oauth2");
-        map.put("T", "SiteStudio");
-        map.put("R", "SitePreview");
-        map.put("S", "SubstituteUser");
-        map.put("B", "TempContentExchange");
-        map.put("G", "TempOauthAccessTokenFrontdoor");
-        map.put("Y", "TempVisualforceExchange");
-        map.put("F", "TempUIFrontdoor");
-        map.put("U", "UI");
-        map.put("E", "UserSite");
-        map.put("V", "Visualforce");
-        map.put("W", "WDC_API");
-        String temp = map.get(ctx.salesforce?.logout?.session?.type);
-        if (temp != null) {
-            ctx.salesforce.logout.session.type = temp;
+        def sessionTypes = [
+          "A": "API",
+          "I": "APIOnlyUser",
+          "N": "ChatterNetworks",
+          "Z": "ChatterNetworksAPIOnly",
+          "C": "Content",
+          "P": "OauthApprovalUI",
+          "O": "Oauth2",
+          "T": "SiteStudio",
+          "R": "SitePreview", 
+          "S": "SubstituteUser",
+          "B": "TempContentExchange",
+          "G": "TempOauthAccessTokenFrontdoor",
+          "Y": "TempVisualforceExchange",
+          "F": "TempUIFrontdoor",
+          "U": "UI",
+          "E": "UserSite",
+          "V": "Visualforce",
+          "W": "WDC_API"
+        ];
+        def type = ctx.salesforce?.logout?.session?.type;
+        if (type != null && sessionTypes.containsKey(type)) {
+          ctx.salesforce.logout.session.type = sessionTypes.get(type);
         }
-- rename: 
+    on_failure:
+    - append:
+        field: error.message
+        value: "Failed to set salesforce.logout.session.type: {{{_ingest.on_failure_message}}}"
+- rename:
     field: json.EVENT_TYPE
     target_field: salesforce.logout.event_type
-    ignore_failure: true
     ignore_missing: true
-- rename: 
-    field: json.SESSION_LEVEL
+- rename:
+    field: json.SESSION_LEVEL  
     target_field: salesforce.logout.session.level
-    ignore_failure: true
     ignore_missing: true
 - script:
     description: Set session level value from session level integer.
     lang: painless
     ignore_failure: true
     source: |
-        Map map = new HashMap();
-        map.put("1", "Standard Session");
-        map.put("2", "High-Assurance Session");
-        String temp = map.get(ctx.salesforce?.logout?.session?.level);
-        if (temp != null){
-            ctx.salesforce.logout.session.level = temp;
-        }
-- rename: 
+      def levels = ["1": "Standard Session", "2": "High-Assurance Session"];
+      def level = ctx.salesforce?.logout?.session?.level;
+      if (level != null && levels.containsKey(level)) {
+        ctx.salesforce.logout.session.level = levels.get(level);
+      }
+    on_failure:
+    - append:  
+        field: error.message
+        value: "Failed to set salesforce.logout.session.level: {{{_ingest.on_failure_message}}}"
+- rename:
     field: json.BROWSER_TYPE
-    target_field: salesforce.logout.browser_type
-    ignore_failure: true
-    ignore_missing: true
-- rename: 
+    target_field: user_agent.name
+    ignore_missing: true    
+- rename:
     field: json.PLATFORM_TYPE
     target_field: salesforce.logout.platform_type
-    ignore_failure: true
     ignore_missing: true
 - script:
     description: Set platform type value from platform type integers.
     lang: painless
-    ignore_failure: true
+    params:
+      platforms:
+        "1000": "Windows"
+        "1008": "Windows 2003"  
+        "1013": "Windows 8.1"
+        "1015": "Windows 10"
+        "2003": "Macintosh/Apple OSX"
+        "4000": "Linux"
+        "5005": "Android"
+        "5006": "iPhone"
+        "5007": "iPad"
+        "5200": "Android 10.0"
     source: |
-        Map map = new HashMap();
-        map.put("1000", "Windows");
-        map.put("1008", "Windows 2003");
-        map.put("1013", "Windows 8.1");
-        map.put("1015", "Windows 10");
-        map.put("2003", "Macintosh/Apple OSX");
-        map.put("4000", "Linux");
-        map.put("5005", "Android");
-        map.put("5006", "iPhone");
-        map.put("5007", "iPad");
-        map.put("5200", "Android 10.0");
-        String temp = map.get(ctx.salesforce?.logout?.platform_type);
-        if (temp != null) {
-            ctx.salesforce.logout.platform_type = temp;
-        }
-- rename: 
+      def platform = ctx.salesforce?.logout?.platform_type; 
+      if (platform != null && params.platforms.containsKey(platform)) {
+        ctx.salesforce.logout.platform_type = params.platforms.get(platform);
+      }
+    on_failure:
+    - append:
+        field: error.message
+        value: "Failed to set salesforce.logout.platform_type: {{{_ingest.on_failure_message}}}"
+- rename:  
     field: json.RESOLUTION_TYPE
     target_field: salesforce.logout.resolution_type
-    ignore_failure: true
     ignore_missing: true
-- rename: 
+- rename:
     field: json.APP_TYPE
     target_field: salesforce.logout.app_type
-    ignore_failure: true
     ignore_missing: true
-- script:
+- script:  
     description: Set application type value from application type integers.
     lang: painless
-    ignore_failure: true
+    params:
+      appTypes: 
+        "1000": "Application"
+        "1007": "SFDC Application"
+        "1014": "Chat"  
+        "2501": "CTI"
+        "2514": "OAuth"
+        "3475": "SFDC Partner Portal"
     source: |
-        Map map = new HashMap();
-        map.put("1000", "Application");
-        map.put("1007", "SFDC Application");
-        map.put("1014", "Chat");
-        map.put("2501", "CTI");
-        map.put("2514", "OAuth");
-        map.put("3475", "SFDC Partner Portal");
-        String temp = map.get(ctx.salesforce?.logout?.app_type);
-        if (temp != null) {
-            ctx.salesforce.logout.app_type = temp;
-        }
-- rename: 
+      def type = ctx.salesforce?.logout?.app_type;
+      if (type != null && params.appTypes.containsKey(type)) { 
+        ctx.salesforce.logout.app_type = params.appTypes.get(type);
+      }  
+    on_failure:
+    - append:
+        field: error.message 
+        value: "Failed to set salesforce.logout.app_type: {{{_ingest.on_failure_message}}}"
+- rename:
     field: json.CLIENT_VERSION
     target_field: salesforce.logout.client_version
-    ignore_failure: true
     ignore_missing: true
-- rename: 
+- rename:  
     field: json.API_TYPE
     target_field: salesforce.logout.api.type
-    ignore_failure: true
     ignore_missing: true
 - script:
     description: Set api type value from api type character.
-    lang: painless
-    ignore_failure: true
+    lang: painless  
+    params:
+      apiTypes:
+        "D": "Apex Class"
+        "E": "SOAP Enterprise" 
+        "I": "SOAP Cross Instance"
+        "M": "SOAP Metadata"
+        "O": "Old SOAP"
+        "P": "SOAP Partner"
+        "S": "SOAP Apex" 
+        "T": "SOAP Tooling"
+        "X": "XmlRPC"
+        "f": "Feed"
+        "l": "Live Agent"
+        "p": "SOAP ClientSync"
     source: |
-        Map map = new HashMap();
-        map.put("D", "Apex Class");
-        map.put("E", "SOAP Enterprise");
-        map.put("I", "SOAP Cross Instance");
-        map.put("M", "SOAP Metadata");
-        map.put("O", "Old SOAP");
-        map.put("P", "SOAP Partner");
-        map.put("S", "SOAP Apex");
-        map.put("T", "SOAP Tooling");
-        map.put("X", "XmlRPC");
-        map.put("f", "Feed");
-        map.put("l", "Live Agent");
-        map.put("p", "SOAP ClientSync");
-        String temp = map.get(ctx.salesforce?.logout?.api?.type);
-        if (temp != null) {
-            ctx.salesforce.logout.api.type = temp;
-        }
-- rename: 
+      def type = ctx.salesforce?.logout?.api?.type;
+      if (type != null && params.apiTypes.containsKey(type)) {
+        ctx.salesforce.logout.api.type = params.apiTypes.get(type);  
+      }
+    on_failure:  
+    - append:
+        field: error.message
+        value: "Failed to set salesforce.logout.api.type: {{{_ingest.on_failure_message}}}" 
+- rename:
     field: json.API_VERSION
     target_field: salesforce.logout.api.version
-    ignore_failure: true
     ignore_missing: true
-- rename: 
+- rename:
     field: json.USER_INITIATED_LOGOUT
-    target_field: salesforce.logout.user_initiated_logout
-    ignore_failure: true
+    target_field: salesforce.logout.user_initiated
     ignore_missing: true
-- rename: 
+- rename:
     field: json.LOGIN_KEY
     target_field: salesforce.logout.login_key
-    ignore_failure: true
     ignore_missing: true
-- rename: 
+- rename:
     field: json.USER_ID
     target_field: salesforce.logout.user_id
-    ignore_failure: true
     ignore_missing: true
-- rename: 
+- rename:
     field: json.ORGANIZATION_ID
     target_field: salesforce.logout.organization_id
-    ignore_failure: true
     ignore_missing: true
 - rename:
     field: json.REQUEST_ID
-    target_field: event.code
-    ignore_failure: true
-    ignore_missing: true
-- rename: 
-    field: json.USER_TYPE
-    target_field: user.roles
-    ignore_failure: true
+    target_field: event.id
+    ignore_missing: true  
+- rename:
+    field: json.USER_TYPE  
+    target_field: salesforce.logout.user.roles
     ignore_missing: true
 - script:
     description: Set user roles value from user roles character.
     lang: painless
-    ignore_failure: true
+    params:
+      userRoles:  
+        "A": "Automated Process"
+        "b": "High Volume Portal"
+        "C": "Customer Portal User"
+        "D": "External Who"  
+        "F": "Self-Service"
+        "G": "Guest"
+        "L": "Package License Manager"
+        "N": "Salesforce to Salesforce"
+        "n": "CSN Only" 
+        "O": "Power Custom"
+        "o": "Custom"  
+        "P": "Partner"
+        "p": "Customer Portal Manager"
+        "S": "Standard"
+        "X": "Salesforce Administrator" 
     source: |
-        Map map = new HashMap();
-        map.put("A", "Automated Process");
-        map.put("b", "High Volume Portal");
-        map.put("C", "Customer Portal User");
-        map.put("D", "External Who");
-        map.put("F", "Self-Service");
-        map.put("G", "Guest");
-        map.put("L", "Package License Manager");
-        map.put("N", "Salesforce to Salesforce");
-        map.put("n", "CSN Only");
-        map.put("O", "Power Custom");
-        map.put("o", "Custom");
-        map.put("P", "Partner");
-        map.put("p", "Customer Portal Manager");
-        map.put("S", "Standard");
-        map.put("X", "Salesforce Administrator");
-        String temp = map.get(ctx.user?.roles);
-        if (temp != null) {
-            ctx.user.roles = [temp];
-        }
-- rename: 
+      def role = ctx.salesforce?.logout?.user?.roles;
+      if (role != null && params.userRoles.containsKey(role)) {
+        ctx.salesforce.logout.user.roles = [params.userRoles.get(role)];
+      }
+    on_failure:
+    - append:  
+        field: error.message
+        value: "Failed to set salesforce.logout.user.roles: {{{_ingest.on_failure_message}}}"
+- rename:
     field: json.USER_ID_DERIVED
     target_field: user.id
-    ignore_failure: true
     ignore_missing: true
 - rename:
     field: json.CLIENT_IP
     target_field: source.ip
-    ignore_failure: true
     ignore_missing: true
-    if: 'ctx.json?.CLIENT_IP != "Salesforce.com IP" && ctx.json?.CLIENT_IP != ""'
+    if: ctx.json?.CLIENT_IP != 'Salesforce.com IP' && ctx.json?.CLIENT_IP != null
 - geoip:
     field: source.ip
     target_field: source.geo

Apr 03 '24 09:04 shmsr

For logout/ingest/object.yml:

diff --git a/x-pack/filebeat/module/salesforce/logout/ingest/object.yml b/x-pack/filebeat/module/salesforce/logout/ingest/object.yml
index 6f8db59a2b..0afeafc782 100644
--- a/x-pack/filebeat/module/salesforce/logout/ingest/object.yml
+++ b/x-pack/filebeat/module/salesforce/logout/ingest/object.yml
@@ -6,23 +6,26 @@ processors:
     target_field: "@timestamp"
     formats:
     - ISO8601
-    ignore_failure: true
-- rename: 
+    on_failure:
+    - append:
+        field: error.message
+        value: "Failed to parse EventDate field: {{{_ingest.on_failure_message}}}"
+- rename:
     field: json.RelatedEventIdentifier
     target_field: salesforce.logout.related_event_identifier
     ignore_missing: true
     ignore_failure: true
-- rename: 
+- rename:
     field: json.LoginKey
     target_field: salesforce.logout.login_key
     ignore_missing: true
     ignore_failure: true
-- rename: 
+- rename:
     field: json.SessionLevel
     target_field: salesforce.logout.session.level
     ignore_missing: true
     ignore_failure: true
-- rename: 
+- rename:
     field: json.SessionKey
     target_field: salesforce.logout.session.key
     ignore_missing: true
@@ -30,25 +33,31 @@ processors:
 - date:
     field: json.CreatedDate
     target_field: event.created
-    formats: 
-    - ISO8601
-    ignore_failure: true
+    formats:
+    - ISO8601 
+    on_failure:
+    - append:
+        field: error.message
+        value: "Failed to parse CreatedDate field: {{{_ingest.on_failure_message}}}"
 - rename:
     field: json.Username
     target_field: user.email
     ignore_failure: true
     ignore_missing: true
 - rename:
-    field: json.UserId
+    field: json.UserId  
     target_field: user.id
     ignore_failure: true
     ignore_missing: true
-- convert:
+- convert: 
     field: json.SourceIp
     type: ip
     target_field: source.ip
     ignore_missing: true
-    ignore_failure: true
+    on_failure:
+    - append:
+        field: error.message
+        value: "Failed to convert SourceIp to IP: {{{_ingest.on_failure_message}}}"
 
 on_failure:
 - set:

Apr 03 '24 10:04 shmsr

This pull request is now in conflicts. Could you fix it? 🙏 To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b refactor_salesforce_module upstream/refactor_salesforce_module
git merge upstream/main
git push upstream refactor_salesforce_module

Apr 05 '24 10:04 mergify[bot]

@shmsr : Since this is a revamp of module, are we mentioning somewhere is the documentation about this for the existing users?

Apr 10 '24 11:04 ishleenk17

@shmsr : Since this is a revamp of module, are we mentioning somewhere is the documentation about this for the existing users?

Thanks. Will mention.

Apr 11 '24 12:04 shmsr

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

Apr 12 '24 07:04 elasticmachine

Given the extensive size of the PR, reviewing all the changes thoroughly might pose a challenge. However, Kush and I have diligently gone through it, addressing numerous bugs along the way. @aliabbas-elastic and @niraj-elastic are helping in the multiple iterations of the review and testing process. Since this marks the initial alpha release, we have a window of opportunity to address any lingering bugs or implement enhancements.

cc: @lalit-satapathy Will require your approval to merge this.

Apr 15 '24 11:04 shmsr

Approving on testing side. Refer here for the testing comments

Thanks @aliabbas-elastic!

Apr 15 '24 14:04 shmsr

beats beats copied to clipboard

x-pack/filebeat/module/salesforce: Refactor a Salesforce module

Description

Checklist

Configuration

Related issues

:broken_heart: Tests Failed

Build stats

Test stats :test_tube:

Test errors

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_183_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_184_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_185_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_186_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_187_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_188_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_189_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_190_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_191_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Steps errors

x-pack/filebeat-pythonIntegTest - mage pythonIntegTest

x-pack/filebeat-pythonIntegTest - mage pythonIntegTest

x-pack/filebeat-pythonIntegTest - mage pythonIntegTest

Error signal

:bug: Flaky test report

Genuine test errors

:robot: GitHub comments

:broken_heart: Build Failed

Build stats

Pipeline error

:grey_exclamation: Flaky test report

:robot: GitHub comments

:broken_heart: Build Failed

Build stats

Pipeline error

:grey_exclamation: Flaky test report

:robot: GitHub comments

:grey_exclamation: Build Aborted

Build stats

Test stats :test_tube:

Test errors

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_250_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_251_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_252_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_253_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_254_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_255_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_256_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_257_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_258_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Steps errors

x-pack/filebeat-pythonIntegTest - mage pythonIntegTest

x-pack/filebeat-windows-2016-windows-2016 - mage build unitTest

x-pack/filebeat-windows-2016-windows-2016 - mage build unitTest

Error signal

:robot: GitHub comments

:green_heart: Build Succeeded

Build stats

:grey_exclamation: Flaky test report

:robot: GitHub comments

:broken_heart: Build Failed

Failed CI Steps

History

:green_heart: Build Succeeded

History

:green_heart: Build Succeeded

History

:green_heart: Build Succeeded

History

:green_heart: Build Succeeded

History

:green_heart: Build Succeeded

History

beats
beats copied to clipboard

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_183_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_184_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_185_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_186_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_187_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_188_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_189_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_190_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_191_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

`Error signal`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_250_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_251_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_252_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_253_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_254_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_255_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_256_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_257_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_258_salesforce – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

`x-pack/filebeat-windows-2016-windows-2016 - mage build unitTest`

`x-pack/filebeat-windows-2016-windows-2016 - mage build unitTest`

`Error signal`