[Security Solution] The Security Solution plugin is unavailable when `config/kibana.yml` is configured with `xpack.fleet.enabled: false`

[andrew-goldstein]

[Security Solution] The Security Solution plugin is unavailable when config/kibana.yml is configured with xpack.fleet.enabled: false

Describe the bug:

A user reported that the Security Solution plugin is unavailable when config/kibana.yml is configured with xpack.fleet.enabled: false.

The appears to be a result of transitive dependencies via the cloudDefend and cloudSecurityPosture plugins.

Please see the Steps to reproduce section below for detailed log output.

Kibana/Elasticsearch Stack version:

8.10.4, 8.11, main

Original install method (e.g. download page, yum, from source, etc.):

self managed

Steps to reproduce:

1. Add the following setting:

xpack.fleet.enabled: false

to

config/kibana.dev.yml.

2. Start Kibana (e.g. yarn start)

Expected result

• The Security Solution is available when (for example) the elastic user logs into Kibana

Actual results

• The Security Solution is NOT available when the elastic user logs into Kibana, per the screenshot below:

Another symptom of this issue is the Security Solution is unavailable in Stack Management > Roles > Create > Kibana privileges, (even when logged in as the elastic user), per the screenshot below:

The Security Solution does not appear in the screenshots above because the securitySolution plugin is disabled at startup.

The apparent root cause of this issue may be observed in Kibana's server log when it starts:

[2023-12-07T12:53:40.423-05:00][INFO ][plugins-service] Plugin "securitySolution" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [cloudDefend, cloudSecurityPosture]

When the cloudDefend plugin starts, it logs the following output:

[2023-12-07T12:53:40.307-05:00][INFO ][plugins-service] Plugin "cloudDefend" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [fleet]

When the cloudSecurityPosture plugin starts, it logs the following output:

[2023-12-07T12:53:40.308-05:00][INFO ][plugins-service] Plugin "cloudSecurityPosture" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [fleet]

The following abridged output from the Kibana log illustrates the effect xpack.fleet.enabled: false has on plugins at startup:

[2023-12-07T12:53:35.220-05:00][INFO ][root] Kibana is starting
[2023-12-07T12:53:35.235-05:00][INFO ][node] Kibana process configured with roles: [background_tasks, ui]
[2023-12-07T12:53:40.298-05:00][INFO ][plugins-service] Plugin "apm" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [observability]
[2023-12-07T12:53:40.307-05:00][INFO ][plugins-service] Plugin "cloudDefend" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [fleet]
[2023-12-07T12:53:40.307-05:00][INFO ][plugins-service] Plugin "cloudChat" is disabled.
[2023-12-07T12:53:40.307-05:00][INFO ][plugins-service] Plugin "cloudExperiments" is disabled.
[2023-12-07T12:53:40.307-05:00][INFO ][plugins-service] Plugin "cloudFullStory" is disabled.
[2023-12-07T12:53:40.307-05:00][INFO ][plugins-service] Plugin "cloudGainsight" is disabled.
[2023-12-07T12:53:40.308-05:00][INFO ][plugins-service] Plugin "cloudSecurityPosture" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [fleet]
[2023-12-07T12:53:40.311-05:00][INFO ][plugins-service] Plugin "datasetQuality" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [fleet]
[2023-12-07T12:53:40.322-05:00][INFO ][plugins-service] Plugin "fleet" is disabled.
[2023-12-07T12:53:40.347-05:00][INFO ][plugins-service] Plugin "infra" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [observability]
[2023-12-07T12:53:40.351-05:00][INFO ][plugins-service] Plugin "logExplorer" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [fleet]
[2023-12-07T12:53:40.372-05:00][INFO ][plugins-service] Plugin "monitoring" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [observability]
[2023-12-07T12:53:40.385-05:00][INFO ][plugins-service] Plugin "observability" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [logExplorer]
[2023-12-07T12:53:40.393-05:00][INFO ][plugins-service] Plugin "observabilityLogExplorer" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [logExplorer, datasetQuality]
[2023-12-07T12:53:40.406-05:00][INFO ][plugins-service] Plugin "observabilityOnboarding" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [observability, fleet]
[2023-12-07T12:53:40.408-05:00][INFO ][plugins-service] Plugin "profiling" is disabled.
[2023-12-07T12:53:40.408-05:00][INFO ][plugins-service] Plugin "profilingDataAccess" is disabled.
[2023-12-07T12:53:40.423-05:00][INFO ][plugins-service] Plugin "securitySolution" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [cloudDefend, cloudSecurityPosture]
[2023-12-07T12:53:40.439-05:00][INFO ][plugins-service] Plugin "securitySolutionEss" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [securitySolution]
[2023-12-07T12:53:40.439-05:00][INFO ][plugins-service] Plugin "securitySolutionServerless" is disabled.
[2023-12-07T12:53:40.439-05:00][INFO ][plugins-service] Plugin "serverless" is disabled.
[2023-12-07T12:53:40.440-05:00][INFO ][plugins-service] Plugin "serverlessObservability" is disabled.
[2023-12-07T12:53:40.440-05:00][INFO ][plugins-service] Plugin "serverlessSearch" is disabled.
[2023-12-07T12:53:40.465-05:00][INFO ][plugins-service] Plugin "synthetics" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [fleet, observability]
[2023-12-07T12:53:40.491-05:00][INFO ][plugins-service] Plugin "uptime" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [fleet, observability]
[2023-12-07T12:53:40.528-05:00][INFO ][plugins-service] Plugin "ux" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [apm]
[2023-12-07T12:53:40.557-05:00][INFO ][http.server.Preboot] http server running at http://localhost:5601
[2023-12-07T12:53:40.616-05:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
[2023-12-07T12:53:40.794-05:00][INFO ][plugins-system.standard] Setting up [134] plugins: [translations,monitoringCollection,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,customBranding,devTools,share,screenshotMode,banners,usageCollection,taskManager,cloud,guidedOnboarding,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,newsfeed,savedObjectsFinder,noDataPage,ftrApis,fieldFormats,expressions,screenshotting,esUiShared,customIntegrations,contentManagement,dataViews,home,searchprofiler,painlessLab,grokdebugger,management,spaces,security,snapshotRestore,logstash,lists,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,encryptedSavedObjects,telemetry,licenseManagement,files,eventLog,actions,notifications,cloudDataMigration,advancedSettings,console,bfetch,data,rollup,fileUpload,ingestPipelines,ecsDataQualityDashboard,apmDataAccess,savedObjectsTagging,savedObjectsManagement,unifiedSearch,alerting,graph,embeddable,uiActionsEnhanced,savedSearch,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,controls,dataViewFieldEditor,dataViewManagement,charts,watcher,visualizations,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,inputControlVis,eventAnnotation,expressionXY,dashboard,triggersActionsUi,transform,stackConnectors,stackAlerts,ruleRegistry,links,dashboardEnhanced,expressionTagcloud,expressionPartitionVis,visTypePie,expressionMetricVis,expressionLegacyMetricVis,expressionHeatmap,expressionGauge,lens,maps,cases,timelines,sessionView,kubernetesSecurity,threatIntelligence,observabilityAIAssistant,logsShared,upgradeAssistant,metricsDataAccess,assetManager,aiops,discover,reporting,canvas,osquery,discoverEnhanced,dataVisualizer,ml,enterpriseSearch,elasticAssistant,visTypeGauge,mockIdpPlugin]
[2023-12-07T12:53:43.735-05:00][INFO ][plugins.alerting] Registering resources for context "ml.anomaly-detection".
[2023-12-07T12:53:44.194-05:00][INFO ][status.core.elasticsearch] elasticsearch service is now available: Elasticsearch is available
[2023-12-07T12:53:44.194-05:00][INFO ][status.core.savedObjects] savedObjects service is now available: SavedObjects service has completed migrations and is available
[2023-12-07T12:53:44.195-05:00][INFO ][plugins-system.preboot] Stopping all plugins.
[2023-12-07T12:53:44.195-05:00][INFO ][plugins-system.preboot] All plugins stopped.
[2023-12-07T12:53:44.195-05:00][INFO ][plugins-system.standard] Starting [134] plugins: [translations,monitoringCollection,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,customBranding,devTools,share,screenshotMode,banners,usageCollection,taskManager,cloud,guidedOnboarding,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,newsfeed,savedObjectsFinder,noDataPage,ftrApis,fieldFormats,expressions,screenshotting,esUiShared,customIntegrations,contentManagement,dataViews,home,searchprofiler,painlessLab,grokdebugger,management,spaces,security,snapshotRestore,logstash,lists,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,encryptedSavedObjects,telemetry,licenseManagement,files,eventLog,actions,notifications,cloudDataMigration,advancedSettings,console,bfetch,data,rollup,fileUpload,ingestPipelines,ecsDataQualityDashboard,apmDataAccess,savedObjectsTagging,savedObjectsManagement,unifiedSearch,alerting,graph,embeddable,uiActionsEnhanced,savedSearch,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,controls,dataViewFieldEditor,dataViewManagement,charts,watcher,visualizations,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,inputControlVis,eventAnnotation,expressionXY,dashboard,triggersActionsUi,transform,stackConnectors,stackAlerts,ruleRegistry,links,dashboardEnhanced,expressionTagcloud,expressionPartitionVis,visTypePie,expressionMetricVis,expressionLegacyMetricVis,expressionHeatmap,expressionGauge,lens,maps,cases,timelines,sessionView,kubernetesSecurity,threatIntelligence,observabilityAIAssistant,logsShared,upgradeAssistant,metricsDataAccess,assetManager,aiops,discover,reporting,canvas,osquery,discoverEnhanced,dataVisualizer,ml,enterpriseSearch,elasticAssistant,visTypeGauge,mockIdpPlugin]
[2023-12-07T12:53:44.453-05:00][INFO ][plugins.alerting] Installing ILM policy .alerts-ilm-policy
[2023-12-07T12:53:44.468-05:00][INFO ][plugins.ruleRegistry] Installing component template .alerts-technical-mappings
[2023-12-07T12:53:44.908-05:00][INFO ][http.server.Kibana] http server running at http://localhost:5601
[2023-12-07T12:53:50.292-05:00][INFO ][status.plugins.alerting] alerting plugin is now available: Alerting is (probably) ready
[2023-12-07T12:53:50.292-05:00][INFO ][status.plugins.licensing] licensing plugin is now available: License fetched
[2023-12-07T12:53:50.292-05:00][INFO ][status.plugins.taskManager] taskManager plugin is now available: Task Manager is healthy
[2023-12-07T12:53:50.372-05:00][INFO ][status] Kibana is now available

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

[elasticmachine]

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

[kfirpeled]

@opauloh after a discussion with @tehilashn we see a risk with fixing it for 8.13 So once you figure out what the fix is, let's assess the potential risk here

[animehart]

@kfirpeled so I spent sometime checking this issue and consulted @opauloh So in order to fix this, we can just make "fleet" plugin to be optional for cloudDefend and cloudSecurityPosture plugins which will fix the transient error. However, by doing this we also introduce other issues in instances where fleet is called in our code such as plugin.fleet where we call this to get some variable from it such as authz. Im not sure on how authz and all the stuff inside plugins.fleet are being used on security solution or if its even replaceable by something else. We also have lots of methods that relies on fleet like this

and by making fleet optional will require us to adapt all the condition checkers. As such this ticket might seem to be bigger than it looks and might be some side effects on resolving this issue

I also tried making cloudDefend and cloudSecurityPosture to be optional plugin for securitySolution but I got the same issue where fleet is called in our code such as plugin.fleet to get some variable from it such as authz.

[kfirpeled]

Thanks for the update @animehart This makes sense to me

Can you proceed and map all the usages of fleet in our plugin and elaborate what is truly needed and what we can workaround?

We should be able to allow our users to use us without fleet permissions (like what you checked as authz) In that specific example, we should check for security solution permissions instead of fleet permissions. So that seems like a technical debt that you can address

[animehart]

@kfirpeled Fleet are used in CSP in the following:

server/routes/detection_engine/get_detection_engine_alerts_count_by_rule_tags.ts
server/routes/benchmark.ts
server/routes/benchmark_rules/get_states/get_states.ts
server/routes/benchmark_rules/find/find.ts
server/routes/benchmark_rules/bulk_action/bulk_action.ts
server/routes/setup_routes.ts

server/lib/telemetry/collectors/indices_stats_collector.ts
server/lib/telemetry/collectors/installation_stats_collector.ts

server/plugin.ts
public/plugin.ts

in Cloud Defend it's used in the following:

server/routes/policies/policies.ts

server/plugin.ts
public/plugin.ts

another issue (which i think the biggest one) is that Security Solution plugin are intended to not start when fleet is not enabled (asked @paul-tavares ) and changing this behaviour where we want to make Security Solution to work without fleet might be a significant task

[caitlinbetz]

hi all -

I'm not sure that this is something we actually want to do. A number of security features rely on fleet, especially from the endpoint/cloud side, which is why we do not currently support using Security without Fleet today.

@kfirpeled / @andrew-goldstein can you share where this request came from? I'm happy to discuss with someone on the product side to understand the background here and clarify what is needed.

cc @dasansol92 @paul-tavares

[kfirpeled]

Hi @caitlinbetz thanks for taking a look at this one

I'm not aware of any requirement here to fill it out. @andrew-goldstein opened the ticket, so I'll address the question to him.

I guess it is worth checking our docs are up to date, showing that fleet must be enabled for the security solution.

[rseldner]

On a related note, xpack.fleet.enabled: false also has the effect of removing Stack Monitoring from the UI (in 8.13)

According to this 7.17 doc, xpack.fleet.enabled: was deprecated in 7.16 and supposedly removed in 8.0; so it is interesting it still works.

Using the current equivalent setting of xpack.fleet.agents.enabled: false does NOT have the effect being reported in this issue.

[kfirpeled]

Closing the issue will be fixed once https://github.com/elastic/kibana/issues/188685 is completed and xpack.fleet.enabled flag will be deprecated