Ask AI Assistant in generic entity flyout

[maxcold]

Summary

This PR adds basic "Ask AI Assistant" functionality to the Generic Entity flyout in Asset Inventory. The Asset Inventory is still in Tech Preview and behind a feature flag, so this functionality when merged will have the same availability. The implementation follows the approach of the same functionality available for Alerts

The feature will be under Technical Preview as Asset Inventory itself is under Technical preview and behind an Advanced Setting

Contributes to:

• https://github.com/elastic/security-team/issues/13836

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
• Flaky Test Runner was used on any tests changed
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
• Review the backport guidelines and apply applicable backport:* labels.

How to test

0. set up AI Assistant LLM Connectors, eg via kibana.dev.yml flag xpack.actions.preconfigured. You will need LLM credentials for that. Reach out to @maxcold if you don't have it. make sure to run elasticsearch cluster without the flag xpack.ml.enabled=false
1. Install Cloud Asset Discovery integration https://localhost:5601/app/integrations/detail/cloud_asset_inventory
2. Reindex Asset Inventory, you will need credentials from an env with asset inventory data, reach out to @maxcold if you don't have it:

POST _reindex?wait_for_completion=true
{
  "conflicts": "proceed", 
  "source": {
    "remote": {
      "host": "${ES_REMOTE_HOST}",
      "username": "${ES_REMOTE_USER}",
      "password": "${ES_REMOTE_PASS}"
    },
    "index": "logs-cloud_asset_inventory*",
    "query": {
      "bool": {
        "must": [
          {
            "range": {
              "@timestamp": {
                "gte": "now-1d"
              }
            }
          }
        ]
      }
    }
  },
  "dest": {
    "op_type": "create",
    "index": "logs-cloud_asset_inventory.asset_inventory-default"
  }
}

3. Enable Asset Inventory on Kibana Settings
4. Enable Asset Inventory on Security -> Inventory page.

Screenshots

Preview mode - hide 'Ask AI Assistant' button - aligned with alert flyout in preview mode:

[elasticmachine]

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

[albertoblaz]

Looks good overall though I lack of enough context to approve this. Only had a minor comment :)

[albertoblaz]

nit: Why not reusing ENTITY_ANALYSIS here? It might be easy to introduce a change in the prompt that does not match the translation

[maxcold]

I believe we can't import server-side code into client-side code. Ofc it's possible, I guess, to move this const to a common package but I implemented the feature the same way it is implemented for Alerts. I think it will be better for future improvements to have the same logic and then get rid of this duplication for all AI Assistant features

[maxcold]

The final prompt will most likely change based on PM's feedback, so I'm holding back on adding it to the security_ai_prompts integration to avoid multiple releases of the integration

[seanrathier]

Look great! Thanks for the templates for future AI assistant work.

On Week Idea
One thing that struck me, is that we have no way to update prompts between releases for Cloud and OnPrem, I wonder if there is a way we can use Integration packages to release the prompt update changes. Or an S3 bucket 😆

[seanrathier]

👏 very nice.

I am just curious, if you know, do we store the context of previous User and AI messages?

[maxcold]

the conversations are stored in ES and available in Kibana in the chat history if that's what you ask

[seanrathier]

Did you try prompting the LLM to format with EUI tables and code blocks?

[stephmilovic]

Markdown is preferred so we don't have to do any html injection.

[niros1]

@maxcold will it better to add a list of specific indices so the ES-QL will be more relevant?

[maxcold]

@niros1 not sure tbh. First I hope we will add the use of ESQL tool quite soon which can validate queries and self correct. Second - index patterns tend to change, especially for asset inventory / entity store which is under active developments. That means constantly update the prompt to keep up witht the changes. Plus users can ask questions about related things, like alerts, misconfigs, vulnerabiltiies, or source events with more entity info. Keeping all these index patterns in the prompt doesn't look like a scalable solution.
While Asset Inventory is under tech preview I'd start with something simple to get going and move to implementing relevant tools and/or leveraging onechat infra/tooling

[maxcold]

@seanrathier thanks for the review

One thing that struck me, is that we have no way to update prompts between releases for Cloud and OnPrem, I wonder if there is a way we can use Integration packages to release the prompt update changes. Or an S3 bucke

This actually is already solved, check this comment out #234324 (comment)

[opauloh]

LGTM - Tested locally, it works nicely!

[maxcold]

Related PR in the integration repo elastic/integrations#15393

[tiansivive]

LGTM from EA

[kibanamachine]

🤖 Prompt Changes Detected

Changes have been detected to one or more prompt files in the Elastic Assistant plugin.

Please remember to update the integrations repository with your prompt changes to ensure consistency across all deployments.

Next Steps:

1. Follow the documentation in x-pack/solutions/security/packages/security-ai-prompts/README.md to update the corresponding prompt files
2. Make the changes in the integrations repository
3. Test your changes in the integrations environment
4. Ensure prompt consistency across all deployments

This is an automated reminder to help maintain prompt consistency across repositories.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: a2d8a06
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-234324-a2d8a0634070

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
securitySolution	8156	8157	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	10.8MB	10.8MB	+4.5KB

History

• 💔 Build #344014 failed e05692d
• 💔 Build #343933 failed 076bcd1
• 💚 Build #341797 succeeded b0855ad
• 💔 Build #341773 failed 236ae04
• 💚 Build #340784 succeeded 8597c3f