Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support TokenCredential-based Authentication for Evaluators🔐✨ #39620

Open
thegovind opened this issue Feb 7, 2025 · 2 comments
Open

Support TokenCredential-based Authentication for Evaluators🔐✨ #39620

thegovind opened this issue Feb 7, 2025 · 2 comments
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Evaluation Issues related to the client library for Azure AI Evaluation feature-request This issue requires a new behavior in the product in order be resolved. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team Service Attention Workflow: This issue is responsible by Azure service team.

Comments

@thegovind
Copy link

Is your feature request related to a problem? Please describe.
Currently, model configuration - see, configs.py supports only API key authentication. This limitation extends to evaluators such as the groundedness evaluator in azure-ai-evaluation SDK, forcing users to rely solely on API keys. This is suboptimal, as one of the design goals for Azure AI Foundry SDKs (and SFI) is to support secure, token-based authentication methods (Entra) over API keys. 😕

Describe the solution you'd like
It would be ideal if the SDK could support authentication via TokenCredential (DefaultAzureCredential/Entra-ID). This enhancement would update the model configuration to accept credential objects in addition to, or instead of, API keys. Doing so would align the azure-ai-evaluation SDK with broader Azure SDK authentication and SFI practices, offering a more secure and consistent experience for users. 🚀

Describe alternatives you've considered

  • Sticking with API Keys: Continuing to use API keys would force users into less secure practices.
  • Conversion Layer Workaround: Implementing a layer that wraps a TokenCredential to generate an API key dynamically is possible, but it adds unnecessary complexity and still does not offer native token-based authentication. 🤷‍♂️

Additional context
Adopting token-based authentication would:

  • Enhance security by reducing reliance on API keys. 🔒
  • Align with Azure’s best practices and the authentication patterns used across other Azure SDKs.
  • Simplify integration for users already standardized on TokenCredential/DefaultAzureCredential for other Azure services. 😊

Feel free to reach out if further details or discussion is needed!
@github-actions github-actions bot added customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Feb 7, 2025
@kristapratico kristapratico added feature-request This issue requires a new behavior in the product in order be resolved. Service Attention Workflow: This issue is responsible by Azure service team. Client This issue points to a problem in the data-plane of the library. Evaluation Issues related to the client library for Azure AI Evaluation and removed question The issue doesn't require a change to the product in order to be resolved. Most issues start as that needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. labels Feb 7, 2025
@github-actions github-actions bot added the needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team label Feb 7, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @luigiw @needuv @singankit.

@singankit
Copy link
Contributor

Thanks @thegovind for reaching out. If ap-key is not passed as part of model config, DefaultAzureCredentials is used. Please give a try and share you feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Evaluation Issues related to the client library for Azure AI Evaluation feature-request This issue requires a new behavior in the product in order be resolved. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@thegovind @singankit @kristapratico