feat: sso with keycloak as well as serveral bug fixes

[patrykattc]

This pull request introduces several significant enhancements that improve the Langflow platform's security, monitoring, and deployment capabilities. The changes span multiple areas of the codebase, from authentication to Docker configurations and monitoring.

Key Improvements

1. SSO Integration with Keycloak

• Added comprehensive Keycloak SSO integration for secure authentication
• Created detailed SSO_INTEGRATION.md documentation explaining architecture, configuration and testing
• Implemented token validation, role mapping, and seamless authentication flow
• Added frontend support for handling OAuth2/OpenID Connect flow
• Enhanced session management with proper logout and page refresh handling

2. Prometheus Monitoring Integration

• Implemented a flexible metrics collection system using OpenTelemetry and Prometheus
• Added support for two metrics deployment modes: inline and separate server
• Created metrics for application performance and usage statistics
• Added gauges for FastAPI and Langflow versions
• Comprehensive PROMETHEUS.md documentation with configuration options and best practices
• Docker Compose integration with Prometheus and Grafana services for visualization

3. Docker and Deployment Improvements

• Enhanced docker/basic-test.sh with proper monitoring functionality, reliable cleanup, and better user interaction
• Created .dockerignore file for the frontend directory to optimize Docker build context
• Improved Docker documentation with detailed READMEs for both frontend and backend configurations
• Created comprehensive Makefile documentation explaining build targets and workflows

4. Logging and Debugging Enhancements

• Improved logging configuration with better formatting and organization
• Enhanced error handling and reporting in critical code paths
• Added more context to log messages for easier debugging

5. Bug Fixes

• Various bug fixes

Technical Details

• Authentication: Implemented Keycloak integration following OAuth 2.0 and OpenID Connect standards
• Monitoring: Used OpenTelemetry for metrics collection and Prometheus for storage/visualization
• Docker: Improved multi-stage builds, layer caching, and startup scripts
• Documentation: Added comprehensive documentation for key features and integrations

Testing

• All new features are covered by unit and integration tests
• Manual testing performed for SSO integration with Keycloak
• Docker builds and deployments tested in multiple environments
• Metrics collection verified with Prometheus queries

Documentation

• SSO_INTEGRATION.md - Complete guide to Keycloak integration
• PROMETHEUS.md - Detailed documentation of Prometheus monitoring
• MAKEFILE.md - Comprehensive explanation of Makefile targets
• docker/README.md - Backend Docker configuration documentation
• docker/frontend/README.md - Frontend Docker configuration guide

These improvements enhance Langflow's security posture, operational visibility, and deployment flexibility, making the platform more robust for production use cases.

[frostronic]

Will other implementations of SSO be support besides Keycloak?

[ogabrielluiz]

Will other implementations of SSO be support besides Keycloak?

What others do you require? Can't Keycloak provide support for the ones you need?

[frostronic]

Will other implementations of SSO be support besides Keycloak?

What others do you require? Can't Keycloak provide support for the ones you need?

If by "Keycloak" you mean "standardized SSO/OIDC/Oauth/SAML support" then yes it probably would. However, as you may know Keycloak is just one of many popular SSO platform, but there are many popular platforms in use such as Authentik, Authelia, Okta, etc.

[ogabrielluiz]

I see. Maybe it should be SAML directly instead, right? @patrykattc any reasons you picked Keycloak out of the gate?

[patrykattc]

I see. Maybe it should be SAML directly instead, right? @patrykattc any reasons you picked Keycloak out of the gate?

@ogabrielluiz @frostronic We are a full-stack development shop, covering everything from AI to Cloud to DevOps to backend to frontend. We provide complete solutions. Ex we build Varcel for clients. Part of our solution is SSO with keycloak. We deployed everything in Kubernetes and provided all the tools for the business application to be production-ready. I understand that many businesses utilize services like Varcel and other services, such as Clerk, for SSO. However, we would like to provide a comprehensive solution. The way we added this SSO feature means it can be extended to other providers with minimal effort. Additionally, this SSO PR is not just SSO; we also provided production-ready Dockerfiles, updated Prometheus observability, and addressed multiple bug fixes. We also have a brand-new, production-ready Helm chart that we are ready to share. We also have many additional features that are waiting for this merge. One of the features is the ability to provide user groups that allow for sharing global variables between users. This means instead of having an API key per user, you have an API key per group of users. We also added an agent-to-agent protocol. We are also iterating fast with additional features. We move fast. (We might also share Terraform/Terragrunt code to deploy the helm charts with databases to build your complete system)

[patrykattc]

To help with the why, here are a few images. Keycloak is also an identity broker.

We use Google login for dev.

And we use many others in production, including different social media platforms and SAML.

[patrykattc]

Since I see the tests running, here is a sneak peek at the group features that we use to manage global variables. We also updated the frontend project to use React Query 5, and you can see the React Query dev tool there at the bottom, which we also added to the project.

[Baptiste-ms]

Hi, I didn't see any infor concerning MCP authentification.
Will there be some integration with MCP server ie. auth and tool calling using the caller's token ?
Thank you for all your work :D

[patrykattc]

@Baptiste-ms, this PR is about integrating SSO with Langflow authentication. The SSO auth creates the same JWT token as the basic login form in Langflow. We have another PR waiting for this one to merge, which addresses additional authentication with the MCP server. Ex. We have an MCP for executing Kubernetes commands, and we only want DevOps users to have specific access to the kubectl commands. Additionally, we have two distinct DevOps roles: administrators and regular users. We control which commands DevOps can execute through the MCP using authentication, and we have different authentication methods based on the role as well. So once this is merged, we will push a lot more code for auth that is related to MCP. The basic mechanism is that we enhanced the LangFlow JWT token.

[Baptiste-ms]

Ok got it! Thank you for the clarification.
Other changes will cascade after this one, good news 🤗

[flefevre]

Hey there,
Do you know how effort is due to integrate this feature?
Do you plan to integrate it for a specific release? We are in 1.4.2

Thanks for your involvement

[ncecere]

I was just wondering if there have been any updates

[ogabrielluiz]

Hey @ncecere

This will be merged soon. We are focused on fixing some bugs but we will look at this very soon.

By the way, do you all have a guide on how to integrate this implementation with Okta?

cc: @patrykattc @flefevre

[ncecere]

Hey @ncecere

This will be merged soon. We are focused on fixing some bugs but we will look at this very soon.

By the way, do you all have a guide on how to integrate this implementation with Okta?

cc: @patrykattc @flefevre

Hey @ncecere

This will be merged soon. We are focused on fixing some bugs but we will look at this very soon.

By the way, do you all have a guide on how to integrate this implementation with Okta?

cc: @patrykattc @flefevre

Just wanted to see if there have been any updates or movement on this

[flefevre]

If I do understand, The primary goal of this PR is to release the new authentication feature using Keycloak with OAuth2.
That said, this implementation could also be a good foundation for improving OAuth-based authentication in MCP (Model Context Protocol) servers, following the official spec here: https://modelcontextprotocol.io/specification/draft/basic/authorization
Definitely something worth considering for future enhancements with the capacity of langflow to generate already Mcp server from flows on demand !

Feature: Support OAuth2 (Keycloak) Authentication for Generated MCP Servers #8883