Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation: External Processing support #5867

Open
wants to merge 50 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
39d0717
init commit for extProc
izturn Sep 28, 2023
98f43de
build pass
izturn Oct 7, 2023
b54c663
Merge remote-tracking branch 'upstream/main' into feat/ext-proc
izturn Oct 8, 2023
ca308a4
insert extProc by phase(TODO)
izturn Oct 9, 2023
f7a1906
make generate
izturn Oct 9, 2023
84bafd4
set default for extProc's params
izturn Oct 11, 2023
922668f
clean up
izturn Oct 12, 2023
9c54782
more todo
izturn Oct 13, 2023
07363a8
more log
izturn Oct 16, 2023
99a3b52
comments & more
izturn Oct 17, 2023
2c9de3f
add filters by phase & refactor
izturn Oct 18, 2023
d1a4011
bugfix & duplicate processing check
izturn Oct 18, 2023
7d2793f
sync & fix conflict
izturn Oct 19, 2023
5dcfcf8
Merge remote-tracking branch 'upstream/main' into feat/extproc
izturn Oct 19, 2023
024a515
fix ut
izturn Oct 19, 2023
04d9840
make lint happy
izturn Oct 19, 2023
707598f
sync & fix conflict
izturn Jan 10, 2024
11396dc
change field's definition & comments
izturn Jan 10, 2024
3c38898
make generate
izturn Jan 11, 2024
de8ad6c
Merge remote-tracking branch 'upstream/main' into feat/extproc
izturn Jan 11, 2024
6928b7e
Merge remote-tracking branch 'upstream/main' into feat/extproc
izturn Jan 18, 2024
56a8fff
merge & fix conflict
izturn Feb 21, 2024
3104a77
make lint happy
izturn Feb 21, 2024
f10908f
revert generate
izturn Feb 21, 2024
fc182ef
remove global ext_proc & add name for ext_proc
izturn Mar 12, 2024
8bac1a0
sync with upstream & fix conflict
izturn Mar 12, 2024
59f5d09
sync with upstream & fix conflict
izturn Mar 13, 2024
d410246
global & vh & route only have one ext_proc
izturn Mar 14, 2024
70aca50
remove deadcode
izturn Mar 14, 2024
daa54f6
fix crash
izturn Mar 15, 2024
9ac6f52
fix crash
izturn Mar 15, 2024
e1e8c19
enable disabled for globalExtProc
izturn Mar 15, 2024
b401806
read extProc config from configMap
izturn Mar 18, 2024
ffb6494
make lint happy
izturn Mar 18, 2024
29936de
Merge remote-tracking branch 'upstream/main' into feat/extproc
izturn Mar 19, 2024
7bb8868
refactor
izturn Mar 19, 2024
80400ad
Merge remote-tracking branch 'upstream/main' into feat/extproc
izturn Mar 21, 2024
23ae148
fix comments
izturn Mar 21, 2024
43acc84
Merge remote-tracking branch 'upstream/main' into feat/extproc
izturn Mar 27, 2024
b1955c6
refactor
izturn Apr 2, 2024
56b3d95
sync with upstream & fix conflict & revert
izturn Apr 3, 2024
ea39c99
make generate
izturn Apr 3, 2024
19e6c3e
refactor
izturn Apr 9, 2024
f48350f
Merge remote-tracking branch 'upstream/main' into feat/extproc
izturn Apr 9, 2024
d455701
Merge remote-tracking branch 'upstream/main' into feat/extproc
izturn Apr 10, 2024
8a321c1
Merge remote-tracking branch 'upstream/main' into feat/extproc
izturn Apr 25, 2024
98e3da5
merge & fix conflict
izturn Jul 30, 2024
263275f
Merge remote-tracking branch 'upstream/main' into feat/extproc
izturn Aug 1, 2024
7da9c30
Merge remote-tracking branch 'upstream/main' into feat/extproc
izturn Nov 4, 2024
f6b0772
fix typo & little refactor
izturn Nov 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions apis/projectcontour/v1/detailedconditions.go
Original file line number Diff line number Diff line change
Expand Up @@ -197,4 +197,7 @@ const (
// ConditionTypeListenerError describes an error condition relating
// to the configuration of Listeners.
ConditionTypeListenerError = "ListenerError"

// ConditionTypeExtProcError describes an error condition related to external processing.
ConditionTypeExtProcError = "ExtProcError"
)
16 changes: 14 additions & 2 deletions apis/projectcontour/v1/helpers.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,15 @@ func (v *VirtualHost) AuthorizationConfigured() bool {
// authorization. If an authorization server is present, the default
// policy is to not disable.
func (v *VirtualHost) DisableAuthorization() bool {
// No authorization, so it is disabled.
if v.AuthorizationConfigured() {
// No policy specified, default is to not disable.
if v.Authorization.AuthPolicy == nil {
return false
}

return v.Authorization.AuthPolicy.Disabled
}

// No authorization, so it is not disabled.
return false
}

Expand All @@ -51,6 +50,19 @@ func (v *VirtualHost) AuthorizationContext() map[string]string {
return nil
}

// ExtProcConfigured returns whether external processing are
// configured on this virtual host.
func (v *VirtualHost) ExtProcConfigured() bool {
return v.ExternalProcessing != nil && v.ExternalProcessing.Processor != nil
}

// ExtProcDisabled returns true if this virtual host disables
// external processing explicit. If an external processor is present, the default
// policy is to not disable.
func (v *VirtualHost) ExtProcDisabled() bool {
return v.ExternalProcessing != nil && v.ExternalProcessing.Disabled
}

// GetPrefixReplacements returns replacement prefixes from the path
// rewrite policy (if any).
func (r *Route) GetPrefixReplacements() []ReplacePrefix {
Expand Down
211 changes: 210 additions & 1 deletion apis/projectcontour/v1/httpproxy.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ type HTTPProxySpec struct {
// +optional
VirtualHost *VirtualHost `json:"virtualhost,omitempty"`
// Routes are the ingress routes. If TCPProxy is present, Routes is ignored.
// +optional
// +optional
Routes []Route `json:"routes,omitempty"`
// TCPProxy holds TCP proxy information.
// +optional
Expand Down Expand Up @@ -311,6 +311,204 @@ type AuthorizationPolicy struct {
Context map[string]string `json:"context,omitempty"`
}

// HeaderSendMode control how headers and trailers are handled.
type HeaderSendMode string

const (
// The default HeaderSendMode depends on which part of the message is being
// processed. By default, request and response headers are sent,
// while trailers are skipped.
ProcessingModeDefault HeaderSendMode = "DEFAULT"

// Send the header or trailer.
ProcessingModeSend HeaderSendMode = "SEND"

// Do not send the header or trailer.
ProcessingModeSkip HeaderSendMode = "SKIP"
)

// BodySendMode control how the request and response bodies are handled
type BodySendMode string

const (
// Do not send the body at all. This is the default.
ProcessingModeNone BodySendMode = "NONE"

// Stream the body to the server in pieces as they arrive at the
// proxy.
ProcessingModeStreamed BodySendMode = "STREAMED"

// Buffer the message body in memory and send the entire body at once.
// If the body exceeds the configured buffer limit, then the
// downstream system will receive an error.
ProcessingModeBuffered BodySendMode = "BUFFERED"

// Buffer the message body in memory and send the entire body in one
// chunk. If the body exceeds the configured buffer limit, then the body contents
// up to the buffer limit will be sent.
ProcessingModeBufferedPartial BodySendMode = "BUFFERED_PARTIAL"
)

// HeaderMutationRules specifies what headers may be manipulated by a processing filter.
// This set of rules makes it possible to control which modifications a filter may make.
type HeaderMutationRules struct {
// By default, certain headers that could affect processing of subsequent
// filters or request routing cannot be modified. These headers are
// ``host``, ``:authority``, ``:scheme``, and ``:method``.
// Setting this parameter to true allows these headers to be modified as well.
//
// +optional
AllowAllRouting bool `json:"allowAllRouting,omitempty"`

// If true, allow modification of envoy internal headers. By default, these
// start with ``x-envoy`` but this may be overridden in the ``Bootstrap`` configuration.
// Default is false.
//
// +optional
AllowEnvoy bool `json:"allowEnvoy,omitempty"`

// If true, prevent modification of any system header, defined as a header
// that starts with a ``:`` character, regardless of any other settings.
// A processing server may still override the ``:status`` of an HTTP response
// using an ``ImmediateResponse`` message.
// Default is false.
//
// +optional
DisallowSystem bool `json:"disallowSystem,omitempty"`

// If true, prevent modifications of all header values, regardless of any
// other settings. A processing server may still override the ``:status``
// of an HTTP response using an ``ImmediateResponse`` message.
// Default is false.
//
// +optional
DisallowAll bool `json:"disallowAll,omitempty"`

// If true, and if the rules in this list cause a header mutation to be
// disallowed, then the filter using this configuration will terminate the
// request with a 500 error. In addition, regardless of the setting of this
// parameter, any attempt to set, add, or modify a disallowed header will
// cause the ``rejected_header_mutations`` counter to be incremented.
// Default is false.
//
// +optional
DisallowIsError bool `json:"disallowIsError,omitempty"`
}

// ProcessingMode describes which parts of an HTTP request and response are sent to a remote server
// and how they are delivered.
type ProcessingMode struct {
// How to handle the request header.
// Default is "SEND".
//
// +kubebuilder:validation:Enum=DEFAULT;SEND;SKIP
// +kubebuilder:default=SEND
// +optional
RequestHeaderMode HeaderSendMode `json:"requestHeaderMode,omitempty"`

// How to handle the response header.
// Default is "SEND".
//
// +kubebuilder:validation:Enum=DEFAULT;SEND;SKIP
// +kubebuilder:default=SEND
// +optional
ResponseHeaderMode HeaderSendMode `json:"responseHeaderMode,omitempty"`

// How to handle the request body.
// Default is "NONE".
//
// +kubebuilder:validation:Enum=NONE;STREAMED;BUFFERED;BUFFERED_PARTIAL
// +kubebuilder:default=NONE
// +optional
RequestBodyMode BodySendMode `json:"requestBodyMode,omitempty"`

// How do handle the response body.
// Default is "NONE".
//
// +kubebuilder:validation:Enum=NONE;STREAMED;BUFFERED;BUFFERED_PARTIAL
// +kubebuilder:default=NONE
// +optional
ResponseBodyMode BodySendMode `json:"responseBodyMode,omitempty"`

// How to handle the request trailers.
// Default is "SKIP".
//
// +kubebuilder:validation:Enum=DEFAULT;SEND;SKIP
// +kubebuilder:default=SKIP
// +optional
RequestTrailerMode HeaderSendMode `json:"requestTrailerMode,omitempty"`

// How to handle the response trailers.
// Default is "SKIP".
//
// +kubebuilder:validation:Enum=DEFAULT;SEND;SKIP
// +kubebuilder:default=SKIP
// +optional
ResponseTrailerMode HeaderSendMode `json:"responseTrailerMode,omitempty"`
}

// ExternalProcessor defines the envoy External Processing filter which allows an external service to act on HTTP traffic in a flexible way
// The external server must implement the v3 Envoy external processing GRPC protocol
// (https://www.envoyproxy.io/docs/envoy/v1.27.0/api-v3/extensions/filters/http/ext_proc/v3/ext_proc.proto).
type ExternalProcessor struct {
// ExtensionServiceRef specifies the extension resource that will handle the client requests.
//
// +optional
ExtensionServiceRef ExtensionServiceReference `json:"extensionRef,omitempty"`

// ResponseTimeout sets how long the proxy should wait for responses.
// Timeout durations are expressed in the Go [Duration format](https://godoc.org/time#ParseDuration).
// Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h".
// The string "infinity" is also a valid input and specifies no timeout.
//
// +optional
// +kubebuilder:validation:Pattern=`^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$`
ResponseTimeout string `json:"responseTimeout,omitempty"`

// If FailOpen is true, the client request is forwarded to the upstream service
// even if the server fails to respond. This field should not be
// set in most cases.
//
// +optional
FailOpen bool `json:"failOpen,omitempty"`

// ProcessingMode describes which parts of an HTTP request and response are sent to a remote server
// and how they are delivered.
//
// +optional
ProcessingMode *ProcessingMode `json:"processingMode,omitempty"`

// MutationRules specifies what headers may be manipulated by a processing filter.
// This set of rules makes it possible to control which modifications a filter may make.
//
// for Overrides is must be nil
//
// +optional
MutationRules *HeaderMutationRules `json:"mutationRules,omitempty"`

// If true, the filter config processingMode can be overridden by the response message from the external processing server `mode_override``.
// If false, `mode_override` API in the response message will be ignored.
//
// +optional
AllowModeOverride bool `json:"allowModeOverride,omitempty"`
}

// ExternalProcessing defines a external processing filter and the policy to act on HTTP traffic in a flexible way.
type ExternalProcessing struct {
// Processor defines a external processing filter which allows an external service to act on HTTP traffic in a flexible way.
//
// +optional
Processor *ExternalProcessor `json:"processor,omitempty"`

// When true, this field disables the external processor for the scope of the policy.
// - for global: no external processing will be append to the filter chain
//
// if both Disabled and Processor are set. use disabled.
//
// +optional
Disabled bool `json:"disabled,omitempty"`
}

// VirtualHost appears at most once. If it is present, the object is considered
// to be a "root".
type VirtualHost struct {
Expand Down Expand Up @@ -358,6 +556,12 @@ type VirtualHost struct {
// Only one of IPAllowFilterPolicy and IPDenyFilterPolicy can be defined.
// The rules defined here may be overridden in a Route.
IPDenyFilterPolicy []IPFilterPolicy `json:"ipDenyPolicy,omitempty"`

izturn marked this conversation as resolved.
Show resolved Hide resolved
// ExternalProcessing defines a external processing filter and the policy
// to act on HTTP traffic in a flexible way.
//
// +optional
ExternalProcessing *ExternalProcessing `json:"externalProcessing,omitempty"`
}

// JWTProvider defines how to verify JWTs on requests.
Expand Down Expand Up @@ -625,6 +829,11 @@ type Route struct {
// Only one of IPAllowFilterPolicy and IPDenyFilterPolicy can be defined.
// The rules defined here override any rules set on the root HTTPProxy.
IPDenyFilterPolicy []IPFilterPolicy `json:"ipDenyPolicy,omitempty"`

// ExternalProcessing override/disable the policy to act on HTTP traffic for the specific route in a flexible way.
//
// +optional
ExternalProcessing *ExternalProcessing `json:"externalProcessing,omitempty"`
}

type JWTVerificationPolicy struct {
Expand Down
Loading
Loading