1.24.0 (October 19, 2022)

Incompatible behavior changes

Changes that are expected to cause an incompatibility if applicable; deployment changes are likely required

  • build: official released binary is now built on Ubuntu 20.04, requires glibc >= 2.30.

  • config: Fixed resource tracking when using the Incremental (Delta-xDS) protocol. The protocol state will be updated after the resources are successfully ingested and an ACK is sent. This behavior change can be reverted by setting the envoy.reloadable_features.delta_xds_subscription_state_tracking_fix runtime flag to false.

  • gcp_authn: Add GCP Authentication filter which can be used to fetch authentication tokens from Google Compute Engine(GCE) metadata server.

  • http: Envoy no longer adds content-length: 0 header when proxying UPGRADE requests without content-length and transfer-encoding headers. This behavior change can be reverted by setting the envoy.reloadable_features.http_skip_adding_content_length_to_upgrade runtime flag to false.

  • http: For HTTP/2 and HTTP/3 codecs, all clients now continue sending data upstream after receiving an end of the server stream. This supports the server half-close semantics for TCP tunneling with CONNECT as well as bi-directional streaming calls. This behavior change can be reverted by setting the envoy.reloadable_features.http_response_half_close runtime flag to false.

  • original_dst: ORIGINAL_DST cluster will not attempt to remove and drain the stale hosts during cleanup if they are still used by the connection pools. For HTTP pools, please set idle_timeout to limit the duration of the upstream connections (the default value is 1h, and the recommended value is 5min). This behavior change can be reverted by setting runtime guard envoy.reloadable_features.original_dst_rely_on_idle_timeout.

  • stats http local_rate_limit: Fixed metric tag extraction so that stat_prefix is properly extracted. This changes the Prometheus name from envoy_http_local_rate_limit_myprefix_rate_limited{} to envoy_http_local_rate_limit_rate_limited{envoy_local_http_ratelimit_prefix=”myprefix”}.

  • stats network local_rate_limit: Fixed metric tag extraction so that stat_prefix is properly extracted. This changes the Prometheus name from envoy_local_rate_limit_myprefix_rate_limited{} to envoy_local_rate_limit_rate_limited{envoy_local_ratelimit_prefix=”myprefix”}.

  • tls: Change TLS and QUIC transport sockets to support asynchronous cert validation extension. This behavior change can be reverted by setting runtime guard envoy.reloadable_features.tls_async_cert_validation to false.

Minor behavior changes

Changes that may cause incompatibilities for some users, but should not for most

  • cache_filter: added a completion callback to insertHeaders and insertTrailers in cache interface. Any external cache implementation extensions will need to also add this callback, and call it on completion.

  • dynamic_forward_proxy: No longer waiting on DNS responses in the dynamic forward proxy filter if upstream proxying is turned on. This behaviorial change can be reverted by setting runtime guard envoy.reloadable_features.skip_dns_lookup_for_proxied_requests to false.

  • health_check: support custom health check address via health_check_config.

  • http: changed shadow requests to more closely behave like the requests they are shadowing. This includes matching the upstream logging for the original request, dynamic stats, suppressing Envoy headers, respecting expected request timeout, suppressing grpc request failure code stats and strict header checks. This behaviorial change can be temporarily reverted by setting runtime guard envoy.reloadable_features.closer_shadow_behavior to false.

  • http: changed the filter callback interfaces to make sure that downstream-only functionality is explicit.

  • http: the upstream remote address is now available to downstream filters via the upstreamRemoteAddress function.

  • local_ratelimit: added virtual host level configuration support for the local ratelimit filter.

  • logging: changed the UPSTREAM_REMOTE_ADDRESS, UPSTREAM_REMOTE_ADDRESS_WITHOUT_PORT, and UPSTREAM_REMOTE_PORT fields to log based on the actual upstream connection rather than the upstream host. This fixes a bug where the address components were not consistently correct for Happy Eyeballs connections and proxied connections, but also means in cases where the host was selected but a connection was not established, the fields will be absent. This change can be temporarily reverted by setting the runtime guard envoy.reloadable_features.correct_remote_address to false.

  • original_dst: transparent listener can use original_dst filter without nf_conntrack enabled.

  • prometheus_stats: removed blank line for being compatible with OpenMetrics.

  • quic: changed the timing of QUIC connection writing data in response to incoming packets in non-Windowns platforms. This change can be reverted by setting runtime guard envoy.reloadable_features.quic_defer_send_in_response_to_packet to false.

  • resource_monitors: changed behavior of the fixed heap monitor to count pages allocated to TCMalloc as free memory if it’s not used by Envoy. This change can be reverted temporarily by setting the runtime guard envoy.reloadable_features.do_not_count_mapped_pages_as_free to true.

  • stats: Default tag extraction rules were changed for worker_id extraction. Previously, worker_ was removed from the original name during the extraction. This led to the same base stat name for both the per-worker and overall stat. For instance, in prometheus stats, the following stats were produced:

    envoy_listener_downstream_cx_total{} 2.
envoy_listener_downstream_cx_total{envoy_worker_id="0"} 1.
envoy_listener_downstream_cx_total{envoy_worker_id="1"} 1.

    This resulted in sum(envoy_listener_downstream_cx_total) producing 4, even though there are only 2 connections. The new behavior results in stats such as this:

    envoy_listener_downstream_cx_total{} 2.
envoy_listener_worker_downstream_cx_total{envoy_worker_id="0"} 1.
envoy_listener_worker_downstream_cx_total{envoy_worker_id="1"} 1.

  • udp_proxy: changed behavior of UDP proxy to connect UDP sockets unless use_original_src_ip is set. This change can be reverted by setting runtime guard envoy.reloadable_features.udp_proxy_connect to false.

Bug fixes

Changes expected to improve the state of the world and are unlikely to have negative effects

  • grpc_transcoder: fixed a bug where a request with a wrong binding type is not rejected if the request body is empty.

  • http: fixed a bug with internal redirects not being performed for drained connections.

  • jwt_authn: fixed a bug where a negative “exp”, “iat”, or “nbf” integer in JWT token readed as a large positive value.

  • listener: fixed a bug that doesn’t handle of an update for a listener with IPv4-mapped address correctly, and that will lead to a memory leak.

  • logger: added the %j and %_ format support for fine-grain loggers to make it consistant with default loggers.

  • router: fixed edge-case interaction between weighted clusters, cluster headers and (request|response)_headers_to_(add|remove).

  • thrift: fixed the routing decision when thrift filters change the value of the cluster header.

  • tls: fixed a bug where, when runtime guard envoy.reloadable_features.tls_async_cert_validation is set to false, the wrong TLS alerts would sometimes be sent in response to certificate validation failures.

  • transport_socket: fixed a bug that prevented the tcp stats to be retrieved when running on kernels different than the kernel where Envoy was built.

  • upstream: fixed a bug where custom transport socket hashes might not be respected by wrapper passthrough sockets. This change can be temporarily reverted by setting runtime guard envoy.reloadable_features.fix_hash_key to false.

Removed config or runtime

Normally occurs at the end of the deprecation period

  • auto_config: removed envoy.reloadable_features.correctly_validate_alpn and legacy code paths.

  • grpc: remove envoy.reloadable_features.enable_grpc_async_client_cache and legacy code paths.

  • hcm: removed envoy.reloadable_features.handle_stream_reset_during_hcm_encoding and legacy code paths.

  • http: removed envoy.reloadable_features.http2_allow_capacity_increase_by_settings and legacy code paths.

  • http: removed envoy.reloadable_features.proxy_120_103 and legacy code paths.

  • http: removed envoy.reloadable_features.sanitize_http_header_referer and legacy code paths.

  • lightstep: removed the Lightstep tracer integration, making way for the native OpenTelemetry integration.

  • listener: removed envoy.reloadable_features.internal_address and legacy code paths.

  • router: removed envoy.reloadable_features.update_expected_rq_timeout_on_retry and legacy code paths.

  • tracing: removed envoy.reloadable_features.update_grpc_response_error_tag and legacy code paths.

New features

  • access_log: added downstream handshake timing to connection streamInfo. Can be accessed by custom access loggers.

  • access_log: added support for number values in substitution format string in json_format.

  • access_log: log duration, upstream_request_attempt_count, connection_termination_details and tls ja3 field in the grpc access log and also log the tls sni and ja3 field in the grpc access log when envoy is configured as a tls forward proxy.

  • access_log: updated command operator %GRPC_STATUS% to suppoprt the snake case.

  • admin: added new /heap_dump endpoint to dump heap profile of Envoy.

  • build: official released binary is now built on Ubuntu 20.04, requires glibc >= 2.30.

  • cluster: added support to override original destination port via setting upstream_port_override.

  • compression: added support for choose_first.

  • cors: added support for cors PNA. This behavioral change can be temporarily reverted by setting runtime guard envoy_reloadable_features_cors_private_network_access to false. More details refer to https://developer.chrome.com/blog/private-network-access-preflight.

  • dns_resolver: added DNS stats for c-ares DNS resolver. Detailed documentation is available here.

  • generic_proxy: added an new network filter generic_proxy filter.

  • grpc_json_transcoder: added support for newline-delimited streams in stream_newline_delimited.

  • grpc_json_transcoder: added support for parsing enum value case insensitively enabled by the config case_insensitive_enum_parsing.

  • grpc_stats: added support for replacing dots of gRPC service name with underscores in the gRPC stats by the config replace_dots_in_grpc_service_name.

  • gzip: added support for max_inflate_ratio.

  • header_formatters: all access log formatters can be used as custom request/response headers. Custom header’s syntax is parsed using access logger’s parser and header values are obtained using access log’s substitution formatters. This feature can be reversed by setting runtime guard envoy.reloadable_features.unified_header_formatter to false.

  • health check: added method support to configure http health check http method.

  • health check: added thrift health check as a custom health check.

  • http: Added HeaderBasedSessionState to manage StatefulSession State via request/response header.

  • http: added envoy.reloadable_features.allow_upstream_filters for experimental upstream filters.

  • http: added default-false envoy.reloadable_features.http1_use_balsa_parser for experimental BalsaParser.

  • http: added the expected receive payload check for HTTP health check. Added response_buffer_size to configure the maximum HTTP health check response buffer size.

  • http: made the admission control work as an upstream filter.

  • listener: added multiple listening addresses in single listener. listener additional addresses.

  • listener: allow network filters other than HTTP Connection Manager to be created for QUIC listeners.

  • listener: expose the implementation of internal listener in xDS API.

  • load balancer: added a new field to subset load balancer config: metadata_fallback_policy.

  • lua: added an alternative function signature to httpCall() with options as an argument. This allows to skip sampling the produced trace span by setting {["trace_sampled"] = false} as the options. And this allows to return multiple header values for same header name by setting {["return_duplicate_headers"] = true} as the options.

  • lua: added new headers method “setHttp1ReasonPhrase” for lua filter, please see lua header wrapper.

  • lua: added stats for lua filter, please see lua filter stats.

  • ratelimit: add support for adding response headers to rate-limited responses.

  • redis: added support for quit command to the redis proxy.

  • redis: added support for redis transactions.

  • subset load balancer: added multiple keys or multiple selectors support for single host per subset mode.

  • tcp_proxy: added support for propagating the response headers in TunnelingConfig to the downstream info filter state.

  • thrift: added stats for downstream connection close to detect SR drop.

  • upstream: added a filter state object to control the destination address in ORIGINAL_DST clusters.

  • upstream: added a new field extra_source_addresses to the BindConfig, it enables to specify multiple source addresses, and the source address selection is based on target host’s address’ version.

  • zipkin: added split_spans_for_request to make Envoy appear as an independent hop for zipkin tracing.

Deprecated