Apache Traffic Control 4.1.0

Release Notes

Note: these are abbreviated release notes. For more information, see the full changelog.

New Features

• Added support for Let's Encrypt
• Support for ATS Slice plugin in Traffic Ops, including new Delivery Service Raw Remap __RANGE_DIRECTIVE__ directive
• Ability to enable EDNS0 client subnet at the delivery service level
• New IPv6 changes:
  • Traffic Portal and Traffic Ops now accept IPv6-only servers
  • Traffic Monitor now polls caches over IPv6 in addition to IPv4, separating the availability status of each (make sure to update the allow_ip6 profile parameter to include the IPv6 addresses of your Traffic Monitors, otherwise they will fail to poll over IPv6 and consider those caches to be unavailable over IPv6)
  • Traffic Router will route IPv4 clients to caches with IPv4 availability and route IPv6 clients to caches with IPv6 availability
• Traffic Router DNSSEC zone diffing performance enhancement
• Traffic Monitor optimistic quorum
• Traffic Ops API 2.0. This new major API version contains several new routes but does not contain many deprecated routes from API 1.x (which will be available until the ATC 5.0 release). API clients should begin migrating to API 2.0 as soon as possible. For the full lists of new or deprecated routes, please see the changelog.
• Ability to choose the TLS version used for Traffic Ops to make requests to Traffic Vault. Note: the default is now TLSv1.1, which may require configuration changes to Riak. See Enabling TLS 1.1

Bug Fixes

This release contains many new bug fixes. For the full list, please see the changelog.

Removals

• The Traffic Ops db/admin.pl script has now been removed. Please use the db/admin binary instead.
• Removed from Traffic Portal the ability to view cache server config files as the contents are no longer reliable through the TO API due to the introduction of atstccfg.
• Traffic Ops Python client no longer supports Python 2.

Apache Traffic Control 4.0.0

Release Notes

Traffic Ops

• Server Capabilities: server capabilities can now be created and assigned to servers. Delivery services can now require certain server capabilities, and servers that lack the required capabilities will not serve those delivery services. As as an example, by default, MID caches will serve all delivery services in a given CDN, but this feature can be used to allow a only a subset of MID caches to serve certain delivery services (based on the server capabilities assigned to the MID caches and required by the delivery services). See the blueprint
• Certificate deletion upon delivery service deletion: Snapshotting the CRConfig now deletes HTTPS certificates in Riak for delivery services which have been deleted in Traffic Ops.
• SSO login using OAuth: Traffic Ops now provides the ability to login using an OAuth provider, and this functionality is now integrated in Traffic Portal. A field is added to cdn.conf to configure whitelisted URLs for Json Key Set URL returned from OAuth provider. Added fields to traffic_portal_properties.json to configure SSO through OAuth for Traffic Portal.
• API rewrite from Perl to Go: A large number of API endpoints were rewritten from Perl to Go
• API Routing Blacklist: via the routing_blacklist field in cdn.conf, enable certain whitelisted Go routes to be handled by Perl instead (via the perl_routes list) in case a regression is found in the Go handler, and explicitly disable any routes via the disabled_routes list. Requests to disabled routes are immediately given a 503 response. Both fields are lists of Route IDs, and route information (ID, version, method, path, and whether or not it can bypass to Perl) can be found by running ./traffic_ops_golang --api-routes. To disable a route or have it bypassed to Perl, find its Route ID using the previous command and put it in the disabled_routes or perl_routes list, respectively.
• Regional Geo-blocking for steering delivery services: Regional Geo-blocking is now supported for steering-based delivery services
• Added pagination support to some Traffic Ops endpoints via three new query parameters, limit and offset/page
• Traffic Ops now supports a "sortOrder" query parameter on some endpoints to return API responses in descending order
• Traffic Ops now uses a consistent format for audit logs across all Go endpoints
• Added an optional SMTP server configuration to the TO configuration file, api now has ability to send emails
• To support reusing a single riak cluster connection, an optional parameter is added to riak.conf: "HealthCheckInterval". This options takes a 'Duration' value (ie: 10s, 5m) which affects how often the riak cluster is health checked. Default is currently set to: "HealthCheckInterval": "5s".
• Fixed a regression where the Expires cookie header was not being set properly in responses. Also, added the Max-Age cookie header in responses.
• Fixed issue #3497: TO API clients that don't specify the latest minor version will overwrite/default any fields introduced in later versions
• Fixed issue #3587: Fixed Traffic Ops Golang reverse proxy and Riak logs to be consistent with the format of other error logs.
• Database migrations have been collapsed. Rollbacks to 3.1 and earlier migrations are no longer possible. As always, backup your database before upgrading.

Deprecations

• The TO API /cachegroup_fallbacks endpoint is now deprecated. That functionality was added to the /cachegroups API.
• The db/admin.pl script is now deprecated. There is a new Go db/admin binary to replace the Perl db/admin.pl script, which will be removed in a future release. The new db/admin binary is essentially a drop-in replacement for db/admin.pl since it supports all of the same commands and options; therefore, it should be used in place of db/admin.pl for all the same tasks.

Breaking changes

• The deprecated Traffic Ops UI has been removed in favor of the Traffic Portal UI
• The /api/1.1/osversions endpoint (used for ISO generation) now expects the Perl osversions.cfg configuration file to be JSON. Added a traffic_ops/app/bin/osversions-convert.pl script to convert the osversions.cfg file from Perl to JSON as part of the /osversions endpoint rewrite.
• traffic_ops/app/bin/checks/ToDnssecRefresh.pl now requires "user" and "pass" parameters of an operations-level user! Update your scripts accordingly! This was necessary to move to an API endpoint with proper authentication, which may be safely exposed.

Traffic Router

• Consistent Hash Query Parameters: Traffic Ops now allows HTTP delivery services to have a set of query parameter keys to be retained for consistent hash generation by Traffic Router. This should be used for query parameters that produce unique content from the origin. For example, if the paths /foo?a=1 and /foo?a=2 each return unique content, you should add a to the list of consistent hash query parameters for that delivery service. This allows clients to be routed to edges for that content more efficiently.
• Client Steering Forced Diversity: force Traffic Router to return more unique edge caches in CLIENT_STEERING results instead of the default behavior which can sometimes return a result of multiple targets using the same edge cache. In the case of edge cache failures, this feature will give clients a chance to retry a different edge cache. This can be enabled with the new client.steering.forced.diversity Traffic Router profile parameter.
• Tunable bounded queue to support DNS request processing.
• Default Certificate: TR now generates a self-signed certificate at startup and uses it as the default TLS cert. The default certificate is used whenever a client attempts an SSL handshake for an SNI host which does not match any of the other certificates.
• TLS certificate validation on certificates imported from Traffic Ops:
  • validates modulus of private and public keys
  • validates current timestamp falls within the certificate date bracket
  • validates certificate subjects against the DS URL
• Fixed a bug which would cause REFUSED DNS answers if the zone priming execution did not complete within the configured zonemanager.init.timeout period.
• Fixed issue #2821: Traffic Router may choose wrong certificate when SNI names overlap
• Modified Traffic Router logging format to include an additional field for DNS log entries, namely rhi. This defaults to '-' and is only used when EDNS0 client subnet extensions are enabled and a client subnet is present in the request. When enabled and a subnet is present, the subnet appears in the chi field and the resolver address is in the rhi field.
• Fixed issue #3476: Traffic Router returns partial result for CLIENT_STEERING Delivery Services when Regional Geoblocking or Anonymous Blocking is enabled.
• Modified Traffic Router API to be available via HTTPS.

Traffic Portal

• Added a context menu in place of the "Actions" column from the following tables in Traffic Portal: cache group tables, CDN tables, delivery service tables, parameter tables, profile tables, server tables.
• Removed the need to specify line breaks using __RETURN__ in delivery service edge/mid header rewrite rules, regex remap expressions, raw remap text and traffic router additional request/response headers.
• Provided the ability to clone delivery service assignments from one cache to another cache of the same type. Issue #2963.
• Delivery service table columns can now be rearranged and their visibility toggled on/off as desired by the user. Hidden table columns are excluded from the table search. These settings are persisted in the browser.
• Server table columns can now be rearranged and their visibility toggled on/off as desired by the user. Hidden table columns are excluded from the table search. These settings are persisted in the browser.
• All tables now include a 'CSV' link to enable the export of table data in CSV format.
• Fixed issue #3275: Improved the snapshot diff performance and experience.
• Disabled TLSv1
• The "Clone Delivery Service Assignments" menu item is now hidden on a cache when the cache has zero delivery service assignments to clone.
• Users with a specified role now have the ability to mark any delivery service request as complete.
• Improved profile comparison view in Traffic Portal.

ORT

• Cache-side ATS config generation: Added cache-side config generator, atstccfg, installed with ORT. Includes all configs. Includes a plugin system.
• Fixed ATS config generation to omit regex remap, header rewrite, URL Sig, and URI Signing files for delivery services not assigned to that server.
• Changed traffic_ops_ort.pl so that hdr_rw-.config files are compared with strict ordering and line duplication when detecting configuration changes.
• Fix to traffic_ops_ort.pl to strip specific comment lines before checking if a file has changed. Also promoted a changed file message from DEBUG to ERROR for report mode.
• ANYMAP override: in traffic_ops_ort.pl added the ability to handle ##OVERRIDE## delivery service ANY_MAP raw remap text to replace and comment out a base delivery service remap rules. Note: this is a temporary feature and may be replaced in the future.

Traffic Monitor

• Traffic Monitor now has "gbps" calculated stat, allowing operators to monitor bandwidth in Gbps.
• Added monitoring.json snapshotting. This stores the monitoring json in the same TO database table as the crconfig snapshot. Snapshotting is now required in order to push out monitoring changes.
• UI updated to support HTTP or HTTPS traffic.
• health/stat time now includes full body ...

Apache Traffic Control 3.0.0

• Removed MySQL-to-Postgres migration tools. This tool is supported for 1.x to 2.x upgrades only and should not be used with 3.x.
• Backup Edge Cache group: If the matched group in the CZF is not available, this list of backup edge cache group configured via Traffic Ops API can be used as backup. In the event of all backup edge cache groups not available, GEO location can be optionally used as further backup. APIs detailed here
• Traffic Ops Golang Proxy Endpoints
  • /api/1.3/origins (GET,POST,PUT,DELETE)
  • /api/1.3/coordinates (GET,POST,PUT,DELETE)
  • /api/1.3/staticdnsentries (GET,POST,PUT,DELETE)
• Delivery Service Origins Refactor: The Delivery Service API now creates/updates an Origin entity on Delivery Service creates/updates, and the org_server_fqdn column in the deliveryservice table has been removed. The org_server_fqdn data is now computed from the Delivery Service's primary origin (note: the name of the primary origin is the xml_id of its delivery service).
• Cachegroup-Coordinate Refactor: The Cachegroup API now creates/updates a Coordinate entity on Cachegroup creates/updates, and the latitude and longitude columns in the cachegroup table have been replaced with coordinate (a foreign key to Coordinate). Coordinates created from Cachegroups are given the name from_cachegroup_\<cachegroup name\>.
• Geolocation-based Client Steering: two new steering target types are available to use for CLIENT_STEERING delivery services: STEERING_GEO_ORDER and STEERING_GEO_WEIGHT. When targets of these types have an Origin with a Coordinate, Traffic Router will order and prioritize them based upon the shortest total distance from client -> edge -> origin. Co-located targets are grouped together and can be weighted or ordered within the same location using STEERING_GEO_WEIGHT or STEERING_GEO_ORDER, respectively.
• Tenancy is now the default behavior in Traffic Ops. All database entries that reference a tenant now have a default of the root tenant. This eliminates the need for the use_tenancy global parameter and will allow for code to be simplified as a result. If all user and delivery services reference the root tenant, then there will be no difference from having use_tenancy set to 0.
• Cachegroup Localization Methods: The Cachegroup API now supports an optional localizationMethods field which specifies the localization methods allowed for that cachegroup (currently 'DEEP_CZ', 'CZ', and 'GEO'). By default if this field is null/empty, all localization methods are enabled. After Traffic Router has localized a client, it will only route that client to cachegroups that have enabled the localization method used. For example, this can be used to prevent GEO-localized traffic (i.e. most likely from off-net/internet clients) to cachegroups that aren't optimal for internet traffic.
• Traffic Monitor Client Update: Traffic Monitor is updated to use the Traffic Ops v13 client.
• Removed previously deprecated traffic_monitor_java
• Added infrastructure/cdn-in-a-box for Apachecon 2018 demonstration
• The CacheURL Delivery service field is deprecated. If you still need this functionality, you can create the configuration explicitly via the raw remap field.

Apache Traffic Control 2.2.0

Changes with Traffic Control 2.2.0
#729 - Traffic Ops Golang Incremental Rewrite App
#875 - TO API - should not be able to create / update a user with a higher role than your role
#932 - [TC-482] Filtered UI View Persistence and Linking
#942 - [TC-458] please add ilo link to servers main page
#944 - [TC-429] TPv2 - remove map due to license compatibility issues
#955 - [TC-26] Prepare Docker Environment for Traffic Portal
#983 - [TC-443] TPv2 - add the ability to view cache config files
#986 - [TC-544] TR should de-dupe Reponse Headers when sending a Steering response.
#993 - [TC-515] Traffic Portal - Show human readable protocol on DS page
#997 - [TC-472] traffic_ops/experimental - failure to assign servers to new Delivery Service
#1022 - [TC-522] "Online Caches" on TP Dashboard is confusing
#1049 - [TC-442] TPv2 - create user registration functionality
#1053 - [TC-436] TPv2 - add the ability to manage DNSSEC keys
#1055 - [TC-431] TPv2 - add the ability to compare 2 profiles
#1059 - [TC-421] TPv2 - create cache health view
#1099 - [TC-509] TO postinstall set default number of secrets to 1
#1109 - disable tm java build; enable tm golang
#1130 - cacheurl is deprecated in ATS 7.x
#1132 - TO - make password reset link configurable
#1135 - Traffic Server administration docs are out of date
#1146 - Traffic Ops Rewrite: /api/1.2/servers endpoint - with sqlx
#1171 - Creating an invalidate content request (purge) on a delivery service should check tenancy (if turned on)
#1173 - 'Multi Site Origin Algorithm' is removed in delivery service UI in traffic_ops in TC-2.1
#1183 - parent cachegroup is required for all cachegroup types in portal
#1187 - api throws db error if no ports provided
#1198 - Updated Traffic Ops ISO Generation process to work with management interfaces
#1201 - TP cache stats (bandwidth and connection) charts wipe out data if API call fails
#1219 - Generating DNSSEC keys for a brand new CDN thru the API creates DNSSEC keys for potentially the wrong delivery services and then results in a 500 internal server error
#1222 - Delivery service sort by last updated column doesn't work
#1226 - Provide the ability to configure defaults for delivery service creation
#1252 - Tenant dropdowns don't visualize the tenant hierarchy
#1268 - Generate ISO
#1269 - TP Dashboard
#1270 - Assign servers to DS
#1271 - Ability to copy a profile
#1273 - Ability to view cache config files
#1275 - Ability to link profiles to parameters
#1276 - Ability to diff CRconfig.json
#1277 - Ability to snapshot CRConfig.json
#1278 - Ability to assign/unassign delivery services to a user
#1279 - Ability to send new user registration
#1281 - Ability to manage steering targets
#1282 - Ability to compare profiles
#1284 - Server checks view
#1285 - Manage federations
#1286 - Ability to clone delivery service assignments from one cache to another
#1295 - Re-add managment interface code to gen iso
#1300 - Add the ability to assign multiple delivery services to a cache
#1304 - Add confirmation dialog when queuing updates on a CDN
#1308 - Need a way to view in the Traffic Portal the Traffic Ops URL
#1309 - Move Generic Go Libs out of Traffic Monitor
#1324 - Generating DNSSEC keys through the API results in internal server error
#1330 - seeds.sql minor correction required
#1350 - TrafficOps: use cdn.conf as JSON instead of perl
#1363 - Profile Parameter Page Should List Profile(s)
#1364 - Traffic Ops golang -- GET /api/1.2/system/info endpoint
#1374 - Only create bonding config if interface name is 'bond0'
#1379 - Failure in launching traffic-portal in dev-environment
#1397 - ORT tries to get ats_uid before installing trafficserver
#1398 - Get ats_uid after packages are installed
#1400 - Traffic Portal dashboard page won't load if any view resolves fail
#1406 - TP - provide a better experience when assigning caches to a delivery service
#1409 - Add URI Signing
#1433 - Fixed the response for a deliveryservice with no urisigning keys.
#1434 - TO API/TPv2 should have some ability to default geo miss long/lat for a delivery service
#1436 - Login can result in no header
#1437 - updated to be idiomatic and include query parameter narrowing
#1438 - Fix for cachegroup query parameter misalignment
#1439 - Updates to the api uri signing service.
#1440 - Breadcrumb navigation doesn't work on ds ssl key generation page
#1441 - Provide warning when assigning/removing parameters from a profile
#1442 - When assigning ds's to a caches, add ds type to table for filtering
#1443 - Hide 'View Config Files' from server form if server.type != EDGE* or MID
#1444 - Cannot delete the CDN if did snapshot before
#1460 - TP - no dot allowed in server hostname
#1463 - surface HTTP response body in HTTPError
#1465 - Portal -- SSL certificates for HTTP* deliveryservices should be created as wildcards
#1466 - False not NULL
#1484 - Handle the case where there is no signingAlgorithm or signed params s…
#1497 - Change cdn.name to cdn.domain_name in DeliveryServiceInfoForDomainList
#1503 - Golang phys locations
#1507 - /api/regions?division=:divId no longer filters regions by division
#1508 - /api/asns?cachegroup=:cgId no longer filters asns by cachgroup ID
#1510 - Provide the ability to show only changed parts (added/removed) in TP of a snapshot diff
#1521 - TP continues to try refresh after 401
#1531 - Traffic Portal - Cloning Delivery Services sort order and dropping existing delivery services
#1533 - TP unique value error trying to store NULL IPv6 address for server
#1537 - We need better feedback for why create/update/save buttons are disabled in Traffic Portal.
#1543 - Loosen up API validation on delivery service create/update
#1551 - [Issue-1550] TO golang -- adds orderby= parameter handling to endpoints
#1554 - TO/TP - query string handling options force cacheurl on option 1
#1560 - Golang Proxy Scrypt module to be compatible with the Perl Crypt::ScryptKDF lib
#1569 - TrafficPortal DNS Bypass TTL storing empty string
#1577 - TO API should not try to insert any non-integer into an integer DB column
#1582 - Attempting to delete a user in Traffic Portal results in "Resource Not Found"
#1584 - When creating a new server, only profiles for the same cdn should be available for selection
#1599 - Ops: Add Multiple custom access logs and Header field for ATS
#1604 - Initial version of the TO API test tool via the Golang TO Client
#1605 - TP - when generating a cert for a DS, the common name field should be editable
#1606 - [Issue-1283] - adds export/import profile functionality to TP and basically just lev…
#1609 - Updated documentation for the Servers API
#1620 - API: Server API sets xmpp_id to null
#1630 - No delivery service prefix after upgrading to 2.2
#1634 - Provide simple/advanced view for creating/editing delivery services in TP
#1638 - Traffic Portal - Gen Iso: Copy server attributes no longer works
#1642 - Attempt to unlink parameter from DS profile incorrectly shows 0 servers affected
#1653 - Add context around added/removed pieces of CDN snapshot when doing a snapshot diff
#1655 - TP: Bug when updating a user role a new password must be entered
#1661 - Dockerize Traffic Ops API tests
#1664 - Traffic Portal doesn't restart using systemctl
#1697 - Traffic Portal: Can't add a new server (Key (status)=(1) is not present)
#1699 - Refreshing Cache Checks table should retain state of the table
#1705 - When creating multiple DS's, the values of the last DS are persisted to the new DS form
#1714 - TP: Need to remove parameter data size limit
#1720 - Adding TXT Records support
#1740 - Read-only users cannot update their own password
#1750 - /traffic_ops/app/public/images/ contains non-free images
#1761 - Traffic OPS ZIPCode
#1768 - Add Traffic Monitor Plugin System for Cache Stats Formats
#1769 - Monitor Stats Format Plugin For Delivery Service Names
#1775 - api/1.3/profiles is slow
#1777 - TO golang -- api/1.3/parameters?orderby=id produces an error
#1792 - changed maxLength of zip code to be longer.
#1797 - TO golang -- time format incorrect
#1800 - Fix deep_caching_type validator regex
#1807 - Replacing ccr during config file generation
#1810 - [Issue 1809] prefetch type for ds type query
#1811 - Traffic Ops: remove unused get_type call that makes extra sql queries
#1814 - response fields are not complete in POST/PUT steering target APIs
#1816 - Value isn't validated for STEERING_WEIGHT type target in POST/PUT steering target APIs
#1825 - updated docs to add some missing data
#1826 - Traffic Stats error when trying to get LastSummaryTime
#1829 - GET /api/1.3/cdns returns wrong response code and response format when unauthorized or forbidden
#1833 - [Issue-1830] - increases portal priv level to above R/O role to better reflect what …
#1844 - DeepCachingType -- "NEVER" is now default
#1851 - browser-icons.png should be removed
#1852 - Document spinner-small.gif license
#1854 - Added 2.0 to 2.2 migration document
#1858 - url parameters for last_updated are not parsed correctly
#1860 - normalize url query parameters handling and where and orderby clause construction
#1862 - [Issue-1855] - messing with the hostname causes mismatches when the ssl keys are upd…
#1873 - License fixes
#1878 - Add docs per-DS routing migration
#1880 - Add Apache license headers, license file refs
#1882 - 2.2.x license headers, missing license file libraries
#1883 - Add docs per-DS routing migration
#1917 - [Issue 1916] extra flag needed for go build in rpm

Apache Traffic Control 2.1.0 (incubating)

Changes with Traffic Control 2.1.0
#878 - [TC-488] Docs - Multi Site Origin not up to date
#879 - [TC-490] mso.qstring_handling parameter is checked but not documented
#880 - [TC-489] Multi Site Origin - Invalid default values for multiple config params
#901 - [TC-377] Default profiles for EDGE and MID are missing after initial install
#906 - [TC-327] ConfigFiles.pm detects blank as not null and tries to gen files GH #1090
#909 - [TC-301] creating https delivery service and not setting to active still looks for cert. Github Issue #1086
#912 - [TC-169] TR download the RGB file continuously when the same RGB file on server
#915 - [TC-116] remap.config order is different on master (postgres) than it is on 1.8.
#980 - [TC-552] Global parameters may be duplicated when seeds.sql is run
#988 - [TC-514] ORT: Change Traffic Ops hostname in middle of ORT run
#1001 - [TC-408] Documentation for creating ssl keys is missing a field.
#1090 - [TC-518] ToCDUCheck and ToCHRCheck: Value formatted as float instead of int
#1115 - [TC-429] - TP - removes map due to license incompatibility
#1118 - POST /api/1.2/deliveryserviceserver doesn't update header rewrite, regex remap and cacheurl
#1167 - [BACKPORT][TC-518] ToCDUCheck and ToCHRCheck: Value formatted as float instead of int #1090
#1168 - [BACKPORT][TC-514] ORT: Change Traffic Ops hostname in middle of ORT run
#1195 - [Issue-1189] - Backport to 2.1.x - delivery service tenancy is forced on creation and update if use_tenancy is on
#1375 - BACKPORT - fix docs for Deliveryservice/sslkeys/generate and deliveryservice/ssl…
#1386 - Traffic Portal V2 main menu has two rows labeled "Tenants"