GRR v1.9.3: Patch me if you can

• Verified with grootfs-release v0.25.0

Minor changes:

• some updates to the experimental windows job
• bumped go version to 1.9
• bosh release now generated (hopefully) with a sha256 digest
• garden now returns a specific ProcessNotFound error when attaching to a process which doesn't exist
• under the covers, work to be able to run without root in real environments is nearing completion

GRR v1.9.2: We did a patch release

• Verified with grootfs-release v0.24.0

Patch release with some bug fixes and some API changes for experimental image and runtime plugins. This release also includes substantial updates for the new rootfs+layer image scheme and the rootless mode which are both still considered undocumented and experimental.

Noteworthy Bug Fixes

• Previously if the container crashed before the network was configured a pid of zero could be passed to the network plugin resulting in confusing error messages (https://www.pivotaltracker.com/story/show/150078138).
• It is now possible to use bind mounts that have a MNT_SHARED propagation type. This would previously be overriden by runc which was a change of behaviour from garden-linux and broke attached disks in bosh-lite (https://www.pivotaltracker.com/story/show/149995674).

Updates to Experimental Features

• A new preloaded+layer:// scheme for the ContainerSpec.Image property allows a client to request a droplet layer be added to the rootfs in the create call (rather than via a separate StreamIn call). This allows the final rootfs to be pre-created via copy-on-write in the layered filesystem rather than by extracting the droplet tarball in to a running container. (Note: this API is currently experimental and subject to change).
• The Experimental Rootless Mode is nearing completion and now supports limiting cpu and memory usage for created containers.
• Numerous updates have been made to the windows job to enable Garden-on-Windows support
• Garden now reports metrics for StreamIn and Create to aid in debugging performance of image plugins (https://www.pivotaltracker.com/story/show/149914457).

Bosh Property Changes

• Garden's default port-pool range now starts from 61001 rather than 60000 to avoid colliding with the top of the ephemeral port range. This reduces the number of ports in the pool to 4534 but avoids potential conflicts with OS-assigned ports. Operators can use the garden.port_pool.start and garden.port_pool.size properties to change this range (https://www.pivotaltracker.com/story/show/117415775).

GRR v1.9.0: Private Registry Reporting For Duty

• Verified with grootfs-release v0.20.0

Resolves a problem where after multiple invocations of Attach some output would be lost. Also fixes private image support for various non-docker docker registries.

Major Fixes:

• Fixes calls to the docker /v2 endpoints, this was previously breaking support for private images in various registries ECR (AWS EC2 Container Registry), GCR (Google Container Registry) and JFrog Artifactory.
• Multiple invocations of Attach no longer lose output (#144412999 )

Minor Fixes:

• http_proxy property is now properly escaped (#147667115)
• Less noisy logging on start (#146489225)

Changed Bosh Properties:

• The http_proxy property is now properly escaped before being used (#147667115)
• The garden_windows job now exposes various new flags (we will start listing these when Windows is officially supported).

GRR v1.8.0: Yes it runs on windows, why do you ask?

• Verified with grootfs-release v0.19.0

Various things and oh yeah GUARDIAN RUNS (EXPERIMENTALLY) ON WINDOWS NOW.

Major Things

• Allows specifying a maximum tcp kernel memory limit for containers.
• Allow specifying a runtime_plugin other than the default (runc)
• Beginnings of support for running on windows

Highlighted Minor Fixes

• Avoid doing unnecessary DNS lookups at startup
• Ensure MTU is never set defaulted higher than 1500, even if the MTU of the external interface happens to be higher than that (see cloudfoundry/guardian#77)
• Don't explode on restart when -1 was specified as ICMP rule type (see #30)

GRR v1.7.0: UDP and Me

• Verified with grootfs-release v0.18.0

Minor release. Allows logging UDP/ICMP packets and stops shipping unused shadow package with the bosh release. Get it while it's hot!

GRR v1.6.0: Oh, you are auth-full

• Verified with grootfs-release v0.17.1

Big Changes

• Support for Authenticated (User/Pass) Docker Repositories in native garden-shed rootfs fetcher! Woop!

Little Changes and Fixes

• RunC output is now streamed to the main garden log rather than collected after execution has finished. This enables better diagnostics for cases where runc hangs or deadlocks
• Fixed a regressed that made /etc/hosts and /etc/resolv.conf not writable by root in the container
• max_containers property is now enforced rather than just being reported in client.Capacity()
• Changed image_plugin api to allow the plugin to return JSON in order to better support rootless containers using overlayfs (the image plugin API is still experimental and subject to change)

GRR v1.5.0: The One With the Less Weighty Wait

• Verified with grootfs-release v0.17.0

Happy spring-time holidays garden fans! v1.5.0 has only a couple small changes, but please note the new process.Wait() behaviour if you create lots of processes in a single container (see below).

Less Weighty Wait

• Wait() no longer silently deletes the process state, which means it's now safe to call attach() and wait() multiple times. The process state is cleaned up when the container is destroyed instead.
• Systems that create large numbers of processes in a single container - for example Cloud Foundry/Diego - will want to opt-back-in to the old behaviour by setting the garden.cleanup_process_dirs_on_wait bosh property to true.

Bug Fixes

• Fixed a regression in resolv.conf processing introduced in 1.3.0

GRR v1.4.0: The One With The Minor Networking Changes

• Verified with grootfs-release v0.15.0

Hi Gardeneers!

GRR v1.4.0 mostly introduces some changes to the network plugin API and introduces a first-pass experimental ability to limit container block IO. Please, contain your enthusiasm.

Network Plugin / Networking Changes

• We now allow the network plugin to return a set of DNS servers which we will ensure are set on the container's resolv.conf. This allows DNS policy to be set by the network plugin if enabled.
• When the network plugin is not enabled, there's now an additional_dns_servers property that allows specifying extra DNS servers which should be appended to any inherited from resolv.conf or set to the dns_servers property.
• /etc/hosts and /etc/resolv.conf are now bind-mounted in to the container rather than being written in a chroot. This interacts better with user namespaces and rootless containers and is generally more simple and secure.
• We no longer set up the built-in networkers iptable chains when an external network plugin is enabled, to avoid confusion caused by two things both creating iptable state on the host
• Log messages from NetOut(log=true) rules now properly truncate handles so that space separation is preserved when the handle is longer than 29 characters

BlockIO Limiting (Experimental)

• We've added an experimental default_container_blockio_weight property to allow assigning a blockio weight to all containers created by garden. This requires the CFQ scheduler to be enabled to be useful and applies to all garden-created containers on the host. The feature is being released experimentally via the bosh property, feedback welcome!

Rootless Mode

• You can now follow the updated experimental rootless mode doc to add networking support to your rootless containers.

GRR v1.3.0: People say I am rootless

• Verified with grootfs-release v0.15.0

Lots of nice little changes and two big ones: CPU maximums support and alpha/experimental support for running containers without root.

Experimental! Support for running containers without root

This is very much a v1 which we would not suggest running in production (yet!). It is now possible to run garden without needing root (other than for one-time setup). This increases the security posture of garden and the number of places where it can potentially be used. Manual instructions and known gaps in the attached doc: https://github.com/cloudfoundry/garden-runc-release/blob/develop/docs/rootless-containers.md.

You can also opt-in using the experimental_rootless_mode bosh property which will do all the work for you!

NOTE: Not ready - or secure - for production use yet, but feedback very welcome!

CPU Maximums

It is now possible to request a maximum CPU limit proportional to the existing fair-share CPU limit. The previous cpu limit just ensured that if two containers competed for cpu they would receive it in proportion to their requested cpu share (in the Cloud Foundry use case, this meant in proportion to their memory limit). This meant, for example, if only one container was active on a host it could potentially consume all of the host's resources -- far more than were paid for.

The new cpu_quota_per_share_in_us property, which - if set - applies to all containers created on a host, enforces an additional maximum amount of cpu usage in each quota period (which unless changed on the system is 100ms) in proportion to the existing shares limit.

Example: Using Cloud Foundry as an example, since CF sets a container's cpu limit (ContainerSpec.Limits.Cpu.LimitInShares) based on the memory limit, if cpu_quota_per_share_in_us is set to 100 on a single-core system then a 64MB container will be able to use a maximum of 6.4ms (64 * 100us = 6400us = 6.4ms) of cpu every 100ms.

A Few More Things..

• Container MTU now defaults to the MTU of the external network interface on the host rather than 1500. This is pretty much always what you want and avoids issues on environments with lower MTUs than the default.
• We now strip all entries starting 127.* from resolv.conf on the host when setting up networking for the container. Since 127.* on the host can never be accessed in the container it's never the right thing for it to survive into the container's resolv.conf (and causes slow DNS lookups when it does).
• Our CI now tracks container creation performance using bucket bench
• Golang bumped to 1.8.0
• Squished some misleading log messages. Feel good knowing you will no longer be misled (as much).

Enjoy!

GRR v1.2.0

• Verified with grootfs-release v0.11.0

Hi garden fans! Today we have a bumper release with lots of nice little fixes and improvements:

API Changes and Deprecations:

• Docker Auth Support: The client.Create call now supports an Image.URI field which deprecates the existing RootfsPath field. It acts in exactly the same way as the existing field, however it is now possible to specify Image.Username and Image.Password which will be passed to the image_plugin if configured. This allows an image_plugin (such as grootfs) to support authenticated Docker images.
• Create-Time NetIn/Out: NetIn and NetOut are now able to be specified on the client.Create call, and we recommend all clients switch to this as the dynamic methods are now deprecated. If specified, they are passed to the network_plugin on create, which allows better integration with CNI up/down hooks. The existing methods will continue to be supported in the built-in kawasaki networker until the next major version bump.
• Image Plugin API: Experimental image_plugin API continues to evolve, now does not needlessly swap uid before running the plugin in unprivileged mode (this is up to the plugin to do if it wishes).
• Grace Time: grace_time now defaults to 0 (i.e. infinity). Most clients were explicitly overriding our default here anyway, and without this default it is impossible to later ask for a container not to have a grace time (since 0 in client.Create means "use the default"). Clients must now explicitly set the grace_time bosh property if they wish containers to have a grace time by default.

General Improvements

• RunC was bumped to the latest version
• Inspector-garden is no longer needed! You can now interact with containers without any extra steps. (Yay!)
• Fixed handling of the bosh release shutdown script to avoid a case where a non-zero exit was returned to bosh even though the server had been killed
• The bosh release now increases pid limits to avoid running out of pids in large deployments (to avoid pid exhaustion from containers we recommend configuring ContainerSpec.Pids.Max in the client.Create call).
• It is possible to opt-out of apparmor (for environments which cannot support it), by specifying an empty string for the apparmor_profile property. It is also possible to request a different apparmor profile than the garden-default be used, so long as this is installed on the host (for example as a bosh pre-start job in an add-on).
• iptables rules now add a descriptive comment containing the container guid for easier debuggability
• Work continues on the experimental rootless mode, there is now a separate setup command which can be run as root, allowing the main guardian server to start up without needing root.