Release Notes for 0.4.1 release. (#839)

Also update Getting Started and Debugging docs to reflect changes in
`Tap` and `Stat`.

Fixes #838

Signed-off-by: Andrew Seigner <siggy@buoyant.io>

Author: siggy

CHANGES.md

@@ -1,3 +1,35 @@
+## v0.4.1
+
+Conduit 0.4.1 builds on the telemetry work from 0.4.0, providing rich,
+Kubernetes-aware observability and debugging.
+
+* Web UI
+  * **New** Automatically-configured Grafana dashboards for Services, Pods,
+    ReplicationControllers, and Conduit mesh health.
+  * **New** `conduit dashboard` Pod and ReplicationController views.
+* Command-line interface
+  * **Breaking change** `conduit tap` now operates on most Kubernetes resources.
+  * `conduit stat` and `conduit tap` now both support kubectl-style resource
+    strings (`deploy`, `deploy/web`, and `deploy web`), specifically:
+    * `namespaces`
+    * `deployments`
+    * `replicationcontrollers`
+    * `services`
+    * `pods`
+* Telemetry
+  * **New** Tap support for filtering by and exporting destination metadata. Now
+    you can sample requests from A to B, where A and B are any resource or group
+    of resources.
+  * **New** TCP-level stats, including connection counts and durations, and
+    throughput, wired through to Grafana dashboards.
+* Service Discovery
+  * The proxy now uses the [trust-dns] DNS resolver. This fixes a number of DNS
+    correctness issues.
+  * The Destination service could sometimes return incorrect, stale, labels for an
+    endpoint. This has been fixed!
+
+[trust-dns]: https://github.com/bluejekyll/trust-dns
+
 ## v0.4.0
 
 Conduit 0.4.0 overhauls Conduit's telemetry system and improves service discovery

cli/cmd/testdata/install_default.golden

@@ -493,7 +493,6 @@ data:
       - role: pod
         namespaces:
           names: ['conduit']
-      # TODO: do something with "conduit.io/control-plane-component"
       relabel_configs:
       - source_labels:
         - __meta_kubernetes_pod_label_conduit_io_control_plane_component

cli/cmd/testdata/install_output.golden

@@ -496,7 +496,6 @@ data:
       - role: pod
         namespaces:
           names: ['Namespace']
-      # TODO: do something with "conduit.io/control-plane-component"
       relabel_configs:
       - source_labels:
         - __meta_kubernetes_pod_label_conduit_io_control_plane_component

cli/install/template.go

@@ -333,7 +333,6 @@ data:
       - role: pod
         namespaces:
           names: ['{{.Namespace}}']
-      # TODO: do something with "conduit.io/control-plane-component"
       relabel_configs:
       - source_labels:
         - __meta_kubernetes_pod_label_conduit_io_control_plane_component

doc/debugging-an-app.md

@@ -15,41 +15,41 @@ running](/getting-started), let's use Conduit to diagnose issues.
 
 First, let's use the `conduit stat` command to get an overview of deployment
 health:
-#### `conduit stat deployments`
+#### `conduit -n emojivoto stat deploy`
 
 ### Your results will be something like:
 ```
-NAME                   REQUEST_RATE   SUCCESS_RATE   P50_LATENCY   P99_LATENCY
-emojivoto/emoji              2.0rps        100.00%           0ms           0ms
-emojivoto/voting             0.6rps         66.67%           0ms           0ms
-emojivoto/web                2.0rps         95.00%           0ms           0ms
+NAME       MESHED   SUCCESS      RPS   LATENCY_P50   LATENCY_P95   LATENCY_P99
+emoji         1/1   100.00%   2.0rps           1ms           4ms           5ms
+vote-bot      1/1         -        -             -             -             -
+voting        1/1    89.66%   1.0rps           1ms           5ms           5ms
+web           1/1    94.92%   2.0rps           5ms          10ms          18ms
 ```
 
 We can see that the `voting` service is performing far worse than the others.
 
 How do we figure out what's going on? Our traditional options are: looking at
 the logs, attaching a debugger, etc. Conduit gives us a new tool that we can use
 - a live view of traffic going through the deployment. Let's use the `tap`
-command to take a look at requests currently flowing through this deployment.
+command to take a look at all requests currently flowing to this deployment.
 
-#### `conduit tap deploy emojivoto/voting`
+#### `conduit -n emojivoto tap deploy --to deploy/voting`
 
 This gives us a lot of requests:
 
 ```
-req id=0:458 src=172.17.0.9:45244 dst=172.17.0.8:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VoteGhost
-rsp id=0:458 src=172.17.0.9:45244 dst=172.17.0.8:8080 :status=200 latency=758µs
-end id=0:458 src=172.17.0.9:45244 dst=172.17.0.8:8080 grpc-status=OK duration=9µs response-length=5B
-req id=0:459 src=172.17.0.9:45244 dst=172.17.0.8:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VoteDoughnut
-rsp id=0:459 src=172.17.0.9:45244 dst=172.17.0.8:8080 :status=200 latency=987µs
-end id=0:459 src=172.17.0.9:45244 dst=172.17.0.8:8080 grpc-status=OK duration=9µs response-length=5B
-req id=0:460 src=172.17.0.9:45244 dst=172.17.0.8:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VoteBurrito
-rsp id=0:460 src=172.17.0.9:45244 dst=172.17.0.8:8080 :status=200 latency=767µs
-end id=0:460 src=172.17.0.9:45244 dst=172.17.0.8:8080 grpc-status=OK duration=18µs response-length=5B
-req id=0:461 src=172.17.0.9:45244 dst=172.17.0.8:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VoteDog
-rsp id=0:461 src=172.17.0.9:45244 dst=172.17.0.8:8080 :status=200 latency=693µs
-end id=0:461 src=172.17.0.9:45244 dst=172.17.0.8:8080 grpc-status=OK duration=10µs response-length=5B
-req id=0:462 src=172.17.0.9:45244 dst=172.17.0.8:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
+req id=0:1624 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VoteDoughnut
+rsp id=0:1624 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :status=200 latency=1603µs
+end id=0:1624 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs grpc-status=OK duration=28µs response-length=5B
+req id=0:1629 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VoteBeer
+rsp id=0:1629 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :status=200 latency=2009µs
+end id=0:1629 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs grpc-status=OK duration=24µs response-length=5B
+req id=0:1634 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VoteDog
+rsp id=0:1634 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :status=200 latency=1730µs
+end id=0:1634 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs grpc-status=OK duration=21µs response-length=5B
+req id=0:1639 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VoteCrossedSwords
+rsp id=0:1639 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :status=200 latency=1599µs
+end id=0:1639 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs grpc-status=OK duration=27µs response-length=5B
 ```
 
 Let's see if we can narrow down what we're looking at. We can see a few
@@ -59,35 +59,35 @@ requests.
 Let's figure out where those are coming from. Let's run the `tap` command again,
 and grep the output for `Unknown`s:
 
-####  ```conduit tap deploy emojivoto/voting | grep Unknown -B 2```
+####  ```conduit -n emojivoto tap deploy --to deploy/voting | grep Unknown -B 2```
 
 ```
-req id=0:212 src=172.17.0.8:58326 dst=172.17.0.10:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
-rsp id=0:212 src=172.17.0.8:58326 dst=172.17.0.10:8080 :status=200 latency=360µs
-end id=0:212 src=172.17.0.8:58326 dst=172.17.0.10:8080 grpc-status=Unknown duration=0µs response-length=0B
+req id=0:2294 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
+rsp id=0:2294 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :status=200 latency=2147µs
+end id=0:2294 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs grpc-status=Unknown duration=0µs response-length=0B
 --
-req id=0:215 src=172.17.0.8:58326 dst=172.17.0.10:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
-rsp id=0:215 src=172.17.0.8:58326 dst=172.17.0.10:8080 :status=200 latency=414µs
-end id=0:215 src=172.17.0.8:58326 dst=172.17.0.10:8080 grpc-status=Unknown duration=0µs response-length=0B
+req id=0:2314 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
+rsp id=0:2314 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :status=200 latency=2405µs
+end id=0:2314 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs grpc-status=Unknown duration=0µs response-length=0B
 --
 ```
 
 We can see that all of the `grpc-status=Unknown`s are coming from the `VotePoop`
 endpoint. Let's use the `tap` command's flags to narrow down our output to just
 this endpoint:
 
-####  ```conduit tap deploy emojivoto/voting --path /emojivoto.v1.VotingService/VotePoop```
+####  ```conduit -n emojivoto tap deploy/voting --path /emojivoto.v1.VotingService/VotePoop```
 
 ```
-req id=0:264 src=172.17.0.8:58326 dst=172.17.0.10:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
-rsp id=0:264 src=172.17.0.8:58326 dst=172.17.0.10:8080 :status=200 latency=696µs
-end id=0:264 src=172.17.0.8:58326 dst=172.17.0.10:8080 grpc-status=Unknown duration=0µs response-length=0B
-req id=0:266 src=172.17.0.8:58326 dst=172.17.0.10:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
-rsp id=0:266 src=172.17.0.8:58326 dst=172.17.0.10:8080 :status=200 latency=667µs
-end id=0:266 src=172.17.0.8:58326 dst=172.17.0.10:8080 grpc-status=Unknown duration=0µs response-length=0B
-req id=0:270 src=172.17.0.8:58326 dst=172.17.0.10:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
-rsp id=0:270 src=172.17.0.8:58326 dst=172.17.0.10:8080 :status=200 latency=346µs
-end id=0:270 src=172.17.0.8:58326 dst=172.17.0.10:8080 grpc-status=Unknown duration=0µs response-length=0B
+req id=0:2724 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
+rsp id=0:2724 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :status=200 latency=1644µs
+end id=0:2724 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs grpc-status=Unknown duration=0µs response-length=0B
+req id=0:2729 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
+rsp id=0:2729 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :status=200 latency=1736µs
+end id=0:2729 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs grpc-status=Unknown duration=0µs response-length=0B
+req id=0:2734 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
+rsp id=0:2734 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs :status=200 latency=1779µs
+end id=0:2734 src=10.1.8.150:56224 dst=voting-6795f54474-6vfbs grpc-status=Unknown duration=0µs response-length=0B
 ```
 
 We can see that none of our `VotePoop` requests are successful. What happens

doc/getting-started.md

@@ -267,6 +267,16 @@ high-level health metrics of deployments in the data plane.
 ### DEPLOYMENTS
 Lists all deployments by requests, success rate, and latency.
 
+### PODS
+Lists all pods by requests, success rate, and latency.
+
+### REPLICATION CONTROLLER
+Lists all replications controllers by requests, success rate, and latency.
+
+### GRAFANA
+For detailed metrics on all of the above resources, click any resource to browse
+to a dynamically-generated Grafana dashboard.
+
 ___
 
 ## Using the CLI 💻
@@ -275,30 +285,31 @@ happening in the Conduit service mesh. The CLI provides several interesting and
 powerful commands that you should experiment with, including `conduit stat` and `conduit tap`.
 
 ### To view details per deployment, run:
-#### `conduit stat deployments`
+#### `conduit -n emojivoto stat deploy`
 
 ### Which should display:
 ```
-NAME                   REQUEST_RATE   SUCCESS_RATE   P50_LATENCY   P99_LATENCY
-emojivoto/emoji              2.0rps        100.00%           0ms           0ms
-emojivoto/voting             0.6rps         66.67%           0ms           0ms
-emojivoto/web                2.0rps         95.00%           0ms           0ms
+NAME       MESHED   SUCCESS      RPS   LATENCY_P50   LATENCY_P95   LATENCY_P99
+emoji         1/1   100.00%   2.0rps           1ms           2ms           3ms
+vote-bot      1/1         -        -             -             -             -
+voting        1/1    81.36%   1.0rps           1ms           1ms           2ms
+web           1/1    90.68%   2.0rps           4ms           5ms           5ms
 ```
 
  
 
 ### To see a live pipeline of requests for your application, run:
-#### `conduit tap deploy emojivoto/voting`
+#### `conduit -n emojivoto tap deploy`
 
 ### Which should display:
 ```
-req id=0:127 src=172.17.0.11:50992 dst=172.17.0.10:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VoteManInTuxedo
-rsp id=0:127 src=172.17.0.11:50992 dst=172.17.0.10:8080 :status=200 latency=588µs
-end id=0:127 src=172.17.0.11:50992 dst=172.17.0.10:8080 grpc-status=OK duration=9µs response-length=5B
-req id=0:128 src=172.17.0.11:50992 dst=172.17.0.10:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePager
-rsp id=0:128 src=172.17.0.11:50992 dst=172.17.0.10:8080 :status=200 latency=601µs
-end id=0:128 src=172.17.0.11:50992 dst=172.17.0.10:8080 grpc-status=OK duration=11µs response-length=5B
-req id=0:129 src=172.17.0.11:50992 dst=172.17.0.10:8080 :method=POST :authority=voting-svc.emojivoto:8080 :path=/emojivoto.v1.VotingService/VotePoop
+req id=0:2900 src=10.1.8.151:51978 dst=10.1.8.150:80 :method=GET :authority=web-svc.emojivoto:80 :path=/api/list
+req id=0:2901 src=10.1.8.150:49246 dst=emoji-664486dccb-97kws :method=POST :authority=emoji-svc.emojivoto:8080 :path=/emojivoto.v1.EmojiService/ListAll
+rsp id=0:2901 src=10.1.8.150:49246 dst=emoji-664486dccb-97kws :status=200 latency=2146µs
+end id=0:2901 src=10.1.8.150:49246 dst=emoji-664486dccb-97kws grpc-status=OK duration=27µs response-length=2161B
+rsp id=0:2900 src=10.1.8.151:51978 dst=10.1.8.150:80 :status=200 latency=5698µs
+end id=0:2900 src=10.1.8.151:51978 dst=10.1.8.150:80 duration=112µs response-length=4558B
+req id=0:2902 src=10.1.8.151:51978 dst=10.1.8.150:80 :method=GET :authority=web-svc.emojivoto:80 :path=/api/vote
 ...
 ```
 

doc/roadmap.md

@@ -15,33 +15,16 @@ overly conservative.
 
 
 ##### Status: alpha
-## [0.4.0: Rich, Kubernetes-aware Grafana & Prometheus](https://github.com/runconduit/conduit/milestone/6)
-#### 2018-04-16
-
-### Visibility
-
-- Rich, Kubernetes-aware `conduit stat`:
-  - Works on deployments & namespaces.
-  - `--from` & `--to` flags filter stats by source & destination.
-- Proxy exposes Prometheus labeled with rich outbound stats.
-- Grafana dashboards for Kubernetes Deployments & Namespaces.
-
-### Reliability
-
-- The proxy properly routes egress traffic to arbitrary DNS names.
-
-
 ## [0.4.1: Rich, Kubernetes-aware debugging](https://github.com/runconduit/conduit/milestone/10)
-#### Late April 2018
+#### 2018-04-26
 
 ### Visibility
 
 - `conduit stat` works on many Kubernetes resources.
   - Per-authority HTTP stats.
   - TCP-level stats
 - `conduit tap` works on many Kubernetes resources, too.
-- `conduit wtf`: what's the failure?
-
+- Grafana dashboards for Kubernetes Pods, Services, & Replication Controllers.
 
 ## [0.5: Stable, private communication](https://github.com/runconduit/conduit/milestone/7)
 #### Mid-May 2018
@@ -58,6 +41,10 @@ overly conservative.
 - Stable Service Discovery semantics.
 - Latency-aware load balancing.
 
+### Visibility
+
+- `conduit wtf`: what's the failure?
+
 
 ## [0.6: Externally accessible](https://github.com/runconduit/conduit/milestone/8)
 #### Early June 2018