engine: Add timeline events for the breathe cycle

.gitignore

@@ -15,3 +15,4 @@ obj
 /src/syscall
 /deps/*.vsn
 /src/programs.inc
+.images

CONTRIBUTING.md

@@ -0,0 +1,18 @@
+Whenever you create a Pull Request (PR), a maintainer will self-assign
+themselves as the *upstream*. The upstream decides if your PR is accepted and
+might require you to amend additional changes before merging. You can expect
+the upstream to communicate clearly if there are any issues preventing your PR
+from being merged, and how they can be fixed. Once merged, the upstream will
+add the `merged` label to your PR.
+
+You are also welcome to submit PRs you would like to receive feedback on, but
+which are not ready to be merged: include the labels `[wip]` in the title of
+PRs that require further work, and `[sketch]` for PRs that are not meant to be
+merged at all.
+
+Our [Documentation Guide](https://github.com/SnabbCo/snabbswitch/blob/master/src/doc/documentation-guide.md)
+gives pointers on how to contribute to the project's documentation.
+
+If you wish to record a copyright notice with your contribution then you can
+optionally do this in the file `src/COPYRIGHT`; copyright notices in other
+files will be rejected.

README.md

@@ -1,13 +1,13 @@
-# Snabb Switch
+# Snabb
 
-Snabb Switch is a simple and fast packet networking toolkit.
+Snabb (formerly "Snabb Switch") is a simple and fast packet networking toolkit.
 
 We are also a grassroots community of programmers and network
 engineers who help each other to build and deploy new network
 elements. We care about practical applications and finding simpler
 ways to do things.
 
-The Snabb Switch community are active in
+The Snabb community are active in
 [applying modern programming techniques](http://blog.ipspace.net/2014/09/snabb-switch-deep-dive-on-software-gone.html),
 [do-it-yourself operator networking](http://blog.ipspace.net/2014/12/l2vpn-over-ipv6-with-snabb-switch-on.html),
 [high-level device drivers](https://github.com/SnabbCo/snabbswitch/blob/master/src/apps/intel/intel10g.lua),
@@ -24,21 +24,21 @@ on.
 
 ## How does it work?
 
-Snabb Switch is written using these main techniques:
+Snabb is written using these main techniques:
 
 - Lua, a high-level programming language that is easy to learn.
 - LuaJIT, a just-in-time compiler that is competitive with C.
 - Ethernet I/O with no kernel overhead ("kernel bypass" mode).
 
-Snabb Switch compiles into a stand-alone executable called
+Snabb compiles into a stand-alone executable called
 `snabb`. This single binary includes multiple applications and runs on
 any modern Linux distribution. (You could think of it as a
 [busybox](http://en.wikipedia.org/wiki/BusyBox#Single_binary) for
 networking.)
 
 ## How is it being used?
 
-The first generation of Snabb Switch applications include:
+The first generation of Snabb applications include:
 
 ### snabbnfv
 
@@ -53,6 +53,17 @@ network.
 You can deploy Snabb NFV stand-alone with QEMU or you can integrate it
 with a cloud computing platform such as OpenStack.
 
+### lwAFTR
+
+[Snabb lwAFTR](src/program/lwaftr/) is the internet-facing component of
+"lightweight 4-over-6" (lw4o6), an IPv6 transition technology.  An ISP
+can use lwAFTR functions to provide its users with access to the IPv4
+internet while maintaining a simple IPv6-only internal network.  An ISP
+deploying Snabb lwAFTR can also configure lw4o6 to share IPv4 addresses
+between multiple different customers, ameliorating the IPv4 address
+space exhaustion problem and lowering costs.  See the [lwAFTR
+documentation](src/program/lwaftr/doc/) for more details.
+
 ### VPWS
 
 VPWS (Virtual Private Wire Service) is a Layer-2 VPN application being
@@ -73,13 +84,13 @@ or even directly on a Device Under Test.
 ### snsh
 
 [snsh](src/program/snsh/) (Snabb Shell) is a tool for interactively
-experimenting with Snabb Switch. It provides direct access to all APIs
+experimenting with Snabb. It provides direct access to all APIs
 using a Lua shell. You can operate snsh either from script files or
 from an interactive shell.
 
 ## How do I get started?
 
-Setting up a Snabb Switch development environment takes around one
+Setting up a Snabb development environment takes around one
 minute:
 
 ```
@@ -103,7 +114,7 @@ $ sudo snabb packetblaster replay capture.pcap 01:00.0
 
 Here are the ways you can get involved:
 
-- Use the Snabb Switch applications in your network.
+- Use the Snabb applications in your network.
 - Join the [snabb-devel mailing list](https://groups.google.com/forum/#!forum/snabb-devel).
 - Send a mail to [introduce yourself](https://groups.google.com/forum/#!searchin/snabb-devel/introduce/snabb-devel/d8t6hGClnQY/flztyLiIGzoJ) to the community (don't be shy!).
 - Create your very own application: [Getting Started](src/doc/getting-started.md).

src/Makefile

@@ -16,7 +16,7 @@ CSRC   = $(shell find . -regex '[^\#]*\.c' -not -regex './arch/.*' -printf '%P '
 CHDR   = $(shell find . -regex '[^\#]*\.h' -printf '%P ')
 ASM    = $(shell find . -regex '[^\#]*\.dasl' -printf '%P ')
 ARCHSRC= $(shell find . -regex '^./arch/[^\#]*\.c' -printf '%P ')
-RMSRC  = $(shell find . -name  README.src.md -printf '%P ')
+RMSRC  = $(shell find . -name '*.md' -not -regex './obj.*' -printf '%P ')
 # regexp is to include program/foo but not program/foo/bar
 PROGRAM = $(shell find program -regex '^[^/]+/[^/]+' -type d -printf '%P ')
 # sort to eliminate potential duplicate of programs.inc
@@ -30,7 +30,7 @@ ARCHOBJ:= $(patsubst %.c,obj/%_c.o,    $(ARCHSRC))
 ASMOBJ := $(patsubst %.dasl,obj/%_dasl.o,   $(ASM))
 JITOBJS:= $(patsubst %,obj/jit_%.o,$(JITSRC))
 EXTRAOBJS := obj/jit_tprof.o obj/jit_vmprof.o obj/strict.o
-RMOBJS := $(patsubst %.src.md,%.md,$(RMSRC))
+RMOBJS := $(patsubst %,obj/%,$(RMSRC))
 INCOBJ := $(patsubst %.inc,obj/%_inc.o, $(INCSRC))
 EXE    := bin/snabb $(patsubst %,bin/%,$(PROGRAM))
 
@@ -54,7 +54,6 @@ snabb: $(LUAOBJ) $(PFLUAOBJ) $(HOBJ) $(COBJ) $(ARCHOBJ) $(ASMOBJ) $(INCOBJ) $(LU
 	    ../lib/luajit/src/libluajit.a \
 	    -lrt -lc -ldl -lm -lpthread
 	@echo -n "BINARY    "
-	@ln -fs snabb snabbswitch
 	@ls -sh snabb
 
 all: $(EXE)
@@ -110,7 +109,7 @@ $(TESTSCRIPTS): testlog snabb
 		) \
 	)
 
-$(OBJDIR) bin testlog:
+$(OBJDIR) bin testlog $(OBJDIR/doc):
 	$(E) "DIR       $@"
 	$(Q) mkdir -p $@
 
@@ -151,10 +150,9 @@ $(JITOBJS): obj/jit_%.o: ../lib/luajit/src/jit/%.lua $(OBJDIR)
 	$(E) "LUA       $@"
 	$(Q) luajit -bg -n $(patsubst obj/jit_%.o, jit.%, $@) $< $@
 
-
-$(RMOBJS): %.md: %.src.md
+$(RMOBJS): obj/%: %
 	$(E) "MARKDOWN  $@"
-	$(Q) scripts/process-markdown $< > $@
+	$(Q) scripts/process-markdown $< $@
 
 $(INCOBJ): obj/%_inc.o: %.inc Makefile | $(OBJDIR)
 	$(E) "INC       $@"
@@ -182,24 +180,24 @@ obj/jit_vmprof.o: extra/vmprof.c | $(OBJDIR)
 	$(E) "C         $@"
 	$(Q) gcc $(DEBUG) -Wl,-E -O2 -I ../lib/luajit/src -c -Wall -Werror -o $@ $<
 
-book: doc/snabbswitch.pdf doc/snabbswitch.html doc/snabbswitch.epub
+book: obj/doc/snabb.pdf obj/doc/snabb.html obj/doc/snabb.epub
 
-doc/snabbswitch.md: markdown Makefile doc/genbook.sh
+obj/doc/snabb.markdown: markdown Makefile doc/genbook.sh
 	(cd doc; ./genbook.sh) > $@
 
-doc/snabbswitch.pdf: doc/snabbswitch.md
+obj/doc/snabb.pdf: obj/doc/snabb.markdown
 	$(E) "PANDOC    $@"
-	$(Q) (cd doc; pandoc --template=template.latex --latex-engine=lualatex -V fontsize=10pt -V monofont=droidsansmono -V monoscale=.70 -V verbatimspacing=.85 -V mainfont=droidserif -V sansfont=droidsans -V documentclass:book -V geometry:top=1.0in -V geometry:bottom=0.75in -S --toc --chapters  -o snabbswitch.pdf snabbswitch.md)
+	$(Q) (cd obj/doc; pandoc --template=template.latex --latex-engine=lualatex -V fontsize=10pt -V monofont=droidsansmono -V monoscale=.70 -V verbatimspacing=.85 -V mainfont=droidserif -V sansfont=droidsans -V documentclass:book -V geometry:top=1.0in -V geometry:bottom=0.75in -S --toc --chapters  -o ../../$@ ../../$<)
 
-doc/snabbswitch.html: doc/snabbswitch.md
+obj/doc/snabb.html: obj/doc/snabb.markdown
 	$(E) "PANDOC    $@"
-	$(Q) (cd doc; pandoc --self-contained --css="style.css" -S -s --toc --chapters -o snabbswitch.html snabbswitch.md)
+	$(Q) (cd obj/doc; pandoc --self-contained --css="../../doc/style.css" -S -s --toc --chapters -o ../../$@ ../../$<)
 
-doc/snabbswitch.epub: doc/snabbswitch.md
+obj/doc/snabb.epub: obj/doc/snabb.markdown
 	$(E) "PANDOC    $@"
-	$(Q) (cd doc; pandoc --self-contained --css="style.css" -S -s --toc --chapters -o snabbswitch.epub snabbswitch.md)
+	$(Q) (cd obj/doc; pandoc --self-contained --css="../../doc/style.css" -S -s --toc --chapters -o ../../$@ ../../$<)
 
-CLEAN = snabb snabbswitch obj bin doc/snabbswitch.* doc/.images/* testlog programs.inc
+CLEAN = snabb obj bin testlog programs.inc
 
 clean:
 	$(E) "RM        $(CLEAN)"

src/README.md

@@ -1,39 +1,65 @@
 # Introduction
 
-*Snabb Switch* is an extensible, virtualized, Ethernet networking
-toolkit.  With Snabb Switch you can implement networking applications
-using the *Lua language*. Snabb Switch includes all the tools you need to
+*Snabb* is an extensible, virtualized, Ethernet networking
+toolkit.  With Snabb you can implement networking applications
+using the *Lua language*. Snabb includes all the tools you need to
 quickly realize your network designs and its really fast too!
-Furthermore, Snabb Switch is extensible and encourages you to grow the
+Furthermore, Snabb is extensible and encourages you to grow the
 ecosystem to match your requirements.
 
-![Architecture](.images/Architecture.png)
-
-The Snabb Switch Core forms a runtime environment (*engine*) which
+    DIAGRAM: Architecture
+            +---------------------+
+            | Your Network Design |
+            +----*-----*-----*----+
+                 |     |     |
+
+    (Built in and custom Apps and Libraries)
+
+                 |     |     |
+           +-----*-----*-----*-----+
+           |      Snabb Core       |
+           +-----------------------+
+
+The Snabb Core forms a runtime environment (*engine*) which
 executes your *design*. A design is simply a Lua script used to drive the
-Snabb Switch stack, you can think of it as your top-level "main" routine.
+Snabb stack, you can think of it as your top-level "main" routine.
 
-In order to add functionality to the Snabb Switch stack you can load
-modules into the Snabb Switch engine. These can be Lua modules as well as
+In order to add functionality to the Snabb stack you can load
+modules into the Snabb engine. These can be Lua modules as well as
 native code objects. We differentiate between two classes of modules,
 namely libraries and *Apps*. Libraries are simple collections of program
 utilities to be used in your designs, apps or other libraries, just as
 you might expect. Apps, on the other hand, are code objects that
-implement a specific interface, which is used by the Snabb Switch engine
+implement a specific interface, which is used by the Snabb engine
 to organize an *App Network*.
 
-![Network](.images/Network.png)
-
-Usually, a Snabb Switch design will create a series of apps, interconnect
+    DIAGRAM: Network
+                   +---------+
+                   |         |
+                +->* Filter0 *--+
+                |  |         |  |
+                |  +---------+  |
+    +---+----+  |               |  +----+---+
+    |        *--+               +->*        |
+    |  NIC0  |                     |  NIC1  |
+    |        *<-+               +--*        |
+    +---+----+  |               |  +----+---+
+                |  +---------+  |
+                |  |         |  |
+                +--* Filter1 *<-+
+                   |         |
+                   +---------+
+
+Usually, a Snabb design will create a series of apps, interconnect
 these in a desired way using *links* and finally pass the resulting app
-network on to the Snabb Switch engine. The engine's job is to:
+network on to the Snabb engine. The engine's job is to:
 
  * Pump traffic through the app network
  * Keep the app network running (e.g. restart failed apps)
  * Report on the network status
 
 
-# Snabb Switch API
+# Snabb API
 
 The core modules defined below  can be loaded using Lua's `require`. For
 example:
@@ -184,7 +210,7 @@ network. The engine updates the app network as follows:
 
 — Function **engine.main** *options*
 
-Run the Snabb Switch engine. *Options* is a table of key/value pairs. The
+Run the Snabb engine. *Options* is a table of key/value pairs. The
 following keys are recognized:
 
  * `duration` - Duration in seconds to run the engine for (as a floating
@@ -198,6 +224,11 @@ following keys are recognized:
    set to boolean values to force or suppress link and app reporting
    individually. By default `engine.main()' will report on links but not
    on apps.
+ * `measure_latency` - By default, the `breathe()` loop is instrumented
+   to record the latency distribution of running the app graph.  This
+   information can be processed by the `snabb top` program.  Passing
+   `measure_latency=false` in the *options* will disable this
+   instrumentation.
  * `no_report` - A boolean value. If `true` no final report will be
    printed.
 
@@ -333,7 +364,7 @@ Allocate packet and fill it with the contents of *string*.
 
 ## Memory (core.memory)
 
-Snabb Switch allocates special
+Snabb allocates special
 [DMA](https://en.wikipedia.org/wiki/Direct_memory_access) memory that
 can be accessed directly by network cards. The important
 characteristic of DMA memory is being located in contiguous physical
@@ -582,7 +613,7 @@ integers *n* respectively.
 
 ## Main
 
-Snabb Switch designs can be run either with:
+Snabb designs can be run either with:
 
     snabb <snabb-arg>* <design> <design-arg>*
 
@@ -591,7 +622,7 @@ or
     #!/usr/bin/env snabb <snabb-arg>*
     ...
 
-The *main* module provides an interface for running Snabb Switch scripts.
+The *main* module provides an interface for running Snabb scripts.
 It exposes various operating system functions to scripts.
 
 — Field **main.parameters**