From 7a0c4c7b1601d9af86b61a254a6d43fa9441b40f Mon Sep 17 00:00:00 2001 From: Michael Crouch Date: Tue, 7 Jun 2016 14:02:24 -0400 Subject: [PATCH 1/5] Whitespace changes - tabs to spaces --- docs/readme.md | 214 ++-- docs/src/WEB-INF/web.xml | 12 +- docs/src/_includes/footer.html | 4 +- docs/src/_includes/head.html | 16 +- docs/src/_includes/submenu.html | 18 +- docs/src/_includes/tocs/docs-development.html | 44 +- docs/src/_includes/tocs/docs-user-guide.html | 38 +- docs/src/_layouts/chapter.html | 36 +- docs/src/_layouts/contact.html | 28 +- docs/src/_layouts/default.html | 44 +- docs/src/_layouts/image.html | 20 +- docs/src/_layouts/section.html | 26 +- docs/src/_layouts/submenu.html | 28 +- docs/src/_sass/_code-block.scss | 8 +- docs/src/contact/sidebar.html | 8 +- docs/src/css/all.scss | 736 ++++++------ docs/src/css/syntax.scss | 2 +- docs/src/demos/sidebar.html | 40 +- .../advanced/custom-tiling/readme.md | 356 +++--- .../advanced/graph-tiling/readme.md | 1042 ++++++++--------- .../advanced/standard-tiling/readme.md | 324 +++-- .../docs/development/api/annotation/readme.md | 986 ++++++++-------- docs/src/docs/development/api/layer/readme.md | 645 +++++----- .../src/docs/development/api/legend/readme.md | 52 +- docs/src/docs/development/api/tile/readme.md | 66 +- .../getting-started/installation/readme.md | 338 +++--- .../getting-started/quick-start/readme.md | 192 +-- .../getting-started/tile-pyramid/readme.md | 113 +- .../development/how-to/app-setup/readme.md | 99 +- .../how-to/standard-tiling/readme.md | 793 +++++++------ .../development/how-to/test-output/readme.md | 142 +-- .../development/how-to/tile-client/readme.md | 244 ++-- .../development/how-to/tile-server/readme.md | 340 +++--- docs/src/docs/user-guide/interface/readme.md | 44 +- docs/src/docs/user-guide/navigation/readme.md | 29 +- docs/src/download/readme.md | 21 +- docs/src/download/sidebar.html | 8 +- docs/src/readme.md | 2 +- docs/src/sidebar.html | 32 +- docs/src/tour/components/sidebar.html | 4 +- docs/src/tour/process/readme.md | 23 +- docs/src/tour/readme.md | 16 +- docs/src/tour/sidebar.html | 4 +- 43 files changed, 3600 insertions(+), 3637 deletions(-) diff --git a/docs/readme.md b/docs/readme.md index d4d8f6b4..125b69da 100644 --- a/docs/readme.md +++ b/docs/readme.md @@ -31,15 +31,15 @@ Before you attempt to build an Aperture site, you should install the following p To build an Aperture site, run the following command in its source docs folder, [docs/src/](docs/src/), which contains a *_config.yml* file: - jekyll build + jekyll build By default, Jekyll will compile the site into a `_site` folder one level above the source docs folder. You can change the destination directory using the following command: - jekyll build --destination + jekyll build --destination To test the site before deploying it, you can run the following command, which will build a preview site at http://localhost:4000/: - jekyll serve + jekyll serve ## Aperture Website Configuration @@ -54,56 +54,56 @@ Each Markdown file (*.md) covers a single topic representing a page on the Apert All Markdown files must contain a frontmatter section so Jekyll will recognize that they should be converted to HTML. The frontmatter must appear at the top of the file between a set of triple-dashed lines as follows: - --- - --- + --- + --- Between these lines, you can define any number of page-specific variables (NOTE: try to keep the variables to a minimum if possible; they are visible at the top of the page in the GitHub repository). The page-specific variables are invoked in the layouts using the [Liquid templating language](http://docs.shopify.com/themes/liquid-basics). The Aperture sites currently use the following page-specific variables: ``` -section (Required) +section (Required) - Name of the section to which the page belongs. Must match the name of one of - the links in the main navigation menu (see callout 1 in the following figure). + Name of the section to which the page belongs. Must match the name of one of + the links in the main navigation menu (see callout 1 in the following figure). - This variable is used by Jekyll when: - - Defining the page title (as it appears in the browser window or tab) - - Determining the navigation bar context - - Retrieving the sidebar content + This variable is used by Jekyll when: + - Defining the page title (as it appears in the browser window or tab) + - Determining the navigation bar context + - Retrieving the sidebar content -subsection (Required for pages with submenus) +subsection (Required for pages with submenus) - Name of the subsection to which the page belongs. Must match the name of one - of the links in the section-specific submenu (see callout 2 in the following - figure). + Name of the subsection to which the page belongs. Must match the name of one + of the links in the section-specific submenu (see callout 2 in the following + figure). - This variable is used by Jekyll when retrieving and outputting the section - submenu and its context. + This variable is used by Jekyll when retrieving and outputting the section + submenu and its context. -chapter (Required for pages in a submenu that represent a chapter) +chapter (Required for pages in a submenu that represent a chapter) - Name of the chapter to which the page belongs. Should match the name of one - of the headings in the submenu-specific table of contents (see callout 3 in - the following figure). + Name of the chapter to which the page belongs. Should match the name of one + of the headings in the submenu-specific table of contents (see callout 3 in + the following figure). -topic (Required for with a submenu-specific table of contents) +topic (Required for with a submenu-specific table of contents) - Name of the topic which the page describes. Should match the name of one - of the links in the submenu-specific table of contents (see callout 4 in the - following figure). + Name of the topic which the page describes. Should match the name of one + of the links in the submenu-specific table of contents (see callout 4 in the + following figure). -permalink (Required) +permalink (Required) - The relative path to which the converted HTML should be written. Upon - conversion, all Markdown files will be renamed to index.html so they are - displayed automatically in a web browser when using relative paths. - -layout (Required) + The relative path to which the converted HTML should be written. Upon + conversion, all Markdown files will be renamed to index.html so they are + displayed automatically in a web browser when using relative paths. + +layout (Required) - Layout used to render the content. Choose from: - - section, for pages without submenus - - submenu, for pages with submenus + Layout used to render the content. Choose from: + - section, for pages without submenus + - submenu, for pages with submenus ``` ![](src/img/resources/SiteComponents.png) @@ -115,107 +115,107 @@ In addition to getting page-specific variables from the Markdown files, Jekyll a ``` _config.yml - File containing Jekyll configuration options such as: + File containing Jekyll configuration options such as: - - The Markdown renderer to use to convert the content. The default renderer - for the Aperture sites is Redcarpet. - - A product variable used to display the correct logo and set part of the - page title (as it appears in the browser window or tab). - - A baseurl variable that indicates where the root site folder is located. - You do not need to set this variable if you are deploying to your server's - root folder + - The Markdown renderer to use to convert the content. The default renderer + for the Aperture sites is kramdown. + - A product variable used to display the correct logo and set part of the + page title (as it appears in the browser window or tab). + - A baseurl variable that indicates where the root site folder is located. + You do not need to set this variable if you are deploying to your server's + root folder _data - Folder containing YAML files (**\*.yaml**) that list the items that should - appear in the main navigation bar, each of the section-specific submenu - navigation bars and the submenu-specific table of contents. + Folder containing YAML files (**\*.yaml**) that list the items that should + appear in the main navigation bar, each of the section-specific submenu + navigation bars and the submenu-specific table of contents. - Items in the main website header menu should be listed in an array named - "main", with each item having a label and a path (with the base URL omitted): + Items in the main website header menu should be listed in an array named + "main", with each item having a label and a path (with the base URL omitted): - main: - - label: Tour - path: tour/ + main: + - label: Tour + path: tour/ - - label: Docs - path: docs/development/ + - label: Docs + path: docs/development/ - - label: Demos - path: demos/ + - label: Demos + path: demos/ - - label: Download - path: download/ + - label: Download + path: download/ - - label: Contact - path: contact/ + - label: Contact + path: contact/ - Items in the section-specific submenu navigation bars should follow the - same format, replacing "main" for the appropriate section label, which - should match one of the item labels in the main menu and the section - variables used in the Markdown files for that section. + Items in the section-specific submenu navigation bars should follow the + same format, replacing "main" for the appropriate section label, which + should match one of the item labels in the main menu and the section + variables used in the Markdown files for that section. - docs: - - label: Development - path: docs/development/getting-started/installation/ + docs: + - label: Development + path: docs/development/getting-started/installation/ - - label: User Guide - path: docs/user-guide/ - - To reference a single item in a file, call the filename followed by the - appropriate array name to specify the menu you want: - - {% for menuItem in site.data.menuItems.main %} + - label: User Guide + path: docs/user-guide/ + + To reference a single item in a file, call the filename followed by the + appropriate array name to specify the menu you want: + + {% for menuItem in site.data.menuItems.main %} - Do not use special characters or spaces in the filename. + Do not use special characters or spaces in the filename. _includes - Contains two types of files: + Contains two types of files: - - Files containing snippets of HTML shared across many pages - (e.g., head.html, submenu.html or footer.html). + - Files containing snippets of HTML shared across many pages + (e.g., head.html, submenu.html or footer.html). - - HTML files for the tables of contents (TOCs) that appear on pages sent to - the chapters layout. Must follow the naming convention: + - HTML files for the tables of contents (TOCs) that appear on pages sent to + the chapters layout. Must follow the naming convention: - {{ page.section }}-{{ page.subtitle }}.html + {{ page.section }}-{{ page.subtitle }}.html - These can be invoked using the following format: + These can be invoked using the following format: - {% include footer.html %} + {% include footer.html %} _layouts - There are five main layouts used to build the site: - - - default.html: Defines the navigation bar at the top of the website, - points to the CSS and JS resources and sets the page title. Invoked by - all of the other layouts (except image.html). - - section.html: Uses a include_relative function to retrieve sidebar content - for pages without submenus. Sidebar content should be stored in a - sidebar.html file in the same folder as the Markdown content. - Calls default.html. - - submenu.html: Gets gets and outputs the submenu items and uses a - include_relative function to retrieve sidebar content. Sidebar content - should be stored in a sidebar.html file in the same folder as the - Markdown content. Calls default.html. - - chapter.html: Gets the sidebar for the pages with submenus (using the - page.section and page.subtitle variables) and gets and outputs the - submenu items. Calls default.html. - - image.html: Simple layout for displaying expanded images that don't fit - in the default layout. - - The layouts are written in HTML and utilize Liquid templating language to - retrieve and manipulate page- and site-specific variables. + There are five main layouts used to build the site: + + - default.html: Defines the navigation bar at the top of the website, + points to the CSS and JS resources and sets the page title. Invoked by + all of the other layouts (except image.html). + - section.html: Uses a include_relative function to retrieve sidebar content + for pages without submenus. Sidebar content should be stored in a + sidebar.html file in the same folder as the Markdown content. + Calls default.html. + - submenu.html: Gets gets and outputs the submenu items and uses a + include_relative function to retrieve sidebar content. Sidebar content + should be stored in a sidebar.html file in the same folder as the + Markdown content. Calls default.html. + - chapter.html: Gets the sidebar for the pages with submenus (using the + page.section and page.subtitle variables) and gets and outputs the + submenu items. Calls default.html. + - image.html: Simple layout for displaying expanded images that don't fit + in the default layout. + + The layouts are written in HTML and utilize Liquid templating language to + retrieve and manipulate page- and site-specific variables. _sass - Contains partial Sass files used to build the final CSS files that apply to - your site. + Contains partial Sass files used to build the final CSS files that apply to + your site. _site - Only created after Jekyll compiles the site. Contains all of the HTML files - and associated resources. + Only created after Jekyll compiles the site. Contains all of the HTML files + and associated resources. ``` \ No newline at end of file diff --git a/docs/src/WEB-INF/web.xml b/docs/src/WEB-INF/web.xml index 1ede4aac..96851f78 100644 --- a/docs/src/WEB-INF/web.xml +++ b/docs/src/WEB-INF/web.xml @@ -3,11 +3,11 @@ "http://java.sun.com/dtd/web-app_2_3.dtd" > - Aperture Tiles Docs + Aperture Tiles Docs - - index.html - index.htm - index.jsp - + + index.html + index.htm + index.jsp + \ No newline at end of file diff --git a/docs/src/_includes/footer.html b/docs/src/_includes/footer.html index 12ff27e8..48896155 100644 --- a/docs/src/_includes/footer.html +++ b/docs/src/_includes/footer.html @@ -1,6 +1,6 @@
- +

- ©2013-{{ site.time | date: '%Y'}} Uncharted Software Inc. + ©2013-{{ site.time | date: '%Y'}} Uncharted Software Inc.

\ No newline at end of file diff --git a/docs/src/_includes/head.html b/docs/src/_includes/head.html index 9b01d14a..35835d1e 100644 --- a/docs/src/_includes/head.html +++ b/docs/src/_includes/head.html @@ -15,13 +15,13 @@ {% if site.googleAnalytics != null %} - + ga('create', '{{ site.googleAnalytics }}', 'auto'); + ga('send', 'pageview'); + {% endif %} \ No newline at end of file diff --git a/docs/src/_includes/submenu.html b/docs/src/_includes/submenu.html index efaf6364..3c0358b2 100644 --- a/docs/src/_includes/submenu.html +++ b/docs/src/_includes/submenu.html @@ -1,13 +1,13 @@ {% comment %} Get filename containing submenu items {% endcomment %} - {% capture submenuSection %}{{ page.section | slugify }}{% endcapture %} + {% capture submenuSection %}{{ page.section | slugify }}{% endcapture %} \ No newline at end of file diff --git a/docs/src/_includes/tocs/docs-development.html b/docs/src/_includes/tocs/docs-development.html index d5d21264..882a4df9 100644 --- a/docs/src/_includes/tocs/docs-development.html +++ b/docs/src/_includes/tocs/docs-development.html @@ -1,24 +1,24 @@
- {% for chapter in site.data.developmentChapters %} -
  {{ chapter.label }}
- <{{ chapter.listType }} class="toc-body"> - {% for topic in chapter.topics %} -
  • - {% if chapter.label != page.chapter or topic.label != page.topic %} - {{topic.label}} - {% else %} - {{topic.label}} - - {% endif %} -
    • - {% endfor %} - - {% if forloop.last == false %} -
    - {% endif %} - {% endfor %} + {% for chapter in site.data.developmentChapters %} +
      {{ chapter.label }}
    + <{{ chapter.listType }} class="toc-body"> + {% for topic in chapter.topics %} +
  • + {% if chapter.label != page.chapter or topic.label != page.topic %} + {{topic.label}} + {% else %} + {{topic.label}} + + {% endif %} +
    • + {% endfor %} + + {% if forloop.last == false %} +
    + {% endif %} + {% endfor %}
    \ No newline at end of file diff --git a/docs/src/_includes/tocs/docs-user-guide.html b/docs/src/_includes/tocs/docs-user-guide.html index 6345c7c7..48f9fe40 100644 --- a/docs/src/_includes/tocs/docs-user-guide.html +++ b/docs/src/_includes/tocs/docs-user-guide.html @@ -1,21 +1,21 @@
    - {% for chapter in site.data.userGuideTopics %} -
      {{ chapter.label }}
    - <{{ chapter.listType }} class="toc-body"> - {% for topic in chapter.topics %} -
  • - {% if topic.label != page.topic %} - {{topic.label}} - {% else %} - {{topic.label}} - - {% endif %} -
    • - {% endfor %} - - {% endfor %} + {% for chapter in site.data.userGuideTopics %} +
      {{ chapter.label }}
    + <{{ chapter.listType }} class="toc-body"> + {% for topic in chapter.topics %} +
  • + {% if topic.label != page.topic %} + {{topic.label}} + {% else %} + {{topic.label}} + + {% endif %} +
    • + {% endfor %} + + {% endfor %}
    \ No newline at end of file diff --git a/docs/src/_layouts/chapter.html b/docs/src/_layouts/chapter.html index a7173cf7..4649d8db 100644 --- a/docs/src/_layouts/chapter.html +++ b/docs/src/_layouts/chapter.html @@ -3,26 +3,26 @@ --- {% comment %} Get filename of TOC constructor {% endcomment %} - {% capture tocFile %}{{ page.section | slugify }}-{{ page.subsection | slugify }}.html{% endcapture %} + {% capture tocFile %}{{ page.section | slugify }}-{{ page.subsection | slugify }}.html{% endcapture %}
    -
    - -
    - {% include footer.html %} -
    -
    -
    - -
    +
    + +
    + {% include footer.html %} +
    +
    +
    + +
    \ No newline at end of file diff --git a/docs/src/_layouts/contact.html b/docs/src/_layouts/contact.html index e964bdb6..d326df3e 100644 --- a/docs/src/_layouts/contact.html +++ b/docs/src/_layouts/contact.html @@ -3,18 +3,18 @@ ---
    -
    - - -
    -
    -
    - {{ content }} -
    -
    +
    + + +
    +
    +
    + {{ content }} +
    +
    \ No newline at end of file diff --git a/docs/src/_layouts/default.html b/docs/src/_layouts/default.html index b43209b5..25650a0f 100644 --- a/docs/src/_layouts/default.html +++ b/docs/src/_layouts/default.html @@ -1,26 +1,24 @@ - - {% include head.html %} - - - -
    - - -
    - {{ content | replace: '

  • ', '

  • ' }} -
    • -
    - + + {% include head.html %} + + + +
    + + +
    + {{ content | replace: '

  • ', '

  • ' }} +
    • +
    + \ No newline at end of file diff --git a/docs/src/_layouts/image.html b/docs/src/_layouts/image.html index 646b4744..1b902bb7 100644 --- a/docs/src/_layouts/image.html +++ b/docs/src/_layouts/image.html @@ -1,13 +1,13 @@ - - {% include head.html %} - - -
    -
    - {{ content }} -
    -
    - + + {% include head.html %} + + +
    +
    + {{ content }} +
    +
    + \ No newline at end of file diff --git a/docs/src/_layouts/section.html b/docs/src/_layouts/section.html index d8aae327..2e474009 100644 --- a/docs/src/_layouts/section.html +++ b/docs/src/_layouts/section.html @@ -3,17 +3,17 @@ ---
    -
    - -
    - {% include footer.html %} -
    -
    -
    -
    - {{ content }} -
    -
    +
    + +
    + {% include footer.html %} +
    +
    +
    +
    + {{ content }} +
    +
    \ No newline at end of file diff --git a/docs/src/_layouts/submenu.html b/docs/src/_layouts/submenu.html index a38d0e6c..6d09ec81 100644 --- a/docs/src/_layouts/submenu.html +++ b/docs/src/_layouts/submenu.html @@ -3,20 +3,20 @@ ---
    -
    - -
    - {% include footer.html %} -
    -
    -
    -
    - {{ content }} -
    -
    +
    + +
    + {% include footer.html %} +
    +
    +
    +
    + {{ content }} +
    +
    \ No newline at end of file diff --git a/docs/src/_sass/_code-block.scss b/docs/src/_sass/_code-block.scss index 71d91009..c3436d46 100644 --- a/docs/src/_sass/_code-block.scss +++ b/docs/src/_sass/_code-block.scss @@ -4,14 +4,14 @@ $block-bg-color: #eee; $txt-block-border: #ccc; code-block { - background: $block-bg-color; - border: 1px solid $txt-block-border; + background: $block-bg-color; + border: 1px solid $txt-block-border; border-radius: 2px; - -moz-border-radius: 2px; + -moz-border-radius: 2px; -webkit-border-radius: 2px; font-family: $family-code; font-size: 15px; - line-height: 1.2em !important; + line-height: 1.2em !important; margin: 10px 0 16px 0; overflow: auto; padding: 0 10px 0 10px; diff --git a/docs/src/contact/sidebar.html b/docs/src/contact/sidebar.html index afaeca2f..1cb70175 100644 --- a/docs/src/contact/sidebar.html +++ b/docs/src/contact/sidebar.html @@ -1,6 +1,6 @@ -
    - -
    Aperture Tiles is a creation of Uncharted Software Inc.
    -
    +
    + +
    Aperture Tiles is a creation of Uncharted Software Inc.
    +
    \ No newline at end of file diff --git a/docs/src/css/all.scss b/docs/src/css/all.scss index 8588fd0f..55e3bc4a 100644 --- a/docs/src/css/all.scss +++ b/docs/src/css/all.scss @@ -41,474 +41,474 @@ $map-border: #987; @import "code-block"; body { - font-family: $family-standard; - font-size: 100%; - line-height: 1.4em; - background: $site-bg-color; - background-image: $site-bg-img; - margin: 0; - padding: 15px 0 0; - color: $color-body; + font-family: $family-standard; + font-size: 100%; + line-height: 1.4em; + background: $site-bg-color; + background-image: $site-bg-img; + margin: 0; + padding: 15px 0 0; + color: $color-body; } ul { - padding-left: 30px; - margin-top: 6px; - &.table { - margin: 0; - padding-left: 26.25px; - } + padding-left: 30px; + margin-top: 6px; + &.table { + margin: 0; + padding-left: 26.25px; + } }​ ol { - padding-left: 30px; - margin-top: 6px; + padding-left: 30px; + margin-top: 6px; }​ li { - margin: 0 0 6px 0; - &.toc-item { - margin: 0 0 3px 0; - } + margin: 0 0 6px 0; + &.toc-item { + margin: 0 0 3px 0; + } } .methodDetail { - margin: 0; - padding-left: 0; + margin: 0; + padding-left: 0; } #MethodDetail { - margin: 0; - padding-left: 0; + margin: 0; + padding-left: 0; } @mixin heading($font-size, $margin-bottom, $color) { - color: $color; - font-family: $family-standard; - font-size: $font-size; - font-weight: normal; - margin-bottom: $margin-bottom; + color: $color; + font-family: $family-standard; + font-size: $font-size; + font-weight: normal; + margin-bottom: $margin-bottom; } h1 { - @include heading(32px,25px,$color-h1); - padding-top: 30px; - .tagline { - font-size: 18px; - color: $color-tagline; - } + @include heading(32px,25px,$color-h1); + padding-top: 30px; + .tagline { + font-size: 18px; + color: $color-tagline; + } } h2 { - @include heading(24px,10px,$color-sub-h1); - margin-top: 30px; - border-bottom-color: $txt-block-border; - border-bottom-style: solid; - border-bottom-width: 1px; - line-height: 33px; - &.sectionTitle { - font-size: 22px; - color: $color-api-section-title; - margin-top: 10px; - padding: 0 10px 10px 0; - } + @include heading(24px,10px,$color-sub-h1); + margin-top: 30px; + border-bottom-color: $txt-block-border; + border-bottom-style: solid; + border-bottom-width: 1px; + line-height: 33px; + &.sectionTitle { + font-size: 22px; + color: $color-api-section-title; + margin-top: 10px; + padding: 0 10px 10px 0; + } } h3 { - @include heading(21.33px,9px,$color-sub-h1); - &.context-info-title { - color: $color-api-sidebar-link; - &:hover { - color: $color-api-sidebar-link-hover; - } - } - &.sectionTitle { - font-size: 20px; - color: $color-api-section-title; - margin-top: 10px; - padding: 0 10px 10px 0; - } + @include heading(21.33px,9px,$color-sub-h1); + &.context-info-title { + color: $color-api-sidebar-link; + &:hover { + color: $color-api-sidebar-link-hover; + } + } + &.sectionTitle { + font-size: 20px; + color: $color-api-section-title; + margin-top: 10px; + padding: 0 10px 10px 0; + } } h4 { - @include heading(18.66px,0px,$color-sub-h1); + @include heading(18.66px,0px,$color-sub-h1); } h5 { - @include heading(16px,0px,$color-sub-h1); + @include heading(16px,0px,$color-sub-h1); } h6 { - @include heading(14.66px,0px,$color-sub-h1); - &.fine { - margin-top: 0; - } - &.procedure { - font-style: italic; - margin-top: 16px; - } + @include heading(14.66px,0px,$color-sub-h1); + &.fine { + margin-top: 0; + } + &.procedure { + font-style: italic; + margin-top: 16px; + } } a { - text-decoration: none; - &:link,&:active,&:visited { - color: $color-body-link; - } - &:hover,&:focus { - color: $color-body-link-hover; - } - &.download-link { - background-image: url("../img/resources/download.png"); - background-position: left; - background-repeat: no-repeat; - padding: 6px 0px 6px 30px; - } - &.context-info-title { - color: $color-api-sidebar-link; - &:hover { - color: $color-api-sidebar-link-hover; - } - } - &:hover.context-info-title { - color: $color-api-sidebar-link-hover; - } + text-decoration: none; + &:link,&:active,&:visited { + color: $color-body-link; + } + &:hover,&:focus { + color: $color-body-link-hover; + } + &.download-link { + background-image: url("../img/resources/download.png"); + background-position: left; + background-repeat: no-repeat; + padding: 6px 0px 6px 30px; + } + &.context-info-title { + color: $color-api-sidebar-link; + &:hover { + color: $color-api-sidebar-link-hover; + } + } + &:hover.context-info-title { + color: $color-api-sidebar-link-hover; + } } img { - border-style: none; - &.home-image { - margin: 20px 0 4px; - border: 1px solid $img-border; - } - &.context-logo { - margin: 20px 0 4px; - max-width: 236px; - } - &.screenshot { - margin: 20px auto 4px; - display: block; - } - &.procedure-screenshot { - margin: 6px auto 6px; - display: block; - } - &.table { - margin: 0px auto 4px; - display: block; - } - &.vid-thumbnail { - @extend .screenshot; - border: 1px solid $img-border; - } - &.thumbs { - position: relative; - width: 120px; - height: 100px; - border: 1px solid $img-border; - border-radius: 2px; - cursor: pointer; - &:hover{ - border: 1px solid $img-hover-border; - } - } + border-style: none; + &.home-image { + margin: 20px 0 4px; + border: 1px solid $img-border; + } + &.context-logo { + margin: 20px 0 4px; + max-width: 236px; + } + &.screenshot { + margin: 20px auto 4px; + display: block; + } + &.procedure-screenshot { + margin: 6px auto 6px; + display: block; + } + &.table { + margin: 0px auto 4px; + display: block; + } + &.vid-thumbnail { + @extend .screenshot; + border: 1px solid $img-border; + } + &.thumbs { + position: relative; + width: 120px; + height: 100px; + border: 1px solid $img-border; + border-radius: 2px; + cursor: pointer; + &:hover{ + border: 1px solid $img-hover-border; + } + } } code{ - font-family: $family-code; - font-size: 0.9em; - &.example { - @extend code-block; - display: block; - } + font-family: $family-code; + font-size: 0.9em; + &.example { + @extend code-block; + display: block; + } } pre { - margin: 10px 0 10px 0; - &.code { - @extend code-block; - } + margin: 10px 0 10px 0; + &.code { + @extend code-block; + } } #content-strip { - position: relative; - max-width: 1000px; - margin: 0 auto; + position: relative; + max-width: 1000px; + margin: 0 auto; } .toc-header { - font-size: 16px; - font-weight: bold; - margin: 0 0 5px 0; + font-size: 16px; + font-weight: bold; + margin: 0 0 5px 0; } .toc { - margin: 20px 0 10px 0; - padding: 5px 30px 5px 0px; - position: static; + margin: 20px 0 10px 0; + padding: 5px 30px 5px 0px; + position: static; } #header { - position: absolute; - background: $header-bg-color; - left: 0px; - right: 0px; - top: 0px; - height: 60px; - z-index: -1; + position: absolute; + background: $header-bg-color; + left: 0px; + right: 0px; + top: 0px; + height: 60px; + z-index: -1; } #header-logo { - position: absolute; + position: absolute; } #header-menu { - position: absolute; - right: 0px; - font-family: $family-standard; - font-size: 14px; - margin: 5px 0 20px; - a { - color: $color-main-nav; - line-height: 28px; - padding: 5px 6px 15px 6px; - text-decoration: none; - text-transform: uppercase; - &.selected { - border-bottom: 3px solid $color-main-nav; - } - &:hover { - color: $color-main-nav-hover; - } - } + position: absolute; + right: 0px; + font-family: $family-standard; + font-size: 14px; + margin: 5px 0 20px; + a { + color: $color-main-nav; + line-height: 28px; + padding: 5px 6px 15px 6px; + text-decoration: none; + text-transform: uppercase; + &.selected { + border-bottom: 3px solid $color-main-nav; + } + &:hover { + color: $color-main-nav-hover; + } + } } #submenu { - height: 60px; - margin-bottom: 40px; - .selected { - background-color: $submenu-bg-selected; - color: white; - } + height: 60px; + margin-bottom: 40px; + .selected { + background-color: $submenu-bg-selected; + color: white; + } } .submenu-item { - cursor: pointer; - float: left; - &:link, &:active, &:visited { - color: $color-body; - } - &:hover { - color: $color-main-nav-hover; - } + cursor: pointer; + float: left; + &:link, &:active, &:visited { + color: $color-body; + } + &:hover { + color: $color-main-nav-hover; + } } .submenu-item-background { - background: $submenu-bg-color; - border-radius: 50%; - -webkit-border-radius: 50%; - color: $color-submenu-nav-num; - float: left; - margin-right: 15px; - padding: 16px; + background: $submenu-bg-color; + border-radius: 50%; + -webkit-border-radius: 50%; + color: $color-submenu-nav-num; + float: left; + margin-right: 15px; + padding: 16px; } .submenu-item-label { - float: left; - font-size: 17px; - font-weight: bold; - margin: 15px 15px 0 0; - text-transform: uppercase; - white-space: nowrap; + float: left; + font-size: 17px; + font-weight: bold; + margin: 15px 15px 0 0; + text-transform: uppercase; + white-space: nowrap; } .submenu-item-number { - font-family: $family-standard; - font-size: 15px; - font-weight: bold; - height: 20px; - width: 20px; - text-align: center; + font-family: $family-standard; + font-size: 15px; + font-weight: bold; + height: 20px; + width: 20px; + text-align: center; } #map { - background-color: $map-bg; - border: 1px solid $map-border; + background-color: $map-bg; + border: 1px solid $map-border; } #all { - position: absolute; - width: 1000px; - top: 70px; - bottom: 0px; - padding-top: 10px; + position: absolute; + width: 1000px; + top: 70px; + bottom: 0px; + padding-top: 10px; } #content { - position: relative; - font-size: 18px; - line-height: 1.4em; - padding-bottom: 10px; + position: relative; + font-size: 18px; + line-height: 1.4em; + padding-bottom: 10px; } .content-context { - padding-left: 20px; - position: absolute; - width: 280px; - font-size: 14px; - margin-top: -20px; + padding-left: 20px; + position: absolute; + width: 280px; + font-size: 14px; + margin-top: -20px; } .uncharted-footer { - padding-top: 28px; - max-width: 150px; + padding-top: 28px; + max-width: 150px; } .footer-padding { - padding-bottom: 20px; + padding-bottom: 20px; } .toc-body { - line-height: 23px; - margin: 0 0 3px; + line-height: 23px; + margin: 0 0 3px; }​ .context-info { - background: $context-info-bg; - border: solid 1px $txt-block-border; - border-radius: 5px; - -moz-border-radius: 5px; - -webkit-border-radius: 5px; - left: -10px; - margin-bottom: 10px; - overflow: hidden; - padding: 10px 15px 25px 15px; - position: relative; + background: $context-info-bg; + border: solid 1px $txt-block-border; + border-radius: 5px; + -moz-border-radius: 5px; + -webkit-border-radius: 5px; + left: -10px; + margin-bottom: 10px; + overflow: hidden; + padding: 10px 15px 25px 15px; + position: relative; } #map { - background-color: $map-bg; - border: 1px solid $map-border; + background-color: $map-bg; + border: 1px solid $map-border; } .content-body { - text-align: justify; - padding: 0 30px; - margin: 41px 0 0 310px; - font-size: 16px; - background: $content-body-bg-color; - border: 1px solid $txt-block-border; - line-height: 1.4em; + text-align: justify; + padding: 0 30px; + margin: 41px 0 0 310px; + font-size: 16px; + background: $content-body-bg-color; + border: 1px solid $txt-block-border; + line-height: 1.4em; } .content-image { - text-align: center; - padding: 0 30px; - margin: 41px; - font-size: 16px; - background: $content-body-bg-color; - border: 1px solid $txt-block-border; - line-height: 1.4em; - display: inline-block; + text-align: center; + padding: 0 30px; + margin: 41px; + font-size: 16px; + background: $content-body-bg-color; + border: 1px solid $txt-block-border; + line-height: 1.4em; + display: inline-block; } .git { - display: none; + display: none; } .content-example { - font-size: 18px; - margin-left: 200px; - padding-left: 30px; - text-align: justify; + font-size: 18px; + margin-left: 200px; + padding-left: 30px; + text-align: justify; } #infoTxt { - font-size: 16px; - line-height: 1.2em; + font-size: 16px; + line-height: 1.2em; } .disclaimer { - color: $color-disclaimer; + color: $color-disclaimer; } sup, sub { - height: 0; - line-height: 1; - position: relative; - vertical-align: baseline; + height: 0; + line-height: 1; + position: relative; + vertical-align: baseline; } sup { - bottom: 1ex; + bottom: 1ex; } sub { - top: .5ex; + top: .5ex; } .props { - @extend code-block; - font-family: $family-standard; + @extend code-block; + font-family: $family-standard; } .innerProps { } dl { - &.detailList { - margin: 7px 0 0 0; - padding: 0 28px 0 0; - } - - &.detailList dt { - font-weight: bold; - font-family: $family-code; - color: #420; - } - - &.detailList dt.heading { - font-family: $family-standard; - font-size: 14px; - color: #6A1; - margin: 6px 0; - } + &.detailList { + margin: 7px 0 0 0; + padding: 0 28px 0 0; + } + + &.detailList dt { + font-weight: bold; + font-family: $family-code; + color: #420; + } - &.inheritsList dd + dt { - margin-top: 10px; - } + &.detailList dt.heading { + font-family: $family-standard; + font-size: 14px; + color: #6A1; + margin: 6px 0; + } - &.inheritsList dd { - display: inline; - } + &.inheritsList dd + dt { + margin-top: 10px; + } + + &.inheritsList dd { + display: inline; + } } dt { - font-size: 14px; - margin: 14px 0 0; + font-size: 14px; + margin: 14px 0 0; } dd { - font-size: 14px; - padding: 0 0 0 28px; - margin: 0 0 14px; + font-size: 14px; + padding: 0 0 0 28px; + margin: 0 0 14px; } .light { - color: #666; + color: #666; } .fixedFont { - font-family: $family-code; + font-family: $family-code; } th { - color: rgb(102, 170, 17); - font-size: 14px; - font-weight: normal; - line-height: 19.6px; - padding: 10px; - margin: 0; + color: rgb(102, 170, 17); + font-size: 14px; + font-weight: normal; + line-height: 19.6px; + padding: 10px; + margin: 0; } table.summaryTable { @@ -528,102 +528,102 @@ table.summaryTable { } td { - &.attributes { - @include table-cell(normal,normal,left); - } - &.property { - @include table-cell(bold,normal,left); - } - &.value { - @include table-cell(normal,italic,left); - } - &.description { - @include table-cell(normal,normal,left); - } - &.caption { - font-size: 14px; - font-weight: bold; - text-align: center; - } - &.nameDescription { - vertical-align: text-top; - } + &.attributes { + @include table-cell(normal,normal,left); + } + &.property { + @include table-cell(bold,normal,left); + } + &.value { + @include table-cell(normal,italic,left); + } + &.description { + @include table-cell(normal,normal,left); + } + &.caption { + font-size: 14px; + font-weight: bold; + text-align: center; + } + &.nameDescription { + vertical-align: text-top; + } } div.description { - padding: 0 10px 10px 10px; - font-size: 14px; - line-height: 19.6px; + padding: 0 10px 10px 10px; + font-size: 14px; + line-height: 19.6px; } .version { - display: block; - text-align: center; - font-size: 14px; + display: block; + text-align: center; + font-size: 14px; } .fine-print { - font-size: 14px; - margin-top: 0; + font-size: 14px; + margin-top: 0; } .procedure-text { - padding-left: 13px; + padding-left: 13px; } .list-paragraph { - margin: 6px 0; + margin: 6px 0; } .list-paragraph-2 { - padding-left: 40px; - margin: 6px 0; + padding-left: 40px; + margin: 6px 0; } hr { - border-style: solid; - border-top-width: 1px; - border-bottom-width: 0; - border-color: $txt-block-border; - margin: 18px 0 15px 0; + border-style: solid; + border-top-width: 1px; + border-bottom-width: 0; + border-color: $txt-block-border; + margin: 18px 0 15px 0; } .right { - float: right; + float: right; } .edit { - font-size: 14px; - padding-top: 30px; - margin-bottom: 25px; - a { - color: #777; - &:hover { - color: $color-body-link-hover; - } - } + font-size: 14px; + padding-top: 30px; + margin-bottom: 25px; + a { + color: #777; + &:hover { + color: $color-body-link-hover; + } + } } .copyright { - margin-top: 0; + margin-top: 0; } figure { - margin: 0; + margin: 0; } .attribution { - font-size: 15px; - font-style: italic; - text-align: center; - margin: 25px auto; - color: $color-sub-h1; - a { - font-style: normal; - font-weight: bold; - color: $color-h1; - &:hover { - color: #4e443c; - } - } + font-size: 15px; + font-style: italic; + text-align: center; + margin: 25px auto; + color: $color-sub-h1; + a { + font-style: normal; + font-weight: bold; + color: $color-h1; + &:hover { + color: #4e443c; + } + } } \ No newline at end of file diff --git a/docs/src/css/syntax.scss b/docs/src/css/syntax.scss index 2c288048..ad0eaa34 100644 --- a/docs/src/css/syntax.scss +++ b/docs/src/css/syntax.scss @@ -9,7 +9,7 @@ $literal-color: #0000FF; $generic-change-color: #000000; .highlight { - @extend code-block; + @extend code-block; } .highlight .hll { background-color: #ffffcc } .highlight .c { color: $comment-color; font-style: italic } /* Comment */ diff --git a/docs/src/demos/sidebar.html b/docs/src/demos/sidebar.html index 04f15589..df66e9d1 100644 --- a/docs/src/demos/sidebar.html +++ b/docs/src/demos/sidebar.html @@ -6,36 +6,36 @@ -
    - Twitter Aperture Tile Demo -
    Twitter big data heatmap
    -
    +
    + Twitter Aperture Tile Demo +
    Twitter big data heatmap
    +
    -
    - Twitter Aperture Tile Demo -
    Twitter heatmap with topics of interest and trend aggregate markers
    -
    +
    + Twitter Aperture Tile Demo +
    Twitter heatmap with topics of interest and trend aggregate markers
    +
    -
    - Twitter Aperture Tile Demo -
    Twitter heatmap with top hashtags and sentiment analysis aggregate markers
    -
    +
    + Twitter Aperture Tile Demo +
    Twitter heatmap with top hashtags and sentiment analysis aggregate markers
    +
    -
    - Bitcoin Aperture Tile Demo -
    Bitcoin exploratory data analysis cross-plot of time vs. amount
    -
    +
    + Bitcoin Aperture Tile Demo +
    Bitcoin exploratory data analysis cross-plot of time vs. amount
    +
    -
    - Julia Set Aperture Tile Demo -
    Julia Set Quick Start example of a fractal dataset
    -
    +
    + Julia Set Aperture Tile Demo +
    Julia Set Quick Start example of a fractal dataset
    +
    \ No newline at end of file diff --git a/docs/src/docs/development/advanced/custom-tiling/readme.md b/docs/src/docs/development/advanced/custom-tiling/readme.md index bc66775f..22061622 100644 --- a/docs/src/docs/development/advanced/custom-tiling/readme.md +++ b/docs/src/docs/development/advanced/custom-tiling/readme.md @@ -23,9 +23,9 @@ If your source data is not character delimited or if it contains non-numeric fie - Combine multiple records that fall in the same tile bin - Read and write to the tile set - [Binning Your Data](#binning-your-data) by specifying how to: - - Parse your source data - - Transform the parsed data into the Avro tile set - - Store the Avro tile set to your preferred location + - Parse your source data + - Transform the parsed data into the Avro tile set + - Store the Avro tile set to your preferred location Once you have created the required components for each process, you can run your custom binner to create the Avro tile set you will use in your application. @@ -68,7 +68,7 @@ A code example is shown in line 40 of [TwitterTopicBinningAnalytic.scala](https: ```scala extends BinningAnalytic[Map[String, TwitterDemoTopicRecord], - JavaList[TwitterDemoTopicRecord]] + JavaList[TwitterDemoTopicRecord]] ``` The processing type is a *map* that adds all similar Twitter message topic records together, while the binning type is a *list* that contains only the topics with the highest counts. @@ -79,7 +79,7 @@ The Binning Analytic should also describe how to convert the processing type int ```scala def finish (value: Map[String, TwitterDemoTopicRecord]): JavaList[TwitterDemoTopicRecord] = - value.values.toList.sortBy(-_.getCountMonthly()).slice(0, 10).asJava + value.values.toList.sortBy(-_.getCountMonthly()).slice(0, 10).asJava ``` The **finish** function: @@ -99,10 +99,10 @@ The Binning Analytic defines how data is aggregated. For example, lines 42-47 of ```scala def aggregate (a: Map[String, TwitterDemoTopicRecord], - b: Map[String, TwitterDemoTopicRecord]): Map[String, TwitterDemoTopicRecord] = { - a ++ b.map{case (k, v) => - k -> a.get(k).map(TwitterDemoTopicRecord.addRecords(_, v)).getOrElse(v) - } + b: Map[String, TwitterDemoTopicRecord]): Map[String, TwitterDemoTopicRecord] = { + a ++ b.map{case (k, v) => + k -> a.get(k).map(TwitterDemoTopicRecord.addRecords(_, v)).getOrElse(v) + } } ``` @@ -112,14 +112,14 @@ Lines 85-93 of [TwitterTopicBinner.scala](https://github.com/unchartedsoftware/a ```scala val minAnalysis: - AnalysisDescription[TileData[JavaList[TwitterDemoTopicRecord]], - List[TwitterDemoTopicRecord]] = - new TwitterTopicListAnalysis(new TwitterMinRecordAnalytic) + AnalysisDescription[TileData[JavaList[TwitterDemoTopicRecord]], + List[TwitterDemoTopicRecord]] = + new TwitterTopicListAnalysis(new TwitterMinRecordAnalytic) val maxAnalysis: - AnalysisDescription[TileData[JavaList[TwitterDemoTopicRecord]], - List[TwitterDemoTopicRecord]] = - new TwitterTopicListAnalysis(new TwitterMaxRecordAnalytic) + AnalysisDescription[TileData[JavaList[TwitterDemoTopicRecord]], + List[TwitterDemoTopicRecord]] = + new TwitterTopicListAnalysis(new TwitterMaxRecordAnalytic) ``` Standard Bin Analytics are available in the [Analytics.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/analytics/Analytics.scala) file in [tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/analytics/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/analytics). @@ -207,19 +207,19 @@ Lines 149 - 164 in [TwitterTopicBinner.scala](https://github.com/unchartedsoftwa ```scala val data = rawDataWithTopics.mapPartitions(i => - { - val recordParser = new TwitterTopicRecordParser(endTimeSecs) - i.flatMap(line => - { - try { - recordParser.getRecordsByTopic(line) - } catch { - // Just ignore bad records, there aren't many - case _: Throwable => Seq[((Double, Double), Map[String, TwitterDemoTopicRecord])]() - } - } - ) - } + { + val recordParser = new TwitterTopicRecordParser(endTimeSecs) + i.flatMap(line => + { + try { + recordParser.getRecordsByTopic(line) + } catch { + // Just ignore bad records, there aren't many + case _: Throwable => Seq[((Double, Double), Map[String, TwitterDemoTopicRecord])]() + } + } + ) + } ).map(record => (record._1, record._2, dataAnalytics.map(_.convert(record)))) data.cache ``` @@ -230,168 +230,164 @@ Lines 191 - 199 of [TwitterTopicBinner.scala](https://github.com/unchartedsoftwa ```scala val tiles = binner.processDataByLevel(data, - new CartesianIndexScheme, - new TwitterTopicBinningAnalytic, - tileAnalytics, - dataAnalytics, - tilePyramid, - levelSet, - xBins=1, - yBins=1) + new CartesianIndexScheme, + new TwitterTopicBinningAnalytic, + tileAnalytics, + dataAnalytics, + tilePyramid, + levelSet, + xBins=1, + yBins=1) ``` **processDataByLevel** is defined on line 231 in the [RDDBinner.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/RDDBinner.scala) file in [tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling). It accepts the following properties: -
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    PropertyRequired?Description
    dataYesDistributed collection of (index, record) pairs as described above.
    indexSchemeYesConverts the index to a set of x/y coordinates that can be plotted. When using a CartesianIndexScheme, the coordinates are taken as given. -
    binAnalyticYesBinning Analytic that, as described above, defines how to aggregate two records, convert them into the form written and determine the extrema of the dataset. -
    tileAnalyticsNoAnalytics used to perform custom aggregations on tile data (e.g., get the minimum and maximum values) and write them to the metadata by level. -
    dataAnalyticsNoAnalytics used to perform custom aggregations on raw data that would otherwise be lost by the processing type (e.g., recording the maximum individual value) and write them to the metadata by level. -
    tileSchemeYes - Projection used to transform from the raw data index into tiles and bins. Two types are predefined: - - -
    levelsYesSpecifies which levels to process at the same time. It is generally recommended you process levels 1-9 together, then run additional levels one at a time afterward. This typically makes effective use of system resources. -
    xBinsNoNumber of bins on the x-axis. Defaults to 256 -
    yBinsNoNumber of bins on the y-axis. Defaults to 256 -
    consolidationPartitionsNoNumber of reducers to use when aggregating data records into bins and tiles. Alter if you encounter problems with the tiling job due to lack of resources. Defaults to the same number of partitions as the original dataset. -
    tileTypeNoSpecifies how data should be stored: sparse or dense. If not specified, a heuristic will use the optimal type for a double-valued tile. For significantly larger-valued types, sparse is recommended. -
    -
    - -### Writing Tiles ### - -Lines 200 - 207 of [TwitterTopicBinner.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/TwitterTopicBinner.scala) specify how to write the tiles created from your transformed data. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    PropertyRequired?Description
    dataYesDistributed collection of (index, record) pairs as described above.
    indexSchemeYesConverts the index to a set of x/y coordinates that can be plotted. When using a CartesianIndexScheme, the coordinates are taken as given.
    binAnalyticYesBinning Analytic that, as described above, defines how to aggregate two records, convert them into the form written and determine the extrema of the dataset.
    tileAnalyticsNoAnalytics used to perform custom aggregations on tile data (e.g., get the minimum and maximum values) and write them to the metadata by level.
    dataAnalyticsNoAnalytics used to perform custom aggregations on raw data that would otherwise be lost by the processing type (e.g., recording the maximum individual value) and write them to the metadata by level.
    tileSchemeYes + Projection used to transform from the raw data index into tiles and bins. Two types are predefined: + +
    levelsYesSpecifies which levels to process at the same time. It is generally recommended you process levels 1-9 together, then run additional levels one at a time afterward. This typically makes effective use of system resources.
    xBinsNoNumber of bins on the x-axis. Defaults to 256
    yBinsNoNumber of bins on the y-axis. Defaults to 256
    consolidationPartitionsNoNumber of reducers to use when aggregating data records into bins and tiles. Alter if you encounter problems with the tiling job due to lack of resources. Defaults to the same number of partitions as the original dataset.
    tileTypeNoSpecifies how data should be stored: sparse or dense. If not specified, a heuristic will use the optimal type for a double-valued tile. For significantly larger-valued types, sparse is recommended.
    + +### Writing Tiles ### + +Lines 199 - 206 of [TwitterTopicBinner.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/TwitterTopicBinner.scala) specify how to write the tiles created from your transformed data. ```scala tileIO.writeTileSet(tilePyramid, - pyramidId, - tiles, - new TwitterTopicAvroSerializer(CodecFactory.bzip2Codec()), - tileAnalytics, - dataAnalytics, - pyramidName, - pyramidDescription) + pyramidId, + tiles, + new TwitterTopicAvroSerializer(CodecFactory.bzip2Codec()), + tileAnalytics, + dataAnalytics, + pyramidName, + pyramidDescription) ``` **tileIO.writeTileSet** is defined on line 173 in the [RDDBinner.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/RDDBinner.scala) file in [tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling). It accepts the following properties: -
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    PropertyRequired?Description
    tileSchemeYesType of projection built from the set of bins and levels. Must match the tileScheme specified in binner.processDataByLevel.
    writeLocationYes - ID to apply to the tile set when writing it. Value will vary depending on where you write your tile set: -
      -
    • For the local filesystem: the base directory into which to write the tiles. -
    • For HBase: the name of the table to write. -
    -
    tilesYesBinned dataset produced by binner.processDataByLevel.
    serializerYesSerializer that determines how to read and write to the tile set.
    tileAnalyticsNoAnalytics used to perform custom aggregations on tile data (e.g., get the minimum and maximum values) and write them to the metadata by level.
    dataAnalyticsNoAnalytics used to perform custom aggregations on raw data that would otherwise be lost by the processing type (e.g., recording the maximum individual value) and write them to the metadata by level.
    nameYesName of the finished pyramid. Stored in the tile metadata.
    descriptionYesDescription of the finished pyramid. Stored in the tile metadata.
    -
    \ No newline at end of file + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    PropertyRequired?Description
    tileSchemeYesType of projection built from the set of bins and levels. Must match the tileScheme specified in binner.processDataByLevel.
    writeLocationYes + ID to apply to the tile set when writing it. Value will vary depending on where you write your tile set: +
      +
    • For the local filesystem: the base directory into which to write the tiles.
      • +
    • For HBase: the name of the table to write.
      • +
    +
    tilesYesBinned dataset produced by binner.processDataByLevel.
    serializerYesSerializer that determines how to read and write to the tile set.
    tileAnalyticsNoAnalytics used to perform custom aggregations on tile data (e.g., get the minimum and maximum values) and write them to the metadata by level.
    dataAnalyticsNoAnalytics used to perform custom aggregations on raw data that would otherwise be lost by the processing type (e.g., recording the maximum individual value) and write them to the metadata by level.
    nameYesName of the finished pyramid. Stored in the tile metadata.
    descriptionYesDescription of the finished pyramid. Stored in the tile metadata.
    \ No newline at end of file diff --git a/docs/src/docs/development/advanced/graph-tiling/readme.md b/docs/src/docs/development/advanced/graph-tiling/readme.md index 41e9d5d1..f58f70c4 100644 --- a/docs/src/docs/development/advanced/graph-tiling/readme.md +++ b/docs/src/docs/development/advanced/graph-tiling/readme.md @@ -42,89 +42,85 @@ Aperture Tiles requires graph data to be in comma- or tab-delimited format (CSV) - Use the following command line syntax: - ```bash - java GraphParseApp –in source.graphml -out output.csv -longIDs true - –nAttr nAttr1, nAttr2 -eAttr eAttr1, eAttr2 -nCoordAttr NO - ``` - - Where: - -
    - -
    + ```bash + java GraphParseApp –in source.graphml -out output.csv -longIDs true + –nAttr nAttr1, nAttr2 -eAttr eAttr1, eAttr2 -nCoordAttr NO + ``` + + Where: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    ArgumentRequired?Description
    -inYesPath and filename of GraphML input file.
    -outYesPath and filename of tab-delimited output file.
    -longIDsNoIndicates whether nodes should be assigned a unique long ID (true) regardless of the ID format in the original file. This ID convention is needed for data processing with Spark's GraphX library. Defaults to false.
    -nAttrNoList of node attributes to parse. Enter as a list of attribute ID tags separated by commas. Defaults to all node attributes.
    -eAttrNoList of edge attributes to parse. Enter as a list of attribute ID tags separated by commas. Defaults to all edge attributes.
    -nCoordAttrNoNode attributes to use for node co-ordinates. Enter as a list of attribute ID tags separated by commas. Defaults to NO, which indicates no co-ordinate data should be associated with nodes.
    #### Output #### The GraphParseApp outputs a file that contains tab-delimited data. The first column denotes whether each record is a *node* or an *edge* object. -
    - -
    + + + + + + + + + + + + + +
    Node object columnsEdge object columns
    +
      +
    • Long ID identifier
      • +
    • Original string name
      • +
    • Any additional node attributes in the source file or just those passed in with the -nAttr argument
      • +
    +     		+
      +
    • ID of the source node
      • +
    • ID of the destination node
      • +
    • Any additional edge attributes in the source file or just those passed in with the -eAttr argument
      • +
    +
    GraphParseApp also creates a README file containing column labels for all node and edge records in the CSV file. @@ -159,38 +155,41 @@ java GraphParseApp –in -out -longIDs true GraphParseApp then outputs a file containing records for all of your nodes with the following format: -
    - -
    - -## Hierarchical Clustering ## + + + + + + + + + + + + + + + + +
    Node columns
    node0n0rhrostralmiddlefrontal10
    + + + + + + + + + + + + + + + +
    Edge columns
    edge36105436432219
    + +## Hierarchical Clustering ## CSV datasets can be hierarchically clustered using the [GraphClusterApp](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/graph/cluster/GraphClusterApp.scala) Scala application in [com.oculusinfo.tilegen.graph.cluster](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/graph/cluster). GraphClusterApp groups nodes into communities using Louvain community detection based on modularity-maximization. @@ -204,117 +203,117 @@ GraphClusterApp's implementation of the Louvain clustering algorithm uses Sotera
    To execute the GraphClusterApp and hierarchically cluster your data
    -- Use the following command line syntax: - - ```bash - spark-submit --class com.oculusinfo.tilegen.graph.cluster.GraphClusterApp - -source -output - -onlyEdges false -nID 1 -nAttr 2 -eSrcID 1 -eDstID 2 - -eWeight 3 -spark local -sparkhome /opt/spark -user - ``` - - Where: - -
    - -
    +- Use the following command line syntax: + + ```bash + spark-submit --class com.oculusinfo.tilegen.graph.cluster.GraphClusterApp + -source -output + -onlyEdges false -nID 1 -nAttr 2 -eSrcID 1 -eDstID 2 + -eWeight 3 -spark local -sparkhome /opt/spark -user + ``` + + Where: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    ArgumentRequired?Description
    -sourceYesHDFS location of the input data.
    -outputYesHDFS location to which to save clustered results.
    -onlyEdgesNoIndicates whether the source data contains only edges (*true*). Defaults to *false*.
    -partsNoNumber of partitions into which to break up the source dataset. Defaults to value chosen automatically by Spark.
    -pNoAmount of parallelism for Spark-based data processing of source data. Defaults to value chosen automatically by Spark.
    -dNoSource dataset delimiter. Defaults to tab-delimited.
    -progMinNoPercent of nodes that must change communities for the algorithm to consider progress relative to total vertices in a level. Defaults to 0.15.
    -progCountNoNumber of times the algorithm can fail to make progress before exiting. Defaults to 1.
    -nIDWhen ‑onlyEdges = falseNumber of the column in the raw data that contains the node IDs. Note that IDs must be of type long.
    -nAttrNoColumn numbers in the raw data that contain additional node metadata that should be parsed and saved with cluster results. Individual attribute tags should be separated by commas.
    -eSrcIDYesNumber of the column in the raw data that contains the edge source IDs. Note that IDs must be of type long.
    -eDstIDYesNumber of the column in the raw data that contains the edge destination IDs. Note that IDs must be of type long.
    -eWeightNoNumber of the column in the raw data that contains the edge weights. Defaults to -1, meaning that no edge weighting is used.
    -sparkYesSpark master location.
    -sparkhomeYesSpark HOME location.
    -userNoSpark/Hadoop username associated with the Spark job.
    #### Input #### GraphClusterApp accepts two types of graph data formats: -- **Node and edge tab-delimited data**, where the first column contains the keyword *node* or *edge* -- **Edge-only tab-delimited data** with different columns for source ID, destination ID and edge weight (*optional*).

    In this case, all nodes will be inferred internally from the parsed edges, but no node attributes or metadata will be associated with the clustered nodes or communities.

    +- **Node and edge tab-delimited data**, where the first column contains the keyword *node* or *edge* +- **Edge-only tab-delimited data** with different columns for source ID, destination ID and edge weight (*optional*). + +

    In this case, all nodes will be inferred internally from the parsed edges, but no node attributes or metadata will be associated with the clustered nodes or communities.

    **NOTE**: GraphClusterApp requires that node IDs and edge weights are of type *long*. @@ -324,36 +323,39 @@ Clustered results are stored sub-directories in the **-output** HDFS location. E Within each hierarchical level, clustered data is stored in the following tab-delimited format for nodes/communities and edges: -
    - -
    + + + + + + + + + + + + + + + + +
    Node columns
    nodeIDparent IDnumber of internal nodesnode degreemetadata (optional)
    + + + + + + + + + + + + + + + +
    Edge columns
    edgesrcIDdstIDedge weight
    The modularity (*q-value*) is also saved for each hierarchical level in a *_qvalues* sub-directory. @@ -370,13 +372,13 @@ spark-submit --class com.oculusinfo.tilegen.graph.cluster.GraphClusterApp In this case, the source dataset contains both nodes and edges (**-onlyEdges** *false*) with the following columns: -- For nodes: - - Column 1 contains the node IDs (**-nID** *1*) - - Column 2 contains node metadata (**-nAttr** *2*) -- For edges - - Column 1 contains source IDs (**-eSrcID** *1*) - - Column 2 contains destination IDs (**-eDstID** *2*) - - Column 3 contains the edge weights (**-eWeight** *3*) +- For nodes: + - Column 1 contains the node IDs (**-nID** *1*) + - Column 2 contains node metadata (**-nAttr** *2*) +- For edges + - Column 1 contains source IDs (**-eSrcID** *1*) + - Column 2 contains destination IDs (**-eDstID** *2*) + - Column 3 contains the edge weights (**-eWeight** *3*) Using the brain connectomics dataset as an example: @@ -398,106 +400,104 @@ The hierarchic force-directed algorithm runs in a distributed manner using Spark
    To execute the ClusterGraphLayoutApp and position your node/community data
    -- User the following command line syntax: - - ```bash - spark-submit --class com.oculusinfo.tilegen.graph.util.ClusteredGraphLayoutApp - -source -output - -i 1000 -maxLevel 4 -layoutLength 256 -nArea 45 - -border 2 -eWeight true -g 0 -spark local -sparkhome /opt/spark - -user - ``` - - Where: - -
    - -
    +- User the following command line syntax: + + ```bash + spark-submit --class com.oculusinfo.tilegen.graph.util.ClusteredGraphLayoutApp + -source -output + -i 1000 -maxLevel 4 -layoutLength 256 -nArea 45 + -border 2 -eWeight true -g 0 -spark local -sparkhome /opt/spark + -user + ``` + + Where: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    ArgumentRequired?Description
    -sourceYesHDFS location of the clustered input graph data.
    -outputYesHDFS location to which to save graph layout results.
    -partsNoNumber of partitions into which to break up the source dataset. Defaults to value chosen automatically by Spark.
    -pNoAmount of parallelism for Spark-based data processing of source data. Defaults to value chosen automatically by Spark.
    -dNoSource dataset delimiter. Defaults to tab-delimited.
    -iNoMaximum number of iterations for the force-directed algorithm. Defaults to 500.
    -maxLevelNoHighest cluster hierarchic level to use for determining the graph layout. Defaults to 0.
    -borderNoPercent of the parent bounding box to leave as whitespace between neighbouring communities during initial layout. Defaults to 2 percent.
    -layoutLengthNoDesired width/height of the total graph layout region. Defaults to 256.0.
    -nAreaNoArea of all node circles with a given parent community. Controls the amount of whitespace in the graph layout. Defaults to 30 percent.
    -eWeightNoIndicates whether to use edge weights to scale force-directed attraction forces (true). Defaults to false.
    -gNoAmount of gravitational force to use for force-directed layout to prevent outer nodes from spreading out too far. Defaults to 0 (no gravity).
    -sparkYesSpark master location.
    -sparkhomeYesSpark HOME location.
    -userNoSpark/Hadoop username associated with the Spark job.
    #### Input #### @@ -507,78 +507,84 @@ The *source* location should correspond to the root directory of hierarchically ClusterGraphLayoutApp requires the source clustered graph data to be a tab-delimited format analogous to that used by the Louvain Clustering application: -
    - -
    + + + + + + + + + + + + + + + + +
    Node columns
    nodeIDparent IDnumber of internal nodesnode degreemetadata (optional)
    + + + + + + + + + + + + + + + +
    Edge columns
    edgesrcIDdstIDedge weight
    #### Output #### Graph layout results are stored separately for each hierarchical level. Each level has the following tab-delimited format: -
    - -
    + + + + + + + + + + + + + + + + + + + + +
    Node columns
    nodeIDXY coordsradiusparent IDparent XY coordsparent XY radiusnum internal nodesdegreemetadata (optional)
    + + + + + + + + + + + + + + + + + + +
    Edge columns
    edgesrcIDsrc XY coordsdstIDdest XY coordsedge weightisInterCommunityEdge
    **NOTE**: The final edge column will be *0* for intra-community edges (both endpoints have the same parent community) or *1* for inter-community edges. @@ -595,11 +601,11 @@ Consider a graph dataset clustered up to hierarchy 4. The expected directory str ```text ../ - /level_4/ - /level_3/ - /level_2/ - /level_1/ - /level_0/ + /level_4/ + /level_3/ + /level_2/ + /level_1/ + /level_0/ ``` The following command line syntax defines a layout for the clustered graph dataset with: @@ -806,108 +812,102 @@ A list of BD file parameters for parsing each of these stats is given below. The following parameters specify the column number of key graph community/node attributes: -
    - -
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    ArgumentDescription
    oculus.binning.graph.x.indexx-axis coordinate
    oculus.binning.graph.y.indexy-axis coordinate
    oculus.binning.graph.id.indexLong ID
    oculus.binning.graph.r.indexRadius
    oculus.binning.graph.numnodes.indexNumber of nodes
    oculus.binning.graph.degree.indexDegree
    oculus.binning.graph.metadata.indexMetadata
    oculus.binning.graph.parentID.indexLong ID of the parent community
    The following parameters specify the column number of key parent community attributes: -
    - -
    + + + + + + + + + + + + + + + + + + + + + + + + + +
    ArgumentDescription
    oculus.binning.graph.parentID.indexLong ID
    oculus.binning.graph.parentR.indexRadius
    oculus.binning.graph.parentX.indexx-axis coordinate
    oculus.binning.graph.parentY.indexy-axis coordinate
    It is possible to save stats on the 10 highest weighted edges incident on a given community using the following parameters. **NOTE**: If these parameters are excluded, no edge analytics information will be saved for each GraphAnalyticsRecord. -
    - -
    + + + + + + + + + + + + + + + + + + + + + +
    ArgumentDescription
    oculus.binning.graph.edge.srcID.indexSource ID of each graph edge
    oculus.binning.graph.edges.dstID.indexDestination ID of each graph edge
    oculus.binning.graph.edges.weight.indexWeight of each graph edge. Defaults to 1 (unweighted).
    The number of communities to store per record can be tuned using the **oculus.binning.graph.maxcommunities** parameter (set to 25 by default). diff --git a/docs/src/docs/development/advanced/standard-tiling/readme.md b/docs/src/docs/development/advanced/standard-tiling/readme.md index 3c950f45..ca7bf27d 100644 --- a/docs/src/docs/development/advanced/standard-tiling/readme.md +++ b/docs/src/docs/development/advanced/standard-tiling/readme.md @@ -20,123 +20,117 @@ The following sections describe advanced properties available for [standard tili The **oculus.binning.parsing.<field>.fieldType** property indicates the type of values stored in a column that the CSVBinner will parse. Possible types include: -
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    ValueDescription
    doubleReal, double-precision floating-point numbers. This is the default type.
    constant0.0. The column does not need to exist.
    zero
    intIntegers
    longDouble-precision integers
    dateDates parsed and transformed into milliseconds since the standard Java start date (using SimpleDateFormatter). Expects the format yyMMddHHmm. Override the default format using the oculus.binning.parsing.<field>.dateFormat property.
    booleanBoolean values (e.g., true/false, yes/no)
    byteBytes
    shortShort integers
    floatFloating-point numbers -
    ipv4Contains an IP address treated as a four-digit base 256 number turned into an array of four bytes
    stringString value
    propertyMapProperty maps. Requires the presence of an additional set of propertyMap fields.
    -
    - -### dateFormat ### + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    ValueDescription
    doubleReal, double-precision floating-point numbers. This is the default type.
    constant0.0. The column does not need to exist.
    zero
    intIntegers
    longDouble-precision integers
    dateDates parsed and transformed into milliseconds since the standard Java start date (using SimpleDateFormatter). Expects the format yyMMddHHmm. Override the default format using the oculus.binning.parsing.<field>.dateFormat property.
    booleanBoolean values (e.g., true/false, yes/no)
    byteBytes
    shortShort integers
    floatFloating-point numbers
    ipv4Contains an IP address treated as a four-digit base 256 number turned into an array of four bytes
    stringString value
    propertyMapProperty maps. Requires the presence of an additional set of propertyMap fields.
    + +### dateFormat ### The following property overrides the default date format (*yyMMddHHmm*) expected when you specify a field type to be *date*. -
    - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.parsing.<field>.dateFormatSpecifies the date format expected by the corresponding field (e.g., yyyy-MM-dd HH:mm:ss).
    -
    - -### propertyMap ### + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.parsing.<field>.dateFormatSpecifies the date format expected by the corresponding field (e.g., yyyy-MM-dd HH:mm:ss).
    + +### propertyMap ### The following properties are required when you specify a field type to be *propertyMap*: -
    - - - - - - - - - - - - - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.parsing.<field>.propertyName of the property
    oculus.binning.parsing.<field>.propertyTypeEquivalent to fieldType
    oculus.binning.parsing.<field>.propertySeparatorCharacter or string used to separate properties
    oculus.binning.parsing.<field>.propertyValueSeparatorCharacter or string used to separate property keys from their values
    -
    + + + + + + + + + + + + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.parsing.<field>.propertyName of the property
    oculus.binning.parsing.<field>.propertyTypeEquivalent to fieldType
    oculus.binning.parsing.<field>.propertySeparatorCharacter or string used to separate properties
    oculus.binning.parsing.<field>.propertyValueSeparatorCharacter or string used to separate property keys from their values
    For example, consider the following propertyMap setup: @@ -153,62 +147,58 @@ In this case, a field value of `id=123;name=foo;description=bar` would yield the The following properties enable you modify the values of a field before it is used for binning: -
    - - - - - - - - - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.parsing.<field>.fieldScaling - How field values should be scaled. The default leaves values as they are. Other possibilities are: -
      -
    • log: take the log of the value. The base of the logarithm is taken from oculus.binning.parsing.<field>.fieldBase. -
    -
    oculus.binning.parsing.<field>.fieldBaseBase of the logarithm used to scale field values.
    oculus.binning.parsing.<field>.fieldAggregation - Method of aggregation used on values of field. Describes how values from multiple data points in the same bin should be aggregated together to create a single value for the bin. - -

    The default is addition. Other possible aggregation types are: -

      -
    • min: find the minimum value
      • -
    • max: find the maximum value
      • -
    • log: treat the number as a logarithmic value; aggregation of a and b is log_base(base^a+base^b). Base is taken from property oculus.binning.parsing.<field>.fieldBase, and defaults to e.
      • -
    -
    -
    - -## Consolidation Partitions ## + + + + + + + + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.parsing.<field>.fieldScaling + How field values should be scaled. The default leaves values as they are. Other possibilities are: +
      +
    • log: take the log of the value. The base of the logarithm is taken from oculus.binning.parsing.<field>.fieldBase.
      • +
    +
    oculus.binning.parsing.<field>.fieldBaseBase of the logarithm used to scale field values.
    oculus.binning.parsing.<field>.fieldAggregation + Method of aggregation used on values of field. Describes how values from multiple data points in the same bin should be aggregated together to create a single value for the bin. + +

    The default is addition. Other possible aggregation types are:

    +
      +
    • min: find the minimum value
      • +
    • max: find the maximum value
      • +
    • log: treat the number as a logarithmic value; aggregation of a and b is log_base(base^a+base^b). Base is taken from property oculus.binning.parsing.<field>.fieldBase, and defaults to e.
      • +
    +
    + +## Consolidation Partitions ## The **oculus.binning.consolidationPartitions** property controls the number of partitions into which data is consolidated when binning. -
    - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.consolidationPartitionsThe number of partitions into which to consolidate data when binning. If not included, Spark automatically selects the number of partitions.
    -
    \ No newline at end of file + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.consolidationPartitionsThe number of partitions into which to consolidate data when binning. If not included, Spark automatically selects the number of partitions.
    \ No newline at end of file diff --git a/docs/src/docs/development/api/annotation/readme.md b/docs/src/docs/development/api/annotation/readme.md index b3c9f9f5..3b27da44 100644 --- a/docs/src/docs/development/api/annotation/readme.md +++ b/docs/src/docs/development/api/annotation/readme.md @@ -18,45 +18,45 @@ The Aperture Tiles Annotation Service REST API is based on the Aperture Tiles Bi A single annotation received from a GET request or submitted in a write, modify or remove POST request.
    -

    Method Detail

    -
    -
    Parameters:
    - -
    - {Integer} level -
    -
    Native zoom level in which the annotation is inserted.
    - -
    - {Real or Array of Real} x -
    -
    Raw x data coordinate of the annotation. Either a Real or an Array containing two Real [min, max] for range-based annotations.
    - -
    - {Real or Array of Real} y -
    -
    Raw y data coordinate of the annotation. Either a Real or an Array containing two Real [min, max] for range-based annotations.
    - -
    - {JSON} range -
    -
    min and max key value pairs indicating the zoom-level aggregation range (inclusive). Given as Integers.
    - -
    - {String} group -
    -
    Exclusive groupings by which annotations can be filtered.
    - -
    - {JSON} data -
    -
    Additional information to be stored within the tile.
    - -
    - {JSON} certificate -
    -
    Contains an RFC4122 version 4 compliant UUID and a Unix timestamp. Returned by the server upon successful write and modify operations.
    -
    +

    Method Detail

    +
    +
    Parameters:
    + +
    + {Integer} level +
    +
    Native zoom level in which the annotation is inserted.
    + +
    + {Real or Array of Real} x +
    +
    Raw x data coordinate of the annotation. Either a Real or an Array containing two Real [min, max] for range-based annotations.
    + +
    + {Real or Array of Real} y +
    +
    Raw y data coordinate of the annotation. Either a Real or an Array containing two Real [min, max] for range-based annotations.
    + +
    + {JSON} range +
    +
    min and max key value pairs indicating the zoom-level aggregation range (inclusive). Given as Integers.
    + +
    + {String} group +
    +
    Exclusive groupings by which annotations can be filtered.
    + +
    + {JSON} data +
    +
    Additional information to be stored within the tile.
    + +
    + {JSON} certificate +
    +
    Contains an RFC4122 version 4 compliant UUID and a Unix timestamp. Returned by the server upon successful write and modify operations.
    +
    Example: @@ -70,7 +70,7 @@ Example: min: 0, max: 6 } - group: "Urgent", + group: "Urgent", data: { comment: … , author: … , @@ -85,54 +85,54 @@ Example: A collection of annotations from a single GET request. Contains all annotations (aggregated by bin-key) that pass through the current filter configuration.
    -

    Method Detail

    -
    -
    Parameters:
    - -
    - {JSON} title -
    -
    The respective tile index containing the level, x index and y index (where 0,0 is bottom left).
    - -
    - {JSON} annotations -
    -
    An array of annotations aggregated by bin-key.
    -
    +

    Method Detail

    +
    +
    Parameters:
    + +
    + {JSON} title +
    +
    The respective tile index containing the level, x index and y index (where 0,0 is bottom left).
    + +
    + {JSON} annotations +
    +
    An array of annotations aggregated by bin-key.
    +
    Example: ```json { - index : { - level: 7, - xIndex: 8, - yIndex: 4 - }, - tile : { - values: [ - { - value: [ - level: 7, - x: 73.35, - y: -125.6, - range: { - min: 0, - max: 9 - } - group: "Urgent", - data: { - comment: … , - author: … , - date: … - }, - certificate: { … } - ] - } - ] - }, - version: "v1.0" + index : { + level: 7, + xIndex: 8, + yIndex: 4 + }, + tile : { + values: [ + { + value: [ + level: 7, + x: 73.35, + y: -125.6, + range: { + min: 0, + max: 9 + } + group: "Urgent", + data: { + comment: … , + author: … , + date: … + }, + certificate: { … } + ] + } + ] + }, + version: "v1.0" } ``` @@ -141,20 +141,20 @@ Example: An Annotation Certificate is comprised of the UUID and timestamp from a successful POST request. The certificate is used during remove and modify operations to ensure client-server coherency in the event of live editing conflicts. When a certificate in a supplied annotation does not match that held by the server, the request is ignored and an error status is returned to the client.
    -

    Method Detail

    -
    -
    Parameters:
    - -
    - {String} uuid -
    -
    An RFC 4122 Version 4 compliant UUID generated by the server.
    - -
    - {String} timestamp -
    -
    A Unix timestamp generated by the server.
    -
    +

    Method Detail

    +
    +
    Parameters:
    + +
    + {String} uuid +
    +
    An RFC 4122 Version 4 compliant UUID generated by the server.
    + +
    + {String} timestamp +
    +
    A Unix timestamp generated by the server.
    +
    Example: @@ -167,7 +167,7 @@ Example: ## Web Services ## -Web service paths are relative to a common base URL. For example, assuming the hostname *localhost* and port *8080*, the URLs of the various services would be: +Web service paths are relative to a common base URL. For example, assuming the hostname *localhost* and port *8080*, the URLs of the various services would be: - `https://localhost:8080/{version}/annotation/{layer}/{uuid}/{level}/{x}/{y}.{ext}` - `https://localhost:8080/annotation/{layer}/{level}/{x}/{y}.{ext}` @@ -183,70 +183,70 @@ Web service paths are relative to a common base URL. For example, assuming the Returns an Annotation Tile for a specified filter configuration UUID, annotation layer, level and tile x and y.
    -

    Method Summary

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    AttributesDescription
    URL -
    /{version}/annotation/{layerId}/{level}/{x}/{y}/{z}.{ext}
    -
    Method -
    `GET`
    -
    Output MIME Type -
    application/json
    -
    Output -
    Annotation Tile
    -
    +

    Method Summary

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributesDescription
    URL +
    /{version}/annotation/{layerId}/{level}/{x}/{y}/{z}.{ext}
    +
    Method +
    `GET`
    +
    Output MIME Type +
    application/json
    +
    Output +
    Annotation Tile
    +
    -

    Method Detail

    -
    -
    URL Parameters:
    - -
    layer
    -
    Annotation layer ID.
    - -
    uuid
    -
    Filter configuration UUID. Submit default to use the default server filter configuration.
    - -
    level
    -
    Zoom level, where 0 is highest level of aggregation.
    - -
    x
    -
    Horizontal tile index, where 0 represents the leftmost column of tiles.
    - -
    y
    -
    Vertical tile index, where 0 represents the bottommost row of tiles.
    - -
    .ext
    -
    The extension of the annotation format. Only JSON is currently supported.
    -
    +

    Method Detail

    +
    +
    URL Parameters:
    + +
    layer
    +
    Annotation layer ID.
    + +
    uuid
    +
    Filter configuration UUID. Submit default to use the default server filter configuration.
    + +
    level
    +
    Zoom level, where 0 is highest level of aggregation.
    + +
    x
    +
    Horizontal tile index, where 0 represents the leftmost column of tiles.
    + +
    y
    +
    Vertical tile index, where 0 represents the bottommost row of tiles.
    + +
    .ext
    +
    The extension of the annotation format. Only JSON is currently supported.
    +
    Example request: @@ -259,34 +259,34 @@ Example response: ```json { - index : { - level: 7, - xIndex: 8, - yIndex: 4 - }, - tile : { - values: [ - { - value: [ - level: 7, - x: 73.35, - y: -125.6, - range: { - min: 0, - max: 9 - } - group: "Urgent", - data: { - comment: … , - author: … , - date: … - }, - certificate: { … } - ] - } - ] - }, - version: "v1.0" + index : { + level: 7, + xIndex: 8, + yIndex: 4 + }, + tile : { + values: [ + { + value: [ + level: 7, + x: 73.35, + y: -125.6, + range: { + min: 0, + max: 9 + } + group: "Urgent", + data: { + comment: … , + author: … , + date: … + }, + certificate: { … } + ] + } + ] + }, + version: "v1.0" } ``` @@ -295,110 +295,110 @@ Example response: Submits a new annotation to be written to the server. Upon success, a certificate containing the UUID and timestamp is returned to the client.
    -

    Method Summary

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    AttributesDescription
    URL -
    /{version}/annotation/{layerId}/
    -
    Method -
    `POST`
    -
    Output MIME Type -
    application/json
    -
    Output -
    AnnotationCertificate
    -
    +

    Method Summary

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributesDescription
    URL +
    /{version}/annotation/{layerId}/
    +
    Method +
    `POST`
    +
    Output MIME Type +
    application/json
    +
    Output +
    AnnotationCertificate
    +
    -

    Method Detail

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Request ParameterDescriptionData TypeSingle/MultipleRequired?
    type -
    "write"
    -     		-
    String
    -     		-
    Single
    -     		-
    Yes
    -
    layer -
    annotation layer id
    -     		-
    String
    -     		-
    Single
    -     		-
    Yes
    -
    annotation -
    annotation
    -     		-
    JSON
    -     		-
    Single
    -     		-
    Yes
    -
    +

    Method Detail

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Request ParameterDescriptionData TypeSingle/MultipleRequired?
    type +
    "write"
    +     		+
    String
    +     		+
    Single
    +     		+
    Yes
    +
    layer +
    annotation layer id
    +     		+
    String
    +     		+
    Single
    +     		+
    Yes
    +
    annotation +
    annotation
    +     		+
    JSON
    +     		+
    Single
    +     		+
    Yes
    +
    Example request: @@ -419,7 +419,7 @@ POST http://localhost:8080/instagram/rest/v1.0/annotation/parlor-annotations/ min: 0, max: 6 } - group: "Urgent", + group: "Urgent", data: { comment: … , author: … , @@ -435,7 +435,7 @@ Example response: { uuid: "f47ac10b-58cc-4372-a567-0e02b2c3d479", timestamp: "1401830862", - version: "v1.0" + version: "v1.0" } ``` @@ -444,110 +444,110 @@ Example response: Submits a modify request to the server. If the certificate in the annotation state is valid, the operation is processed and a new certificate is returned.
    -

    Method Summary

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    AttributesDescription
    URL -
    /{version}/annotation/{layerId}/
    -
    Method -
    `POST`
    -
    Output MIME Type -
    application/json
    -
    Output -
    AnnotationCertificate
    -
    +

    Method Summary

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributesDescription
    URL +
    /{version}/annotation/{layerId}/
    +
    Method +
    `POST`
    +
    Output MIME Type +
    application/json
    +
    Output +
    AnnotationCertificate
    +
    -

    Method Detail

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Request ParameterDescriptionData TypeSingle/MultipleRequired?
    type -
    "modify"
    -     		-
    String
    -     		-
    Single
    -     		-
    Yes
    -
    layer -
    annotation layer id
    -     		-
    String
    -     		-
    Single
    -     		-
    Yes
    -
    current -
    The annotation after modification. Must contain a valid certificate key.
    -     		-
    JSON
    -     		-
    Single
    -     		-
    Yes
    -
    +

    Method Detail

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Request ParameterDescriptionData TypeSingle/MultipleRequired?
    type +
    "modify"
    +     		+
    String
    +     		+
    Single
    +     		+
    Yes
    +
    layer +
    annotation layer id
    +     		+
    String
    +     		+
    Single
    +     		+
    Yes
    +
    current +
    The annotation after modification. Must contain a valid certificate key.
    +     		+
    JSON
    +     		+
    Single
    +     		+
    Yes
    +
    Example request: @@ -568,7 +568,7 @@ POST http://localhost:8080/instagram/rest/v1.0/annotation/parlor-annotations/ min: 0, max: 6 } - group: "Urgent", + group: "Urgent", data: { comment: … , author: … , @@ -588,7 +588,7 @@ Example response: { uuid: "f47ac10b-58cc-4372-a567-0e02b2c3d479", timestamp: "1401830862", - version: "v1.0" + version: "v1.0" } ``` @@ -597,110 +597,110 @@ Example response: Submits a removal request to the server. If the certificate is valid, the operation will be processed and a success status will be returned.
    -

    Method Summary

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    AttributesDescription
    URL -
    /{version}/annotation/{layerId}/
    -
    HTTP Method -
    `POST`
    -
    Output MIME Type -
    application/json
    -
    Output -
    JSON
    -
    +

    Method Summary

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributesDescription
    URL +
    /{version}/annotation/{layerId}/
    +
    HTTP Method +
    `POST`
    +
    Output MIME Type +
    application/json
    +
    Output +
    JSON
    +
    -

    Method Detail

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Request ParameterDescriptionData TypeSingle/MultipleRequired?
    type -
    "write"
    -     		-
    String
    -     		-
    Single
    -     		-
    Yes
    -
    layer -
    annotation layer id
    -     		-
    String
    -     		-
    Single
    -     		-
    Yes
    -
    certificate -
    The certificate of the annotation to be removed.
    -     		-
    JSON
    -     		-
    Single
    -     		-
    Yes
    -
    +

    Method Detail

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Request ParameterDescriptionData TypeSingle/MultipleRequired?
    type +
    "write"
    +     		+
    String
    +     		+
    Single
    +     		+
    Yes
    +
    layer +
    annotation layer id
    +     		+
    String
    +     		+
    Single
    +     		+
    Yes
    +
    certificate +
    The certificate of the annotation to be removed.
    +     		+
    JSON
    +     		+
    Single
    +     		+
    Yes
    +
    Example request: diff --git a/docs/src/docs/development/api/layer/readme.md b/docs/src/docs/development/api/layer/readme.md index 38cf3646..af08368d 100644 --- a/docs/src/docs/development/api/layer/readme.md +++ b/docs/src/docs/development/api/layer/readme.md @@ -14,14 +14,14 @@ layout: chapter All server configurations adhere to the following general format: ```json -[ - { - id: "...", - public: { - } - private :{ - } - } +[ + { + id: "...", + public: { + } + private :{ + } + } ] ``` @@ -31,22 +31,22 @@ The minimal required fields are: ```json [ - { - id: "...", - public: { - pyramid: { - type: "...." - } - } - private :{ - data : { - id: "...", - pyramidio : { - ... + { + id: "...", + public: { + pyramid: { + type: "...." + } + } + private :{ + data : { + id: "...", + pyramidio : { + ... } } - } - } + } + } ] ``` @@ -60,39 +60,39 @@ Return an object containing an array of: Layer information returned is the **public** node of the server-side configuration JSON. The layer IDs and metadata are appended to the returned JSON under the **id** and **meta** attributes, respectively.
    -

    Method Summary

    - - - - - - - - - - - - - - - - - - - - - - - - -
    AttributesDescription
    URL -
    /{version}/layers -
    /layers
    -
    Method -
    `GET`
    -
    Params -
    None
    -
    +

    Method Summary

    + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributesDescription
    URL +
    /{version}/layers +
    /layers
    +
    Method +
    `GET`
    +
    Params +
    None
    +
    Example request: @@ -105,30 +105,30 @@ Example response: ```json { - layers: [ - { - id: "instagram-heatmap", - pyramid: { - type: "WebMercator" - } - meta: { … }, - renderer : { - ramp: "spectral", - coarseness: 2 - }, - valueTransform: { - type: "log10" - } - }, - { - id: "instagram-keyword-heatmap", - pyramid: { - type: "WebMercator" - }, - meta: { … } - } - ], - version: "v1.0" + layers: [ + { + id: "instagram-heatmap", + pyramid: { + type: "WebMercator" + } + meta: { … }, + renderer : { + ramp: "spectral", + coarseness: 2 + }, + valueTransform: { + type: "log10" + } + }, + { + id: "instagram-keyword-heatmap", + pyramid: { + type: "WebMercator" + }, + meta: { … } + } + ], + version: "v1.0" } ``` @@ -137,39 +137,39 @@ Example response: Returns the requested layer information. Layer information returned is the **public** node of the server-side configuration JSON. The layer IDs and metadata are appended to the returned JSON under **id** and **meta** attributes, respectively.
    -

    Method Summary

    - - - - - - - - - - - - - - - - - - - - - - - - -
    AttributesDescription
    URL -
    /{version}/layers/{layerId} -
    /layers/{layerId}
    -
    Method -
    `GET`
    -
    Params -
    None
    -
    +

    Method Summary

    + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributesDescription
    URL +
    /{version}/layers/{layerId} +
    /layers/{layerId}
    +
    Method +
    `GET`
    +
    Params +
    None
    +
    Example request: @@ -182,21 +182,21 @@ Example response: ```json { - layer: { - id: "instagram-heatmap", - pyramid: { - type: "WebMercator" - } - meta: { … }, - renderer : { - ramp: "spectral", - coarseness: 2 - }, - valueTransform: { - type: "log10" - } - }, - version: "v1.0" + layer: { + id: "instagram-heatmap", + pyramid: { + type: "WebMercator" + } + meta: { … }, + renderer : { + ramp: "spectral", + coarseness: 2 + }, + valueTransform: { + type: "log10" + } + }, + version: "v1.0" } ``` @@ -205,39 +205,39 @@ Example response: Store a configuration state on the server that can be accessed at a later time. Returns the SHA-256 hash of the state. This SHA-256 hash can be used later as a 'state' query parameter. Hashes are deterministic and cachable.
    -

    Method Summary

    - - - - - - - - - - - - - - - - - - - - - - - - -
    AttributesDescription
    URL -
    /{version}/layers/{layerId}/states -
    layers/{layerId}/states
    -
    Method -
    `POST`
    -
    Params -
    None
    -
    +

    Method Summary

    + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributesDescription
    URL +
    /{version}/layers/{layerId}/states +
    layers/{layerId}/states
    +
    Method +
    `POST`
    +
    Params +
    None
    +
    Example request: @@ -248,11 +248,11 @@ POST http://localhost:8080/instagram/rest/v1.0//layer/instagram-heatmap/states ```json { - tileTransform: { - data: { - vars: [ … ] - } - } + tileTransform: { + data: { + vars: [ … ] + } + } } ``` @@ -260,50 +260,49 @@ Example response: ```json { - version: "v1.0", - state: "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" + version: "v1.0", + state: "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" } - ``` -## Get Configuration State ## +## Get Configuration State ## Returns a specified configured state stored on the server for a particular layer.
    -

    Method Summary

    - - - - - - - - - - - - - - - - - - - - - - - - -
    AttributesDescription
    URL -
    /{version}/layers/{layerId}/states/{stateId} -
    layers/{layerId}/states/{stateId}
    -
    HTTP Method -
    `GET`
    -
    Params -
    None
    -
    +

    Method Summary

    + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributesDescription
    URL +
    /{version}/layers/{layerId}/states/{stateId} +
    layers/{layerId}/states/{stateId}
    +
    HTTP Method +
    `GET`
    +
    Params +
    None
    +
    Example request: @@ -316,25 +315,25 @@ Example response: ```json { - state:{ - tileTransform:{ + state:{ + tileTransform:{ type:"identity" }, - pyramid:{ + pyramid:{ maxY:0, maxX:0, type:"WebMercator", minX:0, minY:0 }, - valueTransform:{ + valueTransform:{ min:5e-324, max:1.7976931348623157e+308, layerMax:0, layerMin:0, type:"linear" }, - renderer:{ + renderer:{ to:"0x000000", halign:0.5, from-alpha:-1, @@ -347,11 +346,11 @@ Example response: rangeMin:0, opacity:1, rangeMax:100, - components:[ + components:[ ], valign:0.5, - gradients:[ + gradients:[ ] } @@ -365,39 +364,39 @@ Example response: Returns all configured states stored on the server for a particular layer, including the default state.
    -

    Method Summary

    - - - - - - - - - - - - - - - - - - - - - - - - -
    AttributesDescription
    URL -
    /{version}/layers/{layerId}/states -
    layers/{layerId}/states
    -
    HTTP Method -
    `GET`
    -
    Params -
    None
    -
    +

    Method Summary

    + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributesDescription
    URL +
    /{version}/layers/{layerId}/states +
    layers/{layerId}/states
    +
    HTTP Method +
    `GET`
    +
    Params +
    None
    +
    Example request: @@ -410,87 +409,87 @@ Example response: ```json { - states:{ - default:{ - tileTransform:{ - type:"identity" - }, - pyramid:{ - maxY:0, - maxX:0, - type:"WebMercator", - minX:0, - minY:0 - }, - valueTransform:{ - min:5e-324, - max:1.7976931348623157e+308, - layerMax:0, - layerMin:0, - type:"linear" - }, - renderer:{ - to:"0x000000", - halign:0.5, - from-alpha:-1, - theme:"dark", - to-alpha:-1, - from:"0xffffff", - type:"heatmap", - ramp:"spectral", - coarseness:1, - rangeMin:0, - opacity:1, - rangeMax:100, - components:[ - - ], - valign:0.5, - gradients:[ - - ] - } - }, - e5f8e8aa55e008aeb9a3bbf40d93da4a5630112cf280e2f7e12245e219044031:{ - tileTransform:{ - type:"identity" - }, - pyramid:{ - maxY:0, - maxX:0, - type:"WebMercator", - minX:0, - minY:0 - }, - valueTransform:{ - min:5e-324, - max:1.7976931348623157e+308, - layerMax:0, - layerMin:0, - type:"linear" - }, - renderer:{ - to:"0x000000", - halign:0.5, - from-alpha:-1, - theme:"dark", - to-alpha:-1, - from:"0xffffff", - type:"heatmap", - ramp:"hot", - coarseness:1, - rangeMin:0, - opacity:1, - rangeMax:100, - components:[ - - ], - valign:0.5, - gradients:[ - - ] - } - } + states:{ + default:{ + tileTransform:{ + type:"identity" + }, + pyramid:{ + maxY:0, + maxX:0, + type:"WebMercator", + minX:0, + minY:0 + }, + valueTransform:{ + min:5e-324, + max:1.7976931348623157e+308, + layerMax:0, + layerMin:0, + type:"linear" + }, + renderer:{ + to:"0x000000", + halign:0.5, + from-alpha:-1, + theme:"dark", + to-alpha:-1, + from:"0xffffff", + type:"heatmap", + ramp:"spectral", + coarseness:1, + rangeMin:0, + opacity:1, + rangeMax:100, + components:[ + + ], + valign:0.5, + gradients:[ + + ] + } + }, + e5f8e8aa55e008aeb9a3bbf40d93da4a5630112cf280e2f7e12245e219044031:{ + tileTransform:{ + type:"identity" + }, + pyramid:{ + maxY:0, + maxX:0, + type:"WebMercator", + minX:0, + minY:0 + }, + valueTransform:{ + min:5e-324, + max:1.7976931348623157e+308, + layerMax:0, + layerMin:0, + type:"linear" + }, + renderer:{ + to:"0x000000", + halign:0.5, + from-alpha:-1, + theme:"dark", + to-alpha:-1, + from:"0xffffff", + type:"heatmap", + ramp:"hot", + coarseness:1, + rangeMin:0, + opacity:1, + rangeMax:100, + components:[ + + ], + valign:0.5, + gradients:[ + + ] + } + } }, version:"v1.0" } diff --git a/docs/src/docs/development/api/legend/readme.md b/docs/src/docs/development/api/legend/readme.md index 34f28849..617a44e0 100644 --- a/docs/src/docs/development/api/legend/readme.md +++ b/docs/src/docs/development/api/legend/readme.md @@ -14,32 +14,32 @@ layout: chapter Returns an image describing the values that a tile can take, with a specified orientation, width, and height.
    -

    Method Summary

    - - - - - - - - - - - - - - - - - - - -
    AttributesDescription
    URL -
    /{version}/legend/{layer} -
    /legend/{layer}
    -
    Method -
    `GET`
    -
    +

    Method Summary

    + + + + + + + + + + + + + + + + + + + +
    AttributesDescription
    URL +
    /{version}/legend/{layer} +
    /legend/{layer}
    +
    Method +
    `GET`
    +
    Example request: diff --git a/docs/src/docs/development/api/tile/readme.md b/docs/src/docs/development/api/tile/readme.md index efe9a3e3..862c6779 100644 --- a/docs/src/docs/development/api/tile/readme.md +++ b/docs/src/docs/development/api/tile/readme.md @@ -14,39 +14,39 @@ layout: chapter Returns a tile for a given layer at the specified level and index.
    -

    Method Summary

    - - - - - - - - - - - - - - - - - - - - - - - - -
    AttributesDescription
    URL -
    /{version}/tile/{layer}/{level}/{x}/{y}.{ext} -
    /tile/{layer}/{level}/{x}/{y}.{ext}
    -
    Method -
    `GET`
    -
    Params -
    Attributes to override in the "public" node of the server configuration JSON.
    -
    +

    Method Summary

    + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributesDescription
    URL +
    /{version}/tile/{layer}/{level}/{x}/{y}.{ext} +
    /tile/{layer}/{level}/{x}/{y}.{ext}
    +
    Method +
    `GET`
    +
    Params +
    Attributes to override in the "public" node of the server configuration JSON.
    +
    Example request: diff --git a/docs/src/docs/development/getting-started/installation/readme.md b/docs/src/docs/development/getting-started/installation/readme.md index 9b1db7c5..de38b0cc 100644 --- a/docs/src/docs/development/getting-started/installation/readme.md +++ b/docs/src/docs/development/getting-started/installation/readme.md @@ -30,78 +30,74 @@ Compatibility for Windows is available through [Cygwin](https://cygwin.com/) or Aperture Tiles requires the following third-party tools: -
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    ComponentRequiredNotes
    Languages - Scala v2.10.3 -
    Java (JDK v1.7+) -
    Cluster Computing Framework - Apache Spark -
    v1.3 -     		- Specify the version of Hadoop with which Spark will be working (if applicable). -

    Spark may cause class path issues if you compile from source code. We recommend using a pre-built Spark package.

    -
    Build Automation - Node.js - - The Node.js Windows installer has a known issue where it fails to install the following directory. To work around this issue, create the directory manually. -

    C:\Users\<UserName>\AppData\Roaming\npm

    -
    Web ServerApache Tomcat or JettyThe Tile Server and Tile Client, built using the Restlet web framework, require a servlet-compatible web server.
    -
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    ComponentRequiredNotes
    Languages + Scala v2.10.3 +
    Java (JDK v1.7+) +
    Cluster Computing Framework + Apache Spark +
    v1.3 +     		+ Specify the version of Hadoop with which Spark will be working (if applicable). +

    Spark may cause class path issues if you compile from source code. We recommend using a pre-built Spark package.

    +
    Build Automation + Node.js + + The Node.js Windows installer has a known issue where it fails to install the following directory. To work around this issue, create the directory manually. +

    C:\Users\<UserName>\AppData\Roaming\npm

    +
    Web ServerApache Tomcat or JettyThe Tile Server and Tile Client, built using the Restlet web framework, require a servlet-compatible web server.
    If you intend to work with datasets that cannot fit in the memory of a single machine or if you wish to avoid wait times, we recommend you also install the following tools to enable Aperture Tiles to work with particularly large datasets. -
    - - - - - - - - - - - - - -
    ComponentRequiredNotes
    Cluster Computing Framework - A Hadoop distribution: - - Some Hadoop distributions automatically install Apache Spark. Upgrade to v1.3 if the installation is older.
    -
    + + + + + + + + + + + + + +
    ComponentRequiredNotes
    Cluster Computing Framework + A Hadoop distribution: + + Some Hadoop distributions automatically install Apache Spark. Upgrade to v1.3 if the installation is older.
    Aperture Tiles Architecture Diagram @@ -115,15 +111,15 @@ The source code repository for Aperture Tiles is available on [GitHub](https://g 1. Execute the following command to clone the Aperture Tiles repository to an *aperture-tiles/* directory in your Git project folder: - ```bash - git clone https://github.com/unchartedsoftware/aperture-tiles.git - ``` + ```bash + git clone https://github.com/unchartedsoftware/aperture-tiles.git + ``` 2. Check out the *master* branch to work with the most recent stable release: - ```bash - git checkout master - ``` + ```bash + git checkout master + ``` ### Building the Project ### @@ -147,31 +143,29 @@ Aperture Tiles builds with [Gradle](https://gradle.org/).
    To install Gradle and build Aperture Tiles
    -- Execute the following command in your root Aperture Tiles directory: - - ```bash - gradlew install -PbuildType= - ``` - - Where: - -
    - - - - - - - - - - - -
    Sub-ProjectDescription
    buildType - A case in the build.gradle file that specifies which versions of Hadoop/HBase and Spark you are using (e.g., cdh5.1.2). -

    If you do not specify a buildType, the default value (cdh4.6.0) in aperture-tiles/gradle.properties is used.

    -
    -
    +- Execute the following command in your root Aperture Tiles directory: + + ```bash + gradlew install -PbuildType= + ``` + + Where: + + + + + + + + + + + + +
    Sub-ProjectDescription
    buildType + A case in the build.gradle file that specifies which versions of Hadoop/HBase and Spark you are using (e.g., cdh5.1.2). +

    If you do not specify a buildType, the default value (cdh5.4.7) in aperture-tiles/gradle.properties is used.

    +
    This will compile all the project components and install .jar files for each project into a local Gradle repository on your build machine. @@ -179,74 +173,72 @@ This will compile all the project components and install .jar files for each pro Aperture Tiles is made up of the following sub-projects: -
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Sub-ProjectDescription
    math-utilitiesBasic, underlying Java utilities (for angles, linear algebra and statistics) to aid in processing data.
    geometric-utilitiesAdvanced math utilities for processing geometry and geographic problems.
    binning-utilitiesBasic substrate of tiling, bin data structures, bin processing and basic bin storage classes.
    factory-utilitiesFactory system to allow construction and configuration of generic objects.
    tile-generationSpark-based tools to generate tile pyramids from raw data.
    tile-renderingLibrary that handles turning tile data into useful output formats, generally graphical.
    tile-serviceWeb service to serve tiles from tile pyramids to web clients.
    spark-tile-utilitiesOptional services for allowing the tile server to communicate directly with Spark.
    annotation-serviceOptional services for adding annotations to Aperture Tiles visualizations.
    tile-clientSimple web client to display tiles from tile pyramids.
    tile-packagingPackaged assembly of the tile generation service for the Quick Start example on this site.
    tile-examplesExample applications.
    gradleGradle build system support.
    docsSource files for the documentation on this website.
    -
    - -## Packaged Distributions ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Sub-ProjectDescription
    math-utilitiesBasic, underlying Java utilities (for angles, linear algebra and statistics) to aid in processing data.
    geometric-utilitiesAdvanced math utilities for processing geometry and geographic problems.
    binning-utilitiesBasic substrate of tiling, bin data structures, bin processing and basic bin storage classes.
    factory-utilitiesFactory system to allow construction and configuration of generic objects.
    tile-generationSpark-based tools to generate tile pyramids from raw data.
    tile-renderingLibrary that handles turning tile data into useful output formats, generally graphical.
    tile-serviceWeb service to serve tiles from tile pyramids to web clients.
    spark-tile-utilitiesOptional services for allowing the tile server to communicate directly with Spark.
    annotation-serviceOptional services for adding annotations to Aperture Tiles visualizations.
    tile-clientSimple web client to display tiles from tile pyramids.
    tile-packagingPackaged assembly of the tile generation service for the Quick Start example on this site.
    tile-examplesExample applications.
    gradleGradle build system support.
    docsSource files for the documentation on this website.
    + +## Packaged Distributions ## The Aperture Tiles packaged distributions are intended for use with the [Quick Start](../quick-start/) example workflow. These distributions require a subset of the complete installation [prerequisites](#prerequisites). @@ -269,11 +261,11 @@ The Aperture Tiles packaged distributions enable you to create a sample Aperture
    To download the packaged distributions
    -1. Download and save the following files: - - [Tile Generator](../../../../download/#tile-generator): Creates the Julia set data and generates a set of tiles - - [Tile Quick Start Application](../../../../download/#tile-quick-start-application): Serves as an example application that you can quickly deploy after minimal modification -2. Extract the contents of each file to your local machine. -3. See the [Quick Start](../quick-start/) topic for information on configuring each packaged distribution. +1. Download and save the following files: + - [Tile Generator](../../../../download/#tile-generator): Creates the Julia set data and generates a set of tiles + - [Tile Quick Start Application](../../../../download/#tile-quick-start-application): Serves as an example application that you can quickly deploy after minimal modification +2. Extract the contents of each file to your local machine. +3. See the [Quick Start](../quick-start/) topic for information on configuring each packaged distribution. ## Next Steps ## diff --git a/docs/src/docs/development/getting-started/quick-start/readme.md b/docs/src/docs/development/getting-started/quick-start/readme.md index f4acb039..35693f1f 100644 --- a/docs/src/docs/development/getting-started/quick-start/readme.md +++ b/docs/src/docs/development/getting-started/quick-start/readme.md @@ -60,63 +60,67 @@ A preconfigured base properties file is available in the Tile Generator utility.
    To edit the base properties file
    -1. Open the **julia-base.bd** file in the Tile Generator *examples/* folder. -2. Edit the **oculus.binning.source.location** property to specify the location of your Julia set data: - - For the local system: */data/julia* - - For HDFS: *hdfs://hadoop.example.com/data/julia* -3. Edit the following general output properties: -
    - - - - - - - - - - - - - - - -
    PropertyDescription
    oculus.tileio.typeSpecify whether the tiles should be saved locally (file) or to HBase (hbase). Local tile IO is supported only for standalone Spark installations.
    oculus.binning.nameSpecify the name of the output tile set. If you are writing to a file system, use a relative path instead of an absolute path. Use julia for this example.
    -
    -4. If you are using Hadoop/HDFS and HBase, edit the following HBase connection details: -
    - - - - - - - - - - - - - - - - - - - -
    PropertyDescription
    hbase.zookeeper.quorumZookeeper quorum location needed to connect to HBase.
    hbase.zookeeper.portPort through which to connect to zookeeper. Typically defaults to 2181.
    hbase.masterLocation of the HBase master to which to save the tiles.
    -
    -5. Save the **julia-base.bd** file. - -### Tiling Property File Configuration ### - -The Tile Generator utility also contains a tiling properties file (**julia-tiling.bd** in *examples/*). This file should not need to be edited. - -**NOTE**: For a typical Aperture Tiles project, you will need to edit this file to define the layout of the map/plot on which to project your data. For more information on these properties, see the [Standard Tiling Jobs](../../how-to/standard-tiling/) topic.. - -### Execution ### - -After you have edited the properties files, you can use the Tile Generator utility to create the Avro tile set. +1. Open the **julia-base.bd** file in the Tile Generator *examples/* folder. +2. Edit the **oculus.binning.source.location** property to specify the location of your Julia set data: + - For the local system: */data/julia* + - For HDFS: *hdfs://hadoop.example.com/data/julia* +3. Edit the following general output properties: + + + + + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.tileio.typeSpecify whether the tiles should be saved locally (file) or to HBase (hbase). Local tile IO is supported only for standalone Spark installations.
    oculus.binning.nameSpecify the name of the output tile set. If you are writing to a file system, use a relative path instead of an absolute path. Use julia for this example.
    + +4. If you are using Hadoop/HDFS and HBase, edit the following HBase connection details: + + + + + + + + + + + + + + + + + + + + + + +
    PropertyDescription
    hbase.zookeeper.quorumZookeeper quorum location needed to connect to HBase.
    hbase.zookeeper.portPort through which to connect to zookeeper. Typically defaults to 2181.
    hbase.masterLocation of the HBase master to which to save the tiles.
    + +5. Save the **julia-base.bd** file. + +### Tiling Property File Configuration ### + +The Tile Generator utility also contains a tiling properties file (**julia-tiling.bd** in *examples/*). You do not need to edit it. + +**NOTE**: For a typical Aperture Tiles project, you will edit the properties file to define the layout of the map/plot on which to project your data. For more information on these properties, see the [Standard Tiling Jobs](../../how-to/standard-tiling/) topic. + +### Execution ### + +After you edit the properties files, you can use the Tile Generator utility to create the Avro tile set.
    To execute the tile generation job
    @@ -148,42 +152,44 @@ For this example, you only need to edit the layer properties file if you saved y
    To edit the layer properties
    -1. Access the *WEB-INF/classes/layers/***julia-layer.json** file. -2. Make sure the **id** property under the *private* node matches the name given to the HBase table containing your Avro tiles (*julia.x.y.v*). -3. Clear the existing attributes under the **pyramidio** node and add the following HBase connection details: -
    - - - - - - - - - - - - - - - - - - - - - - - -
    PropertyValue
    typehbase
    hbase.zookeeper.quorumZookeeper quorum location needed to connect to HBase. For example: -
      -
    • my-zk-server1.example.com
      • -
    • my-zk-server2.example.com
      • -
    • my-zk-server3.example.com
      • -
    -
    hbase.zookeeper.portPort through which to connect to zookeeper. Typically defaults to 2181.
    hbase.masterLocation of the HBase master in which the tiles are saved (e.g., my-hbase-master.example.com:60000)
    -
    -4. Save the file. +1. Access the *WEB-INF/classes/layers/***julia-layer.json** file. +2. Make sure the **id** property under the *private* node matches the name given to the HBase table containing your tiles (*julia.x.y.v*). +3. Clear the existing attributes under the **pyramidio** node and add the following HBase connection details: + + + + + + + + + + + + + + + + + + + + + + + + + + +
    PropertyValue
    typehbase
    hbase.zookeeper.quorumZookeeper quorum location needed to connect to HBase. For example: +
      +
    • my-zk-server1.example.com
      • +
    • my-zk-server2.example.com
      • +
    • my-zk-server3.example.com
      • +
    +
    hbase.zookeeper.portPort through which to connect to zookeeper. Typically defaults to 2181.
    hbase.masterLocation of the HBase master in which the tiles are saved (e.g., my-hbase-master.example.com:60000)
    + +4. Save the file. For information on additional layer properties, see the *Layers* section of the [Tile Server](../../how-to/tile-server/#layers) topic. diff --git a/docs/src/docs/development/getting-started/tile-pyramid/readme.md b/docs/src/docs/development/getting-started/tile-pyramid/readme.md index a07f8c71..4198e832 100644 --- a/docs/src/docs/development/getting-started/tile-pyramid/readme.md +++ b/docs/src/docs/development/getting-started/tile-pyramid/readme.md @@ -26,30 +26,31 @@ The process of generating a set of Avro tiles from your raw source data is calle As shown in the following diagram, the tile generation process has five main stages: -1. **Raw Data Parsing**: Pass in your raw data and parse its structure.

    For standard tiling jobs, this is the main task; the rest of the applicable stages are kicked off automatically.

    +1. **Raw Data Parsing**: Pass in your raw data and parse its structure. +

    For standard tiling jobs, this is the main task; the rest of the applicable stages are kicked off automatically.

    2. **Bin Analytics**: Create summaries per tile that are partitioned into bins. -
      -
    1. Extract intermediate values from the records in your data (e.g., a count of values and their sum).
      2. -
    2. Map to one tile bin.
      3. -
    3. Aggregate all the values in each bin (e.g., divide sum by count to get mean value).
      4. -
    4. Output the final value written to each bin (e.g., mean value).
      5. -
    -

    Example Use: Create heatmaps distributed across each zoom level in your tile pyramid.

    +
      +
    1. Extract intermediate values from the records in your data (e.g., a count of values and their sum).
      2. +
    2. Map to one tile bin.
      3. +
    3. Aggregate all the values in each bin (e.g., divide sum by count to get mean value).
      4. +
    4. Output the final value written to each bin (e.g., mean value).
      5. +
    +

    Example Use: Create heatmaps distributed across each zoom level in your tile pyramid.

    3. **Data Analytics** (*Optional*) -
      -
    1. Extract intermediate values from the records in your data.
      2. -
    2. Map to one tile.
      3. -
    3. Aggregate all the values mapped to each tile (e.g., determine the maximum individual raw data value).
      4. -
    4. Output tile metadata values.
      5. -
    -

    Example Use: Create custom tile overlay analytics. Not part of the standard tile generation process.

    +
      +
    1. Extract intermediate values from the records in your data.
      2. +
    2. Map to one tile.
      3. +
    3. Aggregate all the values mapped to each tile (e.g., determine the maximum individual raw data value).
      4. +
    4. Output tile metadata values.
      5. +
    +

    Example Use: Create custom tile overlay analytics. Not part of the standard tile generation process.

    4. **Tile Analytics** (*Optional*) -
      -
    1. Extract additional data tile data.
      2. -
    2. Map to one tile.
      3. -
    3. Output tile metadata values.
      4. -
    -

    Example Use: Analyze completed tiles and store information to tile metadata (e.g., extract the minimum and maximum bin values, which are used to apply color ramps).

    +
      +
    1. Extract additional data tile data.
      2. +
    2. Map to one tile.
      3. +
    3. Output tile metadata values.
      4. +
    +

    Example Use: Analyze completed tiles and store information to tile metadata (e.g., extract the minimum and maximum bin values, which are used to apply color ramps).

    5. **Tile Creation**: Assemble the values generated by the various analytics into a completed tile pyramid, first in memory and then to HBase or the local file system. Tile Generation Process @@ -57,42 +58,40 @@ As shown in the following diagram, the tile generation process has five main sta
    You can execute the tile generation process on your source data in any of the following manners: -
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Job TypeDescriptionRequirements
    StandardUses the built-in CSVBinner tool, which automates generation of tiles that aggregate numeric data by summation or take the minimum or maximum value.A set of properties files that describe the source data to analyze and the job options to pass in.
    CustomUses a set of RDDBinner APIs for creating custom tile-based analytics on non-numeric or non-delimited data.Create custom code for some or all of the tile generation stages. See the Twitter Topics demo (tile-examples/twitter-topics/) for an example implementation of custom tiling.
    GraphUses the built-in CSVGraphBinner tool, which automates generation of tiles that aggregate node and edge data.A set of properties files that describe the source data to analyze and the job options to pass in.
    Third PartyUse third-party tools. This approach is not discussed below.Adhere to the Aperture Tiles Avro schema. Basic schema files are in the Aperture Tiles source code (binning-utilities/src/main/resources/).
    -
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Job TypeDescriptionRequirements
    StandardUses the built-in CSVBinner tool, which automates generation of tiles that aggregate numeric data by summation or take the minimum or maximum value.A set of properties files that describe the source data to analyze and the job options to pass in.
    CustomUses a set of RDDBinner APIs for creating custom tile-based analytics on non-numeric or non-delimited data.Create custom code for some or all of the tile generation stages. See the Twitter Topics demo (tile-examples/twitter-topics/) for an example implementation of custom tiling.
    GraphUses the built-in CSVGraphBinner tool, which automates generation of tiles that aggregate node and edge data.A set of properties files that describe the source data to analyze and the job options to pass in.
    Third PartyUse third-party tools. This approach is not discussed below.Adhere to the Aperture Tiles Avro schema. Basic schema files are in the Aperture Tiles source code (binning-utilities/src/main/resources/).
    Before you run a tiling job, make sure you meet all of the prerequisites listed in the following section. diff --git a/docs/src/docs/development/how-to/app-setup/readme.md b/docs/src/docs/development/how-to/app-setup/readme.md index 39c82935..e0b56c2a 100644 --- a/docs/src/docs/development/how-to/app-setup/readme.md +++ b/docs/src/docs/development/how-to/app-setup/readme.md @@ -19,56 +19,55 @@ The fastest way to create these components is to modify an existing example appl
    To begin configuring your Aperture Tiles application
    -1. Choose the one of the following demos to use as a template: -
    - - - - - - - - - - - - - - - - - - - - -
    DemoLocationDescription
    Julia Settile-examples/julia-demo/Server-rendered heatmap on a blank cross-plot baselayer
    Twitter Topicstile-examples/twitter-topics/Server-rendered heatmap with client-rendered word clouds on a geographic Google Maps baselayer
    -
    -2. Copy the folder of the demo you want and give it a unique name (e.g., *new‑project*). -3. Update the Gradle build file (*new-project/build.gradle*) to add or change the following fields: -
    - - - - - - - - - - - - - - - - - - - - - -
    FieldDescription
    descriptionProject description
    groupGroup ID
    versionProject version number
    -
    +1. Choose the one of the following demos to use as a template: + + + + + + + + + + + + + + + + + + + + + +
    DemoLocationDescription
    Julia Settile-examples/julia-demo/Server-rendered heatmap on a blank cross-plot baselayer
    Twitter Topicstile-examples/twitter-topics/Server-rendered heatmap with client-rendered word clouds on a geographic Google Maps baselayer
    + +2. Copy the folder of the demo you want and give it a unique name (e.g., *new‑project*). +3. Update the Gradle build file (*new-project/build.gradle*) to add or change the following fields: + + + + + + + + + + + + + + + + + + + + + + +
    FieldDescription
    descriptionProject description
    groupGroup ID
    versionProject version number
    ## Next Steps ## diff --git a/docs/src/docs/development/how-to/standard-tiling/readme.md b/docs/src/docs/development/how-to/standard-tiling/readme.md index 4d07e1f5..58253c0f 100644 --- a/docs/src/docs/development/how-to/standard-tiling/readme.md +++ b/docs/src/docs/development/how-to/standard-tiling/readme.md @@ -27,17 +27,17 @@ During the tiling job, the CSVBinner writes a set of Avro tile data files to the
    To execute the CSVBinner and run a standard tiling job
    -- Use the **spark-submit** script and pass in the names of the properties files you want to use. For example: +- Use the **spark-submit** script and pass in the names of the properties files you want to use. For example: - ```bash - spark-submit --class com.oculusinfo.tilegen.examples.apps.CSVBinner - lib/tile-generation-assembly.jar -d /data/twitter/dataset-base.bd - /data/twitter/dataset.lon.lat.bd - ``` + ```bash + spark-submit --class com.oculusinfo.tilegen.examples.apps.CSVBinner + lib/tile-generation-assembly.jar -d /data/twitter/dataset-base.bd + /data/twitter/dataset.lon.lat.bd + ``` - Where the `-d` switch specifies the base properties file path, and each subsequent file path specifies a tiling properties file. + Where the `-d` switch specifies the base properties file path, and each subsequent file path specifies a tiling properties file. -## Base Properties Files ## +## Base Properties Files ## The base properties file describes the tiling job, the systems on which it will run and the general characteristics of the source data. The following properties must be defined in the file: @@ -48,209 +48,205 @@ The base properties file describes the tiling job, the systems on which it will Additional properties are described in the advanced [Standard Tiling](../../advanced/standard-tiling/) topic. -### Source Location ### - -- Indicate where your source data is stored: -
    - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.source.location - Filename or directory containing the source data. If set to a directory, all of its contents will be ingested. Example values: -
      -
    • file://data/test.data
      • -
    • hdfs://hadoop-s1/data/julia/500by200
      • -
    • datasets/julia
      • -
        -
    -
    - -### Source Data Format ### - -1. Indicate how the columns in your source data are separated: -
    - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.parsing.separatorCharacter or string used to separate columns in the source data. Defaults to tab character (\t).
    -
    -2. Pass in the fields used to write your data to the tile pyramid. The tile generation job requires two types of fields: - - [Index](#index) fields indicate where on the map/plot your data points are located - - A [value](#value) field contains the value of the data points at the corresponding indexes - - For example: - - ```properties - # Index Fields - oculus.binning.parsing.longitude.index=0 - oculus.binning.parsing.longitude.fieldType=double - oculus.binning.parsing.latitude.index=1 - oculus.binning.parsing.latitude.fieldType=double - - # Value Field - oculus.binning.parsing.value.index=2 - oculus.binning.parsing.value.fieldType=double - ``` - - Where: - -
    - - - - - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.parsing.<field>.index - Column number of the described field in the source data files. This property is mandatory for every field type to be used. -

    NOTE: The oculus.binning.index.type you select in your Tiling properties file determines the number of index files you must include.

    -
    oculus.binning.parsing.<field>.fieldType - Type of value expected in the column specified by oculus.binning.parsing.<field>.index: - - - - - - - -
    -
      -
    • double (*default*)
      • -
    • constant
      • -
    • zero
      • -
    • int
      • -
    • long
      • -
    • date
      • -
    • boolean
      • -
    -     		-
      -
    • byte
      • -
    • short -
    • float -
    • ipv4
      • -
    • string
      • -
    • propertyMap
      • -
    -
    -
    For more information on the supported field types, see the advanced Standard Tiling topic. -
    -
    - -### Source Data Manipulation ### +### Source Location ### + +- Indicate where your source data is stored: + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.source.location + Filename or directory containing the source data. If set to a directory, all of its contents will be ingested. Example values: +
      +
    • file://data/test.data
      • +
    • hdfs://hadoop-s1/data/julia/500by200
      • +
    • datasets/julia
      • +
    +
    + +### Source Data Format ### + +1. Indicate how the columns in your source data are separated: + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.parsing.separatorCharacter or string used to separate columns in the source data. Defaults to tab character (\t).
    + +2. Pass in the fields used to write your data to the tile pyramid. The tile generation job requires two types of fields: + - [Index](#index) fields indicate where on the map/plot your data points are located + - A [value](#value) field contains the value of the data points at the corresponding indexes + + For example: + + ```properties + # Index Fields + oculus.binning.parsing.longitude.index=0 + oculus.binning.parsing.longitude.fieldType=double + oculus.binning.parsing.latitude.index=1 + oculus.binning.parsing.latitude.fieldType=double + + # Value Field + oculus.binning.parsing.value.index=2 + oculus.binning.parsing.value.fieldType=double + ``` + + Where: + + + + + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.parsing.<field>.index + Column number of the described field in the source data files. This property is mandatory for every field type to be used. +

    NOTE: The oculus.binning.index.type you select in your Tiling properties file determines the number of index files you must include.

    +
    oculus.binning.parsing.<field>.fieldType + Type of value expected in the column specified by oculus.binning.parsing.<field>.index: + + + + + + + +
    +
      +
    • double (*default*)
      • +
    • constant
      • +
    • zero
      • +
    • int
      • +
    • long
      • +
    • date
      • +
    • boolean
      • +
    +     		+
      +
    • byte
      • +
    • short
      • +
    • float
      • +
    • ipv4
      • +
    • string
      • +
    • propertyMap
      • +
    +
    +
    For more information on the supported field types, see the advanced Standard Tiling topic. +
    + +### Source Data Manipulation ### Additional properties are available for scaling fields logarithmically before they are used in the tile generation job. For more information, see the advanced Standard Tiling topic. -### Tile Storage ### - -1. Specify where to save your tile set: -
    - - - - - - - - - - - - - -
    PropertyDescription
    oculus.tileio.type - Location to which tiles are written: -
      -
    • file (default): Writes .avro files to the local file system. NOTE: If this type is used on a distributed cluster, the file is saved to the worker node, not the machine that initiates the tiling job.
      • -
    • hbase: Writes to HBase. For further HBase configuration properties, see the following step.
      • -
    -
    -
    -2. If you are saving your tile set to HBase, specify the connection properties. Otherwise, skip to the next step. - -
    - - - - - - - - - - - - - - - - - - - - - - - -
    PropertyDescription
    hbase.zookeeper.quorumZookeeper quorum location needed to connect to HBase.
    hbase.zookeeper.portPort through which to connect to zookeeper. Defaults to 2181.
    hbase.masterLocation of the HBase master to which to write tiles.
    -
    -3. Edit the tile set name and metadata values: -
    - - - - - - - - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.nameName (path) of the output data tile set pyramid. If you are writing to a file system, use a relative path instead of an absolute path. If you are writing to HBase, this is used as a table name. This name is also written to the tile set metadata and used as a plot label.
    oculus.binning.prefixOptional prefix to be added to the name of every pyramid location. Used to distinguish different tile generation runs. If not present, no prefix is used.
    oculus.binning.descriptionDescription to put in the tile metadata.
    -
    - -## Tiling Properties Files ## +### Tile Storage ### + +1. Specify where to save your tile set: + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.tileio.type + Location to which tiles are written: +
      +
    • file (default): Writes .avro files to the local file system. NOTE: If this type is used on a distributed cluster, the file is saved to the worker node, not the machine that initiates the tiling job.
      • +
    • hbase: Writes to HBase. For further HBase configuration properties, see the following step.
      • +
    +
    + +2. If you are saving your tile set to HBase, specify the connection properties. Otherwise, skip to the next step. + + + + + + + + + + + + + + + + + + + + + + + + +
    PropertyDescription
    hbase.zookeeper.quorumZookeeper quorum location needed to connect to HBase.
    hbase.zookeeper.portPort through which to connect to zookeeper. Defaults to 2181.
    hbase.masterLocation of the HBase master to which to write tiles.
    + +3. Edit the tile set name and metadata values: + + + + + + + + + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.nameName (path) of the output data tile set pyramid. If you are writing to a file system, use a relative path instead of an absolute path. If you are writing to HBase, this is used as a table name. This name is also written to the tile set metadata and used as a plot label.
    oculus.binning.prefixOptional prefix to be added to the name of every pyramid location. Used to distinguish different tile generation runs. If not present, no prefix is used.
    oculus.binning.descriptionDescription to put in the tile metadata.
    + +## Tiling Properties Files ## The tiling properties files define the tiling job parameters for each layer in your visual analytic, such as which fields to bin on and how values are binned: @@ -265,148 +261,145 @@ Additional properties are described in the advanced [Standard Tiling](../../adva The projection properties define the area on which your data points are plotted. -1. Choose the map or plot over which to project your data points: -
    - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.projection.type - Type of projection to use when binning data. Possible values are: -
      -
    • areaofinterest or EPSG:4326 (default) - Bin linearly over the whole range of values found.
      • -
    • webmercator, EPSG:900913 or EPSG:3857 - Web-mercator projection. Used for geographic values only.
      • -
    -
    -
    -2. If your projection type is *areaofinterest*, decide whether to manually set the bounds of the projection. -
    - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.projection.autobounds - Indicates how the minimum and maximum bounds should be set: -
      -
    • true (default) - Automatically
      • -
    • false - Manually
      • -
    - Note that this property is not applicable to webmercator projection. -
    -
    -3. If you chose to set the bounds manually, specify the minimum and maximum x- and y-axis values: -
    - - - - - - - - - - - - - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.projection.minxLowest x-axis value
    oculus.binning.projection.maxxHighest x-axis value
    oculus.binning.projection.minyLowest y-axis value
    oculus.binning.projection.maxyHighest y-axis value
    -
    - -### Index ### +1. Choose the map or plot over which to project your data points: + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.projection.type + Type of projection to use when binning data. Possible values are: +
      +
    • areaofinterest or EPSG:4326 (default) - Bin linearly over the whole range of values found.
      • +
    • webmercator, EPSG:900913 or EPSG:3857 - Web-mercator projection. Used for geographic values only.
      • +
    +
    + +2. If your projection type is *areaofinterest*, decide whether to manually set the bounds of the projection. + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.projection.autobounds + Indicates how the minimum and maximum bounds should be set: +
      +
    • true (default) - Automatically
      • +
    • false - Manually
      • +
    + Note that this property is not applicable to webmercator projection. +
    + +3. If you chose to set the bounds manually, specify the minimum and maximum x- and y-axis values: + + + + + + + + + + + + + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.projection.minxLowest x-axis value
    oculus.binning.projection.maxxHighest x-axis value
    oculus.binning.projection.minyLowest y-axis value
    oculus.binning.projection.maxyHighest y-axis value
    + +### Index ### The index properties specify the fields used to locate the binning value on the projection and map them to the corresponding tile bins. -1. Specify the index scheme: -
    - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.index.type - Scheme used to indicate how the binning values are mapped to the projection: -
      -
    • cartesian (default) - Cartesian (x/y) coordinates
      • -
    • ipv4 - IP address (v4)
      • -
    • timerange - Standard time range and cartesian point index
      • -
    -
    The scheme also determines the number of index fields you must specify. -
    -
    -2. Map the index fields you specified in your base properties file to the fields required by the scheme you selected. - -
    - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.index.field.<order>List of fields that satisfy the chosen index scheme: -
    -
    cartesian (default):
    -
    - oculus.binning.index.field.0=<x_Field> - oculus.binning.index.field.1=<y_Field> -
    - -
    ipv4:
    -
    oculus.binning.index.field.0=<IPv4_Field>
    - -
    timerange:
    -
    - oculus.binning.index.field.0=<time_Field> - oculus.binning.index.field.1=<x_Field> - oculus.binning.index.field.2=<y_Field> -
    -
    -
    -
    +1. Specify the index scheme: + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.index.type + Scheme used to indicate how the binning values are mapped to the projection: +
      +
    • cartesian (default) - Cartesian (x/y) coordinates
      • +
    • ipv4 - IP address (v4)
      • +
    • timerange - Standard time range and cartesian point index
      • +
    +
    The scheme also determines the number of index fields you must specify. +
    + +2. Map the index fields you specified in your base properties file to the fields required by the scheme you selected. + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.index.field.<order>List of fields that satisfy the chosen index scheme: +
    +
    cartesian (default):
    +
    + oculus.binning.index.field.0=<x_Field> + oculus.binning.index.field.1=<y_Field> +
    + +
    ipv4:
    +
    oculus.binning.index.field.0=<IPv4_Field>
    + +
    timerange:
    +
    + oculus.binning.index.field.0=<time_Field> + oculus.binning.index.field.1=<x_Field> + oculus.binning.index.field.2=<y_Field> +
    +
    +
    **NOTE**: An additional index scheme (*segment*) is available for tiling node and edge graph data. See the [Graph Tiling](../../advanced/graph-tiling/) topic for more information. @@ -414,60 +407,58 @@ The index properties specify the fields used to locate the binning value on the The value properties specify the field to use as the binning value and how multiple values in the same bin should be combined. -
    - - - - - - - - - - - - - - - - - - - - - - - - - -
    PropertyDescription
    oculus.binning.value.type - Specifies how to determine the values written to each bin: -
      -
    • count (default) - Count the number of records in a bin. If selected, no value field is required.
      • -
    • field - Perform an aggregation on the values of all records in a bin to create a single value for the bin
      • -
    • series - Save the values of all records in a bin to a dense array
      • -
    -
    oculus.binning.value.fieldField to use as the bin value. If no value field is provided, the tiling job will write the count of records in each bin as the bin value.
    oculus.binning.value.valueType - Type of values stored in the value field: -
      -
    • double (default) - Real, double-precision floating-point numbers
      • -
    • int - Integers
      • -
    • long - Double-precision integers
      • -
    • float - Floating-point numbers
      • -
    -
    oculus.binning.value.aggregation - Method of aggregation used on the values of all records in a bin when oculus.binning.value.type = field. Creates a single value for the bin: -
      -
    • sum (default)- Sum the numeric values of all records in the bin.
      • -
    • min - Select the minimum numeric value from the records in the bin.
      • -
    • max - Select the maximum numeric value from the records in the bin.
      • -
    • mean - Calculate the mean numeric value of all records in the bin.
      • -
    • stats - Calculates the mean and standard deviation numeric values of all records in the bin.
      • -
    -
    -
    - -### Levels ### + + + + + + + + + + + + + + + + + + + + + + + + + +
    PropertyDescription
    oculus.binning.value.type + Specifies how to determine the values written to each bin: +
      +
    • count (default) - Count the number of records in a bin. If selected, no value field is required.
      • +
    • field - Perform an aggregation on the values of all records in a bin to create a single value for the bin
      • +
    • series - Save the values of all records in a bin to a dense array
      • +
    +
    oculus.binning.value.fieldField to use as the bin value. If no value field is provided, the tiling job will write the count of records in each bin as the bin value.
    oculus.binning.value.valueType + Type of values stored in the value field: +
      +
    • double (default) - Real, double-precision floating-point numbers
      • +
    • int - Integers
      • +
    • long - Double-precision integers
      • +
    • float - Floating-point numbers
      • +
    +
    oculus.binning.value.aggregation + Method of aggregation used on the values of all records in a bin when oculus.binning.value.type = field. Creates a single value for the bin: +
      +
    • sum (default)- Sum the numeric values of all records in the bin.
      • +
    • min - Select the minimum numeric value from the records in the bin.
      • +
    • max - Select the maximum numeric value from the records in the bin.
      • +
    • mean - Calculate the mean numeric value of all records in the bin.
      • +
    • stats - Calculates the mean and standard deviation numeric values of all records in the bin.
      • +
    +
    + +### Levels ### The **oculus.binning.levels.<order>** array property describes how the tiling job executes the generation of the various zoom levels. For example, if you want to generate levels in three groups, you should include: diff --git a/docs/src/docs/development/how-to/test-output/readme.md b/docs/src/docs/development/how-to/test-output/readme.md index 1121610f..7e71168d 100644 --- a/docs/src/docs/development/how-to/test-output/readme.md +++ b/docs/src/docs/development/how-to/test-output/readme.md @@ -25,76 +25,78 @@ The [BinVisualizer.java](https://github.com/unchartedsoftware/aperture-tiles/blo
    To use the Bin Visualizer
    -1. Execute the following command in your root aperture-tiles directory to start the Bin Visualizer: - - ```bash - gradlew runBinVisualizer - ``` - -2. Use the **I/O** type drop-down list to specify the location (HBase or local file system) of your Avro tiles, then enter the appropriate connection details: -
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Storage LocationConnection DetailsDefault
    HBaseZookeeper quorum -
    Zookeeper port -
    HBase master -
    Local file systemRoot path -
    Tile extensionavro
    -
    -3. In the **Pyramid id** field, enter the name of the tile pyramid you want to view. -4. Set the following coordinates to choose the individual tile you want to view: -
    - - - - - - - - - - - - - - - - - - - - -
    PropertyDescription
    Zoom level - 0 is the highest level (most zoomed out), -
    1 is one further zoom level down, etc. -
    Tile x coordinate0 is the leftmost column of tiles
    Tile y coordinate0 is the bottommost row of tiles
    -
    +1. Execute the following command in your root aperture-tiles directory to start the Bin Visualizer: + + ```bash + gradlew runBinVisualizer + ``` + +2. Use the **I/O** type drop-down list to specify the location (HBase or local file system) of your Avro tiles, then enter the appropriate connection details: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Storage LocationConnection DetailsDefault
    HBaseZookeeper quorum
    Zookeeper port
    HBase master
    Local file systemRoot path
    Tile extensionavro
    + +3. In the **Pyramid id** field, enter the name of the tile pyramid you want to view. +4. Set the following coordinates to choose the individual tile you want to view: + + + + + + + + + + + + + + + + + + + + + + + +
    PropertyDescription
    Zoom level + 0 is the highest level (most zoomed out), +
    1 is one further zoom level down, etc. +
    Tile x coordinate0 is the leftmost column of tiles
    Tile y coordinate0 is the bottommost row of tiles
    + 5. Click **Show tile**. Bin Visualizer diff --git a/docs/src/docs/development/how-to/tile-client/readme.md b/docs/src/docs/development/how-to/tile-client/readme.md index 85890352..970bfde6 100644 --- a/docs/src/docs/development/how-to/tile-client/readme.md +++ b/docs/src/docs/development/how-to/tile-client/readme.md @@ -27,12 +27,12 @@ The map describes the base map upon which your source data is projected: ```json pyramid: { - type : "AreaOfInterest", - minX : -2, - maxX : 2, - minY : -2, - maxY : 2 - }, + type : "AreaOfInterest", + minX : -2, + maxX : 2, + minY : -2, + maxY : 2 + }, ``` **NOTE**: Your layer and map pyramid configurations must match each other. @@ -43,35 +43,35 @@ The BaseLayer parameters use map provider APIs to determine what features to inc ```json baseLayer = new tiles.BaseLayer({ - type: "Google", - options : { - styles : [ - { featureType: "all", - stylers : [ { invert_lightness : true }, - { saturation : -100 }, - { visibility : "simplified" } ] }, - { featureType: "administrative", - elementType: "geometry", - stylers: [ { visibility: "off" } ] }, - { featureType : "landscape.natural.landcover", - stylers : [ { visibility : "off" } ] }, - { featureType : "road", - stylers : [ { visibility : "on" } ] }, - { featureType : "landscape.man_made", - stylers : [ { visibility : "off" } ] }, - { featureType : "landscape", - stylers : [ { lightness : "-100" } ] }, - { featureType : "poi", - stylers : [ { visibility : "off" } ] }, - { featureType : "administrative.country", - elementType : "geometry", - stylers : [ { visibility : "on" }, - { lightness : -56 } ] }, - { elementType : "labels", - stylers : [ { lightness : -46 }, - { visibility : "on" } ] } - ] - } + type: "Google", + options : { + styles : [ + { featureType: "all", + stylers : [ { invert_lightness : true }, + { saturation : -100 }, + { visibility : "simplified" } ] }, + { featureType: "administrative", + elementType: "geometry", + stylers: [ { visibility: "off" } ] }, + { featureType : "landscape.natural.landcover", + stylers : [ { visibility : "off" } ] }, + { featureType : "road", + stylers : [ { visibility : "on" } ] }, + { featureType : "landscape.man_made", + stylers : [ { visibility : "off" } ] }, + { featureType : "landscape", + stylers : [ { lightness : "-100" } ] }, + { featureType : "poi", + stylers : [ { visibility : "off" } ] }, + { featureType : "administrative.country", + elementType : "geometry", + stylers : [ { visibility : "on" }, + { lightness : -56 } ] }, + { elementType : "labels", + stylers : [ { lightness : -46 }, + { visibility : "on" } ] } + ] + } }); ``` @@ -79,20 +79,20 @@ The next example shows a TMS layer configuration (standard OpenLayers.Layer.TMS) ```javascript { - "type": "TMS", - "url" : "http://aperture.oculusinfo.com/map-world-graphite/", - "options" : { - "name" : "Open Graphite", - "layername": "world-graphite", - "osm": 0, - "type": "png", - "serverResolutions": [156543.0339,78271.51695,39135.758475,19567.8792375, - 9783.93961875,4891.96980938,2445.98490469, - 1222.99245234,611.496226172], - "resolutions": [156543.0339,78271.51695,39135.758475,19567.8792375, - 9783.93961875,4891.96980938,2445.98490469, - 1222.99245234,611.496226172] - } + "type": "TMS", + "url" : "http://aperture.oculusinfo.com/map-world-graphite/", + "options" : { + "name" : "Open Graphite", + "layername": "world-graphite", + "osm": 0, + "type": "png", + "serverResolutions": [156543.0339,78271.51695,39135.758475,19567.8792375, + 9783.93961875,4891.96980938,2445.98490469, + 1222.99245234,611.496226172], + "resolutions": [156543.0339,78271.51695,39135.758475,19567.8792375, + 9783.93961875,4891.96980938,2445.98490469, + 1222.99245234,611.496226172] + } } ``` @@ -101,64 +101,64 @@ The next example shows a TMS layer configuration (standard OpenLayers.Layer.TMS) The AxisConfig parameters determine how the X and Y axes are drawn in your cross-plot map.
    -
    -
    position
    -
    Axis type, where bottom denotes the x-axis and left denotes the y-axis.
    - -
    title
    -
    Axis name (e.g., Longitude or Latitude).
    - -
    enabled
    -
    Indicates whether the axes are displayed (true) or hidden (false) when a new session begins.
    - -
    repeat
    -
    Indicates whether the map will repeat when the user scrolls off one end. Most useful for geographic maps.
    - -
    intervals
    -
    -
    -
    type
    -
    How the following increment value is calculated based on the axis range. Accepted values include percentage, %, value or #.
    - -
    increment
    -
    Value or percentage of units by which to increment the intervals. How this is applied is dependent on the specified type.
    - -
    pivot
    -
    Value or percentage from which all other values are incremented. Typically 0.
    - -
    scaleByZoom
    -
    Indicates whether the axis should be scaled by the zoom factor (true/false).
    -
    -
    - -
    units
    -
    -
    -
    type
    -
    - Determines the individual axis label strings formats. Options include: -
      -
    • "billions": 150.25B -
    • "millions": 34.45M -
    • "thousands": 323.26K -
    • "decimal": 234243.32 -
    • "integer": 563554 -
    • "time": MM/DD/YYYY -
    • "degrees": 34.56° -
    -
    - -
    decimals
    -
    Number of decimals to display for each unit. Applicable to billions, millions, thousands and decimal types.
    - -
    stepDown
    -
    Indicates whether the units can step down if they are below range (true/false). Applicable to billions, millions, thousands types.
    -
    -
    -
    +
    +
    position
    +
    Axis type, where bottom denotes the x-axis and left denotes the y-axis.
    + +
    title
    +
    Axis name (e.g., Longitude or Latitude).
    + +
    enabled
    +
    Indicates whether the axes are displayed (true) or hidden (false) when a new session begins.
    + +
    repeat
    +
    Indicates whether the map will repeat when the user scrolls off one end. Most useful for geographic maps.
    + +
    intervals
    +
    +
    +
    type
    +
    How the following increment value is calculated based on the axis range. Accepted values include percentage, %, value or #.
    + +
    increment
    +
    Value or percentage of units by which to increment the intervals. How this is applied is dependent on the specified type.
    + +
    pivot
    +
    Value or percentage from which all other values are incremented. Typically 0.
    + +
    scaleByZoom
    +
    Indicates whether the axis should be scaled by the zoom factor (true/false).
    +
    +
    + +
    units
    +
    +
    +
    type
    +
    + Determines the individual axis label strings formats. Options include: +
      +
    • "billions": 150.25B
      • +
    • "millions": 34.45M
      • +
    • "thousands": 323.26K
      • +
    • "decimal": 234243.32
      • +
    • "integer": 563554
      • +
    • "time": MM/DD/YYYY
      • +
    • "degrees": 34.56°
      • +
    +
    + +
    decimals
    +
    Number of decimals to display for each unit. Applicable to billions, millions, thousands and decimal types.
    + +
    stepDown
    +
    Indicates whether the units can step down if they are below range (true/false). Applicable to billions, millions, thousands types.
    +
    +
    +
    -## Client-Side Rendering ## +## Client-Side Rendering ## The previous sections focus largely on the process of implementing an Aperture Tiles application using server-side tile rendering (where the Server renders the tiles as image files and passes them to the Client). The process of implementing an application using client-side tile rendering (where the Server passes the tiles as JSON data to the Client, which then renders them directly) requires custom code. @@ -168,7 +168,7 @@ Line 33 requests the layer configuration objects from the server, passing them t ```javascript tiles.LayerService.getLayers( function( layers ) { - ... + ... }); ``` @@ -182,9 +182,9 @@ Lines 132-136 instantiate a render theme object that styles the rendered compone ```javascript darkRenderTheme = new tiles.RenderTheme( "dark", { - 'color': "#FFFFFF", - 'color:hover': "#09CFFF", - 'text-shadow': "#000" + 'color': "#FFFFFF", + 'color:hover': "#09CFFF", + 'text-shadow': "#000" }); ``` @@ -194,17 +194,17 @@ Note the **textKey** and **countKey** attributes under the **text** render targe ```javascript wordCloudRenderer = new tiles.WordCloudRenderer({ - text: { - textKey: "topic", - countKey: "countMonthly", - themes: [ darkRenderTheme ] - }, - hook: function( elem, entry, entries, data ) { - elem.onclick = function() { - console.log( elem ); - console.log( entry ); - }; - } + text: { + textKey: "topic", + countKey: "countMonthly", + themes: [ darkRenderTheme ] + }, + hook: function( elem, entry, entries, data ) { + elem.onclick = function() { + console.log( elem ); + console.log( entry ); + }; + } }); ``` @@ -212,8 +212,8 @@ The client-rendered layer is instantiated on lines 186-189, passing the "top-twe ```javascript clientLayer = new tiles.ClientLayer({ - source: layers["top-tweets"], - renderer: wordCloudRenderer + source: layers["top-tweets"], + renderer: wordCloudRenderer }); ``` diff --git a/docs/src/docs/development/how-to/tile-server/readme.md b/docs/src/docs/development/how-to/tile-server/readme.md index 8ccaee23..a1d5e1b4 100644 --- a/docs/src/docs/development/how-to/tile-server/readme.md +++ b/docs/src/docs/development/how-to/tile-server/readme.md @@ -46,22 +46,20 @@ Layer file examples can be found in in the Julia example at [tile-examples\julia The ID parameter uniquely identifies the layer. -
    - - - - - - - - - - - - - -
    PropertyDescription
    idLayer identification string, which is used in all layer-related REST calls. Must conform to JSON property name format.
    -
    + + + + + + + + + + + + + +
    PropertyDescription
    idLayer identification string, which is used in all layer-related REST calls. Must conform to JSON property name format.
    ### Public Parameters ### @@ -75,11 +73,11 @@ For cross-plot maps, the `type` should always be set to *AreaOfInterest*. Also i ```json pyramid: { - type: "AreaOfInterest", - minX : 1, - maxX : 6336769, - minY : 0, - maxY : 500000 + type: "AreaOfInterest", + minX : 1, + maxX : 6336769, + minY : 0, + maxY : 500000 }, ``` @@ -87,126 +85,122 @@ For geographic maps, the `type` should always be set to *WebMercator*. No minimu ```json pyramid: { - type: "WebMercator" + type: "WebMercator" }, ``` **NOTE**: Your layer and map pyramid configurations much match each other. -#### Renderers #### +#### Renderers #### The **renderer** defines how tiles are rendered on the server side. Renderers are dependent on the type of tile data. The renderer options are: -
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    PropertyDescription
    type -
      -
    • 'heatmap': Renders to a heatmap, based on a standard Avro double-valued tile
      • -
    • 'doubleseries': Renders to a series of heatmaps, based on a standard Avro double-series-valued tile
      • -
    • 'doublestatistics': Renders tile's total hit and % coverage as text to the tile
      • -
    • 'textscore': Renders to an image showing scored words, with bars based on their score
      • -
    -
    ramp - Determines the color scale applied to the data points based on their concentration. The default color scales are: -
      -
    • 'hot': Warm orange ramp.
      • -
    • 'neutral': Black-grey-white ramp.
      • -
    • 'cool': Cool blue ramp.
      • -
    • 'spectral': Red-green-yellow ramp.
      • -
    • 'flat': Single color (white) ramp.
      • -
    -
    rangeMinMinimum percentage to clamp the low end of the color ramp.
    rangeMaxMaximum percentage to clamp the low end of the color ramp.
    opacityOpacity of the rendered tile layer expressed as a decimal ranging from 0 (completely transparent) to 1 (completely opaque).
    enabledIndicates whether the layer is enabled on load.
    -
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    PropertyDescription
    type +
      +
    • 'heatmap': Renders to a heatmap, based on a standard Avro double-valued tile
      • +
    • 'doubleseries': Renders to a series of heatmaps, based on a standard Avro double-series-valued tile
      • +
    • 'doublestatistics': Renders tile's total hit and % coverage as text to the tile
      • +
    • 'textscore': Renders to an image showing scored words, with bars based on their score
      • +
    +
    ramp + Determines the color scale applied to the data points based on their concentration. The default color scales are: +
      +
    • 'hot': Warm orange ramp.
      • +
    • 'neutral': Black-grey-white ramp.
      • +
    • 'cool': Cool blue ramp.
      • +
    • 'spectral': Red-green-yellow ramp.
      • +
    • 'flat': Single color (white) ramp.
      • +
    +
    rangeMinMinimum percentage to clamp the low end of the color ramp.
    rangeMaxMaximum percentage to clamp the low end of the color ramp.
    opacityOpacity of the rendered tile layer expressed as a decimal ranging from 0 (completely transparent) to 1 (completely opaque).
    enabledIndicates whether the layer is enabled on load.
    #### Value Transformer #### The **valueTransform** defines the **type** of transformation that can be applied to the data values when determining color: -
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    ValueDescription
    minmaxDo not change the value, but allow capping within min and max values.
    sigmoid - Apply a sigmoid function to each value to make them fall between fixed values. With data that can be infinitely large in either direction, this can be used to bring it into finite, displayable bounds. - -

    Uses the following properties to indicate the minimum and maximum expected values while allowing values infinitely above or below them.

    -
      -
    • layerMin
      • -
    • layerMax
      • -
    - These two values should be symmetric around the desired central value. -
    half-sigmoid - Apply a sigmoid function to each value to make them fall between fixed values. With data that can be infinitely large in one direction only, this can be used to bring it into finite, displayable bounds. - -

    Uses the following properties:

    -
      -
    • layerMin corresponds to the minimum allowed data value -
    • layerMax indicates the maximum expected data value while allowing values infinitely above it -
    -
    log10Take the log (base 10) of each value. Useful when displaying data that can be large (but not infinite) in one direction only. Uses the layerMax property to indicate the maximum.
    -
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    ValueDescription
    minmaxDo not change the value, but allow capping within min and max values.
    sigmoid + Apply a sigmoid function to each value to make them fall between fixed values. With data that can be infinitely large in either direction, this can be used to bring it into finite, displayable bounds. + +

    Uses the following properties to indicate the minimum and maximum expected values while allowing values infinitely above or below them.

    +
      +
    • layerMin
      • +
    • layerMax
      • +
    + These two values should be symmetric around the desired central value. +
    half-sigmoid + Apply a sigmoid function to each value to make them fall between fixed values. With data that can be infinitely large in one direction only, this can be used to bring it into finite, displayable bounds. + +

    Uses the following properties:

    +
      +
    • layerMin corresponds to the minimum allowed data value
      • +
    • layerMax indicates the maximum expected data value while allowing values infinitely above it
      • +
    +
    log10Take the log (base 10) of each value. Useful when displaying data that can be large (but not infinite) in one direction only. Uses the layerMax property to indicate the maximum.
    ### Private Parameters ### @@ -217,51 +211,51 @@ Parameters in the **private** node section of the **layers.json** file are not a The data parameters specify the location of the tiles that you created. If you are using HBase, separate parameters are required.
    -
    -
    id
    -
    - Must match the name of the folder to which the tiles were saved during the generation process, which is composed of the following parameters from the Tiling Property File: - -

    <oculus.binning.name>.<oculus.binning.xField>.<oculus.binning.yField>.<oculus.binning.valueField> - -

    Where oculus.binning.yField is set to 0 if no yField name is specified. -
    - -
    pyramidio
    -
    -
    -
    type
    -
    Indicates the file format of your tiles: -
    -
    hbase
    -
    Tiles are stored in HBase
    - -
    file
    -
    Tiles are stored in an uncompressed directory in a local filesystem
    - -
    zip
    -
    Tiles are stored in compressed file in a local filesystem
    -
    -
    - -
    root.path
    -
    Root path in which the tiles are stored. Not used for HBase.
    - -
    extension
    -
    Name of the compressed file in which tiles are stored. Only used for ZIP - files.
    - -
    hbase.zookeeper.quorum
    -
    Zookeeper quorum location needed to connect to HBase.
    - -
    hbase.zookeeper.port
    -
    Port through which to connect to zookeeper. Typically defaults to 2181.
    - -
    hbase.master
    -
    Location of the HBase master on which the tiles are saved.
    -
    -
    -
    +
    +
    id
    +
    + Must match the name of the folder to which the tiles were saved during the generation process, which is composed of the following parameters from the Tiling Property File: + +

    <oculus.binning.name>.<oculus.binning.xField>.<oculus.binning.yField>.<oculus.binning.valueField> + +

    Where oculus.binning.yField is set to 0 if no yField name is specified. +
    + +
    pyramidio
    +
    +
    +
    type
    +
    Indicates the file format of your tiles: +
    +
    hbase
    +
    Tiles are stored in HBase
    + +
    file
    +
    Tiles are stored in an uncompressed directory in a local filesystem
    + +
    zip
    +
    Tiles are stored in compressed file in a local filesystem
    +
    +
    + +
    root.path
    +
    Root path in which the tiles are stored. Not used for HBase.
    + +
    extension
    +
    Name of the compressed file in which tiles are stored. Only used for ZIP + files.
    + +
    hbase.zookeeper.quorum
    +
    Zookeeper quorum location needed to connect to HBase.
    + +
    hbase.zookeeper.port
    +
    Port through which to connect to zookeeper. Typically defaults to 2181.
    + +
    hbase.master
    +
    Location of the HBase master on which the tiles are saved.
    +
    +
    +
    ## Next Steps ## diff --git a/docs/src/docs/user-guide/interface/readme.md b/docs/src/docs/user-guide/interface/readme.md index 49caea72..943f0aaf 100644 --- a/docs/src/docs/user-guide/interface/readme.md +++ b/docs/src/docs/user-guide/interface/readme.md @@ -11,7 +11,7 @@ layout: chapter The user interface for a typical Aperture Tiles project contains the following components: - [Map](#map) - - [Axes](#axes) + - [Axes](#axes) - [Layers](#layers) - [Heatmap](#heatmap) - [Tile Carousel](#carousel) @@ -63,25 +63,25 @@ Each tile in the Twitter Topics application contains a carousel with multiple vi The Twitter Topics carousels let you cycle through the following pages: - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + +
    Word CloudTopic ListBlank Page
    Tile Carousel Word CloudTile Carousel Topic ListTile Carousel Topic List
    Top 10 topics mentioned in tweets. The larger the word, the more times it was used.Histograms showing usage of the top 8 topics over the last 24 hours.Hide the carousel display for a single tile.
    Word CloudTopic ListBlank Page
    Tile Carousel Word CloudTile Carousel Topic ListTile Carousel Topic List
    Top 10 topics mentioned in tweets. The larger the word, the more times it was used.Histograms showing usage of the top 8 topics over the last 24 hours.Hide the carousel display for a single tile.
    Each carousel page also includes a **Translate** button that translates topics to English. @@ -92,8 +92,8 @@ Clicking on a particular topic on any of the carousel pages refreshes the map to - Highlight instances of the topic in other tiles to illustrate other locations in which the topic is popular. - Display a detailed overview of the topic usage, which includes: - - A total count and expanded histograms of its usage in the past month, week and 24 hours. - - The most recent tweets in which it appeared. + - A total count and expanded histograms of its usage in the past month, week and 24 hours. + - The most recent tweets in which it appeared. Tile Carousel Topic Detail diff --git a/docs/src/docs/user-guide/navigation/readme.md b/docs/src/docs/user-guide/navigation/readme.md index b7a2a134..8fb9b1a1 100644 --- a/docs/src/docs/user-guide/navigation/readme.md +++ b/docs/src/docs/user-guide/navigation/readme.md @@ -15,7 +15,7 @@ The Aperture Tiles user interface works much like a modern web-based map system.
    To zoom in on an area you want to investigate
    - Double click on the region, OR -- Click the **Zoom In** ![Zoom In](../../../img/zoom-in.png) button in the upper right corner of the map. +- Click the **Zoom In** ![Zoom In](../../../img/zoom-in.png) button in the upper right corner of the map.
    To zoom out
    @@ -28,11 +28,11 @@ The Aperture Tiles user interface works much like a modern web-based map system. 1. Click the **Description** control in the upper right corner of the map. - Description + Description 2. Select the appropriate city from the **Zoom to City** drop-down list in the project description. - Zoom to City + Zoom to City The map is redrawn to show the selected city with a zoom factor of five. @@ -45,15 +45,16 @@ The map is redrawn to show the selected city with a zoom factor of five. - Click and drag in the direction you want to move. You can overscroll from the left side of the map to the right and vice versa.
    -

    Interested in Learning More?

    - -
      -
    • Tour: Take our tour to learn more about Aperture Tiles. -
    • Documentation: Learn how to install, implement and test your Aperture Tiles applications. -
        -
      • Quick Start: Our Julia dataset provides an example of the process for generating tiles and visualizing them using Aperture Tiles. -
      -
    • Live Examples: See our demos page to explore live examples of the capabilities of Aperture Tiles. -
    • Download: For details on downloading pre-packaged versions or acquiring the Aperture Tiles source code visit our download page. -
    +

    Interested in Learning More?

    + +
      +
    • Tour: Take our tour to learn more about Aperture Tiles.
      • +
    • Documentation: Learn how to install, implement and test your Aperture Tiles applications. +
        +
      • Quick Start: Our Julia dataset provides an example of the process for generating tiles and visualizing them using Aperture Tiles.
        • +
      +
      • +
    • Live Examples: See our demos page to explore live examples of the capabilities of Aperture Tiles.
      • +
    • Download: For details on downloading pre-packaged versions or acquiring the Aperture Tiles source code visit our download page.
      • +
    \ No newline at end of file diff --git a/docs/src/download/readme.md b/docs/src/download/readme.md index d17517c5..8a3e5399 100644 --- a/docs/src/download/readme.md +++ b/docs/src/download/readme.md @@ -25,14 +25,15 @@ The following pre-built distribution can be used to quickly generate and display

    Tile Quick Start Application
    -

    Interested in Learning More?

    - -
      -
    • Tour: Take our tour to learn more about Aperture Tiles. -
    • Documentation: Learn how to install, implement and test your Aperture Tiles applications. -
        -
      • Quick Start: Our Julia dataset provides an example of the process for generating tiles and visualizing them using Aperture Tiles. -
      -
    • Live Examples: See our demos page to explore live examples of the capabilities of Aperture Tiles. -
    +

    Interested in Learning More?

    + +
      +
    • Tour: Take our tour to learn more about Aperture Tiles.
      • +
    • Documentation: Learn how to install, implement and test your Aperture Tiles applications. +
        +
      • Quick Start: Our Julia dataset provides an example of the process for generating tiles and visualizing them using Aperture Tiles.
        • +
      +
      • +
    • Live Examples: See our demos page to explore live examples of the capabilities of Aperture Tiles.
      • +
    \ No newline at end of file diff --git a/docs/src/download/sidebar.html b/docs/src/download/sidebar.html index ea779058..46f4ff87 100644 --- a/docs/src/download/sidebar.html +++ b/docs/src/download/sidebar.html @@ -1,6 +1,6 @@ -
    - Julia Set Quick Start Example -
    Julia Set Quick Start example
    -
    +
    + Julia Set Quick Start Example +
    Julia Set Quick Start example
    +
    \ No newline at end of file diff --git a/docs/src/readme.md b/docs/src/readme.md index 48b8b5b4..d537b7ab 100644 --- a/docs/src/readme.md +++ b/docs/src/readme.md @@ -47,6 +47,6 @@ Aperture Tiles is under ongoing development and is freely available for download * [Tour](tour/): Take our tour to learn more about Aperture Tiles. * [Documentation](docs/development/getting-started/quick-start/): Learn how to install, implement and test your Aperture Tiles applications. - * [Quick Start](docs/development/getting-started/quick-start/): Our Julia dataset provides an example of the process for generating tiles and visualizing them using Aperture Tiles. + * [Quick Start](docs/development/getting-started/quick-start/): Our Julia dataset provides an example of the process for generating tiles and visualizing them using Aperture Tiles. * [Live Examples](demos/): See our demos page to see live examples of the capabilities of Aperture Tiles. * [Download](download/): For details on downloading pre-packaged versions or acquiring the Aperture Tiles source code visit our download page. \ No newline at end of file diff --git a/docs/src/sidebar.html b/docs/src/sidebar.html index f64c73e3..7cf60f5d 100644 --- a/docs/src/sidebar.html +++ b/docs/src/sidebar.html @@ -1,34 +1,30 @@ - -
    - Twitter Aperture Tile Demo -
    +
    + Twitter Aperture Tile Demo +

    Published Papers:

    - - Tile Based Visual Analytics for Twitter Big Data Exploratory Analysis. Cheng, D. et al, IEEE Big Data Conference, 2013 - + + Tile Based Visual Analytics for Twitter Big Data Exploratory Analysis. Cheng, D. et al., IEEE Big Data Conference, 2013 +

    - - Interactive Data Exploration with "Big Data Tukey Plots". Schretlen, P. et al, IEEE VisWeek, 2013 + + Interactive Data Exploration with "Big Data Tukey Plots". Schretlen, P. et al., IEEE VisWeek, 2013
    - Bitcoin Aperture Tile Demo -
    Just like for maps, tile-based visual analytics technology can "plot all the data" for billions of data points.
    + Bitcoin Aperture Tile Demo +
    Just like for maps, tile-based visual analytics technology can "plot all the data" for billions of data points.
    -
    - Bitcoin Aperture Tile Demo -
    +
    + Bitcoin Aperture Tile Demo +
    -

    - Aperture Tiles Overview Document: pdf -

    \ No newline at end of file +

    Aperture Tiles Overview Document: pdf

    \ No newline at end of file diff --git a/docs/src/tour/components/sidebar.html b/docs/src/tour/components/sidebar.html index 4cb96e14..c981a08f 100644 --- a/docs/src/tour/components/sidebar.html +++ b/docs/src/tour/components/sidebar.html @@ -1,4 +1,4 @@
    - Interactive Aggregation Markers -
    Interactive aggregation markers provide per tile analytics.
    + Interactive Aggregation Markers +
    Interactive aggregation markers provide per tile analytics.
    \ No newline at end of file diff --git a/docs/src/tour/process/readme.md b/docs/src/tour/process/readme.md index b7d5f9d9..2be74768 100644 --- a/docs/src/tour/process/readme.md +++ b/docs/src/tour/process/readme.md @@ -13,7 +13,7 @@ Aperture Tiles uses a multi-stage process to develop tile-based visual analytics Aggregation Stage ----------------- -The aggregation stage projects and bins data into a predefined grid, such as the Tiling Map Service standard. This uses a (z,x,y) coordinate system, where z identifies the zoom level, and x,y identifies a specific tile on the plot for the given zoom level. +The aggregation stage projects and bins data into a predefined grid, such as the Tiling Map Service standard. This uses a (z,x,y) coordinate system, where z identifies the zoom level, and x,y identifies a specific tile on the plot for the given zoom level. Tile Summarization Stage ------------------------ @@ -28,14 +28,15 @@ The rendering stage maps the summary to a visual representation, and renders it ![Aperture Tiles Architectural Overview](../../img/arch-overview.png)
    -

    Interested in Learning More?

    - -
      -
    • Documentation: Learn how to install, implement and test your Aperture Tiles applications. -
        -
      • Quick Start: Our Julia dataset provides an example of the process for generating tiles and visualizing them using Aperture Tiles. -
      -
    • Live Examples: See our demos page to see live examples of the capabilities of Aperture Tiles. -
    • Download: For details on downloading pre-packaged versions or acquiring the Aperture Tiles source code visit our download page. -
    +

    Interested in Learning More?

    + +
      +
    • Documentation: Learn how to install, implement and test your Aperture Tiles applications. +
        +
      • Quick Start: Our Julia dataset provides an example of the process for generating tiles and visualizing them using Aperture Tiles.
        • +
      +
      • +
    • Live Examples: See our demos page to see live examples of the capabilities of Aperture Tiles.
      • +
    • Download: For details on downloading pre-packaged versions or acquiring the Aperture Tiles source code visit our download page.
      • +
    \ No newline at end of file diff --git a/docs/src/tour/readme.md b/docs/src/tour/readme.md index b348b933..478ca25a 100644 --- a/docs/src/tour/readme.md +++ b/docs/src/tour/readme.md @@ -10,15 +10,15 @@ Aperture Tiles Overview Aperture Tiles provides a single, consistent tile-based framework to visualize and analyze data at any scale, including extremely large datasets. Effective use of a tiling approach has multiple advantages: -* You can *plot all the data* without needing aggregation or sampling. -* In a web browser, users can interactively visualize the data and associated computed analytics at multiple resolutions, from very high overviews to very detailed individual data points. -* Due to the increasing exposure of web-based geographic tile applications such as Google Maps, user interactions have become familiar and widespread. -* You can compose powerful visual analytic capabilities by overlaying multiple facets of information (e.g., alerts, patterns, extracted features) on top of the complete data context. +- You can *plot all the data* without needing aggregation or sampling. +- In a web browser, users can interactively visualize the data and associated computed analytics at multiple resolutions, from very high overviews to very detailed individual data points. +- Due to the increasing exposure of web-based geographic tile applications such as Google Maps, user interactions have become familiar and widespread. +- You can compose powerful visual analytic capabilities by overlaying multiple facets of information (e.g., alerts, patterns, extracted features) on top of the complete data context. There are several mature, feature-rich JavaScript libraries that support integration of tile rendering and interaction into web applications. However, these conventional tiling approaches are limited by two critical restraints: -* Tiles can take a long time to generate. -* Data values are baked into pre-rendered tiles, resulting in stale tiles representing static data. +- Tiles can take a long time to generate. +- Data values are baked into pre-rendered tiles, resulting in stale tiles representing static data. Aperture Tiles addresses these drawbacks by using cloud-distributed computation. This method generates data for tiles in far less time, by using defined, reproducible data flows that can regenerate tiles from live data. This not only speeds the process of generating data for tiles, but serves to reduce the staleness of the data in any given tile. Aperture Tiles also separates tile data apportionment from the visual rendering process. This approach allows tile visualization to be altered in real-time, allowing interactions like filtering of data values or the application of new visualization techniques. Most importantly, tile displays are not limited to static images. Tile-based visual analytics can be the centerpiece of highly interactive visual analytic applications using web-based, client-side rendering, such as [Aperture JS](http://aperturejs.com/). @@ -27,10 +27,8 @@ Computing Tile-based Visual Analytics Aperture Tiles uses a multi-stage process to develop tile-based visual analytics. -1. The aggregation stage projects and bins data into a predefined pyramid grid, such as the Tiling Map Service standard. This uses a (z,x,y) coordinate system, where z identifies the zoom level, and x,y identifies a specific tile on the plot for the given zoom level. - +1. The aggregation stage projects and bins data into a predefined pyramid grid, such as the Tiling Map Service standard. This uses a (z,x,y) coordinate system, where z identifies the zoom level, and x,y identifies a specific tile on the plot for the given zoom level. 2. The summary and analytical stage applies one or more summary statistics or other analytics to the data in each tile region, storing the result in a data store. - 3. The rendering stage maps the computed summary and analytics to visual representations, and renders them to an image tile or html at request time. This stage is capable of rendering on-demand interactions such as filtering, or rich dynamic interactions such as brushing and drill-downs, if using client-side rendering with html and JavaScript. ## Next Steps diff --git a/docs/src/tour/sidebar.html b/docs/src/tour/sidebar.html index f037148c..6cd7327d 100644 --- a/docs/src/tour/sidebar.html +++ b/docs/src/tour/sidebar.html @@ -1,4 +1,4 @@
    - Twitter Heatmap -
    Millions of Tweets across Europe
    + Twitter Heatmap +
    Millions of Tweets across Europe
    \ No newline at end of file From c1f726310057f0d7793d179ae27a847397996a1a Mon Sep 17 00:00:00 2001 From: Michael Crouch Date: Tue, 7 Jun 2016 14:19:59 -0400 Subject: [PATCH 2/5] Switch jekyll build to use default kramdown converter and automatically generate header ids --- docs/readme.md | 13 +++--- docs/src/_config.yml | 16 +++---- docs/src/_data/developmentChapters.yaml | 26 +++++------ docs/src/demos/readme.md | 18 +++----- .../advanced/custom-tiling/readme.md | 44 +++++++++---------- .../advanced/graph-tiling/readme.md | 16 +++---- .../advanced/standard-tiling/readme.md | 4 +- .../docs/development/api/annotation/readme.md | 21 ++++----- docs/src/docs/development/api/layer/readme.md | 13 +++--- .../src/docs/development/api/legend/readme.md | 5 ++- docs/src/docs/development/api/tile/readme.md | 5 ++- .../getting-started/installation/readme.md | 22 +++++----- .../getting-started/quick-start/readme.md | 21 ++++----- .../getting-started/tile-pyramid/diagram.md | 2 +- .../getting-started/tile-pyramid/readme.md | 8 ++-- .../how-to/standard-tiling/readme.md | 10 ++--- .../development/how-to/test-output/readme.md | 6 +-- .../development/how-to/tile-client/readme.md | 8 ++-- .../development/how-to/tile-server/readme.md | 18 ++++---- docs/src/docs/user-guide/interface/readme.md | 22 +++++----- docs/src/docs/user-guide/navigation/readme.md | 6 +-- docs/src/docs/user-guide/readme.md | 2 +- docs/src/readme.md | 10 ++--- docs/src/tour/components/readme.md | 13 +++--- docs/src/tour/process/readme.md | 9 ++-- docs/src/tour/readme.md | 5 +-- 26 files changed, 165 insertions(+), 178 deletions(-) diff --git a/docs/readme.md b/docs/readme.md index 125b69da..27e5ff96 100644 --- a/docs/readme.md +++ b/docs/readme.md @@ -1,4 +1,5 @@ -# Building Aperture Sites with Jekyll +Building Aperture Sites with Jekyll +=================================== **NOTE**: This topic describes how to build the Aperture Tiles website. To view the Aperture Tiles website content, visit or see the [src/](src/) folder. @@ -16,7 +17,7 @@ Note that the latest versions of the pre-built Aperture websites are available f - Aperture Tiles: [http://aperturetiles.com/](http://aperturetiles.com/) - Influent: [http://community.influent.org/](http://community.influent.org/) -## Prerequisites +## Prerequisites ## Before you attempt to build an Aperture site, you should install the following prerequisites. Note that while other versions of the third-party tools below may work, only the listed versions have been tested with the latest site build. @@ -27,7 +28,7 @@ Before you attempt to build an Aperture site, you should install the following p - **Site Generator**: [Jekyll](http://jekyllrb.com/) (version 2.5.3) - **Syntax Highlighter**: [Rouge](https://github.com/jneen/rouge) (version 1.7.7) -## Building the Site +## Building the Site ## To build an Aperture site, run the following command in its source docs folder, [docs/src/](docs/src/), which contains a *_config.yml* file: @@ -41,14 +42,14 @@ To test the site before deploying it, you can run the following command, which w jekyll serve -## Aperture Website Configuration +## Aperture Website Configuration ## This section describes how the Aperture websites are configured for the Jekyll conversion. - Markdown Files - Jekyll Resources -### Markdown Files +### Markdown Files ### Each Markdown file (*.md) covers a single topic representing a page on the Aperture website. There should only be one Markdown file per folder. Each file should be named **readme.md** so that it is natively displayed in the GitHub repository when viewing the corresponding folder. @@ -108,7 +109,7 @@ layout (Required) ![](src/img/resources/SiteComponents.png) -### Jekyll Resources +### Jekyll Resources ### In addition to getting page-specific variables from the Markdown files, Jekyll also gets global variables and assets from the following locations: diff --git a/docs/src/_config.yml b/docs/src/_config.yml index 3071e61d..70745e42 100644 --- a/docs/src/_config.yml +++ b/docs/src/_config.yml @@ -1,18 +1,12 @@ # Conversion -markdown: redcarpet - -redcarpet: - extensions: ["no_intra_emphasis", "fenced_code_blocks", "tables"] +markdown: kramdown highlighter: rouge -# Uncomment if using alternative renderer -# rdiscount: -# extensions: -# - autolink - -# kramdown: -# input: GFM +kramdown: + input: GFM + syntax_highlighter: rouge + auto_ids: true # Serving destination: ../_site/ diff --git a/docs/src/_data/developmentChapters.yaml b/docs/src/_data/developmentChapters.yaml index f8b0a8f9..a7dab966 100644 --- a/docs/src/_data/developmentChapters.yaml +++ b/docs/src/_data/developmentChapters.yaml @@ -40,13 +40,13 @@ path: tile-pyramid bookmarks: - label: Tile Pyramid Hierarchy - anchor: pyramid-hierarchy + anchor: tile-pyramid-hierarchy - label: Tile Generation Process - anchor: tile-gen-process + anchor: tile-generation-process - label: Tiling Job Prerequisites - anchor: prerequisites + anchor: tiling-job-prerequisites - label: spark-submit anchor: spark-submit @@ -63,10 +63,10 @@ anchor: csvbinner - label: Base Properties File - anchor: base-properties + anchor: base-properties-files - label: Tiling Properties Files - anchor: tiling-properties + anchor: tiling-properties-files - label: Test Tiling Job Output path: test-output @@ -80,10 +80,10 @@ path: tile-server bookmarks: - label: Web XML - anchor: webxml + anchor: web-xml - label: Tile Properties - anchor: tileproperties + anchor: tile-properties - label: Layers anchor: layers @@ -95,7 +95,7 @@ anchor: application-javascript - label: Client-Side Rendering - anchor: clientside + anchor: client-side-rendering - label: Deploy the Application path: deploy @@ -122,10 +122,10 @@ path: graph-tiling bookmarks: - label: Convert GraphML to CSV - anchor: convert-graphml + anchor: converting-graphml-data-to-csv-data - label: Hierarchical Clustering - anchor: clustering + anchor: hierarchical-clustering - label: Graph Layout anchor: graph-layout @@ -140,10 +140,10 @@ path: custom-tiling bookmarks: - label: Custom Tiling Jobs - anchor: custom-tiling + anchor: custom-tiling-jobs - label: Define Data Structure - anchor: define-data-structure + anchor: defining-your-data-structure - label: Bin Data anchor: binning-your-data @@ -179,7 +179,7 @@ anchor: request-specific-layer - label: Store a Configuration State - anchor: store-configuration-state + anchor: store-a-configuration-state - label: Get a Configuration State anchor: get-configuration-state diff --git a/docs/src/demos/readme.md b/docs/src/demos/readme.md index 2066a6d7..8e8d6f3b 100644 --- a/docs/src/demos/readme.md +++ b/docs/src/demos/readme.md @@ -17,8 +17,7 @@ The following demos illustrate the utility of tile-based visual analytics and th - Integration with advanced analytics incorporated in generated tile data - Use of Aperture tiles for exploratory data analysis by plotting *all the data* -Tile-based Metrics of NYC Taxi Trip Data ---------------------------------------- +## Tile-Based Metrics of NYC Taxi Trip Data ## This demo, which was presented at [Strata and Hadoop World 2014](http://strataconf.com/stratany2014), enables the exploration of data for 187 million NYC taxi trips taken between Jan 1, 2013 - Jan 3, 2014. Trip data is displayed over a variety of metrics (both on a [Google Maps](https://maps.google.com) base layer and in cross-plot form) with the goal of answering the following questions: @@ -38,15 +37,13 @@ The source dataset for this demo was acquired from the [NYC Taxi and Limousine C > Explore the [NYC Taxi Trip demo](http://aperturetiles.com/nyc-taxi/?map=6&baselayer=0) -Tile-based Heatmap of Twitter Messages --------------------------------------- +## Tile-Based Heatmap of Twitter Messages ## This demo shows the scalability of map-oriented big data plots using heatmaps. The application contains a heatmap layer of Twitter messages by location on a [Google Maps](https://maps.google.com) base layer. > Explore the [Twitter Heatmap demo](http://aperturetiles.com/twitter-heatmap/) -Tile-based Topic Tracking in Twitter Messages ---------------------------------------------- +## Tile-Based Topic Tracking in Twitter Messages ## This demo shows map-oriented tile applications using multiple layers of tile data and tile carousels to flip through multiple per-tile analytics. The application shows a heatmap layer of Twitter messages by location on a [Google Maps](https://maps.google.com) base layer. An "aggregate marker" layer provides an interactive carousel of visual analytic summaries of each tile region. These aggregate markers summarize and provide interactive visualizations that support analysis and integrate with advanced analytics. @@ -55,23 +52,20 @@ In this demo, topics of interest are extracted from Twitter messages across Sout > Explore the [Twitter Topics demo](http://aperturetiles.com/twitter-topics/)
    > Access the source code in the [tile-examples](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-examples) directory of the project -Tile-based Sentiment Analysis of Twitter Messages -------------------------------------------------- +## Tile-Based Sentiment Analysis of Twitter Messages ## This Aperture tile demo shows a heatmap layer of Twitter messages by location on a [Google Maps](https://maps.google.com) base layer. An "aggregate marker" layer provides an interactive carousel of visual analytic summaries of each tile region summarizing the top topics discussed in the Twitter messages and the sentiment of the Tweet (positive, neutral and negative). Histograms of topics show sentiment trends. > Explore the [Twitter Sentiment demo](http://aperturetiles.com/twitter-sentiment/) -Big Data Plots of Bitcoin for Exploratory Data Analysis -------------------------------------------------------- +## Big Data Plots of Bitcoin for Exploratory Data Analysis ## This demo enables exploratory data analysis of [Bitcoin](http://bitcoin.org/) transactions, a well-known digital currency. Using an interactive cross-plot of transaction amounts against time, Aperture Tiles aids the characterization and understanding of the underlying distribution, quality and gaps of the source data. A heat map is applied to the data points to summarize frequency distribution, effectively highlighting interesting data structures, such as the "grinding down" of bitcoins through micro-transfers of pools of bitcoins. > Explore the [Bitcoin demo](http://aperturetiles.com/bitcoin-demo/) -Julia Set Example ------------------ +## Julia Set Example ## This demo is used in the [Quick Start guide](../docs/development/getting-started/quick-start/) to illustrate the process of generating tiles and configuring a tile client to display them in a web browser. diff --git a/docs/src/docs/development/advanced/custom-tiling/readme.md b/docs/src/docs/development/advanced/custom-tiling/readme.md index 22061622..2285335f 100644 --- a/docs/src/docs/development/advanced/custom-tiling/readme.md +++ b/docs/src/docs/development/advanced/custom-tiling/readme.md @@ -12,16 +12,16 @@ Custom Tiling Jobs The following sections describe how to generate a tile pyramid from your raw data by creating your own custom generation code. -## Custom Tiling Jobs ## +## Custom Tiling Jobs ## If your source data is not character delimited or if it contains non-numeric fields, you may need to create custom code to parse it and create a tile pyramid. Creating custom tile generation code involves: -- [Defining Your Data Structure](#define-data-structure) by specifying how to: - - Structure your source data while it is: - - Processed by the tiling job - - Written to the Avro tile set - - Combine multiple records that fall in the same tile bin - - Read and write to the tile set +- [Defining Your Data Structure](#defining-your-data-structure) by specifying how to: + - Structure your source data while it is: + - Processed by the tiling job + - Written to the Avro tile set + - Combine multiple records that fall in the same tile bin + - Read and write to the tile set - [Binning Your Data](#binning-your-data) by specifying how to: - Parse your source data - Transform the parsed data into the Avro tile set @@ -34,25 +34,25 @@ For an example of custom code written for tile generation, see the Twitter Topic - Heatmap of Twitter messages by location - Word cloud and bar chart overlays of the top words mentioned in tweets for each tile and bin -## Defining Your Data Structure ## +## Defining Your Data Structure ## The following custom modules are required to determine how your data should be structured, aggregated and written: - [Binning Analytic](#binning-analytic): defines how to process and aggregate your source data - [Serializer](#serializer): defines how to read and write your source data to the tile set -### Binning Analytic ### +### Binning Analytic ### The Binning Analytic is used throughout the tiling process. It should define: - Two [data types](#data-types) used during the tiling job: - 1. Processing type - 2. Binning type -- The method of [aggregation](#data-aggregation) for joining individual data records in a bin + 1. Processing type + 2. Binning type +- The method of [aggregation](#data-aggregation-and-record-creation) for joining individual data records in a bin For an example of a custom Bin Analytic, see the [TwitterTopicBinningAnalytic.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/TwitterTopicBinningAnalytic.scala) file in [tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen). -#### Data Types #### +#### Data Types #### During a tiling job, the Binning Analytic uses two types of data: @@ -90,7 +90,7 @@ The **finish** function: While the remaining topics are eventually discarded, they are necessary during processing. For example, a topic that is recorded as the 11th most used topic on several different machines is likely to be in the top 10 once all the data is aggregated together. -#### Data Aggregation and Record Creation #### +#### Data Aggregation and Record Creation #### The Binning Analytic defines how data is aggregated. For example, lines 42-47 of [TwitterTopicBinningAnalytic.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/TwitterTopicBinningAnalytic.scala) compare two maps and creates a new map that contains: @@ -106,7 +106,7 @@ def aggregate (a: Map[String, TwitterDemoTopicRecord], } ``` -##### Custom Aggregation Methods ##### +##### Custom Aggregation Methods ##### Lines 85-93 of [TwitterTopicBinner.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/TwitterTopicBinner.scala) (found in the [same folder](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen) as the Binning Analytic) are used to calculate the minimum and maximum values and write them to the metadata by level. @@ -124,7 +124,7 @@ val maxAnalysis: Standard Bin Analytics are available in the [Analytics.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/analytics/Analytics.scala) file in [tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/analytics/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/analytics). -### Serializer ### +### Serializer ### The Serializer determines how to read and write to the tile set. The Tile Server requires the following supporting classes to use the Serializer: @@ -134,7 +134,7 @@ The Serializer determines how to read and write to the tile set. The Tile Server See the following sections for examples of each custom Serializer component. -#### Serializer #### +#### Serializer Class #### The Serializer implements the [com.oculusinfo.binning.io.serialization.TileSerializer](https://github.com/unchartedsoftware/aperture-tiles/blob/master/binning-utilities/src/main/java/com/oculusinfo/binning/io/serialization/TileSerializer.java) interface. To read and write the Avro tiles that are most commonly used, it should inherit from: @@ -159,20 +159,20 @@ For records that aren't list types, inherit from the [GenericAvroSerializer.java The definition of the Avro schema can be based on the template in the [doubleData.avsc](https://github.com/unchartedsoftware/aperture-tiles/blob/master/binning-utilities/src/main/resources/doubleData.avsc) file ([binning-utilities/src/main/resources/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/binning-utilities/src/main/resources)), where the **name** is set to *recordType*. -#### Serialization Factory #### +#### Serialization Factory #### The Serialization Factory gets configuration information (e.g., the Avro compression codec) and hands back the serializer of choice when needed. It also produces the factory and can be injected by Guice. For an example Serialization Factory, see the [TwitterTileSerializationFactory.java](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/java/com/oculusinfo/twitter/init/TwitterTileSerializationFactory.java) in [tile-examples/twitter-topics/twitter-topics-utilities/src/main/java/com/oculusinfo/twitter/init/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/java/com/oculusinfo/twitter/init). -#### Serialization Factory Module #### +#### Serialization Factory Module #### The Factory Module tells Guice which factory providers to use to create serialization factories. For an example Serialization Factory, see the [TwitterSerializationFactoryModule.java](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/java/com/oculusinfo/twitter/init/TwitterSerializationFactoryModule.java) file in [tile-examples/twitter-topics/twitter-topics-utilities/src/main/java/com/oculusinfo/twitter/init/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/java/com/oculusinfo/twitter/init). -## Binning Your Data ## +## Binning Your Data ## There are three steps in binning your data: @@ -182,7 +182,7 @@ There are three steps in binning your data: For an example of a custom Binner, see the [TwitterTopicBinner.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/TwitterTopicBinner.scala) file in [tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen). -### Parsing your Data ### +### Parsing your Data ### The Binner expects your data as pairs of **(index, record)**, where: @@ -224,7 +224,7 @@ val data = rawDataWithTopics.mapPartitions(i => data.cache ``` -### Binning ### +### Binning ### Lines 191 - 199 of [TwitterTopicBinner.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/TwitterTopicBinner.scala) transform the data into tiles: diff --git a/docs/src/docs/development/advanced/graph-tiling/readme.md b/docs/src/docs/development/advanced/graph-tiling/readme.md index f58f70c4..d0ae7fb4 100644 --- a/docs/src/docs/development/advanced/graph-tiling/readme.md +++ b/docs/src/docs/development/advanced/graph-tiling/readme.md @@ -12,22 +12,22 @@ Graph Tiling Jobs In addition to [heatmap layers](../../how-to/standard-tiling/) of individual data points, Aperture Tiles supports visualizations of graph datasets that contain edge and node information. Graph visualizations illustrate the relationships between nodes and communities across multiple zoom levels. The process of generating the [tile pyramid](../../getting-started/tile-pyramid/) that represents this type of layer is a graph tiling job. -Graph tiling jobs comprise several configuration and generation phases as described in the [Graph Tiling Process](#process) section. +Graph tiling jobs comprise several configuration and generation phases as described in the [Graph Tiling Process](#graph-tiling-process) section. **NOTE**: The graph tiling capabilities of Aperture Tiles are under ongoing development and considered experimental. -## Graph Tiling Process ## +## Graph Tiling Process ## -Aperture Tiles requires graph data to be in a comma- or tab-delimited format (CSV). If your data is in a GraphML format, the first step in the graph tiling process is to [convert the GraphML data to a CSV format](#convert-graphml). +Aperture Tiles requires graph data to be in a comma- or tab-delimited format (CSV). If your data is in a GraphML format, the first step in the graph tiling process is to [convert the GraphML data to a CSV format](#converting-graphml-data-to-csv-data). Once your source data is in a valid delimited format, you can choose to apply either of the following optional layout styles before proceeding with the tile generation: -1. [Hierarchical Clustering](#clustering): Hierarchically group nodes into communities using Louvain detection based on modularity-maximization. +1. [Hierarchical Clustering](#hierarchical-clustering): Hierarchically group nodes into communities using Louvain detection based on modularity-maximization. 2. [Graph Layout](#graph-layout): Compute node locations using a force-directed algorithm. Once your data is prepared, you can execute your tile generation job using [standard](#tile-generation) or [customized](#graph-analytics) methods. -## Converting GraphML Data to CSV Data ## +## Converting GraphML Data to CSV Data ## Aperture Tiles requires graph data to be in comma- or tab-delimited format (CSV). GraphML data can be converted to CSV using the following tools in [com.oculusinfo.tilegen.graph.util](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-generation/src/main/java/com/oculusinfo/tilegen/graph/util): @@ -385,7 +385,7 @@ Using the brain connectomics dataset as an example: - Graph community *X* at hierarchy level 2 contains 100 communities (from hierarchy 1) - If the child community with the highest weighted degree has ID *0* and metadata *rh-rostralmiddlefrontal*, community *X* at hierarchy 2 will be labeled with the same ID and metadata. -## Graph Layout ## +## Graph Layout ## Node positions can be computed using a hierarchical force-directed algorithm with the following tools in [com.oculusinfo.tilegen.graph.util](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/graph/util): @@ -623,7 +623,7 @@ spark-submit --class com.oculusinfo.tilegen.graph.util.ClusteredGraphLayoutApp -eWeight true -g 0 -spark local -sparkhome /opt/spark -user ``` -## Tile Generation ## +## Tile Generation ## Once a graph dataset has been converted to CSV format and the nodes have been positioned (using raw positions for geo-located data or using the hierarchical force-directed algorithm), tile generation can be performed. Standard heatmaps of nodes and edges can be generated using the [CSVGraphBinner](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/examples/apps/CSVGraphBinner.scala) application in [com.oculusinfo.tilegen.examples.apps](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/examples/apps) @@ -793,7 +793,7 @@ oculus.binning.graph.data=edges oculus.binning.graph.edges.type=all ``` -## Graph Analytics ## +## Graph Analytics ## You can also create custom analytics of key communities at each hierarchy for client-side rendering to display labels, metadata and add interactive features as desired. Custom analytics can be created using the [GraphAnalyticsBinner](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/graph/analytics/GraphAnalyticsBinner.scala) in [com.oculusinfo.tilegen.graph.analytics](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/graph/analytics). diff --git a/docs/src/docs/development/advanced/standard-tiling/readme.md b/docs/src/docs/development/advanced/standard-tiling/readme.md index ca7bf27d..e9c5fad6 100644 --- a/docs/src/docs/development/advanced/standard-tiling/readme.md +++ b/docs/src/docs/development/advanced/standard-tiling/readme.md @@ -16,7 +16,7 @@ The following sections describe advanced properties available for [standard tili - [Field Scaling](#field-scaling) - [Consolidation Partitions](#consolidation-partitions) -## Source Data Format ## +## Source Data Format ## The **oculus.binning.parsing.<field>.fieldType** property indicates the type of values stored in a column that the CSVBinner will parse. Possible types include: @@ -143,7 +143,7 @@ oculus.binning.parsing.<field>.propertyValueSeparator=, In this case, a field value of `id=123;name=foo;description=bar` would yield the value *foo*. -## Field Scaling ## +## Field Scaling ## The following properties enable you modify the values of a field before it is used for binning: diff --git a/docs/src/docs/development/api/annotation/readme.md b/docs/src/docs/development/api/annotation/readme.md index 3b27da44..ee19bd06 100644 --- a/docs/src/docs/development/api/annotation/readme.md +++ b/docs/src/docs/development/api/annotation/readme.md @@ -7,13 +7,14 @@ permalink: docs/development/api/annotation/ layout: chapter --- -# Annotation Service # +Annotation Service +================== The Aperture Tiles Annotation Service REST API is based on the Aperture Tiles Binning API. Annotations are submitted into tiles and aggregated through a given range of levels based on bin-index. Annotations can be written, modified and removed in real time. -## Data Types ## +## Data Types ## -### Annotation ### +### Annotation ### A single annotation received from a GET request or submitted in a write, modify or remove POST request. @@ -80,7 +81,7 @@ Example: } ``` -### Annotation Tile ### +### Annotation Tile ### A collection of annotations from a single GET request. Contains all annotations (aggregated by bin-key) that pass through the current filter configuration. @@ -136,7 +137,7 @@ Example: } ``` -### Annotation Certificate ### +### Annotation Certificate ### An Annotation Certificate is comprised of the UUID and timestamp from a successful POST request. The certificate is used during remove and modify operations to ensure client-server coherency in the event of live editing conflicts. When a certificate in a supplied annotation does not match that held by the server, the request is ignored and an error status is returned to the client. @@ -165,7 +166,7 @@ Example: timestamp: "1401830862" ``` -## Web Services ## +## Web Services ## Web service paths are relative to a common base URL. For example, assuming the hostname *localhost* and port *8080*, the URLs of the various services would be: @@ -178,7 +179,7 @@ Web service paths are relative to a common base URL. For example, assuming the h - `https://localhost:8080/{version}/layer` - `https://localhost:8080/layer` -### Read Annotation Tile ### +### Read Annotation Tile ### Returns an Annotation Tile for a specified filter configuration UUID, annotation layer, level and tile x and y. @@ -290,7 +291,7 @@ Example response: } ``` -### Write Annotation ### +### Write Annotation ### Submits a new annotation to be written to the server. Upon success, a certificate containing the UUID and timestamp is returned to the client. @@ -439,7 +440,7 @@ Example response: } ``` -### Modify Annotation ### +### Modify Annotation ### Submits a modify request to the server. If the certificate in the annotation state is valid, the operation is processed and a new certificate is returned. @@ -592,7 +593,7 @@ Example response: } ``` -### Remove Annotation ### +### Remove Annotation ### Submits a removal request to the server. If the certificate is valid, the operation will be processed and a success status will be returned. diff --git a/docs/src/docs/development/api/layer/readme.md b/docs/src/docs/development/api/layer/readme.md index af08368d..034747a9 100644 --- a/docs/src/docs/development/api/layer/readme.md +++ b/docs/src/docs/development/api/layer/readme.md @@ -7,9 +7,10 @@ permalink: docs/development/api/layer/ layout: chapter --- -# Layer Service # +Layer Service +============= -## Configuration ## +## Configuration ## All server configurations adhere to the following general format: @@ -50,7 +51,7 @@ The minimal required fields are: ] ``` -## Request All Layers ## +## Request All Layers ## Return an object containing an array of: @@ -132,7 +133,7 @@ Example response: } ``` -## Request Specific Layer ## +## Request Specific Layer ## Returns the requested layer information. Layer information returned is the **public** node of the server-side configuration JSON. The layer IDs and metadata are appended to the returned JSON under **id** and **meta** attributes, respectively. @@ -200,7 +201,7 @@ Example response: } ``` -## Store a Configuration State ## +## Store a Configuration State ## Store a configuration state on the server that can be accessed at a later time. Returns the SHA-256 hash of the state. This SHA-256 hash can be used later as a 'state' query parameter. Hashes are deterministic and cachable. @@ -359,7 +360,7 @@ Example response: } ``` -## Get Configuration States ## +## Get Configuration States ## Returns all configured states stored on the server for a particular layer, including the default state. diff --git a/docs/src/docs/development/api/legend/readme.md b/docs/src/docs/development/api/legend/readme.md index 617a44e0..52e7cf99 100644 --- a/docs/src/docs/development/api/legend/readme.md +++ b/docs/src/docs/development/api/legend/readme.md @@ -7,9 +7,10 @@ permalink: docs/development/api/legend/ layout: chapter --- -# Legend Service # +Legend Service +============== -## Get Encoded Image Key ## +## Get Encoded Image Key ## Returns an image describing the values that a tile can take, with a specified orientation, width, and height. diff --git a/docs/src/docs/development/api/tile/readme.md b/docs/src/docs/development/api/tile/readme.md index 862c6779..7a6635e0 100644 --- a/docs/src/docs/development/api/tile/readme.md +++ b/docs/src/docs/development/api/tile/readme.md @@ -7,9 +7,10 @@ permalink: docs/development/api/tile/ layout: chapter --- -# Tile Service # +Tile Service +============ -## Get Tile ## +## Get Tile ## Returns a tile for a given layer at the specified level and index. diff --git a/docs/src/docs/development/getting-started/installation/readme.md b/docs/src/docs/development/getting-started/installation/readme.md index de38b0cc..5aca5e26 100644 --- a/docs/src/docs/development/getting-started/installation/readme.md +++ b/docs/src/docs/development/getting-started/installation/readme.md @@ -14,7 +14,7 @@ The following installation instructions are intended for developers who want to For developers who want to quickly install a pre-configured example Aperture Tiles application with minimal modification, a set of [packaged distributions](#packaged-distributions) are available as an alternative. -## Supported Platforms ## +## Supported Platforms ## Aperture Tiles is compatible with Linux and OS X. @@ -26,7 +26,7 @@ Compatibility for Windows is available through [Cygwin](https://cygwin.com/) or 1. **Spark release**: 1.3.0 2. **Package type**: Pre-built for CDH4 -## Prerequisites ## +## Prerequisites ## Aperture Tiles requires the following third-party tools: @@ -101,9 +101,9 @@ If you intend to work with datasets that cannot fit in the memory of a single ma Aperture Tiles Architecture Diagram -## Complete Install ## +## Complete Install ## -### Source Code ### +### Source Code ### The source code repository for Aperture Tiles is available on [GitHub](https://github.com/unchartedsoftware/). @@ -121,14 +121,14 @@ The source code repository for Aperture Tiles is available on [GitHub](https://g git checkout master ``` -### Building the Project ### +### Building the Project ### Once you have cloned the Aperture Tiles repository, you can build the project. -1. [Specify your Hadoop/HBase version](#hbase-version) (if applicable) -2. [Compile Aperture Tiles](#compiling) +1. [Specify your Hadoop/HBase version](#specifying-your-hadoophbase-version) (if applicable) +2. [Compile Aperture Tiles](#compiling-the-aperture-tiles-projects) -#### Specifying Your Hadoop/HBase Version #### +#### Specifying Your Hadoop/HBase Version #### **NOTE**: If you plan to run Apache Spark only in standalone mode on single machine, you can skip this step. @@ -137,7 +137,7 @@ Prior to building the project, you must specify which version of Hadoop and/or H 1. Review the *Deployment Variants* section of the [build.gradle](https://github.com/unchartedsoftware/aperture-tiles/blob/master/build.gradle) file in [aperture-tiles/](https://github.com/unchartedsoftware/aperture-tiles/tree/master) to check for valid settings for your version. 2. If your version is not included, you must build a new case for it. See the comments in the file for more details. -#### Compiling the Aperture Tiles Projects #### +#### Compiling the Aperture Tiles Projects #### Aperture Tiles builds with [Gradle](https://gradle.org/). @@ -169,7 +169,7 @@ Aperture Tiles builds with [Gradle](https://gradle.org/). This will compile all the project components and install .jar files for each project into a local Gradle repository on your build machine. -##### Project Structure ##### +##### Project Structure ##### Aperture Tiles is made up of the following sub-projects: @@ -267,6 +267,6 @@ The Aperture Tiles packaged distributions enable you to create a sample Aperture 2. Extract the contents of each file to your local machine. 3. See the [Quick Start](../quick-start/) topic for information on configuring each packaged distribution. -## Next Steps ## +## Next Steps ## For a guide on quickly configuring and deploying an example Aperture application, see the [Quick Start](../quick-start) topic. \ No newline at end of file diff --git a/docs/src/docs/development/getting-started/quick-start/readme.md b/docs/src/docs/development/getting-started/quick-start/readme.md index 35693f1f..1c36dc6b 100644 --- a/docs/src/docs/development/getting-started/quick-start/readme.md +++ b/docs/src/docs/development/getting-started/quick-start/readme.md @@ -19,13 +19,13 @@ At the end of this guide, you will have an example Aperture Tiles application th Aperture Tiles Julia Set Project -## Prerequisites ## +## Prerequisites ## Before beginning this Quick Start example, install the prerequisites described in the [Packaged Distribution](../installation/#packaged-distributions) section of the [Installation](../installation/) topic. Once you have installed all the prerequisites, you can generate the [Julia set data](#julia-set-data-generation), from which you will later create a set of tiles that will be used in your Aperture Tiles project. -### Julia Set Data Generation ### +### Julia Set Data Generation ###
    To use the Tile Generator utility to create the Julia set data
    @@ -43,7 +43,7 @@ Once you have installed all the prerequisites, you can generate the [Julia set d **NOTE**: For typical Aperture Tiles projects, these steps are unnecessary. You will instead begin with your own custom dataset. -## Tile Generation ## +## Tile Generation ## The first step in building any Aperture Tiles project is to create a set of [Avro](http://avro.apache.org/) tiles that aggregate your source data across the plot/map and its various zoom levels. @@ -52,7 +52,7 @@ For delimited numeric data sources like the Julia set, we use the CSVBinner tool - [Base properties file](#base-property-file-configuration), which describes the general characteristics of your data - [Tiling properties files](#tiling-property-file-configuration), each of which describes a specific attribute you want to plot and the number of zoom levels -### Base Property File Configuration ### +### Base Property File Configuration ### A preconfigured base properties file is available in the Tile Generator utility. You only need to edit this file if you intend to save your Avro tiles to HBase. Otherwise, you can skip ahead to the [execution](#execution) of the tile generation job. @@ -131,11 +131,12 @@ After you edit the properties files, you can use the Tile Generator utility to c .CSVBinner --master local[2] --driver-memory 1G lib/tile-generation-assembly.jar -d examples/julia-base.bd examples/julia-tiling.bd ``` -2. Check your output location (*julia.x.y.v*) for six subfolders, each corresponding to a zoom level (0, being the highest, through 5, being the lowest). Across all the folders, you should have a total of 1,365 Avro tile files. -## Tile Server Configuration ## +2. Check your output location (*julia.x.y.v*) for six subfolders, each corresponding to a zoom level (0 through 5 from highest to lowest). Across all the folders, you should have a total of 1,365 Avro tile files. -The Tile Server renders your generated tiles as layers in your Aperture Tiles visualization and passes them to the client. For this example, a preconfigured example server application has been provided as part of the Tile Quick Start Application ([tile-quickstart.war](../../../../download/#tile-quick-start-application)). +## Tile Server Configuration ## + +The Tile Server renders your generated tiles as layers in your Aperture Tiles visualization and passes them to the client. For this example, a preconfigured example server application is part of the Tile Quick Start Application ([tile-quickstart.war](../../../../download/#tile-quick-start-application)).
    To make tiles on the local file system available to Tile Quick Start Application
    @@ -193,13 +194,13 @@ For this example, you only need to edit the layer properties file if you saved y For information on additional layer properties, see the *Layers* section of the [Tile Server](../../how-to/tile-server/#layers) topic. -## Tile Client Application ## +## Tile Client Application ## For this example, a preconfigured example client application has been provided as part of the Tile Quick Start Application ([tile-quickstart.war](../../../../download/#tile-quick-start-application)). The client displays the base map or plot and any layers passed in from the server. -For information on map properties (e.g., for boundaries and axes), see the *Maps* section of the [Tile Client](../../how-to/tile-client/#maps) topic. +For information on map properties (e.g., for boundaries and axes), see the *Maps* section of the [Tile Client](../../how-to/tile-client/#map) topic. -## Deployment ## +## Deployment ##
    To deploy your application
    diff --git a/docs/src/docs/development/getting-started/tile-pyramid/diagram.md b/docs/src/docs/development/getting-started/tile-pyramid/diagram.md index c9f75269..d73c0b05 100644 --- a/docs/src/docs/development/getting-started/tile-pyramid/diagram.md +++ b/docs/src/docs/development/getting-started/tile-pyramid/diagram.md @@ -11,4 +11,4 @@ layout: image --- -[< Back](./#tile-gen-process) \ No newline at end of file +[< Back](./#tile-generation-process) \ No newline at end of file diff --git a/docs/src/docs/development/getting-started/tile-pyramid/readme.md b/docs/src/docs/development/getting-started/tile-pyramid/readme.md index 4198e832..cdffb435 100644 --- a/docs/src/docs/development/getting-started/tile-pyramid/readme.md +++ b/docs/src/docs/development/getting-started/tile-pyramid/readme.md @@ -12,7 +12,7 @@ Understanding the Tile Pyramid Using a distributed framework built on the [Apache Spark](https://spark.apache.org/) engine, Aperture Tiles enables you to create a set of data tiles that summarize and aggregate your large-scale data at various levels in a interactive, pyramid-style visualization. -## Tile Pyramid Hierarchy ## +## Tile Pyramid Hierarchy ## At the highest level in the tile set pyramid (level 0), a single tile summarizes all of your data. On each lower level, there are up to *4z* tiles, where *z* is the zoom level (with lower numbers indicating higher levels). At each level, the tiles are laid out row-wise across the base map or plot, starting at the lower left. @@ -20,7 +20,7 @@ Each tile summarizes the data in the region of the base map/plot to which it cor Tile Pyramid -## Tile Generation Process ## +## Tile Generation Process ## The process of generating a set of Avro tiles from your raw source data is called a tiling job. The tiles created by a job can be served and browsed in any modern Web browser. @@ -95,7 +95,7 @@ As shown in the following diagram, the tile generation process has five main sta Before you run a tiling job, make sure you meet all of the prerequisites listed in the following section. -## Tiling Job Prerequisites ## +## Tiling Job Prerequisites ## Aperture Tiles uses a distributed framework built on the Apache Spark engine to generate Avro tiles. Before you run a tiling job, you must install and configure the following prerequisites: @@ -118,7 +118,7 @@ Perform one of the following actions: - [Download](http://spark.apache.org/downloads.html) the correct version directly. - If no version is listed for your flavor of Hadoop, [build](http://spark.apache.org/docs/latest/building-with-maven.html) Spark to support it. -## spark-submit ## +## spark-submit ## The Aperture Tiles source code relies on the standard [spark-submit](http://spark.apache.org/docs/1.3.0/submitting-applications.html) script to build your tiles. The script simplifies the process of running Spark jobs by including all the necessary libraries and setting various parameters. diff --git a/docs/src/docs/development/how-to/standard-tiling/readme.md b/docs/src/docs/development/how-to/standard-tiling/readme.md index 58253c0f..e3612383 100644 --- a/docs/src/docs/development/how-to/standard-tiling/readme.md +++ b/docs/src/docs/development/how-to/standard-tiling/readme.md @@ -16,12 +16,12 @@ Standard tiling jobs can be executed with the [CSVBinner](#csvbinner), which is For information on creating custom tiling jobs that ingest data that is not character delimited or contains non-numeric fields, see the [Run Custom Tiling Jobs](../../advanced/custom-tiling/) topic. -## CSVBinner ## +## CSVBinner ## The CSVBinner ingests numeric, character-separated (e.g., CSV) tabular data. To define the tile set you want to create, you must pass two types of properties files to the CSVBinner: -- A [base properties](#base-properties) file, which describes the general characteristics of the data -- [Tiling properties](#tiling-properties) files, each of which describes the specific attributes you want to tile +- A [base properties](#base-properties-files) file, which describes the general characteristics of the data +- [Tiling properties](#tiling-properties-files) files, each of which describes the specific attributes you want to tile During the tiling job, the CSVBinner writes a set of Avro tile data files to the location of your choice (HBase or local file system). @@ -257,7 +257,7 @@ The tiling properties files define the tiling job parameters for each layer in y Additional properties are described in the advanced [Standard Tiling](../../advanced/standard-tiling/) topic. -### Projection ### +### Projection ### The projection properties define the area on which your data points are plotted. @@ -403,7 +403,7 @@ The index properties specify the fields used to locate the binning value on the **NOTE**: An additional index scheme (*segment*) is available for tiling node and edge graph data. See the [Graph Tiling](../../advanced/graph-tiling/) topic for more information. -### Value ### +### Value ### The value properties specify the field to use as the binning value and how multiple values in the same bin should be combined. diff --git a/docs/src/docs/development/how-to/test-output/readme.md b/docs/src/docs/development/how-to/test-output/readme.md index 7e71168d..edc2f6be 100644 --- a/docs/src/docs/development/how-to/test-output/readme.md +++ b/docs/src/docs/development/how-to/test-output/readme.md @@ -13,11 +13,11 @@ Testing Tiling Job Output The Aperture Tiles source code contains two utilities for testing the output of tiling jobs. Both utilities are Java applications that can help you quickly identify problems with your tiling job before you deploy your application. - [Bin Visualizer](#bin-visualizer): Renders basic representations of individual Avro tiles in your tile pyramid. -- [JSON Tile Visualizer](#json-visualizer): Generates a list of data or bin values that appear in individual Avro tiles in your tile pyramid. +- [JSON Tile Visualizer](#json-tile-visualizer): Generates a list of data or bin values that appear in individual Avro tiles in your tile pyramid. If you notice any inconsistencies with the tiled data, you should review your tiling job parameters, fix any errors and run the job again. -## Bin Visualizer ## +## Bin Visualizer ## The [BinVisualizer.java](https://github.com/unchartedsoftware/aperture-tiles/blob/master/binning-utilities/src/main/java/com/oculusinfo/binning/visualization/BinVisualizer.java) file in [binning-utilities/src/main/java/com/oculusinfo/binning/visualization/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/binning-utilities/src/main/java/com/oculusinfo/binning/visualization) allows you to preview tiling job output before you configure your application. @@ -101,7 +101,7 @@ The [BinVisualizer.java](https://github.com/unchartedsoftware/aperture-tiles/blo Bin Visualizer -## JSON Tile Visualizer ## +## JSON Tile Visualizer ## **NOTE**: The JSON Tile Visualizer currently only supports heatmap layers. diff --git a/docs/src/docs/development/how-to/tile-client/readme.md b/docs/src/docs/development/how-to/tile-client/readme.md index 970bfde6..47f7f7ab 100644 --- a/docs/src/docs/development/how-to/tile-client/readme.md +++ b/docs/src/docs/development/how-to/tile-client/readme.md @@ -14,11 +14,11 @@ Once you have customized your Tile Server, you should customize the Tile Client, **NOTE**: The Tile Client can also be configured to receive tiles as JSON data from the Server and then render them directly. -## Application JavaScript ## +## Application JavaScript ## The application JavaScript file (*/src/main/webapp/js/***app.js**) should request layers from the server and return an array of layer configuration objects. Among the objects it should instantiate are the map, its baselayer and axis configurations. -### Map ### +### Map ### The map describes the base map upon which your source data is projected: @@ -37,7 +37,7 @@ pyramid: { **NOTE**: Your layer and map pyramid configurations must match each other. -### BaseLayer ### +### BaseLayer ### The BaseLayer parameters use map provider APIs to determine what features to include on the base map. In the following example, the Google Maps API (OpenLayers.Layer.Google) is used to define the style of the base map. @@ -96,7 +96,7 @@ The next example shows a TMS layer configuration (standard OpenLayers.Layer.TMS) } ``` -### Axes ### +### Axes ### The AxisConfig parameters determine how the X and Y axes are drawn in your cross-plot map. diff --git a/docs/src/docs/development/how-to/tile-server/readme.md b/docs/src/docs/development/how-to/tile-server/readme.md index a1d5e1b4..82396297 100644 --- a/docs/src/docs/development/how-to/tile-server/readme.md +++ b/docs/src/docs/development/how-to/tile-server/readme.md @@ -14,20 +14,20 @@ Once you have copied an existing Aperture Tiles application as a template, you s The Tile Server in your new template relies on the following configuration files: -- [Web XML](#webxml), which defines which modules Guice will use. -- [Tile Properties](#tileproperties), which specifies constants used by Guice during initialization, including the location of your Spark installation, and the location of some server configuration files. +- [Web XML](#web-xml), which defines which modules Guice will use. +- [Tile Properties](#tile-properties), which specifies constants used by Guice during initialization, including the location of your Spark installation, and the location of some server configuration files. - [Layers](#layers), which defines the individual layers of data that can be overlaid on your base map. The layers file also indicates whether rendering should be performed by the server or the client. The remainder of the configuration, which includes instantiation of the map, its base features and axis configuration, should be handled in the [Application JavaScript](../tile-client/#application-javascript) (*/src/main/webapp/js/***app.js**). -## Web XML ## +## Web XML ## Edit the client **web.xml** file in *new-project/src/main/webapp/WEB-INF/*: 1. If you performed a custom tile generation, edit the guice-modules parameter to pass in any custom modules you created (e.g., your custom Tile Serialization Factory). 2. If required, uncomment the relevant Spark lines in the guice-modules to enable live tiling or drill-through to raw data. - -## Tile Properties ## + +## Tile Properties ## Edit the client **tile.properties** file in *new-project/src/main/resources/*. This file specifies parameters for use by Guice, such as the location of your layer and annotation directories. Additional Spark parameters are available for on-demand tile generation and data drill down: @@ -36,13 +36,13 @@ Edit the client **tile.properties** file in *new-project/src/main/resources/*. T - The home directory of Spark - Any JARs you want to add to the Spark context -## Layers # +## Layers ## The layers file describes the tile layers to be made available to the server and client application. Parameters in the layers file are split into two sections: those in the public node are accessible from the client, while those under the private note are not. Layer file examples can be found in in the Julia example at [tile-examples\julia-demo\src\main\resources\layers](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-examples/julia-demo/src/main/resources/layers) and the Twitter Topics example at [tile-examples\twitter-topics\twitter-topics-client\src\main\resources\layers](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-examples/twitter-topics/twitter-topics-client/src/main/resources/layers). -### ID ### +### ID ### The ID parameter uniquely identifies the layer. @@ -65,7 +65,7 @@ The ID parameter uniquely identifies the layer. Parameters in the **public** node section of the **layers.json** file are accessible from the client. -#### Pyramid #### +#### Pyramid #### The pyramid parameters describe the extent of the data in the layer. The values that you provide in this section must match the values in your data source and in your map configuration. @@ -206,7 +206,7 @@ The **valueTransform** defines the **type** of transformation that can be applie Parameters in the **private** node section of the **layers.json** file are not accessible from the client. -#### Data #### +#### Data #### The data parameters specify the location of the tiles that you created. If you are using HBase, separate parameters are required. diff --git a/docs/src/docs/user-guide/interface/readme.md b/docs/src/docs/user-guide/interface/readme.md index 943f0aaf..9616c284 100644 --- a/docs/src/docs/user-guide/interface/readme.md +++ b/docs/src/docs/user-guide/interface/readme.md @@ -13,12 +13,12 @@ The user interface for a typical Aperture Tiles project contains the following c - [Map](#map) - [Axes](#axes) - [Layers](#layers) - - [Heatmap](#heatmap) - - [Tile Carousel](#carousel) - - [Controls](#controls) + - [Heatmap](#heatmap-layer) + - [Tile Carousel](#tile-carousel) + - [Controls](#controls) - [Description](#description) -## Map ## +## Map ## The map is the base layer upon which your data is projected. Aperture Tiles supports geographic and cross-plot maps. At each zoom level, the map is automatically scaled to show more or less detail. @@ -32,19 +32,19 @@ The map for the Twitter Topics application is a Google Maps layer that supports Just like [layers](#layers), the base map can be configured or toggled using the [Controls](#controls) dialog. -### Axes ### +### Axes ### The x and y axes are labeled to describe the units in your base map. Click either axis to show or hide its units. Axis -## Layers ## +## Layers ## In addition to the map (base layer), Aperture Tiles supports various overlays that summarize your data. These layers can be toggled and configured to enable exploration and customization of any subset of data. -Aperture Tiles supports both [heatmap](#heatmap) layers and [tile-based data summary](#carousel) layers. Layers can be configured or toggled using the [Controls](#controls) dialog. +Aperture Tiles supports both [heatmap](#heatmap-layer) layers and [tile-based data summary](#tile-carousel) layers. Layers can be configured or toggled using the [Controls](#controls) dialog. -### Heatmap Layer ### +### Heatmap Layer ### A typical Aperture Tiles project has a heatmap layer showing the concentration of the source data. The heatmap layer in the Twitter Topics application shows the concentration of Twitter messages across South America. @@ -56,7 +56,7 @@ The color of the data points on this layer indicate the concentration of tweets While the heatmap in this application shows an aggregation of the individual tweets in a region, other Aperture Tiles applications can be configured to incorporate layers that show only the minimum or maximum value on tile. -### Tile Carousel ### +### Tile Carousel ### Each tile in the Twitter Topics application contains a carousel with multiple views that summarize the data contained in each region. As they are highly specific to the source data, each tile carousel implementation generally requires custom coding. @@ -97,7 +97,7 @@ Clicking on a particular topic on any of the carousel pages refreshes the map to Tile Carousel Topic Detail -### Controls ### +### Controls ### The Controls panel is an expandable dialog with configurable settings that let you: @@ -111,7 +111,7 @@ You can also use the Controls to change the color scheme (ramp) of your heatmap Layer Settings -## Description ## +## Description ## The Description panel is an expandable dialog that briefly defines the source data and its representation in the Aperture Tiles application. diff --git a/docs/src/docs/user-guide/navigation/readme.md b/docs/src/docs/user-guide/navigation/readme.md index 8fb9b1a1..2b743434 100644 --- a/docs/src/docs/user-guide/navigation/readme.md +++ b/docs/src/docs/user-guide/navigation/readme.md @@ -10,7 +10,7 @@ layout: chapter The Aperture Tiles user interface works much like a modern web-based map system. Your source data is projected on a set of tiles in an x/y cross-plot or a geographic map. In either case, your data is aggregated at several zoom levels so you can view the data at a high level or drill down to see more detailed distribution. -## Zoom ## +## Zoom ##
    To zoom in on an area you want to investigate
    @@ -22,7 +22,7 @@ The Aperture Tiles user interface works much like a modern web-based map system. - Click the **Zoom Out** ![Zoom Out](../../../img/zoom-out.png) button in the upper right corner of the map, OR - Use the scroll button on your mouse to quickly zoom in or out -### Zoom to City ### +### Zoom to City ###
    To quickly zoom to a major city in South America
    @@ -38,7 +38,7 @@ The map is redrawn to show the selected city with a zoom factor of five. **NOTE**: This is customized control that requires specialized code not available in the standard Aperture Tiles template or base libraries. -## Explore ## +## Explore ##
    To view other areas of the map
    diff --git a/docs/src/docs/user-guide/readme.md b/docs/src/docs/user-guide/readme.md index 6d15de9e..b48fe237 100644 --- a/docs/src/docs/user-guide/readme.md +++ b/docs/src/docs/user-guide/readme.md @@ -12,7 +12,7 @@ This page is designed for analysts who want to understand how to navigate the Ap **NOTE**: Fully functional example Aperture Tiles applications are available on the the [Demos](../../demos/) page. Copies of some of these demos are also available for your use in the Aperture Tiles source code (in [tile-examples/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-examples)). -## Twitter Latin American Topic Summarization Data ## +## Twitter Latin American Topic Summarization Data ## The example Twitter Topics application described in this guide explores a heatmap of 66,996,907 Twitter messages originating in South American between February and April, 2014. Tiles at every zoom level contain a carousel that summarizes and overlays the top words used in the tweets in that region. diff --git a/docs/src/readme.md b/docs/src/readme.md index d537b7ab..74fc69d8 100644 --- a/docs/src/readme.md +++ b/docs/src/readme.md @@ -5,7 +5,7 @@ layout: section --- Aperture Tiles tile-based visual analytics for big data -======================================================= +====================================================================================
    Aperture Tiles is powered by Salt, an open
    source library for scalable, analytic layered tiles.
    @@ -19,7 +19,7 @@ Tailored analytic overlays (e.g., alerts, pattern detection, feature extraction) Aperture Tiles is an open source software product managed by Uncharted Software Inc. The MIT License is used. -### Easy-to-Use Big Data Visual Analytics +### Easy-to-Use Big Data Visual Analytics ### The widespread adoption of web-based geographic maps provides a familiar set of zoom/pan interactions that can be similarly used for working with extremely large, abstract data spaces. Aperture Tiles builds on these techniques to provide tools for big data visual analytics that use tile-based rendering and analytics. @@ -31,11 +31,11 @@ Tile-based geographic maps provide an interactive experience and solid framework With widespread use, map interactions have become familiar and make exploration of large data spaces easy and even enjoyable. -### Tile-Based Visual Analytics +### Tile-Based Visual Analytics ### Tile-based visual analytics (TBVA) is a technique that divides the data into evenly sized hierarchical tiles, and then calculates and applies an analytic to the data contained in each tile. For example, if examining a large Twitter dataset, the data would be divided into equally sized tiles across the geographic area. Then, an analytic is applied to each tile, such as the top five Twitter hashtags for the bounded region. As the user zooms into the map, Aperture Tiles presents a new layer of tiles that each contain a smaller portion of the full dataset. In each new zoom layer, the analytic applied to each tile becomes more localized. -### Extensible and Open Source +### Extensible and Open Source ### The Aperture Tiles services and API are designed to be extensible, allowing a broad community to leverage and extend its capabilities in creative ways. The extensible Avro tile format allows generation of tiles by third parties, which can be served using the Aperture Tiles server. Aperture Tiles leverages open standards such as [Tile Map Services (TMS)](http://en.wikipedia.org/wiki/Tile_Map_Service) that are widely supported by web mapping clients and servers. @@ -43,7 +43,7 @@ Aperture Tile generation service builds on the [Apache Spark](http://spark.incub Aperture Tiles is under ongoing development and is freely available for download under [The MIT License](http://www.opensource.org/licenses/MIT) open source licensing. Unlike GNU General Public License (GPL), MIT freely permits distribution of derivative work under proprietary license, without requiring the release of source code. -###Interested in Learning More? +### Interested in Learning More? ### * [Tour](tour/): Take our tour to learn more about Aperture Tiles. * [Documentation](docs/development/getting-started/quick-start/): Learn how to install, implement and test your Aperture Tiles applications. diff --git a/docs/src/tour/components/readme.md b/docs/src/tour/components/readme.md index 128cd71e..12d73a38 100644 --- a/docs/src/tour/components/readme.md +++ b/docs/src/tour/components/readme.md @@ -13,22 +13,19 @@ Aperture Tiles implements a data flow that breaks the visualization process into - Using the Aperture Tiles Avro tile data format, third parties can produce custom tile data using their own analytics. - Using the Aperture Tile Server standards-based [Tile Map Services](http://en.wikipedia.org/wiki/Tile_Map_Service), developers can integrate a third-party tile client. -Distributed Tile Generation ---------------------------- +## Distributed Tile Generation ## The Aperture Tiles generation analytic leverages cluster computing with [Apache Spark](http://spark.incubator.apache.org/) to rapidly compute tiles from terabytes of "big data" (investigations are ongoing to work with petabyte-sized datasets). The tile generation analytic transforms raw data into Avro data tile files with metadata descriptor files, summarizing all tile content. Avro data files allow for language neutrality and support multiple summaries per bin, allowing a flexible and extensible analytic. Finished tiles are stored in HBase to allow efficient scalability to billions of tiles. -###Tile Set Size and Resolution +### Tile Set Size and Resolution ### ![Billions of Tiles](../../img/billions-of-tiles.png) -Tile Server ------------ +## Tile Server ## The Aperture Tile server provides services to serve rendered tiles or raw tile data. The Aperture Tile Server defines a REST API for requesting tiles and rendering properties. The tile server works by retrieving Avro data tile files and rendering PNG images for display in the tile client or external visualization applications. The Aperture Tile Server also has the ability to directly serve raw tile data via JSON, supporting data services or rich client side rendering capabilities. -Tile Client ------------ +## Tile Client ## The Aperture Tile client is a full-featured, interactive web application for displaying layered tiles with zoom, pan, and filtering interactions. Example client side rendering (of aggregation marker visualization techniques) is provided to demonstrate some of the possibilities. The tile client web application is designed to promote: @@ -37,6 +34,6 @@ The Aperture Tile client is a full-featured, interactive web application for dis This is handled by displaying high-level patterns and summary statistics while preserving fine detail at the appropriate zoom level. -## Next Steps +## Next Steps ## For more information on the process of developing a custom visual analytic with Aperture Tiles, see the [Process](../process/) topic. \ No newline at end of file diff --git a/docs/src/tour/process/readme.md b/docs/src/tour/process/readme.md index 2be74768..341c340e 100644 --- a/docs/src/tour/process/readme.md +++ b/docs/src/tour/process/readme.md @@ -10,18 +10,15 @@ Aperture Tiles Process Aperture Tiles uses a multi-stage process to develop tile-based visual analytics. For a detailed description of the steps required to develop custom visual analytics, see the [Quick Start Guide](../../docs/development/getting-started/quick-start/). -Aggregation Stage ------------------ +## Aggregation Stage ## The aggregation stage projects and bins data into a predefined grid, such as the Tiling Map Service standard. This uses a (z,x,y) coordinate system, where z identifies the zoom level, and x,y identifies a specific tile on the plot for the given zoom level. -Tile Summarization Stage ------------------------- +## Tile Summarization Stage ## The summarization stage applies one or more summary statistics or other analytics to the data in each tile region, storing the result in a data store. -Tile Rendering Stage --------------------- +## Tile Rendering Stage ## The rendering stage maps the summary to a visual representation, and renders it to an image tile or html at request time. This stage is capable of rendering on-demand to support interactions such as filtering, or rich dynamic interactions such as brushing and drill-downs, if using client-side rendering with html and JavaScript. diff --git a/docs/src/tour/readme.md b/docs/src/tour/readme.md index 478ca25a..3f49407e 100644 --- a/docs/src/tour/readme.md +++ b/docs/src/tour/readme.md @@ -22,8 +22,7 @@ There are several mature, feature-rich JavaScript libraries that support integra Aperture Tiles addresses these drawbacks by using cloud-distributed computation. This method generates data for tiles in far less time, by using defined, reproducible data flows that can regenerate tiles from live data. This not only speeds the process of generating data for tiles, but serves to reduce the staleness of the data in any given tile. Aperture Tiles also separates tile data apportionment from the visual rendering process. This approach allows tile visualization to be altered in real-time, allowing interactions like filtering of data values or the application of new visualization techniques. Most importantly, tile displays are not limited to static images. Tile-based visual analytics can be the centerpiece of highly interactive visual analytic applications using web-based, client-side rendering, such as [Aperture JS](http://aperturejs.com/). -Computing Tile-based Visual Analytics -------------------------------------- +## Computing Tile-based Visual Analytics ## Aperture Tiles uses a multi-stage process to develop tile-based visual analytics. @@ -31,6 +30,6 @@ Aperture Tiles uses a multi-stage process to develop tile-based visual analytics 2. The summary and analytical stage applies one or more summary statistics or other analytics to the data in each tile region, storing the result in a data store. 3. The rendering stage maps the computed summary and analytics to visual representations, and renders them to an image tile or html at request time. This stage is capable of rendering on-demand interactions such as filtering, or rich dynamic interactions such as brushing and drill-downs, if using client-side rendering with html and JavaScript. -## Next Steps +## Next Steps ## For more information on the components that make up Aperture Tiles, see the [System Description](components/) topic. \ No newline at end of file From 469f09f3a51200b9219f405ec32e8c3670164b98 Mon Sep 17 00:00:00 2001 From: Michael Crouch Date: Tue, 7 Jun 2016 14:21:19 -0400 Subject: [PATCH 3/5] Update installation instructions --- docs/readme.md | 11 +++----- .../getting-started/installation/readme.md | 26 +++++++++---------- .../getting-started/quick-start/readme.md | 22 ++++++++-------- .../getting-started/tile-pyramid/readme.md | 8 +++--- 4 files changed, 32 insertions(+), 35 deletions(-) diff --git a/docs/readme.md b/docs/readme.md index 27e5ff96..5e3e8688 100644 --- a/docs/readme.md +++ b/docs/readme.md @@ -3,7 +3,7 @@ Building Aperture Sites with Jekyll **NOTE**: This topic describes how to build the Aperture Tiles website. To view the Aperture Tiles website content, visit or see the [src/](src/) folder. -Uncharted Software's Aperture websites (Aperture JS, Aperture Tiles and Influent) contain detailed documentation for both developers and end users. Each site is built using [Jekyll](http://jekyllrb.com/), a static website generator. All of the website content is written in Markdown (*.md), a plain text formatting syntax, and stored separately from the website layout (HTML) and styles (Sass/CSS). Jekyll generates the sites by parsing the Markdown contents, converting them to HTML and inserting them into the appropriate layout. +Uncharted Software's Aperture websites (Aperture Tiles and Influent) contain detailed documentation for both developers and end users. Each site is built using [Jekyll](http://jekyllrb.com/), a static website generator. All of the website content is written in Markdown (*.md), a plain text formatting syntax, and stored separately from the website layout (HTML) and styles (Sass/CSS). Jekyll generates the sites by parsing the Markdown contents, converting them to HTML and inserting them into the appropriate layout. You can use Jekyll to build the website source files if you want to: @@ -13,7 +13,6 @@ You can use Jekyll to build the website source files if you want to: Note that the latest versions of the pre-built Aperture websites are available for browsing at the following locations: -- Aperture JS: [http://aperturejs.com/](http://aperturejs.com/) - Aperture Tiles: [http://aperturetiles.com/](http://aperturetiles.com/) - Influent: [http://community.influent.org/](http://community.influent.org/) @@ -21,12 +20,10 @@ Note that the latest versions of the pre-built Aperture websites are available f Before you attempt to build an Aperture site, you should install the following prerequisites. Note that while other versions of the third-party tools below may work, only the listed versions have been tested with the latest site build. -- **Operating System**: Linux, Unix or Mac OS X. (Windows support with [Cygwin](https://www.cygwin.com/) version 1.7.33) -- **Language**: [Ruby](https://www.ruby-lang.org/) (version 1.9.3) - - **Package Manager**: [RubyGems](https://rubygems.org/) (version 1.8.29) +- **Language**: [Ruby](https://www.ruby-lang.org/) (version 2.1.5) + - **Package Manager**: [RubyGems](https://rubygems.org/) (version 2.2.2) - **Gems**: - - **Site Generator**: [Jekyll](http://jekyllrb.com/) (version 2.5.3) - - **Syntax Highlighter**: [Rouge](https://github.com/jneen/rouge) (version 1.7.7) + - **Site Generator**: [Jekyll](http://jekyllrb.com/) (version 3.1.6) ## Building the Site ## diff --git a/docs/src/docs/development/getting-started/installation/readme.md b/docs/src/docs/development/getting-started/installation/readme.md index 5aca5e26..d38492e9 100644 --- a/docs/src/docs/development/getting-started/installation/readme.md +++ b/docs/src/docs/development/getting-started/installation/readme.md @@ -22,9 +22,9 @@ Compatibility for Windows is available through [Cygwin](https://cygwin.com/) or
    To use Aperture Tiles with Apache Spark in standalone mode in Windows
    -- [Download](https://spark.apache.org/downloads.html) and install a pre-built version of Apache Spark with the following properties: - 1. **Spark release**: 1.3.0 - 2. **Package type**: Pre-built for CDH4 +- [Download](https://spark.apache.org/downloads.html) and install a pre-built version of Apache Spark with the following properties: + 1. **Spark release**: 1.3.0 + 2. **Package type**: Pre-built for Hadoop 2.4 ## Prerequisites ## @@ -107,15 +107,15 @@ If you intend to work with datasets that cannot fit in the memory of a single ma The source code repository for Aperture Tiles is available on [GitHub](https://github.com/unchartedsoftware/). -
    To work with the Aperture Tiles source code
    +
    To work with the most recent stable release of the Aperture Tiles source code
    -1. Execute the following command to clone the Aperture Tiles repository to an *aperture-tiles/* directory in your Git project folder: +1. Clone the Aperture Tiles repository to an *aperture-tiles/* directory in your Git project folder: ```bash git clone https://github.com/unchartedsoftware/aperture-tiles.git ``` -2. Check out the *master* branch to work with the most recent stable release: +2. Check out the *master* branch: ```bash git checkout master @@ -135,7 +135,7 @@ Once you have cloned the Aperture Tiles repository, you can build the project. Prior to building the project, you must specify which version of Hadoop and/or HBase you have installed (if applicable): 1. Review the *Deployment Variants* section of the [build.gradle](https://github.com/unchartedsoftware/aperture-tiles/blob/master/build.gradle) file in [aperture-tiles/](https://github.com/unchartedsoftware/aperture-tiles/tree/master) to check for valid settings for your version. -2. If your version is not included, you must build a new case for it. See the comments in the file for more details. +2. If your version is not included, build a new case for it. See the comments in the file for more details. #### Compiling the Aperture Tiles Projects #### @@ -248,12 +248,12 @@ The packaged distributions of Aperture Tiles require only a subset of the [prere
    To install the prerequisites for the packaged distributions
    -1. Install [Apache Spark](http://spark.incubator.apache.org/). -2. If you want to understand how Aperture Tiles works with particularly large datasets, install your preferred flavor of [Hadoop](http://hadoop.apache.org/): - - [Cloudera](http://www.cloudera.com/content/www/en-us/downloads/cdh.html) v4.6 (*recommended*) - - [Apache](http://hadoop.apache.org/docs/r1.2.1/index.html) - - [MapR](http://www.mapr.com/products/apache-hadoop) - - [HortonWorks](http://hortonworks.com/) +1. Install [Apache Spark](http://spark.incubator.apache.org/). +2. If you want to understand how Aperture Tiles works with particularly large datasets, install your preferred flavor of [Hadoop](http://hadoop.apache.org/): + - [Cloudera](http://www.cloudera.com/content/www/en-us/downloads/cdh.html) v5.4.7 (*recommended*) + - [Apache](http://hadoop.apache.org/docs/r1.2.1/index.html) + - [MapR](http://www.mapr.com/products/apache-hadoop) + - [HortonWorks](http://hortonworks.com/) ### Download the Distributions ### diff --git a/docs/src/docs/development/getting-started/quick-start/readme.md b/docs/src/docs/development/getting-started/quick-start/readme.md index 1c36dc6b..470d1793 100644 --- a/docs/src/docs/development/getting-started/quick-start/readme.md +++ b/docs/src/docs/development/getting-started/quick-start/readme.md @@ -23,13 +23,13 @@ At the end of this guide, you will have an example Aperture Tiles application th Before beginning this Quick Start example, install the prerequisites described in the [Packaged Distribution](../installation/#packaged-distributions) section of the [Installation](../installation/) topic. -Once you have installed all the prerequisites, you can generate the [Julia set data](#julia-set-data-generation), from which you will later create a set of tiles that will be used in your Aperture Tiles project. +Once you install all the prerequisites, you can generate the [Julia set data](#julia-set-data-generation), from which you will later create a set of tiles that will be used in your Aperture Tiles project. ### Julia Set Data Generation ###
    To use the Tile Generator utility to create the Julia set data
    -1. Execute the standard [spark-submit](http://spark.apache.org/docs/1.3.0/submitting-applications.html) script using the following command, changing the **output** URI to specify where you want to save the data (HDFS or local file system): +1. Execute the standard [spark-submit](http://spark.apache.org/docs/1.3.0/submitting-applications.html) script, changing the **output** URI to specify where you want to save the data (HDFS or local file system): ```bash $SPARK_HOME/bin/spark-submit --class com.oculusinfo.tilegen.examples.datagen @@ -47,16 +47,16 @@ Once you have installed all the prerequisites, you can generate the [Julia set d The first step in building any Aperture Tiles project is to create a set of [Avro](http://avro.apache.org/) tiles that aggregate your source data across the plot/map and its various zoom levels. -For delimited numeric data sources like the Julia set, we use the CSVBinner tool to create these tiles. The CSVBinner tool requires two types of input: +To create tiles for delimited numeric data sources like the Julia set, we use the CSVBinner tool. The CSVBinner tool requires two types of input: - [Base properties file](#base-property-file-configuration), which describes the general characteristics of your data - [Tiling properties files](#tiling-property-file-configuration), each of which describes a specific attribute you want to plot and the number of zoom levels ### Base Property File Configuration ### -A preconfigured base properties file is available in the Tile Generator utility. You only need to edit this file if you intend to save your Avro tiles to HBase. Otherwise, you can skip ahead to the [execution](#execution) of the tile generation job. +A preconfigured base properties file is available in the Tile Generator utility. You only need to edit this file if you want to save your tiles to HBase. Otherwise, skip ahead to the [execution](#execution) of the tile generation job. -**NOTE**: For a typical Aperture Tiles project, you will need to edit the additional properties files to define the types of fields in your source data. For more information on these properties, see the [Standard Tiling Jobs](../../how-to/standard-tiling/) topic. +**NOTE**: For a typical Aperture Tiles project, you will edit the additional properties files to define the types of fields in your source data. For more information on these properties, see the [Standard Tiling Jobs](../../how-to/standard-tiling/) topic.
    To edit the base properties file
    @@ -124,7 +124,7 @@ After you edit the properties files, you can use the Tile Generator utility to c
    To execute the tile generation job
    -1. Execute the standard **spark-submit** script again, invoking the CSVBinner and using the *-d* switch to pass your edited base properties file. Tiling properties files can be passed in without a switch. +1. Execute the standard **spark-submit** script again, invoking the CSVBinner and using the *-d* switch to pass your edited base properties file. Tiling properties files can be passed in without a switch. ```bash $SPARK_HOME/bin/spark-submit --class com.oculusinfo.tilegen.examples.apps @@ -143,13 +143,13 @@ The Tile Server renders your generated tiles as layers in your Aperture Tiles vi 1. Zip the *julia.x.y.v* directory produced during the Tile Generation stage. 2. Copy the ZIP file to the *WEB-INF/classes/* directory of the Tile Quick Start Application. -**NOTE**: For typical Aperture Tiles projects, you will need to edit the */WEB-INF/***web.xml** and *WEB-INF/classes/***tile.properties** files in the Tile Server. For more information on editing these files, see the [Tile Server](../../how-to/tile-server/) topic. +**NOTE**: For typical Aperture Tiles projects, you will edit the */WEB-INF/***web.xml** and *WEB-INF/classes/***tile.properties** files in the Tile Server. For more information on editing these files, see the [Tile Server](../../how-to/tile-server/) topic. ### Layer Properties ### -Layer properties (within the **tile-quickstart.war** at *WEB-INF/classes/layers/***julia-layer.json**) specify the layers that can be overlaid on your base map or plot. +Layer properties (within the **tile-quickstart.war** at *WEB-INF/classes/layers/***julia-layer.json**) specify the layers that can be overlaid on your base map or plot. -For this example, you only need to edit the layer properties file if you saved your Avro tiles to HBase. Otherwise, you can skip ahead to the configuration of the [Tile Client Application](#tile-client-application). +For this example, you only need to edit the layer properties file if you saved your tiles to HBase. Otherwise, skip ahead to the configuration of the [Tile Client Application](#tile-client-application).
    To edit the layer properties
    @@ -196,7 +196,7 @@ For information on additional layer properties, see the *Layers* section of the ## Tile Client Application ## -For this example, a preconfigured example client application has been provided as part of the Tile Quick Start Application ([tile-quickstart.war](../../../../download/#tile-quick-start-application)). The client displays the base map or plot and any layers passed in from the server. +For this example, a preconfigured example client application is part of the Tile Quick Start Application ([tile-quickstart.war](../../../../download/#tile-quick-start-application)). The client displays the base map or plot and any layers passed in from the server. For information on map properties (e.g., for boundaries and axes), see the *Maps* section of the [Tile Client](../../how-to/tile-client/#map) topic. @@ -205,7 +205,7 @@ For information on map properties (e.g., for boundaries and axes), see the *Maps
    To deploy your application
    1. Copy the **tile-quickstart.war** to the *webapps/* directory of your web server (e.g., Apache Tomcat or Jetty). -2. Restart the server, if necessary +2. If necessary, restart the server. 3. Access your application in web browser at http://localhost:8080/julia-demo. The Julia set application data is plotted on an X/Y chart with six layers of zoom available. diff --git a/docs/src/docs/development/getting-started/tile-pyramid/readme.md b/docs/src/docs/development/getting-started/tile-pyramid/readme.md index cdffb435..a6a02eae 100644 --- a/docs/src/docs/development/getting-started/tile-pyramid/readme.md +++ b/docs/src/docs/development/getting-started/tile-pyramid/readme.md @@ -16,13 +16,13 @@ Using a distributed framework built on the [Apache Spark](https://spark.apache.o At the highest level in the tile set pyramid (level 0), a single tile summarizes all of your data. On each lower level, there are up to *4z* tiles, where *z* is the zoom level (with lower numbers indicating higher levels). At each level, the tiles are laid out row-wise across the base map or plot, starting at the lower left. -Each tile summarizes the data in the region of the base map/plot to which it corresponds. Tiles are [Avro](http://avro.apache.org/) record objects partitioned into a variable number of bins (typically 256 x 256 for heatmaps). Each bin contains an aggregation of all the data points in the tile partition it represents. +Each tile summarizes the data in the region of the base map/plot to which it corresponds. Tiles are serialized objects partitioned into a variable number of bins (typically 256 x 256 for heatmaps). Each bin contains an aggregation of all the data points in the tile partition it represents. Tile Pyramid ## Tile Generation Process ## -The process of generating a set of Avro tiles from your raw source data is called a tiling job. The tiles created by a job can be served and browsed in any modern Web browser. +The process of generating a set of tiles from your raw source data is called a tiling job. The tiles created by a job can be served and browsed in any modern Web browser. As shown in the following diagram, the tile generation process has five main stages: @@ -97,14 +97,14 @@ Before you run a tiling job, make sure you meet all of the prerequisites listed ## Tiling Job Prerequisites ## -Aperture Tiles uses a distributed framework built on the Apache Spark engine to generate Avro tiles. Before you run a tiling job, you must install and configure the following prerequisites: +Aperture Tiles uses a distributed framework built on the Apache Spark engine to generate tiles. Before you run a tiling job, you must install and configure the following prerequisites: - [Scala](http://www.scala-lang.org/) v2.10.3 - [Apache Spark](http://spark.incubator.apache.org/) v1.3 If you intend to work with datasets that cannot fit in the memory of a single machine or if you wish to avoid wait times, we recommend you also install your preferred [Hadoop](http://hadoop.apache.org/) distribution to enable Aperture Tiles to work with particularly large datasets: -- [Cloudera](http://hadoop.apache.org/) v4.6 (*recommended*) +- [Cloudera](http://hadoop.apache.org/) v5.4.7 (*recommended*) - [Apache](http://hadoop.apache.org/docs/r1.2.1/index.html) - [MapR](http://www.mapr.com/products/apache-hadoop) - [HortonWorks](http://hortonworks.com/) From 4c5ea44fee5413c1b96249d88b94c65f3c22f477 Mon Sep 17 00:00:00 2001 From: Michael Crouch Date: Tue, 7 Jun 2016 14:21:56 -0400 Subject: [PATCH 4/5] Miscellaneous docs clean up --- docs/src/_includes/head.html | 9 +++++--- docs/src/contact/readme.md | 4 +++- docs/src/css/all.scss | 23 +++++++++++-------- docs/src/demos/readme.md | 3 +-- docs/src/demos/sidebar.html | 10 ++++---- .../advanced/custom-tiling/readme.md | 6 ++--- .../advanced/graph-tiling/readme.md | 2 +- .../docs/development/how-to/deploy/readme.md | 2 +- docs/src/download/readme.md | 10 ++++---- docs/src/readme.md | 8 +++---- docs/src/tour/components/readme.md | 2 +- 11 files changed, 43 insertions(+), 36 deletions(-) diff --git a/docs/src/_includes/head.html b/docs/src/_includes/head.html index 35835d1e..72839be3 100644 --- a/docs/src/_includes/head.html +++ b/docs/src/_includes/head.html @@ -6,9 +6,12 @@ {{ page.section }} | {{ site.product }} - - - + + +{% if page.subsection == "Development" or page.subsection == "User Guide" %} + + +{% endif %} diff --git a/docs/src/contact/readme.md b/docs/src/contact/readme.md index 491ce628..fb95f7f8 100644 --- a/docs/src/contact/readme.md +++ b/docs/src/contact/readme.md @@ -6,7 +6,9 @@ permalink: contact/ # Overview # -Aperture Tiles is an active open source product developed and managed by Uncharted under DARPA's XDATA program. For more information about the product, or for assistance in creating custom tile-based visual analytics with your organization's big data and integrating with your enterprise systems, please contact us at: +Aperture Tiles is an open source product developed and managed by Uncharted under DARPA's XDATA program. Aperture Tiles is powered by Salt, an open
    source library for scalable, analytic layered tiles. + +For more information, or for assistance in creating custom tile-based visual analytics with your organization's big data and integrating with your enterprise systems, please contact us at: diff --git a/docs/src/css/all.scss b/docs/src/css/all.scss index 55e3bc4a..9e8ac1ec 100644 --- a/docs/src/css/all.scss +++ b/docs/src/css/all.scss @@ -512,19 +512,22 @@ th { } table.summaryTable { - left: -10px; - position: relative; - border-spacing: 0; + @extend code-block; + font-family: $family-standard; + position: relative; + border-spacing: 0; + width: 100%; + padding: 5px; } @mixin table-cell($font-weight, $font-style, $text-align) { - padding: 0 10px 10px 10px; - font-size: 14px; - line-height: 19.6px; - vertical-align: top; - font-weight: $font-weight; - font-style: $font-style; - text-align: $text-align; + padding: 5px; + font-size: 14px; + line-height: 19.6px; + vertical-align: top; + font-weight: $font-weight; + font-style: $font-style; + text-align: $text-align; } td { diff --git a/docs/src/demos/readme.md b/docs/src/demos/readme.md index 8e8d6f3b..7688bda5 100644 --- a/docs/src/demos/readme.md +++ b/docs/src/demos/readme.md @@ -19,8 +19,7 @@ The following demos illustrate the utility of tile-based visual analytics and th ## Tile-Based Metrics of NYC Taxi Trip Data ## -This demo, which was presented at [Strata and Hadoop World 2014](http://strataconf.com/stratany2014), enables the exploration of data for 187 million NYC taxi trips taken between Jan 1, 2013 - Jan 3, 2014. -Trip data is displayed over a variety of metrics (both on a [Google Maps](https://maps.google.com) base layer and in cross-plot form) with the goal of answering the following questions: +This demo, which was presented at [Strata and Hadoop World 2014](http://strataconf.com/stratany2014), enables the exploration of data for 187 million NYC taxi trips taken between Jan 1, 2013 - Jan 3, 2014. Trip data is displayed over a variety of metrics (both on a [Google Maps](https://maps.google.com) base layer and in cross-plot form) with the goal of answering the following questions: - Where is the best place to catch a cab? - How can a cab driver make the most money? diff --git a/docs/src/demos/sidebar.html b/docs/src/demos/sidebar.html index df66e9d1..8df2366a 100644 --- a/docs/src/demos/sidebar.html +++ b/docs/src/demos/sidebar.html @@ -1,8 +1,8 @@ - -
    - NYC Taxi Trip Demo -
    NYC taxi trip data
    -
    +     +
    + NYC Taxi Trip Demo +
    NYC taxi trip data
    +
    diff --git a/docs/src/docs/development/advanced/custom-tiling/readme.md b/docs/src/docs/development/advanced/custom-tiling/readme.md index 2285335f..d2228e16 100644 --- a/docs/src/docs/development/advanced/custom-tiling/readme.md +++ b/docs/src/docs/development/advanced/custom-tiling/readme.md @@ -180,7 +180,7 @@ There are three steps in binning your data: 2. Running the Binner to transform the data into tiles 3. Writing the tiles -For an example of a custom Binner, see the [TwitterTopicBinner.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/TwitterTopicBinner.scala) file in [tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen). +For an example of a custom Binner, see the [TwitterTopicBinner.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/TwitterTopicBinner.scala) file in [tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/). ### Parsing your Data ### @@ -189,7 +189,7 @@ The Binner expects your data as pairs of **(index, record)**, where: - **index** is an object indicating where in space the record lies - **record** is a data record of the processing type your Binning Analytic defines -Several predefined index types are defined in the [IndexingScheme.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/IndexingScheme.scala) file in [tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/IndexingScheme.scala). +Several predefined index types are defined in the [IndexingScheme.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/IndexingScheme.scala) file in [tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/](https://github.com/unchartedsoftware/aperture-tiles/tree/master/tile-generation/src/main/scala/com/oculusinfo/tilegen/tiling/). - Cartesian: the index type is a pair of doubles. - IPv4: the index type is an array of 4 bytes: the 4 values in an IPv4 address. @@ -226,7 +226,7 @@ data.cache ### Binning ### -Lines 191 - 199 of [TwitterTopicBinner.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/TwitterTopicBinner.scala) transform the data into tiles: +Lines 190 - 198 of [TwitterTopicBinner.scala](https://github.com/unchartedsoftware/aperture-tiles/blob/master/tile-examples/twitter-topics/twitter-topics-utilities/src/main/scala/com/oculusinfo/twitter/tilegen/TwitterTopicBinner.scala) transform the data into tiles: ```scala val tiles = binner.processDataByLevel(data, diff --git a/docs/src/docs/development/advanced/graph-tiling/readme.md b/docs/src/docs/development/advanced/graph-tiling/readme.md index d0ae7fb4..41940ee7 100644 --- a/docs/src/docs/development/advanced/graph-tiling/readme.md +++ b/docs/src/docs/development/advanced/graph-tiling/readme.md @@ -14,7 +14,7 @@ In addition to [heatmap layers](../../how-to/standard-tiling/) of individual dat Graph tiling jobs comprise several configuration and generation phases as described in the [Graph Tiling Process](#graph-tiling-process) section. -**NOTE**: The graph tiling capabilities of Aperture Tiles are under ongoing development and considered experimental. +**NOTE**: The graph tiling capabilities of Aperture Tiles are considered experimental. ## Graph Tiling Process ## diff --git a/docs/src/docs/development/how-to/deploy/readme.md b/docs/src/docs/development/how-to/deploy/readme.md index 3cde9eaf..74b4b8e0 100644 --- a/docs/src/docs/development/how-to/deploy/readme.md +++ b/docs/src/docs/development/how-to/deploy/readme.md @@ -23,7 +23,7 @@ Once you have finished customizing the Tile Server and Tile Client, you can depl
    To deploy your application
    1. Copy the **new-project.war** to the *webapps/* directory of your web server (e.g., Apache Tomcat or Jetty). -2. Restart the server, if necessary +2. If necessary, restart the server. 3. Access your application in web browser at http://localhost:8080/new-project. ## Next Steps ## diff --git a/docs/src/download/readme.md b/docs/src/download/readme.md index 8a3e5399..de5c3571 100644 --- a/docs/src/download/readme.md +++ b/docs/src/download/readme.md @@ -9,7 +9,7 @@ Download ## Source Code ## -The Aperture Tiles source code is available on GitHub. Aperture Tiles is under ongoing development and is freely available for download under [The MIT License](http://www.opensource.org/licenses/MIT) (MIT) open source licensing. Unlike GNU General Public License (GPL), MIT freely permits distribution of derivative work under proprietary license, without requiring the release of source code. +The Aperture Tiles source code is available on GitHub. Aperture Tiles is freely available for download under [The MIT License](http://www.opensource.org/licenses/MIT) (MIT) open source licensing. Unlike GNU General Public License (GPL), MIT freely permits distribution of derivative work under proprietary license, without requiring the release of source code.     Aperture Tiles Source Code @@ -19,10 +19,10 @@ For information on full installations of Aperture Tiles, see the [Installation]( The following pre-built distribution can be used to quickly generate and display tiles without having to build the software. See the [Quick Start](../docs/development/getting-started/quick-start/) documentation to help you quickly learn how to process a dataset, transform it and create an Aperture Tiles project that allows you to visualize it in a web browser. -- **Tile Generator**: Enables you to generate a set of tiles that can be viewed using the Tile Client Application available below. This version was built for Apache Spark 1.3.0 and CDH 4.6. -

    Tile Generator

    -- **Tile Quick Start Application**: An example Tile Client application to display tiles in a browser. You can quickly copy and deploy this application to your web server after minimal modification. -

    Tile Quick Start Application +- **Tile Generator**: Enables you to generate a set of tiles that can be viewed using the Tile Client Application available below. This version was built for Apache Spark 1.3.0 and CDH 5.4.7. +

    Tile Generator

    +- **Tile Quick Start Application**: An example Tile Client application to display tiles in a browser. You can quickly copy and deploy this application to your web server after minimal modification. +

    Tile Quick Start Application

    Interested in Learning More?

    diff --git a/docs/src/readme.md b/docs/src/readme.md index 74fc69d8..e771d8be 100644 --- a/docs/src/readme.md +++ b/docs/src/readme.md @@ -9,9 +9,9 @@ Aperture Tiles tile-based visual analytics for big dataAperture Tiles is powered by Salt, an open
    source library for scalable, analytic layered tiles. -Aperture Tiles provides the ability to create browser-based, interactive tools any analyst can use to explore datasets containing billions of data points (or more). +Aperture Tiles provides the ability to create interactive, browser-based tools to explore datasets containing billions of data points or more. -Business and government leaders, operators and analysts today have more data than ever at hand to make smart, informed decisions. This information comes from various places, including social media, sensor data, financial transactions, cyber and open-source data. However making sense of such massive amounts of data is an extremely challenging task. As a result data is often not used to its full potential. +To make informed decisions, business and government leaders, operators and analysts increasingly rely on more data than ever. This information can come from various sources, including social media, sensor data, financial transactions, cyber and open-source data. Making sense of such massive amounts of data is extremely challenging. As a result, data is often not used to its full potential. Aperture Tiles uses a pyramid of tiles to structure, analyze, visualize and interact with a user interface similar to web-based geographic map applications. This approach allows analysts to easily see and navigate all of the data without losing detail, and seamlessly "zoom in" to a more localized portion of that data. @@ -33,7 +33,7 @@ With widespread use, map interactions have become familiar and make exploration ### Tile-Based Visual Analytics ### -Tile-based visual analytics (TBVA) is a technique that divides the data into evenly sized hierarchical tiles, and then calculates and applies an analytic to the data contained in each tile. For example, if examining a large Twitter dataset, the data would be divided into equally sized tiles across the geographic area. Then, an analytic is applied to each tile, such as the top five Twitter hashtags for the bounded region. As the user zooms into the map, Aperture Tiles presents a new layer of tiles that each contain a smaller portion of the full dataset. In each new zoom layer, the analytic applied to each tile becomes more localized. +Tile-based visual analytics (TBVA) is a technique that divides the data into evenly sized hierarchical tiles, and then calculates and applies an analytic to the data contained in each tile. For example, if examining a large Twitter dataset, the data would be divided into equally sized tiles across the geographic area. Then, an analytic is applied to each tile, such as the top five Twitter hashtags for the bounded region. As the user zooms into the map, Aperture Tiles presents a new layer of tiles that each contain a smaller portion of the full dataset. In each new zoom layer, the analytic applied to each tile becomes more localized. ### Extensible and Open Source ### @@ -41,7 +41,7 @@ The Aperture Tiles services and API are designed to be extensible, allowing a br Aperture Tile generation service builds on the [Apache Spark](http://spark.incubator.apache.org/) and [Hadoop](http://hadoop.apache.org/) cluster computing systems, providing the ability to scale up to billions (or more) data points. Tile sets involving billions of tiles are efficiently handled using [Apache HBase](http://hbase.apache.org/) for distributed data storage. -Aperture Tiles is under ongoing development and is freely available for download under [The MIT License](http://www.opensource.org/licenses/MIT) open source licensing. Unlike GNU General Public License (GPL), MIT freely permits distribution of derivative work under proprietary license, without requiring the release of source code. +Aperture Tiles is freely available for download under [The MIT License](http://www.opensource.org/licenses/MIT) open source licensing. Unlike GNU General Public License (GPL), MIT freely permits distribution of derivative work under proprietary license, without requiring the release of source code. ### Interested in Learning More? ### diff --git a/docs/src/tour/components/readme.md b/docs/src/tour/components/readme.md index 12d73a38..9f41ede4 100644 --- a/docs/src/tour/components/readme.md +++ b/docs/src/tour/components/readme.md @@ -15,7 +15,7 @@ Aperture Tiles implements a data flow that breaks the visualization process into ## Distributed Tile Generation ## -The Aperture Tiles generation analytic leverages cluster computing with [Apache Spark](http://spark.incubator.apache.org/) to rapidly compute tiles from terabytes of "big data" (investigations are ongoing to work with petabyte-sized datasets). The tile generation analytic transforms raw data into Avro data tile files with metadata descriptor files, summarizing all tile content. Avro data files allow for language neutrality and support multiple summaries per bin, allowing a flexible and extensible analytic. Finished tiles are stored in HBase to allow efficient scalability to billions of tiles. +The Aperture Tiles generation analytic leverages cluster computing with [Apache Spark](http://spark.incubator.apache.org/) to rapidly compute tiles from terabytes of "big data". The tile generation analytic transforms raw data into Avro data tile files with metadata descriptor files, summarizing all tile content. Avro data files allow for language neutrality and support multiple summaries per bin, allowing a flexible and extensible analytic. Finished tiles are stored in HBase to allow efficient scalability to billions of tiles. ### Tile Set Size and Resolution ### From 089f5b263a4d42c3c2016ebfa49d4704dad1788e Mon Sep 17 00:00:00 2001 From: Michael Crouch Date: Tue, 7 Jun 2016 15:36:09 -0400 Subject: [PATCH 5/5] Update download links and version number --- docs/src/_config.yml | 2 +- docs/src/download/readme.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/_config.yml b/docs/src/_config.yml index 70745e42..dbbffb2d 100644 --- a/docs/src/_config.yml +++ b/docs/src/_config.yml @@ -15,4 +15,4 @@ exclude: ["img/resources/SiteComponents.png","sidebar.html","*/sidebar.html"] # Variables product: Aperture Tiles -version: 0.6.0 \ No newline at end of file +version: 0.7.0 \ No newline at end of file diff --git a/docs/src/download/readme.md b/docs/src/download/readme.md index de5c3571..7b3f9b24 100644 --- a/docs/src/download/readme.md +++ b/docs/src/download/readme.md @@ -20,9 +20,9 @@ For information on full installations of Aperture Tiles, see the [Installation]( The following pre-built distribution can be used to quickly generate and display tiles without having to build the software. See the [Quick Start](../docs/development/getting-started/quick-start/) documentation to help you quickly learn how to process a dataset, transform it and create an Aperture Tiles project that allows you to visualize it in a web browser. - **Tile Generator**: Enables you to generate a set of tiles that can be viewed using the Tile Client Application available below. This version was built for Apache Spark 1.3.0 and CDH 5.4.7. -

    Tile Generator

    +

    Tile Generator

    - **Tile Quick Start Application**: An example Tile Client application to display tiles in a browser. You can quickly copy and deploy this application to your web server after minimal modification. -

    Tile Quick Start Application +

    Tile Quick Start Application

    Interested in Learning More?