Restructure, better logging, docker and new custom keyword (#13)

* Add docker configuration and instructions. * Update node version to 8.11.1 * Add empty string validation to schema. * Fix leadind dot on error message. Improve ValidationError constructor. * Configured logging with winston * Code clean up and reorganization. * Add CustomAjvError object * Implemented isValidTerm custom keyword * Fix issues found by codacy. * SUBS-1343 (#11) * Update heroku app url * Fixed bug on async keywords validation output. * SUBS-1346 (#12) * Change term (ontology term uri) representation from string to object. * Update README accordingly.
HumanCellAtlas · Jun 18, 2018 · ced0b1c · ced0b1c
1 parent e403b8f
commit ced0b1c
Show file tree

Hide file tree

Showing 34 changed files with 3,375 additions and 2,934 deletions.
diff --git a/.dockerignore b/.dockerignore
@@ -0,0 +1,2 @@
+node_modules
+npm-debug.log
diff --git a/.gitignore b/.gitignore
@@ -1,3 +1,4 @@
 node_modules/
 .idea/
-.DS_Store
+.DS_Store
+log/
diff --git a/.travis.yml b/.travis.yml
@@ -1,4 +1,4 @@
 language: node_js
 node_js:
-  - "9.7.1"
+  - "8.11.1"
 script: npm run travis-test
diff --git a/Dockerfile b/Dockerfile
@@ -0,0 +1,17 @@
+FROM node:carbon
+
+# Create app directory
+WORKDIR /usr/src/app
+
+# Install app dependencies
+# A wildcard is used to ensure both package.json AND package-lock.json are copied
+# where available (npm@5+)
+COPY package*.json ./
+
+RUN npm install
+
+# Bundle app source
+COPY . .
+
+EXPOSE 3000
+CMD [ "npm", "start" ]
diff --git a/README.md b/README.md
@@ -3,17 +3,18 @@
 [![tested with jest](https://img.shields.io/badge/tested_with-jest-99424f.svg)](https://github.com/facebook/jest)
 
 This repository contains a [JSON Schema](http://json-schema.org/) validator for the EMBL-EBI Submissions Project. This validator runs as a standalone node server that receives validation requests and gives back it's results.
-The validation is done using the [AJV](https://github.com/epoberezkin/ajv) library version 6.0.0 that fully supports the JSON Schema **draft-07**.
+The validation is done using the [AJV](https://github.com/epoberezkin/ajv) library version ^6.0.0 that fully supports the JSON Schema **draft-07**.
 
-Deployed for tests purposes on heroku: https://usi-json-schema-validator.herokuapp.com/validate
+Deployed for tests purposes on heroku: https://subs-json-schema-validator.herokuapp.com/validate
 
-:arrow_right: [Getting Started](README.md#getting-started)
+## Contents
+- [Getting Started](README.md#getting-started)
 
-:arrow_right: [Validator API](README.md#validator-api)
+- [Validation API](README.md#validation-api)
 
-:arrow_right: [Custom keywords](README.md#custom-keywords)
+- [Custom keywords](README.md#custom-keywords)
 
-:arrow_right: [License](README.md#license)
+- [License](README.md#license)
 
 ## Getting Started
 These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
@@ -25,7 +26,7 @@ npm is distributed with Node.js which means that when you download Node.js, you
 ### Installing
 
 #### Node.js / npm
-- Get Node.js: https://nodejs.org/en/
+- Get Node.js: https://nodejs.org/en/ (v8.11.1 LTS)
 
 - If you use [Homebrew](https://brew.sh/) you can install node by doing:
 ```
@@ -55,82 +56,68 @@ node test
 ```
 node src/server
 ```
-The node server will run on port **3000** and will expose one endpoint: **/validate**.
+The node server will run on port **3020** and will expose one endpoint: **/validate**.
+
+### Executing with Docker
+1. Build docker image:
+```
+docker build -t subs/json-schema-validator .
+```
+2. Run docker image:
+```
+docker run -p 3020:3020 -d subs/json-schema-validator
+```
 ### Development
 For development purposes using [nodemon](https://nodemon.io/) is useful. It reloads the application everytime something has changed on save time.
 ```
 nodemon src/server
 ```
 
-## Validator API
-The validator exposes one single endpoint that will accept POST requests. When running on you local machine it will look like: **http://localhost:3000/validate**.
+## Validation API
+This validator exposes one single endpoint that will accept POST requests. When running on you local machine it will look like: **http://localhost:3020/validate**.
 
 ### Usage
 The endpoint will expect the body to have the following structure:
-```json
+```js
 {
   "schema": {},
   "object": {}
 }
 ```
-Where the schema should be a valid json schema object to validate the submittable against.
+Where the schema should be a valid json schema to validate the object against.
 
 **Example:**
 Sending a POST request with the following body:
-```json
+```js
 {
   "schema": {
-    "$schema": "http://json-schema.org/draft-07/schema#",
-    "title": "Attributes",
-    "description": "USI submittable attributes schema.",
-
+    "$schema": "http://json-schema.org/draft-07/schema#"
+
     "type": "object",
     "properties": {
-      "attributes": {
-        "type": "object",
-        "properties": {},
-        "patternProperties": {
-          "^.*$": {
-            "type": "array",
-            "minItems": 1,
-            "items": {
-              "properties": {
-                "value": {
-                  "type": "string",
-                  "minLength": 1
-                },
-                "units": {"type": "string"},
-                "terms": {
-                  "type": "array",
-                  "items": {
-                    "type": "string",
-                    "format": "uri"
-                  }
-                }
-              },
-              "required": ["value"]
-            }
-          }
-        }
+      "alias": {
+        "description": "A sample unique identifier in a submission.",
+        "type": "string"
+      },
+      "taxonId": {
+        "description": "The taxonomy id for the sample species.",
+        "type": "integer"
+      },
+      "taxon": {
+        "description": "The taxonomy name for the sample species.",
+        "type": "string"
+      },
+      "releaseDate": {
+        "description": "Date from which this sample is released publicly.",
+        "type": "string",
+        "format": "date"
       }
-    }
+    },  
+    "required": ["alias", "taxonId" ]
   },
   "object": {
-    "attributes": {
-      "age": [{
-        "value": "3",
-        "units": "days",
-        "terms":[
-          "http://purl.obolibrary.org/obo/UO_0000033"
-        ]
-      }],
-      "sex": [{
-        "value": "female",
-        "terms":[
-          "http://purl.obolibrary.org/obo/PATO_0000383"
-        ]
-      }]
-    }
+    "alias": "MA456",
+    "taxonId": 9606
   }
 }
 ```
@@ -140,7 +127,7 @@ HTTP status code `200`
 ```json
 []
 ```
-An invalid validation response will look like:
+An example of a validation response with errors:
 
 HTTP status code `200`
 ```json
@@ -172,38 +159,69 @@ HTTP status code `400`
 }
 ```
 ## Custom keywords
-The AJV library supports the implementation of custom json schema keywords to address validation scenarios that json schema is not capable of addressing.
+The AJV library supports the implementation of custom json schema keywords to address validation scenarios that go beyond what json schema can handle.
+This validator has two custom keywords implemented, `isChildTermOf` and `isValidTerm`.
 
 ### isChildTermOf
-This custom keyword *evaluates if an ontology term is child of other*. This keyword is applied to an array of strings (url) and **passes validation if at least one of the terms in the array is child of the term defined in the schema**.
+This custom keyword *evaluates if an ontology term is child of other*. This keyword is applied to a string (url) and **passes validation if the term is a child of the term defined in the schema**.
 The keyword requires the **parent term** and the **ontology id**, both of which should exist in [OLS - Ontology Lookup Service](https://www.ebi.ac.uk/ols).
-This keyword works by doing an asynchronous call to OLS API that will respond with the required information to know if a given term is child of another. Being an async validation step, whenever used is a schema it should have the flag: `"$async": true`
+
+This keyword works by doing an asynchronous call to the [OLS API](https://www.ebi.ac.uk/ols/api/) that will respond with the required information to know if a given term is child of another. 
+Being an async validation step, whenever used is a schema, the schema must have the flag: `"$async": true` in it's object root.
+
 #### Usage
 Schema:
-```json
+```js
 {
   "$schema": "http://json-schema.org/draft-07/schema#",
   "$async": true,
-
-  (...)
-
-    "isChildTermOf": {
-      "parentTerm": "http://purl.obolibrary.org/obo/PATO_0000047",
-      "ontologyId": "pato"
+  "properties": {
+    "term": { 
+      "type": "string", 
+      "format": "uri",
+      "isChildTermOf": {
+        "parentTerm": "http://purl.obolibrary.org/obo/PATO_0000047",
+        "ontologyId": "pato"
+      } 
     }
+  }
 }
 ```
 JSON object:
-```json
+```js
+{
+  "term": "http://purl.obolibrary.org/obo/PATO_0000383"
+}
+```
+
+### isValidTerm
+This custom keyword *evaluates if a given ontology term url exists in OLS* ([Ontology Lookup Service](https://www.ebi.ac.uk/ols)). It is applied to a string (url) and **passes validation if the term exists in OLS**. It can be aplied to any string defined in the schema.
+
+This keyword works by doing an asynchronous call to the [OLS API](https://www.ebi.ac.uk/ols/api/) that will respond with the required information to determine if the term exists in OLS or not. 
+Being an async validation step, whenever used is a schema, the schema must have the flag: `"$async": true` in it's object root.
+
+#### Usage
+Schema:
+```js
 {
-  "attributes": {
-    "sex": [{
-      "value": "female",
-      "terms":["http://purl.obolibrary.org/obo/PATO_0000383"]
-    }]
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "$async": true,
+
+  "properties": {
+    "url": { 
+      "type": "string", 
+      "format": "uri",
+      "isValidTerm": true 
+    } 
   }
 }
 ```
+JSON object:
+```js
+{
+  "url": "http://purl.obolibrary.org/obo/PATO_0000383"
+}
+```
 
 ## License
  For more details about licensing see the [LICENSE](LICENSE.md).
diff --git a/examples/objects/faang-cellCulture-sample.json b/examples/objects/faang-cellCulture-sample.json
@@ -13,11 +13,11 @@
     }],
     "material": [{
       "value": "cell culture",
-      "terms": ["http://purl.obolibrary.org/obo/OBI_0001876", "http://purl.obolibrary.org/obo/BTO_0000214"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/OBI_0001876"}, {"url": "http://purl.obolibrary.org/obo/BTO_0000214"}]
     }],
     "cellType": [{
       "value": "epitheloid cell of small intestine",
-      "terms": ["http://purl.obolibrary.org/obo/CL_0002254"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/CL_0002254"}]
     }],
     "cellCultureProtocol": [{
       "value": "http://www.ncbi.nlm.nih.gov/pubmed/16215741"
@@ -27,7 +27,7 @@
     }],
     "cultureType": [{
       "value": "cell culture",
-      "terms": ["http://purl.obolibrary.org/obo/BTO_0000214"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/BTO_0000214"}]
     }],
     "cultureConditions": [{
       "value": "cells were cultured conventionally with 10% fetal calf serum (FBS) and 1% penicillin /streptomycin (both PAA, Colb, Germany) in Dulbecco's Modified Eagle Medium (DMEM) Sigma, Steinheim , Germany"

diff --git a/examples/objects/faang-cellLine-sample.json b/examples/objects/faang-cellLine-sample.json
@@ -13,21 +13,21 @@
     }],
     "material": [{
       "value": "cell line",
-      "terms": ["http://purl.obolibrary.org/obo/CLO_0000031", "http://www.ebi.ac.uk/efo/EFO_0000322"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/CLO_0000031"}, {"url": "http://www.ebi.ac.uk/efo/EFO_0000322"}]
     }],
     "biomaterialProvider": [{
       "value": "ATCC"
     }],
     "sex": [{
       "value": "female genotypic sex",
-      "terms": ["http://purl.obolibrary.org/obo/PATO_0020002"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/PATO_0020002"}]
     }],
     "cellLine": [{
       "value": "3D4/2"
     }],
     "cellType": [{
       "value": "alveolar macrophage",
-      "terms": ["http://purl.obolibrary.org/obo/CL_0000235", "http://purl.obolibrary.org/obo/CL_0000583"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/CL_0000235"}, {"url": "http://purl.obolibrary.org/obo/CL_0000583"}]
     }],
     "publication": [{
       "value": "doi://10.1016/S0166-0934(02)00085-X"
@@ -37,7 +37,7 @@
     }],
     "breed": [{
       "value": "Landrace",
-      "terms": ["http://purl.obolibrary.org/obo/LBO_0000692"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/LBO_0000692"}]
     }],
     "dateEstablished": [{
       "value": "2013-08",

diff --git a/examples/objects/faang-cellSpecimen-sample.json b/examples/objects/faang-cellSpecimen-sample.json
@@ -10,14 +10,14 @@
   "attributes": {
     "material": [{
       "value": "cell specimen",
-      "terms": ["http://purl.obolibrary.org/obo/OBI_0001468"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/OBI_0001468"}]
     }],
     "project": [{
       "value": "FAANG"
     }],
     "cellType": [{
       "value": "bone marrow cell",
-      "terms": ["http://purl.obolibrary.org/obo/CL_0002092"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/CL_0002092"}]
     }],
     "purificationProtocol": [{
       "value": "http://ftp.faang.ebi.ac.uk/ftp/protocols/samples/ ROSLIN_SOP_Isolation_of_Macrophages_from_Lungs_Blood_and_Bone_Marrow_of_Large_Animals_20160516.pdf"

diff --git a/examples/objects/faang-organism-sample.json b/examples/objects/faang-organism-sample.json
@@ -9,27 +9,27 @@
   "attributes": {
     "material": [{
       "value": "organism",
-      "terms": ["http://purl.obolibrary.org/obo/OBI_0100026"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/OBI_0100026"}]
     }],
     "organism": [{
       "value": "Ovis aries",
-      "terms": ["http://purl.obolibrary.org/obo/NCBITaxon_9940"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/NCBITaxon_9940"}]
     }],
     "sex": [{
       "value": "female",
-      "terms": ["http://purl.obolibrary.org/obo/PATO_0000383"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/PATO_0000383"}]
     }],
     "birthDate": [{
       "value": "2009",
       "units": "YYYY"
     }],
     "breed": [{
       "value": "Spanish Churra",
-      "terms": ["http://purl.obolibrary.org/obo/LBO_0000704"]
+      "terms": [{"url": "http://purl.obolibrary.org/obo/LBO_0000704"}]
     }],
     "healthStatus": [{
       "value": "normal",
-      "terms":["http://purl.obolibrary.org/obo/PATO_0000461"]
+      "terms":[{"url": "http://purl.obolibrary.org/obo/PATO_0000461"}]
     }],
     "project": [{
       "value": "FAANG"