Validation and docs for datetime

Author: greeshmaswaminathan

README.md

@@ -82,6 +82,7 @@ These directives are currently available:
 | [Parse as Simple Date](wrangler-docs/directives/parse-as-simple-date.md)        | Parses date strings                                              |
 | [Parse XML To JSON](wrangler-docs/directives/parse-xml-to-json.md)              | Parses an XML document into a JSON structure                     |
 | [Parse as Currency](wrangler-docs/directives/parse-as-currency.md)              | Parses a string representation of currency into a number.        |
+| [Parse as Datetime](wrangler-docs/directives/parse-as-datetime.md)              | Parses strings with datetime values to CDAP datetime type        |
 | **Output Formatters**                                                  |                                                                  |
 | [Write as CSV](wrangler-docs/directives/write-as-csv.md)                        | Converts a record into CSV format                                |
 | [Write as JSON](wrangler-docs/directives/write-as-json-map.md)                  | Converts the record into a JSON map                              |
@@ -115,6 +116,11 @@ These directives are currently available:
 | [Diff Date](wrangler-docs/directives/diff-date.md)                              | Calculates the difference between two dates                      |
 | [Format Date](wrangler-docs/directives/format-date.md)                          | Custom patterns for date-time formatting                         |
 | [Format Unix Timestamp](wrangler-docs/directives/format-unix-timestamp.md)      | Formats a UNIX timestamp as a date                               |
+| **DateTime Transformations**                                                    |                                                                  |
+| [Current DateTime](wrangler-docs/directives/current-datetime.md)                | Generates the current datetime using the given zone or UTC by default|
+| [Datetime To Timestamp](wrangler-docs/directives/datetime-to-timestamp.md)      | Converts a datetime value to timestamp with the given zone       |
+| [Format Datetime](wrangler-docs/directives/format-datetime.md)                  | Formats a datetime value to custom date time pattern strings     |
+| [Timestamp To Datetime](wrangler-docs/directives/timestamp-to-datetime.md)      | Converts a timestamp value to datetime                           |
 | **Lookups**                                                            |                                                                  |
 | [Catalog Lookup](wrangler-docs/directives/catalog-lookup.md)                    | Static catalog lookup of ICD-9, ICD-10-2016, ICD-10-2017 codes   |
 | [Table Lookup](wrangler-docs/directives/table-lookup.md)                        | Performs lookups into Table datasets                             |

wrangler-core/src/main/java/io/cdap/wrangler/utils/RecordConvertor.java

@@ -35,6 +35,7 @@
 import java.time.LocalDateTime;
 import java.time.LocalTime;
 import java.time.ZonedDateTime;
+import java.time.format.DateTimeParseException;
 import java.util.ArrayList;
 import java.util.List;
 import java.util.Map;
@@ -112,17 +113,34 @@ public StructuredRecord decodeRecord(Row row, Schema schema) throws RecordConver
   private Object decode(String name, Object object, Schema schema) throws RecordConvertorException {
     // Extract the type of the field.
     Schema.Type type = schema.getType();
-    Schema.LogicalType logicalType = schema.getLogicalType();
+    Schema.LogicalType logicalType =
+        schema.isNullable() ? schema.getNonNullable().getLogicalType() :
+            schema.getLogicalType();
 
     if (logicalType != null) {
       switch (logicalType) {
+        case DATETIME:
+          if (schema.isNullable() && object == null || object instanceof LocalDateTime) {
+            return object;
+          }
+          if (object == null) {
+            throw new UnexpectedFormatException(
+                String.format("Datetime field %s should have a non null value", name));
+          }
+          try {
+            LocalDateTime.parse((String) object);
+          } catch (DateTimeParseException exception) {
+            throw new UnexpectedFormatException(
+                String.format("Datetime field '%s' with value '%s' is not in ISO-8601 format.",
+                    name, object), exception);
+          }
+          return object;
         case DATE:
         case TIME_MILLIS:
         case TIME_MICROS:
         case TIMESTAMP_MILLIS:
         case TIMESTAMP_MICROS:
         case DECIMAL:
-        case DATETIME:
           return object;
         default:
           throw new UnexpectedFormatException("field type " + logicalType + " is not supported.");

wrangler-docs/directives/current-datetime.md

@@ -0,0 +1,21 @@
+# Current Datetime
+
+The CURRENT-DATETIME directive generates the current datetime using the given zone or UTC by default.
+
+
+## Syntax
+```
+current-datetime <colname> 'timezone'
+```
+
+
+## Usage Notes
+
+The CURRENT-DATETIME directive generates the current datetime using the given zone or UTC by default.
+Zone can be region based string like America/Los_Angeles, Europe/Paris ,
+ simple offsets like +08:00 , or prefix and offset like UTC+01:00, GMT+08:00, UT+04:00 etc
+
+## Examples
+current-datetime :col1 'UTC-08:00'
+
+current-datetime :col2 'America/Los_Angeles'

wrangler-docs/directives/datetime-to-timestamp.md

@@ -0,0 +1,24 @@
+# Datetime To Timestamp
+
+The DATETIME-TO-TIMESTAMP directive converts a datetime value to timestamp with the given zone
+
+
+## Syntax
+```
+datetime-to-timestamp <colname> 'timezone'
+```
+
+
+## Usage Notes
+
+The DATETIME-TO-TIMESTAMP directive converts datetime values to 
+timestamp values using the given time zone (UTC by default).
+Zone can be region based string like America/Los_Angeles, Europe/Paris ,
+ simple offsets like +08:00 , or prefix and offset like UTC+01:00, GMT+08:00, UT+04:00 etc
+ 
+ If the column is `null` applying this directive is a no-op. 
+
+## Examples
+datetime-to-timestamp :col1 'UTC-08:00'
+
+datetime-to-timestamp :col2 'America/Los_Angeles'

wrangler-docs/directives/format-datetime.md

@@ -0,0 +1,23 @@
+# Format Datetime
+
+The FORMAT-DATETIME directive formats CDAP datetime values to custom pattern strings.
+
+
+## Syntax
+```
+format-datetime <datetime_column> "<pattern>"
+```
+
+
+## Usage Notes
+
+The FORMAT-DATETIME directive will format CDAP datetime values to custom pattern strings.
+Pattern is the format for the output string. 
+
+
+If the column is `null` applying this directive is a no-op. 
+The column to be formatted should be of type datetime.
+
+
+## Examples
+See [FORMAT-DATE](format-date.md) for an explanation and examples the pattern strings.

wrangler-docs/directives/parse-as-datetime.md

@@ -0,0 +1,23 @@
+# Parse as Datetime
+
+The PARSE-AS-DATETIME directive parses strings with datetime values to CDAP datetime type.
+
+
+## Syntax
+```
+parse-as-datetime <column> "<pattern>"
+```
+
+
+## Usage Notes
+
+The PARSE-AS-DATETIME directive will parse strings with datetime values to CDAP
+datetime type. Pattern is the format of the `input` strings. 
+The input values and pattern should have a date and time component.
+
+If the column is `null` or is already a datetime field, applying this directive
+is ano-op. The column to be parsed as a datetime should be of type string.
+
+
+## Examples
+See [FORMAT-DATE](format-date.md) for an explanation and examples of these pattern strings.

wrangler-docs/directives/timestamp-to-datetime.md

@@ -0,0 +1,20 @@
+# Timestamp To Datetime
+
+The TIMESTAMP-TO-DATETIME directive converts a timestamp value to datetime
+
+
+## Syntax
+```
+timestamp-to-datetime <colname>
+```
+
+
+## Usage Notes
+
+The TIMESTAMP-TO-DATETIME directive converts timestamp values to 
+datetime values .
+ 
+If the column is `null` applying this directive is a no-op. 
+
+## Examples
+timestamp-to-datetime :col1