Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update march 2 #13

Merged
merged 1 commit into from
Mar 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 25 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,34 +18,42 @@
</a>
</p>

![quary_core_image](https://github.com/quarylabs/quary/assets/132601011/20024c62-6ad6-42e8-937e-37a708af9c0c)
![quary_core_image](./assets/diagram.jpg)

## What is Quary?

Quary enables teams to model, test and deploy data transformations. This core is a fast and lightweight SQL transformation engine written in Rust.
Quary enables teams to design, document, test and deploy data transformations to your SQL data stores. Quary is a SQL
engine and CLI tool that lets you manage your data transformation projects with ease.

Teams use Quary to transform their raw data tables into actionable and trustworthy insights. Teams do so by layering
well documented and tested transformations into useful insights ready for consumption. A Quary project is formed of a
collection of these SQL `SELECT` statements, that build on top of each other.

- Visit [our website](https://www.quary.dev) to learn more
- Visit [our documentation](https://www.quary.dev/docs) to learn how to use Quary

## Understanding Quary
## Getting Started

Teams use Quary to transform their raw data tables in a data warehouse into actionable trustworthy insights. Users can easily transform their data by writing SQL select statements, which Quary then converts into tables and views within the data warehouse.
### Installation

A Quary project is formed by a collection of these select statements, also known as models. These models often build upon one another, creating a structured flow of data transformation. Quary simplifies the process of visualizing dependencies from the source (raw table) to the final insight. Additionally, it provides testing capabilities for the models at each stage, ensuring data integrity and accuracy.
The following Quary repository contains the core Quary library as well as the Quary CLI tool. The Quary CLI tool can be
installed as following

## Getting Started
#### Homebrew installation for macOS

### Installation
Quary can be installed using Homebrew on macOS using the following command:

#### Homebrew installation
```
brew install quarylabs/quary/quary
```
#### Linux/Mac installation
Header over to [releases](https://github.com/quarylabs/quary/releases/latest) to get the latest version of Quary for your system.

#### Other installations

Other builds are available in the [releases page](https://github.com/quarylabs/quary/releases/latest) to download.

### Usage

Check out our template [here](https://github.com/quarylabs/template) of an example Quary project. The following commands will show you how to get started, and run some basic commands.
Once installed, a sample project can be created and run as follows:

```shell
mkdir example # create an empty project folder
Expand All @@ -56,14 +64,12 @@ quary test -s # run defined tests against target database
quary build # build and execute the model views/seeds against target database
```

Note that you will most likely want to use our [Visual Studio Code extension](https://marketplace.visualstudio.com/items?itemName=Quary.quary-extension)

### Commands
```
quary help
```

Note that you will most likely want to use
our [Visual Studio Code extension](https://marketplace.visualstudio.com/items?itemName=Quary.quary-extension) to
visualise the project.

## Support

If you run into any problems using Quary, please let us know. We want Quary to be easy-to-use, so if you are getting confused, it is our fault, not yours. [Create an issue](https://github.com/quarylabs/quary/issues) and we'll be happy to help you out.
If you run into any problems using Quary, please let us know. We want Quary to be easy-to-use, so if you are getting
confused, it is our fault, not yours. [Create an issue](https://github.com/quarylabs/quary/issues) and we'll be happy to
help you out.
Binary file added assets/diagram.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
90 changes: 90 additions & 0 deletions proto/gen/es/quary/service/v1/query_result_pb.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
// @generated by protoc-gen-es v1.7.2 with parameter "target=ts"
// @generated from file quary/service/v1/query_result.proto (package quary.service.v1, syntax proto3)
/* eslint-disable */
// @ts-nocheck

import type { BinaryReadOptions, FieldList, JsonReadOptions, JsonValue, PartialMessage, PlainMessage } from "@bufbuild/protobuf";
import { Message, proto3 } from "@bufbuild/protobuf";

/**
* QueryResult is the result of a ran query.
*
* @generated from message quary.service.v1.QueryResult
*/
export class QueryResult extends Message<QueryResult> {
/**
* @generated from field: repeated quary.service.v1.QueryResultColumn columns = 1;
*/
columns: QueryResultColumn[] = [];

constructor(data?: PartialMessage<QueryResult>) {
super();
proto3.util.initPartial(data, this);
}

static readonly runtime: typeof proto3 = proto3;
static readonly typeName = "quary.service.v1.QueryResult";
static readonly fields: FieldList = proto3.util.newFieldList(() => [
{ no: 1, name: "columns", kind: "message", T: QueryResultColumn, repeated: true },
]);

static fromBinary(bytes: Uint8Array, options?: Partial<BinaryReadOptions>): QueryResult {
return new QueryResult().fromBinary(bytes, options);
}

static fromJson(jsonValue: JsonValue, options?: Partial<JsonReadOptions>): QueryResult {
return new QueryResult().fromJson(jsonValue, options);
}

static fromJsonString(jsonString: string, options?: Partial<JsonReadOptions>): QueryResult {
return new QueryResult().fromJsonString(jsonString, options);
}

static equals(a: QueryResult | PlainMessage<QueryResult> | undefined, b: QueryResult | PlainMessage<QueryResult> | undefined): boolean {
return proto3.util.equals(QueryResult, a, b);
}
}

/**
* @generated from message quary.service.v1.QueryResultColumn
*/
export class QueryResultColumn extends Message<QueryResultColumn> {
/**
* @generated from field: string name = 1;
*/
name = "";

/**
* @generated from field: repeated string values = 2;
*/
values: string[] = [];

constructor(data?: PartialMessage<QueryResultColumn>) {
super();
proto3.util.initPartial(data, this);
}

static readonly runtime: typeof proto3 = proto3;
static readonly typeName = "quary.service.v1.QueryResultColumn";
static readonly fields: FieldList = proto3.util.newFieldList(() => [
{ no: 1, name: "name", kind: "scalar", T: 9 /* ScalarType.STRING */ },
{ no: 2, name: "values", kind: "scalar", T: 9 /* ScalarType.STRING */, repeated: true },
]);

static fromBinary(bytes: Uint8Array, options?: Partial<BinaryReadOptions>): QueryResultColumn {
return new QueryResultColumn().fromBinary(bytes, options);
}

static fromJson(jsonValue: JsonValue, options?: Partial<JsonReadOptions>): QueryResultColumn {
return new QueryResultColumn().fromJson(jsonValue, options);
}

static fromJsonString(jsonString: string, options?: Partial<JsonReadOptions>): QueryResultColumn {
return new QueryResultColumn().fromJsonString(jsonString, options);
}

static equals(a: QueryResultColumn | PlainMessage<QueryResultColumn> | undefined, b: QueryResultColumn | PlainMessage<QueryResultColumn> | undefined): boolean {
return proto3.util.equals(QueryResultColumn, a, b);
}
}

44 changes: 41 additions & 3 deletions proto/gen/es/quary/service/v1/test_result_pb.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

import type { BinaryReadOptions, FieldList, JsonReadOptions, JsonValue, PartialMessage, PlainMessage } from "@bufbuild/protobuf";
import { Empty, Message, proto3 } from "@bufbuild/protobuf";
import { QueryResult } from "./query_result_pb.js";

/**
* TestResult is the result of a test serializable so it can be shared
Expand Down Expand Up @@ -227,9 +228,9 @@ export class Failed extends Message<Failed> {
*/
reason: {
/**
* @generated from field: google.protobuf.Empty ran = 1;
* @generated from field: quary.service.v1.FailedRunResults ran = 1;
*/
value: Empty;
value: FailedRunResults;
case: "ran";
} | {
/**
Expand All @@ -253,7 +254,7 @@ export class Failed extends Message<Failed> {
static readonly runtime: typeof proto3 = proto3;
static readonly typeName = "quary.service.v1.Failed";
static readonly fields: FieldList = proto3.util.newFieldList(() => [
{ no: 1, name: "ran", kind: "message", T: Empty, oneof: "reason" },
{ no: 1, name: "ran", kind: "message", T: FailedRunResults, oneof: "reason" },
{ no: 2, name: "inferred_from_tests", kind: "message", T: InferredChain, oneof: "reason" },
{ no: 3, name: "inferred_through_tests_operation", kind: "message", T: InferredChainWithOperation, oneof: "reason" },
]);
Expand All @@ -275,3 +276,40 @@ export class Failed extends Message<Failed> {
}
}

/**
* @generated from message quary.service.v1.FailedRunResults
*/
export class FailedRunResults extends Message<FailedRunResults> {
/**
* @generated from field: quary.service.v1.QueryResult query_result = 1;
*/
queryResult?: QueryResult;

constructor(data?: PartialMessage<FailedRunResults>) {
super();
proto3.util.initPartial(data, this);
}

static readonly runtime: typeof proto3 = proto3;
static readonly typeName = "quary.service.v1.FailedRunResults";
static readonly fields: FieldList = proto3.util.newFieldList(() => [
{ no: 1, name: "query_result", kind: "message", T: QueryResult },
]);

static fromBinary(bytes: Uint8Array, options?: Partial<BinaryReadOptions>): FailedRunResults {
return new FailedRunResults().fromBinary(bytes, options);
}

static fromJson(jsonValue: JsonValue, options?: Partial<JsonReadOptions>): FailedRunResults {
return new FailedRunResults().fromJson(jsonValue, options);
}

static fromJsonString(jsonString: string, options?: Partial<JsonReadOptions>): FailedRunResults {
return new FailedRunResults().fromJsonString(jsonString, options);
}

static equals(a: FailedRunResults | PlainMessage<FailedRunResults> | undefined, b: FailedRunResults | PlainMessage<FailedRunResults> | undefined): boolean {
return proto3.util.equals(FailedRunResults, a, b);
}
}

45 changes: 43 additions & 2 deletions proto/gen/json_schema/Failed.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
"Failed": {
"properties": {
"ran": {
"additionalProperties": true,
"type": "object"
"$ref": "#/definitions/quary.service.v1.FailedRunResults",
"additionalProperties": true
},
"inferred_from_tests": {
"$ref": "#/definitions/quary.service.v1.InferredChain",
Expand All @@ -21,6 +21,17 @@
"type": "object",
"title": "Failed"
},
"quary.service.v1.FailedRunResults": {
"properties": {
"query_result": {
"$ref": "#/definitions/quary.service.v1.QueryResult",
"additionalProperties": true
}
},
"additionalProperties": true,
"type": "object",
"title": "Failed Run Results"
},
"quary.service.v1.InferredChain": {
"properties": {
"inferred_chain": {
Expand Down Expand Up @@ -49,6 +60,36 @@
"additionalProperties": true,
"type": "object",
"title": "Inferred Chain With Operation"
},
"quary.service.v1.QueryResult": {
"properties": {
"columns": {
"items": {
"$ref": "#/definitions/quary.service.v1.QueryResultColumn"
},
"type": "array"
}
},
"additionalProperties": true,
"type": "object",
"title": "Query Result",
"description": "QueryResult is the result of a ran query."
},
"quary.service.v1.QueryResultColumn": {
"properties": {
"name": {
"type": "string"
},
"values": {
"items": {
"type": "string"
},
"type": "array"
}
},
"additionalProperties": true,
"type": "object",
"title": "Query Result Column"
}
}
}
47 changes: 47 additions & 0 deletions proto/gen/json_schema/FailedRunResults.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"$ref": "#/definitions/FailedRunResults",
"definitions": {
"FailedRunResults": {
"properties": {
"query_result": {
"$ref": "#/definitions/quary.service.v1.QueryResult",
"additionalProperties": true
}
},
"additionalProperties": true,
"type": "object",
"title": "Failed Run Results"
},
"quary.service.v1.QueryResult": {
"properties": {
"columns": {
"items": {
"$ref": "#/definitions/quary.service.v1.QueryResultColumn"
},
"type": "array"
}
},
"additionalProperties": true,
"type": "object",
"title": "Query Result",
"description": "QueryResult is the result of a ran query."
},
"quary.service.v1.QueryResultColumn": {
"properties": {
"name": {
"type": "string"
},
"values": {
"items": {
"type": "string"
},
"type": "array"
}
},
"additionalProperties": true,
"type": "object",
"title": "Query Result Column"
}
}
}
36 changes: 36 additions & 0 deletions proto/gen/json_schema/QueryResult.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"$ref": "#/definitions/QueryResult",
"definitions": {
"QueryResult": {
"properties": {
"columns": {
"items": {
"$ref": "#/definitions/quary.service.v1.QueryResultColumn"
},
"type": "array"
}
},
"additionalProperties": true,
"type": "object",
"title": "Query Result",
"description": "QueryResult is the result of a ran query."
},
"quary.service.v1.QueryResultColumn": {
"properties": {
"name": {
"type": "string"
},
"values": {
"items": {
"type": "string"
},
"type": "array"
}
},
"additionalProperties": true,
"type": "object",
"title": "Query Result Column"
}
}
}
Loading
Loading