Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AML v2 #322

Merged
merged 96 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from 74 commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
68ae1f6
Better management of whitespaces
loicknuchel Aug 31, 2024
61d5209
Basic AML generation
loicknuchel Aug 31, 2024
3f586e1
Add constraints
loicknuchel Aug 31, 2024
0807d12
Handle nested attributes
loicknuchel Sep 1, 2024
025d0bc
Allow multi-line comments
loicknuchel Sep 1, 2024
1a432cd
Allow v1 compatibility
loicknuchel Sep 1, 2024
ad807d8
Fix snippet checks
loicknuchel Sep 1, 2024
b72037d
Add AML parsing for demo files
loicknuchel Sep 1, 2024
272ce68
Implement type rules
loicknuchel Sep 2, 2024
e497953
Fix doc errors
loicknuchel Sep 2, 2024
e723eda
Rename serde-aml to aml
loicknuchel Sep 3, 2024
2ea678b
Start aml page
loicknuchel Sep 3, 2024
f900e03
Migrate phoenix assets to TypeScript
loicknuchel Sep 3, 2024
d7bc4a2
tmp
loicknuchel Sep 4, 2024
d056d57
Improve monaco editor
loicknuchel Sep 4, 2024
bde1991
Merge branch 'main' into aml-serde
loicknuchel Sep 4, 2024
9d8dc6c
fix
loicknuchel Sep 4, 2024
5e72a75
Create bundle for aml lib
loicknuchel Sep 5, 2024
cc0b1f9
Use AMLv2 in AmlSidebar in Azimutt
loicknuchel Sep 6, 2024
554c23b
fix
loicknuchel Sep 6, 2024
47330cf
fix2
loicknuchel Sep 6, 2024
3d6dbf2
handle types in aml
loicknuchel Sep 7, 2024
48caea7
fix
loicknuchel Sep 7, 2024
ff6de88
Improve legacy compatibility
loicknuchel Sep 7, 2024
c1202f1
more robust parser
loicknuchel Sep 7, 2024
edda074
Refactor AML ast
loicknuchel Sep 8, 2024
5d3a1e1
fix
loicknuchel Sep 8, 2024
78f8081
fix2
loicknuchel Sep 8, 2024
bc39a58
Improve parser robustness & monaco completions
loicknuchel Sep 9, 2024
7206810
Add more warnings in ast & improve monaco editor
loicknuchel Sep 9, 2024
91e3114
Add monaco quickfixes
loicknuchel Sep 10, 2024
ae72bc4
Merge branch 'main' into aml-serde
loicknuchel Sep 10, 2024
5922f63
fix regex
loicknuchel Sep 10, 2024
d052f04
Merge branch 'main' into aml-serde
loicknuchel Sep 11, 2024
fa5ac70
Add duplicate checks
loicknuchel Sep 11, 2024
70407ab
WIP: Monaco web component
loicknuchel Sep 11, 2024
cfbbcfb
Remove Elm AML parser
loicknuchel Sep 14, 2024
59d7593
Generate AML & JSON exports from JS
loicknuchel Sep 15, 2024
83883e0
Improve AML generation
loicknuchel Sep 15, 2024
16f6708
Create convert command in CLI
loicknuchel Sep 16, 2024
5b32b14
Add indentation warning
loicknuchel Sep 16, 2024
a06a246
Fix doc snippets
loicknuchel Sep 16, 2024
b0e1ed4
Fix AML warnings
loicknuchel Sep 17, 2024
8dc60cb
Create converter pages
loicknuchel Sep 18, 2024
574f9f8
fix
loicknuchel Sep 18, 2024
cf54d49
Handle AML properties
loicknuchel Sep 18, 2024
f2d6fcc
fix
loicknuchel Sep 18, 2024
990eb9b
fix2
loicknuchel Sep 18, 2024
ac17b32
fix3
loicknuchel Sep 19, 2024
eb90eee
Improve AML doc
loicknuchel Sep 19, 2024
8514de2
Improve AML doc
loicknuchel Sep 20, 2024
ff10ec7
Rename serde to parser
loicknuchel Sep 20, 2024
b74d04c
Docker build
loicknuchel Sep 21, 2024
75f06b9
Add Open in Azimutt button
loicknuchel Sep 21, 2024
44871df
Server AML schema
loicknuchel Sep 21, 2024
3a623e7
Improve edge cases
loicknuchel Sep 21, 2024
1c4e0c7
Improve AML gen
loicknuchel Sep 21, 2024
36eb1bb
Make all snippets aligned with generation
loicknuchel Sep 22, 2024
2c2a299
Add rollup to DBML parser
loicknuchel Sep 22, 2024
1af1156
Add rollup to SQL parser
loicknuchel Sep 22, 2024
6b9a0a0
Add rollup to Prisma parser
loicknuchel Sep 22, 2024
be4e960
Generate PostgreSQL
loicknuchel Sep 22, 2024
f2e4f2a
code review
loicknuchel Sep 22, 2024
3aa9b08
Create Mermaid generator
loicknuchel Sep 23, 2024
ebeedc9
Create Markdown generator
loicknuchel Sep 23, 2024
158a6ad
fix
loicknuchel Sep 23, 2024
2de1b4c
Update complex schema in AML lib
loicknuchel Sep 23, 2024
b63b6dc
Improve complex test
loicknuchel Sep 23, 2024
0df930d
fix
loicknuchel Sep 23, 2024
e5abae3
Improve demo doc
loicknuchel Sep 24, 2024
1ea8677
Refactor ParserError structure
loicknuchel Sep 24, 2024
5a76b4e
Improve gen AMLv1
loicknuchel Sep 24, 2024
2b5558f
Improve syntax highlight
loicknuchel Sep 24, 2024
744a942
Refactor AML generator
loicknuchel Sep 25, 2024
5a4c57c
Improve Monaco auto-complete
loicknuchel Sep 25, 2024
af5f3fa
Fix check parsing
loicknuchel Sep 25, 2024
fca1e96
Review doc
loicknuchel Sep 26, 2024
c5d7045
fix
loicknuchel Sep 26, 2024
40b52f8
Improve handling of namespace, comments & composite checks
loicknuchel Sep 27, 2024
9500a2e
Handle alias in relations
loicknuchel Sep 27, 2024
7af3af0
Add autocomplete for properties
loicknuchel Sep 27, 2024
720f95f
fix
loicknuchel Sep 27, 2024
84f1382
Small refacto
loicknuchel Sep 27, 2024
afaaa2b
Refactor relations
loicknuchel Sep 28, 2024
c4139e0
Merge duplicates and warn on bad relations
loicknuchel Sep 29, 2024
7cec31a
Write post
loicknuchel Sep 29, 2024
cf0f66b
fix
loicknuchel Sep 29, 2024
2841a05
upgrade libs
loicknuchel Sep 29, 2024
2f8c3f1
publish connectors
loicknuchel Sep 29, 2024
4662913
fix migrations
loicknuchel Sep 29, 2024
11f300e
fix cli
loicknuchel Sep 29, 2024
b9a57b1
fix cli 2 + update dbml
loicknuchel Sep 29, 2024
edbf183
publish parsers
loicknuchel Sep 29, 2024
9205b14
publish gateway & cli
loicknuchel Sep 29, 2024
8a21b17
unpublish cli
loicknuchel Sep 29, 2024
43f3695
publish cli
loicknuchel Sep 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
.DS_Store
.env
.clever.json
/node_modules/
node_modules
addon-manifest.json
addon-manifest-prod.json
addon-manifest-staging.json
Expand Down
32 changes: 16 additions & 16 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -78,33 +78,33 @@ repos:
pass_filenames: false
files: libs/connector-sqlserver/.*\.ts*$

- id: libs-serde-aml
name: 'libs/serde-aml'
- id: libs-aml
name: 'libs/aml'
language: system
entry: bash -c 'cd libs/serde-aml && exec npm run build && exec npm test'
entry: bash -c 'cd libs/aml && exec npm run build && exec npm test'
pass_filenames: false
files: libs/serde-aml/.*\.ts*$
files: libs/aml/.*\.ts*$

- id: libs-serde-dbml
name: 'libs/serde-dbml'
- id: libs-parser-dbml
name: 'libs/parser-dbml'
language: system
entry: bash -c 'cd libs/serde-dbml && exec npm run build && exec npm test'
entry: bash -c 'cd libs/parser-dbml && exec npm run build && exec npm test'
pass_filenames: false
files: libs/serde-dbml/.*\.ts*$
files: libs/parser-dbml/.*\.ts*$

- id: libs-serde-prisma
name: 'libs/serde-prisma'
- id: libs-parser-prisma
name: 'libs/parser-prisma'
language: system
entry: bash -c 'cd libs/serde-prisma && exec npm run build && exec npm test'
entry: bash -c 'cd libs/parser-prisma && exec npm run build && exec npm test'
pass_filenames: false
files: libs/serde-prisma/.*\.ts*$
files: libs/parser-prisma/.*\.ts*$

- id: libs-serde-sql
name: 'libs/serde-sql'
- id: libs-parser-sql
name: 'libs/parser-sql'
language: system
entry: bash -c 'cd libs/serde-sql && exec npm run build && exec npm test'
entry: bash -c 'cd libs/parser-sql && exec npm run build && exec npm test'
pass_filenames: false
files: libs/serde-sql/.*\.ts*$
files: libs/parser-sql/.*\.ts*$

- id: mix-format
name: 'Elixir format'
Expand Down
1 change: 1 addition & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ RUN npm run build:docker
COPY backend/lib lib

# compile assets
RUN cd assets && npm install
RUN mix assets.deploy

RUN mix compile
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<picture>
<source media="(prefers-color-scheme: dark)" srcset="assets/azimutt-logo-light.png">
<source media="(prefers-color-scheme: light)" srcset="assets/azimutt-logo-dark.png">
<img alt="Azimutt logo" src="assets/azimutt-logo-dark.png">
<img alt="Azimutt logo" src="assets/azimutt-logo-dark.png" width="500">
</picture>
</a>
</p>
Expand Down Expand Up @@ -38,7 +38,7 @@ Azimutt started as a schema exploration tool for databases with hundreds of tabl

[![Azimutt roadmap](docs/_assets/roadmap.png)](https://mm.tt/map/2434161843?t=N2yWZj1pc1)

- Design your schema using [AML](docs/aml/README.md) for a fast diagramming
- Design your schema using [AML](libs/aml/README.md) for a fast diagramming
- Explore your database schema using search everywhere, display only useful tables/columns and follow relations
- Query your data like never before, follow foreign keys and display entities in diagram
- Document using table/column notes and tags and layouts and memos for use cases, features or team scopes
Expand Down
2 changes: 2 additions & 0 deletions backend/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -31,5 +31,7 @@ azimutt-*.tar
# In case you use Node.js/npm, you want to ignore these.
npm-debug.log
/node_modules/

/priv/static/swagger*.json
/assets/js/*.js
/uploads
26 changes: 26 additions & 0 deletions backend/assets/js/aml.hljs.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import {HLJSApi, Language} from "highlight.js"

// keep in sync with libs/aml/src/extensions/monaco.ts
const entityRegex = /^[a-zA-Z_][a-zA-Z0-9_#]*/
const attributeNameRegex = /^ +[a-zA-Z_][a-zA-Z0-9_#]*/
const attributeTypeRegex = /\b(uuid|(var|n)?char2?|character( varying)?|(tiny|medium|long|ci)?text|(tiny|small|big)?int(eger)?(\d+)?|numeric|float|double( precision)?|bool(ean)?|timestamp( with(out)? time zone)?|date(time)?|time( with(out)? time zone)?|interval|json|string|number)\b/
const notesRegex = /\|[^#\n]*/
const commentRegex = /#.*/

// see https://highlightjs.readthedocs.io/en/latest/language-guide.html
export function language(hljs: HLJSApi): Language {
return {
name: 'aml',
case_insensitive: true,
keywords: ['namespace', 'nullable', 'pk', 'index', 'unique', 'check', 'fk', 'rel', 'type'],
contains: [
{scope: 'title', begin: entityRegex},
{scope: 'title.class', begin: attributeNameRegex},
// FIXME: {scope: 'type', beginScope: 'title.class', begin: / [a-zA-Z]+/, end: / |\(|=|\n/}, // attribute type
{scope: 'type', begin: attributeTypeRegex},
{scope: 'subst', begin: /-> |fk /, end: /[) \n]/}, // inline relation
{scope: 'string', begin: notesRegex},
{scope: 'comment', begin: commentRegex},
]
}
}
40 changes: 26 additions & 14 deletions backend/assets/js/app.js → backend/assets/js/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,31 +20,36 @@

// Include phoenix_html to handle method=PUT/DELETE in forms and buttons.
import "phoenix_html"
import Alpine from "../vendor/alpine"
import Hljs from "../vendor/highlight.min"
import topbar from "../vendor/topbar"

// Establish Phoenix Socket and LiveView configuration.
import {Socket} from "phoenix"
import {LiveSocket} from "phoenix_live_view"

window.hljs = Hljs;
hljs.highlightAll();
window.Alpine = Alpine;
Alpine.start();
let hooks = {};
let csrfToken = document.querySelector("meta[name='csrf-token']").getAttribute("content")
let liveSocket = new LiveSocket("/live", Socket, {
import Alpine from "alpinejs"
import Hljs from "highlight.js"
import topbar from "topbar"
import {language as hljsAml} from "./aml.hljs"

Hljs.registerLanguage('aml', hljsAml)
Hljs.configure({cssSelector: 'code.hljs'})
Hljs.highlightAll()


Alpine.start()
const hooks = {}
const csrfToken = document.querySelector("meta[name='csrf-token']")?.getAttribute("content")
const liveSocket = new LiveSocket("/live", Socket, {
params: { _csrf_token: csrfToken },
hooks: hooks,
dom: {
onBeforeElUpdated(from, to) {
if (from._x_dataStack) {
window.Alpine.clone(from, to);
onBeforeElUpdated(from: HTMLElement, to: HTMLElement): boolean {
if ((from as any)._x_dataStack) {
Alpine.cloneNode(from, to)
}
return false
},
},
});
})


// Show progress bar on live navigation and form submits
Expand All @@ -60,3 +65,10 @@ liveSocket.connect()
// >> liveSocket.enableLatencySim(1000) // enabled for duration of browser session
// >> liveSocket.disableLatencySim()
window.liveSocket = liveSocket


declare global {
export interface Window {
liveSocket: LiveSocket
}
}
92 changes: 92 additions & 0 deletions backend/assets/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

26 changes: 26 additions & 0 deletions backend/assets/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"name": "@azimutt/phoenix-app",
"version": "0.0.1",
"description": "App managed by Phoenix assets and loaded in every page",
"author": {
"name": "Loïc Knuchel",
"email": "[email protected]",
"url": "https://loicknuchel.fr"
},
"license": "MIT",
"main": "js/app.ts",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"dependencies": {
"alpinejs": "^3.14.1",
"highlight.js": "^11.10.0",
"topbar": "^3.0.0"
},
"devDependencies": {
"@types/alpinejs": "^3.13.10",
"@types/phoenix": "^1.6.5",
"@types/phoenix_live_view": "^0.18.5",
"typescript": "^5.5.4"
}
}
14 changes: 14 additions & 0 deletions backend/assets/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"compilerOptions": {
"target": "es2017",
"module": "es2022",
"moduleResolution": "node",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true
},
"include": [
"js/**/*"
]
}
5 changes: 0 additions & 5 deletions backend/assets/vendor/alpine.js

This file was deleted.

Loading
Loading