Skip to content

Commit 3b1ac1f

Browse files
authored
Merge pull request #258 from constructive-io/devin/1767262837-pgsql-types-narrowed
feat(pgsql-types): add new package for narrowed PostgreSQL AST types
2 parents 529f487 + 2a40481 commit 3b1ac1f

File tree

10 files changed

+8820
-5553
lines changed

10 files changed

+8820
-5553
lines changed

packages/pgsql-types/README.md

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
# pgsql-types
2+
3+
<p align="center" width="100%">
4+
<img height="250" src="https://raw.githubusercontent.com/constructive-io/constructive/refs/heads/main/assets/outline-logo.svg" />
5+
</p>
6+
7+
<p align="center" width="100%">
8+
<a href="https://github.com/constructive-io/pgsql-parser/actions/workflows/run-tests.yaml">
9+
<img height="20" src="https://github.com/constructive-io/pgsql-parser/actions/workflows/run-tests.yaml/badge.svg" />
10+
</a>
11+
<a href="https://github.com/constructive-io/pgsql-parser/blob/main/LICENSE-MIT"><img height="20" src="https://img.shields.io/badge/license-MIT-blue.svg"/></a>
12+
<a href="https://www.npmjs.com/package/pgsql-types"><img height="20" src="https://img.shields.io/github/package-json/v/constructive-io/pgsql-parser?filename=packages%2Fpgsql-types%2Fpackage.json"/></a>
13+
</p>
14+
15+
Narrowed TypeScript type definitions for PostgreSQL AST nodes.
16+
17+
> **Experimental:** This package provides narrowed types inferred from real SQL usage patterns. For production use, see [`@pgsql/types`](https://www.npmjs.com/package/@pgsql/types). However, please kick the tires and let us know what you think!
18+
19+
## Overview
20+
21+
This package provides TypeScript type definitions for PostgreSQL AST nodes with **narrowed `Node` unions**. Instead of generic `Node` types that could be any of ~250 node types, fields are narrowed to only the specific types that actually appear in practice.
22+
23+
The narrowed types are inferred by parsing thousands of SQL statements from PostgreSQL's test suite and tracking which node types appear in each field.
24+
25+
## Installation
26+
27+
```bash
28+
npm install pgsql-types
29+
```
30+
31+
## The Problem
32+
33+
With `@pgsql/types`, the `arg` field in `DefElem` is typed as `Node` - a union of all possible AST node types:
34+
35+
```typescript
36+
// @pgsql/types
37+
export interface DefElem {
38+
defnamespace?: string;
39+
defname?: string;
40+
arg?: Node; // Could be any of ~250 types!
41+
defaction?: DefElemAction;
42+
location?: number;
43+
}
44+
```
45+
46+
When processing the AST, you have no guidance on what types to actually handle.
47+
48+
## The Solution
49+
50+
With `pgsql-types`, the same field is narrowed to only the types that actually appear:
51+
52+
```typescript
53+
// pgsql-types
54+
export interface DefElem {
55+
defnamespace?: string;
56+
defname?: string;
57+
arg?: { A_Const: A_Const }
58+
| { A_Star: A_Star }
59+
| { Boolean: Boolean }
60+
| { Float: Float }
61+
| { Integer: Integer }
62+
| { List: List }
63+
| { String: String }
64+
| { TypeName: TypeName }
65+
| { VariableSetStmt: VariableSetStmt };
66+
defaction?: DefElemAction;
67+
location?: number;
68+
}
69+
```
70+
71+
Now you know exactly which cases to handle when processing `DefElem.arg`.
72+
73+
## Usage
74+
75+
```typescript
76+
import { DefElem, SelectStmt, CreateStmt } from 'pgsql-types';
77+
78+
function processDefElem(elem: DefElem) {
79+
if (elem.arg) {
80+
// TypeScript knows arg can only be one of 9 specific types
81+
if ('String' in elem.arg) {
82+
console.log('String value:', elem.arg.String.sval);
83+
} else if ('Integer' in elem.arg) {
84+
console.log('Integer value:', elem.arg.Integer.ival);
85+
} else if ('List' in elem.arg) {
86+
console.log('List with', elem.arg.List.items?.length, 'items');
87+
}
88+
// ... handle other cases
89+
}
90+
}
91+
```
92+
93+
## How It Works
94+
95+
The narrowed types are generated by:
96+
97+
1. Parsing all SQL fixtures from PostgreSQL's regression test suite (~70,000 statements)
98+
2. Walking each AST and tracking which node types appear in each `Node`-typed field
99+
3. Generating TypeScript interfaces with narrowed unions based on the observed types
100+
101+
This approach ensures the narrowed types reflect real-world usage patterns from PostgreSQL's own test suite.
102+
103+
## Exports
104+
105+
This package exports:
106+
107+
- All narrowed interfaces (e.g., `SelectStmt`, `CreateStmt`, `DefElem`, etc.)
108+
- The `Node` type from `@pgsql/types`
109+
- All enums from `@pgsql/enums`
110+
111+
## Limitations
112+
113+
- The narrowed types are based on SQL fixtures and may not cover every possible valid AST structure
114+
- Some rarely-used node combinations may not be included in the narrowed unions
115+
- For maximum type safety in production, consider using `@pgsql/types` with runtime validation
116+
117+
## Related Packages
118+
119+
- [`@pgsql/types`](https://www.npmjs.com/package/@pgsql/types) - Production TypeScript types for PostgreSQL AST
120+
- [`@pgsql/enums`](https://www.npmjs.com/package/@pgsql/enums) - PostgreSQL enum definitions
121+
- [`pgsql-parser`](https://www.npmjs.com/package/pgsql-parser) - Parse SQL to AST
122+
- [`pgsql-deparser`](https://www.npmjs.com/package/pgsql-deparser) - Convert AST back to SQL
123+
124+
## License
125+
126+
MIT

packages/pgsql-types/package.json

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
{
2+
"name": "pgsql-types",
3+
"version": "17.0.0",
4+
"author": "Constructive <[email protected]>",
5+
"description": "Narrowed PostgreSQL AST type definitions with specific Node unions",
6+
"main": "index.js",
7+
"module": "esm/index.js",
8+
"types": "index.d.ts",
9+
"homepage": "https://github.com/constructive-io/pgsql-parser",
10+
"license": "MIT",
11+
"publishConfig": {
12+
"access": "public",
13+
"directory": "dist"
14+
},
15+
"repository": {
16+
"type": "git",
17+
"url": "https://github.com/constructive-io/pgsql-parser"
18+
},
19+
"bugs": {
20+
"url": "https://github.com/constructive-io/pgsql-parser/issues"
21+
},
22+
"scripts": {
23+
"copy": "makage assets",
24+
"clean": "makage clean dist",
25+
"prepublishOnly": "npm run build",
26+
"build": "npm run infer && npm run clean && tsc && tsc -p tsconfig.esm.json && npm run copy",
27+
"build:dev": "npm run clean && tsc --declarationMap && tsc -p tsconfig.esm.json && npm run copy",
28+
"infer": "ts-node scripts/infer-field-metadata.ts",
29+
"generate": "ts-node scripts/generate-types.ts",
30+
"lint": "eslint . --fix",
31+
"test": "jest",
32+
"test:watch": "jest --watch"
33+
},
34+
"devDependencies": {
35+
"makage": "^0.1.8",
36+
"libpg-query": "17.7.3"
37+
},
38+
"dependencies": {
39+
"@pgsql/types": "^17.6.2",
40+
"@pgsql/enums": "^17.6.2",
41+
"@pgsql/utils": "^17.8.9"
42+
},
43+
"keywords": [
44+
"sql",
45+
"postgres",
46+
"postgresql",
47+
"pg",
48+
"ast",
49+
"types",
50+
"typescript"
51+
]
52+
}
Lines changed: 204 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
import { runtimeSchema, NodeSpec, FieldSpec } from '../../utils/src/runtime-schema';
2+
import * as fs from 'fs';
3+
import * as path from 'path';
4+
5+
interface FieldMetadata {
6+
nullable: boolean;
7+
tags: string[];
8+
isArray: boolean;
9+
}
10+
11+
interface NodeFieldMetadata {
12+
[fieldName: string]: FieldMetadata;
13+
}
14+
15+
interface AllFieldMetadata {
16+
[nodeName: string]: NodeFieldMetadata;
17+
}
18+
19+
const schemaMap = new Map<string, NodeSpec>(
20+
runtimeSchema.map((spec: NodeSpec) => [spec.name, spec])
21+
);
22+
23+
const primitiveTypeMap: Record<string, string> = {
24+
'string': 'string',
25+
'bool': 'boolean',
26+
'int32': 'number',
27+
'int64': 'number',
28+
'uint32': 'number',
29+
'uint64': 'number',
30+
'float': 'number',
31+
'double': 'number',
32+
'bytes': 'Uint8Array',
33+
};
34+
35+
function isPrimitiveType(type: string): boolean {
36+
return type in primitiveTypeMap;
37+
}
38+
39+
function isEnumType(type: string): boolean {
40+
return !isPrimitiveType(type) && !schemaMap.has(type) && type !== 'Node';
41+
}
42+
43+
function getTsType(type: string): string {
44+
return primitiveTypeMap[type] || type;
45+
}
46+
47+
function collectEnumTypes(): Set<string> {
48+
const enumTypes = new Set<string>();
49+
for (const nodeSpec of runtimeSchema) {
50+
for (const field of nodeSpec.fields) {
51+
if (isEnumType(field.type)) {
52+
enumTypes.add(field.type);
53+
}
54+
}
55+
}
56+
return enumTypes;
57+
}
58+
59+
function generateWrappedUnion(tags: string[]): string {
60+
if (tags.length === 0) {
61+
return 'Node';
62+
}
63+
64+
const sortedTags = [...tags].sort();
65+
return sortedTags.map(tag => `{ ${tag}: ${tag} }`).join(' | ');
66+
}
67+
68+
function generateTypeAlias(nodeName: string, fieldName: string, tags: string[]): string {
69+
const aliasName = `${nodeName}_${fieldName}`;
70+
const union = generateWrappedUnion(tags);
71+
return `type ${aliasName} = ${union};`;
72+
}
73+
74+
function generateInterface(
75+
nodeSpec: NodeSpec,
76+
fieldMetadata: NodeFieldMetadata | undefined
77+
): string {
78+
const lines: string[] = [];
79+
lines.push(`export interface ${nodeSpec.name} {`);
80+
81+
for (const field of nodeSpec.fields) {
82+
const tsType = getFieldType(nodeSpec.name, field, fieldMetadata);
83+
const optional = field.optional ? '?' : '';
84+
lines.push(` ${field.name}${optional}: ${tsType};`);
85+
}
86+
87+
lines.push('}');
88+
return lines.join('\n');
89+
}
90+
91+
function getFieldType(
92+
nodeName: string,
93+
field: FieldSpec,
94+
fieldMetadata: NodeFieldMetadata | undefined
95+
): string {
96+
let baseType: string;
97+
98+
if (field.type === 'Node') {
99+
const meta = fieldMetadata?.[field.name];
100+
if (meta && meta.tags.length > 0) {
101+
baseType = `${nodeName}_${field.name}`;
102+
} else {
103+
baseType = 'Node';
104+
}
105+
} else if (isPrimitiveType(field.type)) {
106+
baseType = getTsType(field.type);
107+
} else {
108+
if (schemaMap.has(field.type)) {
109+
baseType = `{ ${field.type}: ${field.type} }`;
110+
} else {
111+
baseType = field.type;
112+
}
113+
}
114+
115+
if (field.isArray) {
116+
if (baseType.includes('|') || baseType.includes('{')) {
117+
return `(${baseType})[]`;
118+
}
119+
return `${baseType}[]`;
120+
}
121+
122+
return baseType;
123+
}
124+
125+
function generateTypes(metadata: AllFieldMetadata): string {
126+
const lines: string[] = [];
127+
128+
lines.push('/**');
129+
lines.push(' * This file was automatically generated by pgsql-types.');
130+
lines.push(' * DO NOT MODIFY IT BY HAND.');
131+
lines.push(' * ');
132+
lines.push(' * These types provide narrowed Node unions based on actual usage');
133+
lines.push(' * patterns discovered by parsing SQL fixtures.');
134+
lines.push(' */');
135+
lines.push('');
136+
137+
const enumTypes = collectEnumTypes();
138+
const sortedEnums = [...enumTypes].sort();
139+
140+
lines.push("import type { Node } from '@pgsql/types';");
141+
if (sortedEnums.length > 0) {
142+
lines.push(`import { ${sortedEnums.join(', ')} } from '@pgsql/enums';`);
143+
}
144+
lines.push("export type { Node } from '@pgsql/types';");
145+
lines.push("export * from '@pgsql/enums';");
146+
lines.push('');
147+
148+
const typeAliases: string[] = [];
149+
for (const nodeName of Object.keys(metadata).sort()) {
150+
const nodeMetadata = metadata[nodeName];
151+
for (const fieldName of Object.keys(nodeMetadata).sort()) {
152+
const fieldMeta = nodeMetadata[fieldName];
153+
if (fieldMeta.tags.length > 0) {
154+
typeAliases.push(generateTypeAlias(nodeName, fieldName, fieldMeta.tags));
155+
}
156+
}
157+
}
158+
159+
if (typeAliases.length > 0) {
160+
lines.push('// Internal type aliases for narrowed Node-typed fields (not exported)');
161+
lines.push(typeAliases.join('\n'));
162+
lines.push('');
163+
}
164+
165+
lines.push('// Interfaces with narrowed Node types');
166+
for (const nodeSpec of runtimeSchema) {
167+
const nodeMetadata = metadata[nodeSpec.name];
168+
lines.push(generateInterface(nodeSpec, nodeMetadata));
169+
lines.push('');
170+
}
171+
172+
return lines.join('\n');
173+
}
174+
175+
async function main() {
176+
const metadataPath = path.resolve(__dirname, '../src/field-metadata.json');
177+
const outputPath = path.resolve(__dirname, '../src/types.ts');
178+
179+
if (!fs.existsSync(metadataPath)) {
180+
console.error('Field metadata not found. Run "npm run infer" first.');
181+
process.exit(1);
182+
}
183+
184+
const metadata: AllFieldMetadata = JSON.parse(fs.readFileSync(metadataPath, 'utf-8'));
185+
186+
console.log('Generating narrowed types...');
187+
const typesContent = generateTypes(metadata);
188+
189+
fs.writeFileSync(outputPath, typesContent);
190+
console.log(`Wrote narrowed types to ${outputPath}`);
191+
192+
let totalAliases = 0;
193+
for (const nodeName of Object.keys(metadata)) {
194+
for (const fieldName of Object.keys(metadata[nodeName])) {
195+
if (metadata[nodeName][fieldName].tags.length > 0) {
196+
totalAliases++;
197+
}
198+
}
199+
}
200+
201+
console.log(`Generated ${totalAliases} narrowed type aliases`);
202+
}
203+
204+
main().catch(console.error);

0 commit comments

Comments
 (0)