A Go library for accurately splitting Oracle PL/SQL scripts into individual statements with precise boundary detection using ANTLR4 for parsing.
This project was created with the help of AI assistance:
- Development performed using Cursor IDE, an AI-powered code editor
- Project structure and development methodology based on the VAN Memory Bank framework
The goal of this library is to help developers extract individual SQL statements from PL/SQL scripts with 100% accurate boundary detection. It uses ANTLR4 for parsing and provides precise source location tracking for each statement.
- Split PL/SQL scripts into individual statements with accurate boundary detection
- Track line and column numbers for each statement
- Properly handle both single-line and multi-line comments
- Process input from both files and strings
- Provide detailed syntax error reporting
- JSON marshalling support for all output structures
- Go 1.21 or later
- ANTLR4 runtime for Go
go get github.com/zodimo/go-plsql-statement-splitterBasic usage example:
package main
import (
"fmt"
"log"
"github.com/zodimo/go-plsql-statement-splitter/pkg/splitter"
)
func main() {
// Split statements from a file
statements, err := splitter.SplitFile("path/to/script.sql")
if err != nil {
log.Fatalf("Error splitting file: %v", err)
}
for i, stmt := range statements {
fmt.Printf("Statement %d: %s\n", i+1, stmt.Content)
fmt.Printf(" Position: %d:%d to %d:%d\n",
stmt.StartLine, stmt.StartColumn,
stmt.EndLine, stmt.EndColumn)
}
// Split statements from a string
sqlContent := `
SELECT * FROM employees;
CREATE OR REPLACE PROCEDURE hello_world IS
BEGIN
DBMS_OUTPUT.PUT_LINE('Hello, World!');
END;
/
`
statements, err = splitter.SplitString(sqlContent)
if err != nil {
log.Fatalf("Error splitting string: %v", err)
}
for i, stmt := range statements {
fmt.Printf("Statement %d: %s\n", i+1, stmt.Content)
}
}MIT
Contributions are welcome! Please feel free to submit a Pull Request.
This library uses ANTLR4 for PL/SQL parsing and statement boundary detection. It incorporates the ANTLR4 grammar from the zodimo/plsql-parser repository, which provides comprehensive support for Oracle PL/SQL syntax.
- Accurate statement boundary detection using a formal grammar approach
- Support for complex PL/SQL constructs
- Detailed positional information for each statement
- Statement type classification (SELECT, INSERT, etc.)
To build the project from source, you need to have ANTLR4 installed:
# Generate the parser code
cd internal/parser
./generate.shThe library provides several configuration options to customize the behavior of the splitter:
package main
import (
"fmt"
"log"
"github.com/zodimo/go-plsql-statement-splitter/pkg/splitter"
)
func main() {
// Create a splitter with custom options
s := splitter.NewSplitter(
splitter.WithPositionInfo(true), // Include position information (default: true)
splitter.WithVerboseErrors(true), // Include detailed error messages (default: false)
splitter.WithMaxErrors(5), // Maximum number of errors to report (default: 1)
splitter.WithErrorContext(true), // Include error context (default: false)
splitter.WithErrorStatement(true), // Include statement causing error (default: false)
)
// Split statements from a file with custom options
statements, err := s.SplitFile("path/to/script.sql")
if err != nil {
syntaxErr, isSyntaxErr := err.(*splitter.SyntaxError)
if isSyntaxErr {
fmt.Printf("Syntax error at line %d, column %d: %s\n",
syntaxErr.Line, syntaxErr.Column, syntaxErr.Message)
} else {
log.Fatalf("Error splitting file: %v", err)
}
}
for i, stmt := range statements {
fmt.Printf("Statement %d (%s): %s\n", i+1, stmt.Type, stmt.Content)
}
}The library supports reading from an io.Reader:
package main
import (
"fmt"
"log"
"os"
"github.com/zodimo/go-plsql-statement-splitter/pkg/splitter"
)
func main() {
// Open a file
file, err := os.Open("path/to/script.sql")
if err != nil {
log.Fatalf("Error opening file: %v", err)
}
defer file.Close()
// Split statements from an io.Reader
statements, err := splitter.SplitReader(file)
if err != nil {
log.Fatalf("Error splitting file: %v", err)
}
for i, stmt := range statements {
fmt.Printf("Statement %d: %s\n", i+1, stmt.Content)
}
}To get all syntax errors in a script:
package main
import (
"fmt"
"log"
"github.com/zodimo/go-plsql-statement-splitter/pkg/splitter"
)
func main() {
s := splitter.NewSplitter(
splitter.WithErrorContext(true),
splitter.WithErrorStatement(true),
)
// Get all syntax errors in a script
content := `
SELECT * FROM;
INSERT INTO employees (id name) VALUES (1, 'John');
`
errors, err := s.GetAllSyntaxErrors(content)
if err != nil {
log.Fatalf("Error getting syntax errors: %v", err)
}
for i, syntaxErr := range errors {
fmt.Printf("Error %d: Line %d, Column %d: %s\n",
i+1, syntaxErr.Line, syntaxErr.Column, syntaxErr.Message)
if syntaxErr.Context != "" {
fmt.Printf(" Context: %s\n", syntaxErr.Context)
}
}
}The library includes a command-line interface for splitting SQL scripts:
# Basic usage
go run cmd/splitter/main.go script.sql
# Specify output format
go run cmd/splitter/main.go -format=json script.sql
# Save output to a file
go run cmd/splitter/main.go -format=json -output=output.json script.sql
# Include verbose error messages
go run cmd/splitter/main.go -verbose-errors script.sql
# Show all syntax errors with context
go run cmd/splitter/main.go -all-errors -error-context script.sqlAvailable CLI options:
-all-errors
Show all errors, ignoring max-errors setting
-error-context
Include context lines for errors
-error-statement
Include full statement with errors
-format string
Output format: text or json (default "text")
-indent string
Indentation for JSON output (default " ")
-max-errors int
Maximum number of errors to report (default 5)
-no-position
Don't include position information
-output string
Output file (works with any format)
-pretty
Pretty print JSON output (default true)
-print-statements
Print the statements (default true)
-print-types
Print statement types (default true)
-verbose-errors
Show detailed error information
The library uses a two-phase approach:
- ANTLR4 parsing of the entire PL/SQL script
- Visitor/listener pattern to extract individual statements with position information
The library can identify the following statement types:
- DML: SELECT, INSERT, UPDATE, DELETE, MERGE
- DDL: CREATE_TABLE, CREATE_VIEW, CREATE_INDEX, etc.
- PL/SQL: PLSQL_BLOCK, CREATE_PROCEDURE, CREATE_FUNCTION, etc.
- Transaction control: COMMIT, ROLLBACK, SAVEPOINT
- Other: EXPLAIN_PLAN, LOCK_TABLE, etc.
- github.com/antlr4-go/antlr/v4: ANTLR4 runtime for Go
- zodimo/plsql-parser: PL/SQL grammar files (embedded in the project)
The project has a comprehensive testing workflow to ensure code quality:
# Run the test script with all checks
./scripts/test.sh -a
# Run basic tests
go test ./...
# Run tests with race detector
go test -race ./...
# Run tests with coverage
go test -cover ./...For more details about testing, see docs/testing.md.
This project uses GitHub Actions for continuous integration:
- Test workflow: Runs tests and linting on push and pull requests
- Coverage workflow: Generates and uploads code coverage reports
- Security scanning: Checks for security issues in the code and dependencies
When adding new features or fixing bugs, please include appropriate tests:
- Add unit tests for new functionality
- Ensure existing tests continue to pass
- Consider adding benchmark tests for performance-critical code
Planned enhancements for future releases:
- Streaming support for processing very large files
- Enhanced error recovery for incomplete statements
- Support for additional Oracle-specific syntax
- Performance optimizations for large scripts
The library provides detailed error reporting capabilities to help diagnose and fix syntax errors:
package main
import (
"fmt"
"log"
"github.com/zodimo/go-plsql-statement-splitter/pkg/splitter"
)
func main() {
// Create a splitter with enhanced error reporting
s := splitter.NewSplitter(
splitter.WithVerboseErrors(true), // Include detailed error messages
splitter.WithMaxErrors(5), // Maximum number of errors to report
splitter.WithErrorContext(true), // Include error context
splitter.WithErrorContextLines(5), // Show 5 lines before and after each error
splitter.WithErrorStatement(true), // Include statement causing error
)
// Try to split statements and handle errors
_, err := s.SplitFile("path/to/script.sql")
if err != nil {
syntaxErr, isSyntaxErr := err.(*splitter.SyntaxError)
if isSyntaxErr {
// This will print the error with context showing 5 lines before and after
fmt.Printf("Syntax error detected:\n%s\n", syntaxErr.Error())
} else {
log.Fatalf("Error: %v", err)
}
}
}An example of the enhanced error output:
Syntax error at line 42, column 10: mismatched input 'END' expecting {';', ','}
40 | FOR emp IN (SELECT * FROM employees) LOOP
41 | DBMS_OUTPUT.PUT_LINE('Employee: ' || emp.name)
42 | END LOOP
^
43 | END;
44 | /
The error output includes:
- Error message with line and column number
- Context showing several lines before and after the error
- A marker (^) pointing precisely to the error position
- Cursor - An advanced IDE powered by AI that was used for the development of this project.
This project utilizes the structured memory bank system developed by vanzan01 - cursor-memory-bank, which provides an organized framework for AI-assisted software development.
