Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[config] [dsl] fixes #32 - RXT scraper definition language #38

Open
wants to merge 87 commits into
base: develop
Choose a base branch
from
Open

[config] [dsl] fixes #32 - RXT scraper definition language #38

wants to merge 87 commits into from

Conversation

olemis
Copy link
Contributor

@olemis olemis commented Nov 21, 2018 •
edited by stdevAlDen
Loading

Fixes #32

Changes:

  • Core interfaces
  • Lexer and parser for RXT scraper definition language

Does this change need to mentioned in CHANGELOG.md?
yes

TODO :

    • Exclude generated files from linter rules
    • Add lexer golden files containing lexical errors
    • Test cases for matching sample RXT files against corresponding lexer golden files
    • Naive make test-grammar for start ...
    • Run RXT lexer golden file test cases in make test-grammar
    • Implement core interfaces for obtaining AST
    • Naive tests for AST
    • Run RXT AST golden file test cases in make test-grammar
    • Implement core interfaces for obtaining scraper
    • Error handling in lexer and parser
    • Configure REXT parser to log to stderr

@olemis
Copy link
Contributor Author

olemis commented Nov 23, 2018 •
edited
Loading

@stdevAlDen this is how I am testing the lexer now (refs item 3)

$ nex -s src/rxt/grammar/lexer.nex
$ go run cmd/rxtc/lexer.go < src/rxt/testdata/skyexample.rxt 2> src/rxt/testdata/skyexample.golden.orig
$ diff -u src/rxt/testdata/skyexample.golden src/rxt/testdata/skyexample.golden.orig

This is how I used to do it before

$ nex -s src/config/rxt/lexer.nex
$ go run src/config/rxt/lexer.nn.go < src/config/rxt/testdata/skyexample.rxt 2> src/config/rxt/testdata/skyexample.golden.orig
$ diff -u src/config/rxt/testdata/skyexample.golden src/config/rxt/testdata/skyexample.golden.orig

olemis and others added 16 commits November 23, 2018 17:35
@olemis
[config] [dsl] refs #32 Boundary conditions related to EOB token
d9f1df4
@olemis
[config] [dsl] refs #32 - RXT AST data types
dc6283e
@olemis
[rxt] refs #32 - Move src/config/rxt => src/rxt
3bc397c 
Autogenerated grammar files in src/rxt/grammar
@olemis
[cmd] [rxt] refs #32 - Lexer command based on (reusable) LexTheRxt fu…
82bf2bb 
…nction
@olemis
[core] refs #32 - Core interfaces with methods needed for parsing REX…
9b04cfb 
…T DSL
@olemis
[core] refs #32 - Some methods in core interfaces may return error
0a5628e
@olemis
[rxt] refs #32 - CTX token. TokenHandler interface
f2feb01
@olemis
[rxt] refs #32 - make test-grammar outputs tokens to file and diff -u…
d60f9bd 
… against golden file
ref #32 Merge branch 'denisacostaq_t54_automatically_generated_labels…
ad2ace0 
…_and_time_series' into stdevAlDen_t32_rxt_dsl
ref #32 skip autogenerated file lexer.nn.go from linter
ed7f01b
ref #32 fix some build errors
bfb0a19
@olemis
[rxt] refs #32 - TokenHandler lexer interface
003121a
ref #32 Merge remote-tracking branch 'olemis/olemis_t32_rxt_dsl' into…
a7a7112 
… stdevAlDen_t32_rxt_dsl
@olemis
[config] [rxt] [util] refs #32 - Parser grammar ... WiP
1f4a05b 
- Changes in AST classes to cope with grammar aspects
- Key/value store implementation using map[string]string
- Library function : MergeStoresInplace
- TokenWriter prints token pairs to stdout (used for tokenizing files to be compared against golden lexer checks)
ref #32 Merge branch 'denisacostaq_t54_automatically_generated_labels…
08dbd9f 
…_and_time_series' into stdevAlDen_t32_rxt_dsl
ref #32 Merge remote-tracking branch 'olemis/olemis_t32_rxt_dsl' into…
c034e0a 
… stdevAlDen_t32_rxt_dsl
Alvaro Denis and others added 30 commits December 21, 2018 14:44
ref #32 validations and tests for RextAuth
f72881c
ref #32 validations and tests for RextResourceDef
7b86c03
ref #32 igore mocks folder
c6a861c
ref #32 validations and tests for RextService
18b5d5f
ref #32 refactor source to resource
b892977
ref #32 validations and tests for RextNodeSolver
f03ad3d
ref #32 refactor resource in RextService
0d82784
ref #32 validations and tests for RextLabel
262f030
ref #32 validations and tests for RextDecoder
8af8886
ref #32 validations and tests for RextMetric
8676163
ref #32 validations and tests for RextRoot
281f213
ref #32 fixes, refactor, source code comments
2e37771 
- refactor in core interfaces definition, add Clone and Validate methods
- some source code comment on validations
- refactor some variable names
- fix some bugs
ref #32 install github.com/vektra/mockery from travis script
bc0f5fe
ref #32 add rule to makefile, generate mocks.
75b90dc
ref #32 delete file from config
3b7d6cf 
	deleted:    src/config/config.go
	deleted:    src/config/config_render.go
	deleted:    src/config/metric.go
	deleted:    src/config/metric_test.go
	deleted:    src/config/server.go
	deleted:    src/config/server_test.go
	deleted:    src/config/service.go
	deleted:    src/config/service_test.go
	deleted:    src/config/serviceconfigfromfile.go
	deleted:    src/config/serviceconfigreader.go
ref #32 crate a core.RextRoot config from a toml config and fix some …
c404d0b 
…bugs
ref #32 fix bug on metrics validations
7960875
ref #32 refactor and little fixes
3cc99ef
ref #32 a complete rework over integration tests
dd09434
ref #32 track toml config reader
c17eb19
ref #32 url validation
6c59e31
ref #32 fix bug on url validation, ignored result
719f67d
ref #32 refactor based on linter and delete key_val_4test
243ae18
ref #32 update vendor
13326e5
ref #32 skip autogenerated mocks folder
c712c06
ref #32 keep the mocks folders even without content
6e4723b
ref #32 Merge remote-tracking branch 'upstream/develop' into stdevAlD…
9d5d595 
…en_t32_rxt_dsl
@olemis
Merge remote-tracking branch 'simelo/develop' into olemis_t32_rxt_dsl
7ab550e
ref #32 Merge remote-tracking branch 'olemis/olemis_t32_rxt_dsl' into…
d0481a1 
… stdevAlDen_t32_rxt_dsl
@olemis
Merge pull request #1 from denisacostaq/stdevAlDen_t32_rxt_dsl
83db570 
Implement RXT DSL interfaces for scraper configuration
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@olemis olemis

@stdevAlDen stdevAlDen

Labels
config stdev wip
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@olemis @stdevAlDen