[config] [dsl] fixes #32 - RXT scraper definition language by olemis · Pull Request #38 · simelo/rextporter

olemis · 2018-11-21T07:13:30Z

Fixes #32

Changes:

Core interfaces
Lexer and parser for RXT scraper definition language

Does this change need to mentioned in CHANGELOG.md?
yes

TODO :

- Exclude generated files from linter rules
- Add lexer golden files containing lexical errors
- Test cases for matching sample RXT files against corresponding lexer golden files
- Naive make test-grammar for start ...
- Run RXT lexer golden file test cases in make test-grammar
- Implement core interfaces for obtaining AST
- Naive tests for AST
- Run RXT AST golden file test cases in make test-grammar
- Implement core interfaces for obtaining scraper
- Error handling in lexer and parser
- Configure REXT parser to log to stderr

…ping

…icsExtractor for runtime. RextMetric for measured data. Base URL

…n tokens

Change in RXT sample AT => FROM

Golden test files match lexer output when sample RXT spec is the input

olemis · 2018-11-23T16:37:08Z

@stdevAlDen this is how I am testing the lexer now (refs item 3)

$ nex -s src/rxt/grammar/lexer.nex
$ go run cmd/rxtc/lexer.go < src/rxt/testdata/skyexample.rxt 2> src/rxt/testdata/skyexample.golden.orig
$ diff -u src/rxt/testdata/skyexample.golden src/rxt/testdata/skyexample.golden.orig

This is how I used to do it before

$ nex -s src/config/rxt/lexer.nex
$ go run src/config/rxt/lexer.nn.go < src/config/rxt/testdata/skyexample.rxt 2> src/config/rxt/testdata/skyexample.golden.orig
$ diff -u src/config/rxt/testdata/skyexample.golden src/config/rxt/testdata/skyexample.golden.orig

Autogenerated grammar files in src/rxt/grammar

…nction

…T DSL

… against golden file

…_and_time_series' into stdevAlDen_t32_rxt_dsl

… stdevAlDen_t32_rxt_dsl

- Changes in AST classes to cope with grammar aspects - Key/value store implementation using map[string]string - Library function : MergeStoresInplace - TokenWriter prints token pairs to stdout (used for tokenizing files to be compared against golden lexer checks)

…_and_time_series' into stdevAlDen_t32_rxt_dsl

… stdevAlDen_t32_rxt_dsl

- refactor in core interfaces definition, add Clone and Validate methods - some source code comment on validations - refactor some variable names - fix some bugs

deleted: src/config/config.go deleted: src/config/config_render.go deleted: src/config/metric.go deleted: src/config/metric_test.go deleted: src/config/server.go deleted: src/config/server_test.go deleted: src/config/service.go deleted: src/config/service_test.go deleted: src/config/serviceconfigfromfile.go deleted: src/config/serviceconfigreader.go

…bugs

…en_t32_rxt_dsl

… stdevAlDen_t32_rxt_dsl

Implement RXT DSL interfaces for scraper configuration

olemis added 4 commits November 20, 2018 12:39

[core] refs #32 - Core interfaces for datasets, and sources for scrap…

c98a68e

…ping

[config] [dsl] refs #32 - RextScrapingRule => RextMetricDef. RextMetr…

9d9dc52

…icsExtractor for runtime. RextMetric for measured data. Base URL

Merge remote-tracking branch 'simelo/develop' into olemis_t32_rxt_dsl

1d18f14

[config] [dsl] refs #32 - Lexer for RXT language

a0ed99e

olemis assigned olemis and stdevAlDen Nov 21, 2018

olemis added stdev wip config labels Nov 21, 2018

olemis added 4 commits November 21, 2018 22:22

[config] [dsl] refs #32 - Lexer tokens BLK, EOL, EOB, BIE, PCT

57c39c1

[config] [dsl] refs #32 - Remove whitespace while emitting punctuatio…

48d6c02

…n tokens

[config] [dsl] refs #32 - Correct lexer golden file for sample RXT spec.

8dd72bb

Change in RXT sample AT => FROM

[config] [dsl] refs #32 - Fix matching of RXT indentation tokens.

1164bbb

Golden test files match lexer output when sample RXT spec is the input

olemis and others added 16 commits November 23, 2018 17:35

[config] [dsl] refs #32 Boundary conditions related to EOB token

d9f1df4

[config] [dsl] refs #32 - RXT AST data types

dc6283e

[rxt] refs #32 - Move src/config/rxt => src/rxt

3bc397c

Autogenerated grammar files in src/rxt/grammar

[cmd] [rxt] refs #32 - Lexer command based on (reusable) LexTheRxt fu…

82bf2bb

…nction

[core] refs #32 - Core interfaces with methods needed for parsing REX…

9b04cfb

…T DSL

[core] refs #32 - Some methods in core interfaces may return error

0a5628e

[rxt] refs #32 - CTX token. TokenHandler interface

f2feb01

[rxt] refs #32 - make test-grammar outputs tokens to file and diff -u…

d60f9bd

… against golden file

ref #32 Merge branch 'denisacostaq_t54_automatically_generated_labels…

ad2ace0

…_and_time_series' into stdevAlDen_t32_rxt_dsl

ref #32 skip autogenerated file lexer.nn.go from linter

ed7f01b

ref #32 fix some build errors

bfb0a19

[rxt] refs #32 - TokenHandler lexer interface

003121a

ref #32 Merge remote-tracking branch 'olemis/olemis_t32_rxt_dsl' into…

a7a7112

… stdevAlDen_t32_rxt_dsl

ref #32 Merge branch 'denisacostaq_t54_automatically_generated_labels…

08dbd9f

…_and_time_series' into stdevAlDen_t32_rxt_dsl

ref #32 Merge remote-tracking branch 'olemis/olemis_t32_rxt_dsl' into…

c034e0a

… stdevAlDen_t32_rxt_dsl

Alvaro Denis and others added 30 commits December 21, 2018 14:44

ref #32 validations and tests for RextAuth

f72881c

ref #32 validations and tests for RextResourceDef

7b86c03

ref #32 igore mocks folder

c6a861c

ref #32 validations and tests for RextService

18b5d5f

ref #32 refactor source to resource

b892977

ref #32 validations and tests for RextNodeSolver

f03ad3d

ref #32 refactor resource in RextService

0d82784

ref #32 validations and tests for RextLabel

262f030

ref #32 validations and tests for RextDecoder

8af8886

ref #32 validations and tests for RextMetric

8676163

ref #32 validations and tests for RextRoot

281f213

ref #32 fixes, refactor, source code comments

2e37771

- refactor in core interfaces definition, add Clone and Validate methods - some source code comment on validations - refactor some variable names - fix some bugs

ref #32 install github.com/vektra/mockery from travis script

bc0f5fe

ref #32 add rule to makefile, generate mocks.

75b90dc

ref #32 crate a core.RextRoot config from a toml config and fix some …

c404d0b

…bugs

ref #32 fix bug on metrics validations

7960875

ref #32 refactor and little fixes

3cc99ef

ref #32 a complete rework over integration tests

dd09434

ref #32 track toml config reader

c17eb19

ref #32 url validation

6c59e31

ref #32 fix bug on url validation, ignored result

719f67d

ref #32 refactor based on linter and delete key_val_4test

243ae18

ref #32 update vendor

13326e5

ref #32 skip autogenerated mocks folder

c712c06

ref #32 keep the mocks folders even without content

6e4723b

ref #32 Merge remote-tracking branch 'upstream/develop' into stdevAlD…

9d5d595

…en_t32_rxt_dsl

Merge remote-tracking branch 'simelo/develop' into olemis_t32_rxt_dsl

7ab550e

ref #32 Merge remote-tracking branch 'olemis/olemis_t32_rxt_dsl' into…

d0481a1

… stdevAlDen_t32_rxt_dsl

Merge pull request #1 from denisacostaq/stdevAlDen_t32_rxt_dsl

83db570

Implement RXT DSL interfaces for scraper configuration

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[config] [dsl] fixes #32 - RXT scraper definition language#38

[config] [dsl] fixes #32 - RXT scraper definition language#38
olemis wants to merge 87 commits intosimelo:developfrom
olemis:olemis_t32_rxt_dsl

olemis commented Nov 21, 2018 •

edited by stdevAlDen

Loading

Uh oh!

olemis commented Nov 23, 2018 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

olemis commented Nov 21, 2018 • edited by stdevAlDen Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

olemis commented Nov 23, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

olemis commented Nov 21, 2018 •

edited by stdevAlDen

Loading

olemis commented Nov 23, 2018 •

edited

Loading