Skip to content

Fix testing #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Jun 13, 2025
Merged

Fix testing #29

merged 28 commits into from
Jun 13, 2025

Conversation

alvarolivie
Copy link
Contributor

@alvarolivie alvarolivie commented Apr 24, 2025

This branch proposes a new way to handle testing that includes schema and schematron validation per type. Testing showed some errors that needed to be fixed. It also adds a new context for Chorus Pro (depends on invopop/gobl#544).

Testing Framework

Originally, this repo only handled schema validation for base CII and did not take into consideration other schemas like FacturX or XRechnung. The repo also ignored schematrons completely. Schemas validate the syntactic aspects while schematrons look at semantics.

The new testing structure in tools/ provides comprehensive validation for different CII implementations:

  • Schema validation for EN16931, FacturX, XRechnung, and PEPPOL formats
  • Schematron validation using compiled XSLT stylesheets for business rule validation
  • xslt3 integration for fast XML transformations without Java dependencies

See the tools README for details on available schemas, schematrons, and how to use the validation tools.

Main Changes

Running the new validation tests revealed several issues that have been fixed as well as some changes to simplify the implementation.

  • agreement.go: Updated the TaxRepresentative to include only the required info and moved email -> URIUniversalComunication creation
  • allowance_charge.go: Base must always be set with percent, this is enforced by gobl so we should remove the total sum as base only adds value with percent. Tax category percent must be set to 0 if null, even for exempts.
  • cii.go: Added zugferd specific guideline and ChorusPro context.
  • cmd/gobl.cii/convert.go: Added a flag to select context. Default is base CII.
  • examples_test.go: Added tests for all contexts, can be called with the --validate flag to test against schema and schematron.
  • invoice.go: Modified GuidelineID is set to adapt to ChorusPro.
  • lines.go: Percent now set inside makeTaxCategory().
  • party.go: Added LegalOrganization which is required by ChorusPro. Added omitempty tags to ensure no blank fields are created. Email handled here now.
  • settlement.go: Added PartialPaymentPercent to terms for parsing. Added notes to description if provided in each inv.Payment.Terms.DueDates. DuePayableAmount needs to be updated with due to take into account payments. The same as before, RateApplicablePercent must be set to 0, not null. newPayee updated due to rules.
    -settlement_parse.go: We now parse the percent which may be present instead of only the amount.

Most fixes relate to schematron validation rules that enforce semantic correctness beyond basic XML schema validation.

@samlown samlown marked this pull request as draft April 28, 2025 15:46
@codecov-commenter
Copy link

codecov-commenter commented May 22, 2025

Codecov Report

Attention: Patch coverage is 74.65753% with 37 lines in your changes missing coverage. Please review.

Project coverage is 72.78%. Comparing base (b2e1798) to head (39fb418).

Files with missing lines Patch % Lines
cmd/gobl.cii/convert.go 9.52% 19 Missing ⚠️
settlement.go 80.00% 6 Missing and 2 partials ⚠️
settlement_parse.go 44.44% 2 Missing and 3 partials ⚠️
allowance_charge.go 78.94% 2 Missing and 2 partials ⚠️
party.go 97.36% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #29      +/-   ##
==========================================
+ Coverage   71.88%   72.78%   +0.89%     
==========================================
  Files          23       23              
  Lines        1615     1683      +68     
==========================================
+ Hits         1161     1225      +64     
- Misses        371      376       +5     
+ Partials       83       82       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@alvarolivie alvarolivie marked this pull request as ready for review June 11, 2025 17:32
@alvarolivie alvarolivie requested review from samlown and cavalle June 11, 2025 17:32
Copy link

@cavalle cavalle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EPIC PR, @alvarolivie!! 💪 Of course, I can't evaluate the correctness of the logic (I'd just suggest thorough manual testing), but the code looks pretty good. I didn't spot any obvious issue.

I'm very glad we're now using the specific validations for each CII variant the library supports. And the conditional logic because of the variants, which was one of my concerns, is minimal so far, which supports the idea of keeping everything together in the same package.

@alvarolivie alvarolivie requested review from Copilot June 12, 2025 13:20
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adjusts the testing and validation workflows and adapts invoice conversion to support multiple contexts (e.g. Chorus Pro, Factur-X, XRechnung, and PEPPOL). Key changes include modifying how payment term descriptions are concatenated, adding context-based logic in invoice conversion, and updating allowance/charge calculations with adjusted function signatures and conditions.

Reviewed Changes

Copilot reviewed 165 out of 165 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
settlement_parse.go Changed description assignment to accumulate text
settlement.go Updated payment term handling and tax treatments
party.go Enhanced party creation for Chorus Pro support
invoice.go Added context flag and guideline extraction
examples_test.go Updated tests to use new context-specific JSON file paths
go.mod Updated dependencies
cmd/gobl.cii/convert.go Added flag for selecting conversion context
allowance_charge.go Revised allowance/charge creation logic and parameters
Comments suppressed due to low confidence (2)

allowance_charge.go:118

  • This condition assigns a percentage only when Base is non-nil; please confirm that this assumption is intended and document the requirement that a Base must always be provided if a Percent is available.
if c.Percent != nil && c.Base != nil {

agreement.go:41

  • [nitpick] The change from 'N/A' to 'NA' may be ambiguous; consider adding a comment or documentation to clarify that 'NA' signifies a default placeholder value.
defaultBuyerReference = "NA"

Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

This comment was marked as outdated.

@samlown samlown requested a review from Copilot June 12, 2025 16:37
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances the testing framework to run schema and schematron validations across multiple CII contexts (EN16931, FacturX, XRechnung, Peppol, ChorusPro) and fixes related data-handling issues uncovered during validation.

  • Added Chorus Pro context support and dynamic guideline extraction
  • Reworked settlement terms parsing to include both amount and percent fields
  • Overhauled examples_test.go to drive per-format tests with --validate flag and external xslt3 integration

Reviewed Changes

Copilot reviewed 165 out of 165 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
settlement_parse.go Changed how term descriptions are concatenated and partial payments are parsed
settlement.go Added Amount/Percent fields to Terms and updated summary/due‐date logic
party.go Introduced LegalOrganization for ChorusPro, moved email handling to newURIUniversalCommunication
examples_test.go Refactored tests for multi‐format validation, added schema/schematron checks
invoice.go Added extractGuidelineID for ChorusPro and imported slices for addon detection
cmd/gobl.cii/convert.go Added --context flag and mapped formats to correct Context values
allowance_charge.go Removed redundant base sum parameter, adjusted percent/base pairing in line charges
Comments suppressed due to low confidence (3)

examples_test.go:286

  • [nitpick] Function isrootFolder uses inconsistent capitalization. Consider renaming to isRootFolder to follow Go naming conventions and improve readability.
for !isrootFolder(cwd) {

examples_test.go:259

  • [nitpick] The local variable schemaPath shadows the package-level schemaPath constant, which can be confusing. Consider renaming one to avoid shadowing and improve clarity.
schemaPath := filepath.Join(rootFolder(), schemaPath, defaultFormat, schemaFile)

settlement.go:118

  • Guarding creation of PaymentTerms on non-empty descriptions changes prior behavior, which always produced a default term entry. If an empty term is still required, consider removing this guard or explicitly handling the empty case.
if description != "" {

@alvarolivie alvarolivie merged commit 5c4530f into main Jun 13, 2025
5 checks passed
@alvarolivie alvarolivie deleted the fix-testing branch June 13, 2025 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants