Modify kbc group nv pdf importer to support

[Nirus2000]

https://forum.portfolio-performance.info/t/pdf-import-from-kbc-bank-nv/22455/10
https://forum.portfolio-performance.info/t/pdf-import-from-kbc-bank-nv/22455/11

[Nirus2000]

Hey @buchen
I see the problem. As soon as we have two booking currencies, the test cases fail.
We once made this change so that all currencies that we identify as booking currencies are checked and accepted and, if missing booking currencies are recognized, an error is thrown.
It should also be recognized that if I insert an incorrect currency, e.g. “ZAR” here, but there is no transaction, this should also be output with a message.

Single currency:

new AssertImportActions().check(results, “EUR”);

Multiple currency:

new AssertImportActions().check(results, “EUR”, “USD”); <-- not working with forex

---

Thats what i want

new AssertImportActions().check(results, “EUR”); <-- for single currency transaction
new AssertImportActions().check(results, hasCurrencyCodes("EUR", "USD")); for multiple currency transactions (forex)

also, if i write ...
new AssertImportActions().check(results, hasCurrencyCodes("EUR", "USD", "ZAR")); in an example
The error is... Transaction with currency code ZAR not found.

with.... ExtractorMatchers.java like

@SafeVarargs
public static Matcher<Transaction> hasCurrencyCodes(String... currencyCodes) {
    return new TypeSafeDiagnosingMatcher<Transaction>() {
        @Override
        protected boolean matchesSafely(Transaction transaction, Description mismatchDescription) {
            String currencyCode = transaction.getMonetaryAmount().getCurrencyCode();
            boolean codeFound = false;

            for (String code : currencyCodes) {
                if (code.equals(currencyCode)) {
                    codeFound = true;
                    break;
                }
            }

            if (!codeFound) {
                mismatchDescription.appendText("currencyCode was ").appendText(currencyCode);
                return false;
            }

            // If the currency code is found, we can ensure there's a corresponding transaction
            boolean hasMatchingTransaction = results.stream()
                .filter(item -> item instanceof TransactionItem)
                .anyMatch(item -> {
                    Transaction trans = ((TransactionItem) item).getSubject();
                    return trans.getMonetaryAmount().getCurrencyCode().equals(currencyCode);
                });

            if (!hasMatchingTransaction) {
                mismatchDescription.appendText("but no matching transaction found for currency code: ").appendText(currencyCode);
                return false;
            }

            return true;
        }

        @Override
        public void describeTo(Description description) {
            description.appendText("a Transaction with one of currency codes ").appendText(String.join(", ", currencyCodes));
        }
    };
}

[buchen]

Hey @Nirus2000,

I am not sure what to look at.

The error message from the log

Failures:
KBCGroupNVPDFExtractorTest.testRekeninguittreksel03:268 Imported gross value (USD 60,665.50) and calculated gross value (USD 69,725.55) do not match
Jun 12, 2023, 11:55 AM Sell USD 69,606.12 ISHARES PLC CORE MSC E.M.IM UC Rekeninguittreksel03.txt

is correct, isn't it? The gross value does not match the calculated gross value so that transaction is not consistent.

69,606.12 + 72.80 (taxes) + 46.63 (fees) is a gross value of 69.725,55 which does not fit to the configured gross value of $60665.50. That is off by some 9000.

About

new AssertImportActions().check(results, “EUR”, “USD”);

where do I find the failure?

[Nirus2000]

HeyHo @buchen
The "Debug" output is in the console.

The problem is that it is displayed via:
new AssertImportActions().check(results, “EUR”, “USD”);
thinks the conversion should be in USD, but it is in EUR.
It simply calculates incorrectly, as I have one transaction that goes to a EUR account and one to a USD account.
The correct results are stored in the test case testRekeninguittreksel03().

[buchen]

The correct results are stored in the test case testRekeninguittreksel03().

Looking at the debug output of this test case "03", the transaction is not created correctly.
As you can see, the calculated gross value is 69.725,55 USD.
But that does not fit to the gross value in the unit, which is 60.665,50 USD.

There is no calculation (or currency conversion) at this point in time, because all these values are stored with the transaction directly.

DEBUG: transaction: 12.06.2023, 11:55 SELL              USD 69.606,12 ISHARES PLC CORE MSC E.M.IM UC
DEBUG: grossValueUnit: Optional[GROSS_VALUE       USD 60.665,50 EUR 65.046,30 0.932651]
DEBUG: grossValueUnit.get(): GROSS_VALUE       USD 60.665,50 EUR 65.046,30 0.932651
DEBUG: grossValueUnit.get().getForex(): EUR 65.046,30
DEBUG: unitValue: USD 60.665,50
DEBUG: calculatedValue: USD 69.725,55
DEBUG: unitValue.equals(calculatedValue): false

[buchen]

I think base and term currency code are flipped in the importer (below the changed version).
After applying the change, the values differ by 2 cents (did not check where that comes from, I guess from converting fees and taxes to USD).

DEBUG: unitValue: USD 69.743,45
DEBUG: calculatedValue: USD 69.743,43

                        .section("fxGross", "baseCurrency", "exchangeRate", "termCurrency").optional() //
                        .match("^.* aan .* (?<fxGross>[\\.,\\d]+) (?<fxCurrency>[\\w]{3})$") //
                        .match("^[\\.,\\d]+ (?<baseCurrency>[\\w]{3}) = (?<exchangeRate>[\\.,\\d]+) (?<termCurrency>[\\w]{3})$") //
                        .assign((t, v) -> {
                            ExtrExchangeRate rate = asExchangeRate(v);
                            type.getCurrentContext().putType(rate);

                            Money fxGross = Money.of(asCurrencyCode(v.get("fxCurrency")), asAmount(v.get("fxGross")));
                            Money gross = rate.convert(t.getAccountTransaction().getCurrencyCode(), fxGross);
                            
                            checkAndSetGrossUnit(gross, fxGross, t, type.getCurrentContext());
                        })

[Nirus2000]

The correct results are stored in the test case testRekeninguittreksel03().

Looking at the debug output of this test case "03", the transaction is not created correctly. As you can see, the calculated gross value is 69.725,55 USD. But that does not fit to the gross value in the unit, which is 60.665,50 USD.

There is no calculation (or currency conversion) at this point in time, because all these values are stored with the transaction directly.

DEBUG: transaction: 12.06.2023, 11:55 SELL              USD 69.606,12 ISHARES PLC CORE MSC E.M.IM UC
DEBUG: grossValueUnit: Optional[GROSS_VALUE       USD 60.665,50 EUR 65.046,30 0.932651]
DEBUG: grossValueUnit.get(): GROSS_VALUE       USD 60.665,50 EUR 65.046,30 0.932651
DEBUG: grossValueUnit.get().getForex(): EUR 65.046,30
DEBUG: unitValue: USD 60.665,50
DEBUG: calculatedValue: USD 69.725,55
DEBUG: unitValue.equals(calculatedValue): false

Thx.. done 👍🏻