Skip to content

Few Date fields are populating incorrectly #752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
suryagits opened this issue Mar 10, 2025 · 3 comments
Open

Few Date fields are populating incorrectly #752

suryagits opened this issue Mar 10, 2025 · 3 comments
Labels
question Further information is requested

Comments

@suryagits
Copy link

suryagits commented Mar 10, 2025 •
edited
Loading

Hi @yruslan

Good morning.

Our Mainframe dataset Copybook (copybook layout/indentation is incorrect. Column names are intentionally changed to dummy) :

05 :DUMY:-RECORD. 00150002
10 :DUMY:-KEY. 00150102
15 :DUMY:-RECORD-ID PIC X(01). 00150203
15 :DUMY:-RECORD-SUB-ID PIC X(01). 00150303
88 :DUMY:-COLM-RECORD VALUE 'A'. 00150403
88 :DUMY:-ABCD-RECORD VALUE 'P'. 00150503
A49824 15 :DUMY:-COLM-ID PIC 9(11). 00150603
15 :DUMY:-APPL-SUFFIX PIC 9(02). 00150700
AUG018 10 :DUMY:-DATA PIC X(385). 00150901
10 :DUMY:-COLM-DATA REDEFINES :DUMY:-DATA. 00150903
15 :DUMY:-COLM-ACTION-CODE PIC X(01). 00151003
15 :DUMY:-COLM-TPC PIC X(02). 00151103
15 :DUMY:-COLM-CPC PIC X(03). 00152002
15 :DUMY:-COLM-ORG-CODE PIC 9(05). 00153002
15 :DUMY:-COLM-TYPEC-TYPE PIC X(02). 00154002
15 :DUMY:-COLM-CD-12 PIC X(01). 00155002
15 :DUMY:-COLM-CD-40 PIC X(04). 00156002
15 :DUMY:-COLM-CODE-48 PIC X(04). 00157002
15 :DUMY:-COLM-DN-FIRST-DLQ PIC S9(07) COMP-3.00158002
15 :DUMY:-COLM-CREDIT-LIM PIC S9(11) COMP-3.00159002
15 :DUMY:-COLM-MONEY-AVAILABLE PIC S9(11)V99 COMP-3.00160002
15 :DUMY:-COLM-NUM-DAYS-CREDIT-BAL PIC S9(03) COMP-3.00170002
15 :DUMY:-COLM-OVERDRAFT-PROTECTION PIC X(01). 00180002
15 :DUMY:-COLM-SAMPL-CHARGEOFF PIC X(02). 00190002
15 :DUMY:-COLM-CHARGEOFF-OPERATOR PIC X(06). 00200002
15 :DUMY:-COLM-DN-CHARGEOFF-COMPLT PIC S9(07) COMP-3.00210002
15 :DUMY:-COLM-SAMPL-CLOSED PIC X(02). 00220002
15 :DUMY:-COLM-DN-SAMPL-CLOSED-MNT PIC S9(07) COMP-3.00230002
15 :DUMY:-COLM-SAMPL-CREDIT-REVOKED PIC X(02). 00240002
15 :DUMY:-COLM-SAMPL-CURR-OVERLIMIT PIC X(02). 00250002
15 :DUMY:-COLM-WAIVE-FCHG-FLAG PIC X(01). 00260005
15 :DUMY:-COLM-SAMPL-CURR-PASTDUE PIC X(02). 00270002
15 :DUMY:-COLM-SAMPL-DLQ PIC X(02). 00280002
15 :DUMY:-COLM-SAMPL-SECURITY-FRD PIC X(02). 00290002
15 :DUMY:-COLM-SAMPL-FRD PIC X(01). 00300002
15 :DUMY:-COLM-DN-SECURITY-FRD-STAT PIC S9(07) COMP-3.00310002
15 :DUMY:-COLM-SAMPL-POT-CHARGEOFF PIC X(01). 00320002
15 :DUMY:-COLM-SAMPL-ASSISTANCE-PGM PIC X(02). 00330002
15 :DUMY:-COLM-DN-PROD-CHNG PIC S9(07) COMP-3.00340002
15 :DUMY:-COLM-BALANCE-CURR PIC S9(11)V99 COMP-3.00350002
15 :DUMY:-COLM-DN-LAST-CASH PIC S9(07) COMP-3.00360002
15 :DUMY:-COLM-DN-LAST-PURCHASE PIC S9(07) COMP-3.00370002
15 :DUMY:-COLM-DN-PAYMENT-DUE PIC S9(07) COMP-3.00380002
15 :DUMY:-COLM-TOTAL-PASTDUE PIC S9(11)V99 COMP-3.00390002
15 :DUMY:-COLM-DN-LAST-TRANS PIC S9(07) COMP-3.00400002
15 :DUMY:-COLM-MERCH-APR PIC S9(4)V9(7) COMP-3.00410004
15 :DUMY:-COLM-AMT-AUTH-OUT PIC S9(11)V99 COMP-3.00420002
15 :DUMY:-COLM-DN-LAST-REIS PIC S9(07) COMP-3.00430002
15 :DUMY:-COLM-MULTI-DEFG-NUM-IND PIC X(01). 00440002
A49824 15 :DUMY:-SAMPL-TIERED-AUTH PIC X(2). 00441000
A49824 15 :DUMY:-DATE-COLM-STAT-MAINT PIC S9(7) COMP-3. 00442000
A49824 15 :DUMY:-ACTUAL-DAYS-PASTDUE PIC S9(5) COMP-3. 00443000
A49824 15 :DUMY:-BANKRUPTCY-TYPE PIC 9(2). 00444002
REGZ01 15 :DUMY:-COLM-SAMPL-LITIGATION PIC X(02). 00444003
JULY13 15 :DUMY:-COLM-PRIN-BAL-AM PIC S9(11)V99 COMP-3.00444004
AUG018 15 :DUMY:-LATE-FEE-CN PIC 9(01) 00444005
AUG018 OCCURS 13 TIMES. 00444006
AUG018 15 :DUMY:-LAST-LATE-FEE-DN PIC S9(07) COMP-3. 00444007
AUG018 15 :DUMY:-NSF-FEE-CN PIC 9(01) 00444008
AUG018 OCCURS 13 TIMES. 00444009
AUG018 15 :DUMY:-DATE-LAST-NSF-PAYMENT PIC S9(07) COMP-3. 00444010
AUG018 15 :DUMY:-CURR-LOW-CR-LMT-AM PIC S9(11) COMP-3. 00444020
AUG018 15 :DUMY:-LTD-NET-FEE-ASES-AM PIC S9(11)V99 00444030
AUG018 COMP-3. 00444040
AUG018 15 :DUMY:-LTD-FRGN-TXN-FEE-AM PIC S9(11)V99 COMP-3.00444050
AUG018 15 :DUMY:-LTD-SPRS-RFND-AMF-AM PIC S9(11)V99 COMP-3.00444060
AUG018 15 :DUMY:-LTD-SPRS-RFND-BT-FEE-AM PIC S9(11)V99 COMP-3.00444070
AUG018 15 :DUMY:-LTD-SPRS-RFND-ACCK-FEE-AM 00444080
AUG018 PIC S9(11)V99 COMP-3.00444081
AUG018 15 :DUMY:-LTD-SPRS-RFND-CADV-FEE-AM 00444090
AUG018 PIC S9(11)V99 COMP-3.00444091
AUG018 15 :DUMY:-LTD-SPRS-RFND-RWD-FEE-AM PIC S9(11)V99 COMP-3.00444100
AUG018 15 :DUMY:-LTD-SPRS-RFND-DACC-FEE-AM 00444200
AUG018 PIC S9(11)V99 COMP-3.00444210
AUG018 15 :DUMY:-LTD-SPRS-RFND-MIN-FCS-AM PIC S9(11)V99 COMP-3.00444300
AUG018 15 :DUMY:-MAX-FEE-SPRS-THLD-PC PIC 9V99. 00444400
AUG018 15 :DUMY:-SPRS-FEE-DAY-CN PIC 9(03). 00444500
AUG018 15 :DUMY:-LTR-ID PIC X(06). 00444600
AUG018 15 :DUMY:-EXCL-OPT-SET-ID PIC X(05). 00444700
AUG018 15 :DUMY:-SPRS-FEE-THLD-IN PIC X(01). 00444800
AUG018 15 :DUMY:-SPRS-EXPIRY-DATE PIC S9(07) COMP-3. 00444900
AUG018 15 :DUMY:-SPRS-EXPIRY-CYCLE PIC S9(07) COMP-3. 00445000
SLC419 15 :DUMY:-LTD-SPRS-RFND-SLC-FEE-AM PIC S9(11)V99 COMP-3.00445100
MOB121 15 :DUMY:-SB-FLEX-PMT-CRLT-AM PIC S9(11) COMP-3. 00445200
MOB121 15 :DUMY:-SB-FLEX-PMT-AVAL-CRLT-AM PIC S9(11)V99 COMP-3.00445300
10 :DUMY:-ABCD-DATA REDEFINES :DUMY:-DATA. 00450002
15 :DUMY:-ABCD-KEY. 00460002
20 :DUMY:-ABCD-CUST-TYPE PIC 9(01). 00470002
20 :DUMY:-ABCD-CUST-ID PIC 9(09). 00480002
20 :DUMY:-ABCD-COLM-NUM PIC X(19). 00490002
15 :DUMY:-ABCD-ACTION-CODE PIC X(01). 00500004
15 :DUMY:-ABCD-COLM-REL-STAT PIC X(01). 00650002
15 :DUMY:-ABCD-DN-TRANSFER PIC S9(07) COMP-3.00651002
15 :DUMY:-ABCD-COLM-NUM-TRANS-FROM PIC X(19). 00660002
15 :DUMY:-ABCD-COLM-NUM-TRANS-TO PIC X(19). 00670002
15 :DUMY:-ABCD-SAMPL-DEFG-WATCH PIC X(02). 00680002
15 :DUMY:-ABCD-CHECKING1 PIC X(17). 00690002
15 :DUMY:-ABCD-SPENDING-AVAIL PIC S9(11)V99 COMP-3.00700002
15 :DUMY:-ABCD-SPENDING-LIMIT PIC S9(11)V99 COMP-3.00710002
15 :DUMY:-ABCD-CASH-AVAIL PIC S9(11)V99 COMP-3.00720002
15 :DUMY:-ABCD-CASH-LIMIT PIC S9(11)V99 COMP-3.00730002
102869 15 :DUMY:-ABCD-DEFG-REF-NB PIC 9(09). 00730003




## Question

For certain columns like DUMY:-COLM-DN-PAYMENT-DUE , DUMY:-COLM-DN-LAST-TRANS which contain date columns in Mainframe System , the year 2025 is being shown as 2015 however 2024 is coming up correctly.
Suppose Mainframe value = 2025062 then CoBrix value is showing as 2015062

Please note that we are doing Zipping of the file and then encrypting the file before sending it to Cobrix. We are then decrypting, unzipping and then using Cobrix for ASCII conversion.

We are not able to understand why only few columns especially these date columns are getting parsed incorrectly. Could you kindly provide some guidance please.

Thank you
@suryagits suryagits added the question Further information is requested label Mar 10, 2025
@yruslan
Copy link
Collaborator

yruslan commented Mar 12, 2025

Hi @suryagits , this is very strange. Could you please add this option:

.option("debug", "true")

and send an example of correct values and incorrect values for the column and corresponding _debug column. Form example:

  • Correct DUMY:-COLM-DN-PAYMENT-DUE = 2025062 , DUMY:-COLM-DN-PAYMENT-DUE_debug = 2025062C.
    ...

@suryagits
Copy link
Author

Hi @yruslan ,

One query, Does Cobrix support ingesting zipped (.gz extension to be precise) files directly or do we need to de-compress first and then pass the EBCDIC source file ?

@yruslan
Copy link
Collaborator

yruslan commented Mar 19, 2025

Unpacking .gz is not supported at the moment, you need to decompress and point Cobrix to uncompressed binary files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yruslan @suryagits