table parser fails on format value with comma

[kyleam]

@callistosp reported a case where nmrec failed to parse a table record, complaining that list1 is repeated in the table.

The failure is triggered by a comma being in the FORMAT value:

ctl <- nmrec::parse_ctl(c("$prob p", "$tab num format=1PE12.5"))
ctl$records[[2]]$parse()
ctl$records[[2]]$get_options()
#> [[1]]
#> $tab
#>
#> [[2]]
#> num
#>
#> [[3]]
#> format=1PE12.5

ctl <- nmrec::parse_ctl(c("$prob p", "$tab num format=,1PE12.5"))
ctl$records[[2]]$parse()
#> Error in `private$parse_fn()`:
#> ! list1 repeated in table
#> • $tab num format=,1PE12.5

So it's seeing that comma as marking the end of the format value.

Which is fairly reasonable given that NM-TRAN does usually take a comma as a field delimiter. For example, NM-TRAN generates an identical FCON for these two records (note the comma in the second record after the format value):

$TABLE  NUM IPRED NOPRINT ONEHEADER FILE=o.tab FORMAT=,1PE12.5 RANMETHOD=P

$TABLE  NUM IPRED NOPRINT ONEHEADER FILE=o.tab FORMAT=,1PE12.5,RANMETHOD=P

At the moment, FORMAT parsing is handled with the general machinery for parsing record options. However, this isn't likely to be a general option parser fix because NM-TRAN does not consistently treat a comma at the beginning of a value as part of the value (i.e. it still considers it a delimiter). For example, with

$TABLE  NUM IPRED NOPRINT ONEHEADER FILE=,o.tab FORMAT=,1PE12.5 RANMETHOD=P

NM-TRAN fails with

AN ERROR WAS FOUND ON LINE 47 AT THE APPROXIMATE POSITION NOTED:
 $TABLE  NUM IPRED NOPRINT ONEHEADER FILE=,o.tab FORMAT=,1PE12.5 RANMETHOD=P
                                         X
   19  A FILE NAME IS REQUIRED.

Aside from revisiting what NONMEM guides IV and VIII say on this, coming up with the fix will likely take a fair amount of NM-TRAN/FCON testing to find out what the behavior should be here (especially where the edges are) and to which options it should apply.

---

For anyone hitting into this, a workaround is to quote the FORMAT value.

$TABLE NUM IPRED NPDE CWRES NOPRINT ONEHEADER FILE=1.tab FORMAT=,1PE12.5

and

$TABLE NUM IPRED NPDE CWRES NOPRINT ONEHEADER FILE=1.tab FORMAT=',1PE12.5'

produce the same result, and nmrec can parse the latter.

ctl <- nmrec::parse_ctl(
  c(
    "$prob p",
    "$TABLE NUM IPRED NPDE CWRES NOPRINT ONEHEADER FILE=1.tab FORMAT=',1PE12.5'"
  )
)
ctl$records[[2]]$parse()
ctl$records[[2]]$get_options()
#> [[1]]
#> $TABLE
#>
#> [[2]]
#> NUM IPRED NPDE CWRES
#>
#> [[3]]
#> NOPRINT
#>
#> [[4]]
#> ONEHEADER
#>
#> [[5]]
#> FILE=1.tab
#>
#> [[6]]
#> FORMAT=',1PE12.5'