-
Notifications
You must be signed in to change notification settings - Fork 4
Open
Description
Email from user indicated following error when using ddf.gof()
Mark-recapture component:
Capture History 10
[0,50] (50,100] (100,200] (200,300] Total
Observed 131 118 160 81 490
Expected 132 109 178 70 489
Chisquare 0 1 2 2 4
Capture History 01
[0,50] (50,100] (100,200] (200,300] Total
Observed 136 112 179 54 481
Expected 134 112 178 58 482
Chisquare 0 0 0 0 0
Capture History 11
[0,50] (50,100] (100,200] (200,300] Total
Observed 126 78 128 21 353
Expected 127 87 112 28 353
Chisquare 0 1 2 2 5
Total chi-square = 17.214 P = NaN with -32 degrees of freedom
Warning message: In pchisq(chisq.1 + chisq.2, 3 * nc - length(model$par) - 1) : NaNs produced
As shown, data were binned with 4 intervals, fitted mrmodel
had 29 estimated parameters:
> summary(MRDS.6ii7)
Summary for io.fi object
Number of observations : 1324
Number seen by primary : 843
Number seen by secondary : 834
Number seen by both : 353
AIC : 2705.31
Conditional detection function parameters:
estimate se
(Intercept) -1.0796625270 0.277260949
obsnameBP 1.1046744569 0.361828611
obsnameBW 0.5433139316 0.394464395
obsnameCF 1.1961084816 0.382167121
obsnameCS 1.1934733554 0.396189048
obsnameDH 0.7954464197 0.343565299
obsnameEC 1.1975425396 0.395574213
obsnameMG 0.5859840295 0.362524121
obsnameMH 2.1259248539 0.409690893
obsnamePL 1.5807107511 0.425884100
obsnamePOB 0.5848155343 0.401523354
obsnameRL 2.5931041668 0.495230694
obsnameRT 1.7839252838 0.421544945
obsnameSD 1.4995302978 0.439472284
distance -0.0036643313 0.002600023
obsnameBP:distance -0.0099988195 0.003096288
obsnameBW:distance 0.0078620272 0.003474921
obsnameCF:distance -0.0067997640 0.003229027
obsnameCS:distance -0.0007569504 0.003252560
obsnameDH:distance -0.0017379043 0.002718869
obsnameEC:distance -0.0030372291 0.003149064
obsnameMG:distance 0.0046375639 0.003172420
obsnameMH:distance -0.0114362985 0.003412340
obsnamePL:distance -0.0102762333 0.003554344
obsnamePOB:distance 0.0068440041 0.003242920
obsnameRL:distance -0.0089213506 0.003706149
obsnameRT:distance -0.0079749867 0.003662896
obsnameSD:distance 0.0011171377 0.004304027
distance:size 0.0008886445 0.000333803
Solution should be to have ddf.gof
test the number of parameters against the number of bins and not attempt a test if there are insufficient remaining degrees of freedom.
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
Select code repository
Activity
lenthomas commentedon Nov 25, 2024
My first suggestion was: "I suggest to have a look at the Distance for Windows message in this case and reproduce the same kind of output."
After discussion, it seems clear this is a poor suggestion since DistWin calls
mrds
to do its calculations so this won't produce any useful insights - sorry, my bad!LHMarshall commentedon Jan 20, 2025
This seems to be an issue only related to mr models... I will try now simulate some double observer data.
LHMarshall commentedon Jan 21, 2025
Following on from above code to generate a second set of observations
LHMarshall commentedon Jan 22, 2025
This needs more investigation... why does it think there is 1 degree of freedom for the MR model?
LHMarshall commentedon Jan 22, 2025
@lenthomas @erex
Do either of you know where these calculations for degrees of freedom come from?
Also should there be a distance sampling component output for the io.fi model?
For the io model there is 1 degree of freedom for the MR model as the calculation is:
For an io.fi model the calculation for degrees of freedom is 3*3-3-1 = 5
(there was an error when trying to fit with factors with only 2 levels so had to remove sex & col)
Also isn't it strange that there is still a Chisquare table output for the distance sampling component?
erex commentedon Jan 27, 2025
@LHMarshall Best description of GOF tests is on p.5 of David's 2006 Biometrics paper
borchers et al(2006).pdf
David might be the source of answers for MRDS GOF questions.
For the io configuration that threw the error for the user to generate this issue in 2023, I think the problem lies with the overly ambitious
return
statement that endsgof.io.R
. The return statement carries out a call topchisq
without checking whether the degrees of freedom argument is legitimate.mrds/R/gof.io.R
Line 88 in 50d9c0c
Breaking down that
return
and checking that degrees of freedom is positive and gracefully reporting what happens if inadmissible, seems the simplest solution.Issue #96 gof dfs for MR models
LHMarshall commentedon Jan 27, 2025
@erex yes, I'd already fixed that (sorry hadn't yet pushed as I was still constructing a test for the test suite) but was just checking the df calculations as I didn't uderstand them. I think this looks like an issue for other MR models also but my test dataset was failing for the io.fi model as it had factors with 2 covariate levels. I also noticed that the io.fi model gave 2 chi quare outputs the first still referring to the 'Distance Sampling Component' this seemed strange as there is no ds component for fi models I thought?
@lenthomas should the 'distance sampling component' in the chi square output be removed for fi models?
LHMarshall commentedon Jan 27, 2025
In print.ddf.gof line 33-35 it prints out Distance sampling component heading if chi2 is not NULL
Firstly, I can't currently see a situation where this would ever be NULL as there always seems to be a chi2 component. If there are insufficient degrees of freedom then the $df and $p components are NA. Hmm ok it is NULL if it is a gof test for a ds model... ok so I guess in that case there is no header.
Secondly, the code following this does not print out the chi2 component but it prints out the chi1 table: lines 37:44
lenthomas commentedon Jan 28, 2025
Although the distance sampling component is not part of the likelihood in fi models, there is no reason why we cannot check the fit of a component we didn't fit to. For example, in mark recapture models, it's common to do goodness of fit tests to look for a trap-shyness effect even if that wasn't part of the model fit -- if the GOF test fails then one goes ahead and fits a trap shyness model. Here I would think if the distance sampling component means the fi GOF fails then that's a spur to fit a pi model.
However, I suggest double checking with David Borchers for his thoughts.
fix other MR gof total chi sq df and p values
lenthomas commentedon May 7, 2025
Just to clarify, the thing I suggested checking with David is how to calculate the degrees of freedom for the Distance sampling component of the test when it's a full independence model. I can see how the test statistic is constructed, but not easily how to calculate the degrees of freedom for the test.
LHMarshall commentedon Jul 2, 2025
Add the table to the help, suggested location ddf.gof
The table below is currently being checked, an incinsistency is highlighted
Fix rem model gof degrees of freedom