Blobs function error

[OneSkyknight]

General information:

• emcee version: 3.1.1
• platform: Centos7
• installation method (pip/conda/source/other?): Conda-forge

Problem description:

Expected behavior:

Blobs function

Actual behavior:

After running normally for some time, an error is reported:

378012 Traceback (most recent call last):
378013 File "/home/yuebb/.local/lib/python3.9/site-packages/emcee/ensemble.py", line 492, in compute_log_prob
378014 log_prob = np.array([float(l[0]) for l in results])
378015 File "/home/yuebb/.local/lib/python3.9/site-packages/emcee/ensemble.py", line 492, in
378016 log_prob = np.array([float(l[0]) for l in results])
378017 TypeError: 'float' object is not subscriptable
378018
378019 During handling of the above exception, another exception occurred:
378020
378021 Traceback (most recent call last):
378022 File "/public/data/yuebb/Neutrino/SkyWalker/bin/JUNOMCMC/JUNO_C13_Pool.py", line 146, in
378023 sampler.run_mcmc(starting_guesses, nsteps)
378024 File "/home/yuebb/.local/lib/python3.9/site-packages/emcee/ensemble.py", line 443, in run_mcmc
378025 for results in self.sample(initial_state, iterations=nsteps, **kwargs):
378026 File "/home/yuebb/.local/lib/python3.9/site-packages/emcee/ensemble.py", line 402, in sample
378027 state, accepted = move.propose(model, state)
378028 File "/home/yuebb/.local/lib/python3.9/site-packages/emcee/moves/red_blue.py", line 93, in propose
378029 new_log_probs, new_blobs = model.compute_log_prob_fn(q)
378030 File "/home/yuebb/.local/lib/python3.9/site-packages/emcee/ensemble.py", line 495, in compute_log_prob
378031 log_prob = np.array([float(l) for l in results])
378032 File "/home/yuebb/.local/lib/python3.9/site-packages/emcee/ensemble.py", line 495, in
378033 log_prob = np.array([float(l) for l in results])
378034 TypeError: float() argument must be a string or a number, not 'tuple'

What have you tried so far?:

Minimal example:

import emcee

def log_posterior(theta):
    deltaMSq21, sinSqTheta12, B8Flux, B8NuShift, JUNO_B12, JUNO_Li8, JUNO_C10, JUNO_Be11, JUNO_He6, JUNO_U238, JUNO_Th232, JUNO_ReactorES, JUNO_CCBG, C13CCScale, C13NCScale, ESScale, sinSqThetaW, JUNODetectorESNCEff, JUNODetectorESCCEff = theta
    
    model.setParValue("deltaMSq21", deltaMSq21*1e-5)
    model.setParValue("sinSqTheta12", sinSqTheta12)
    model.setParValue("B8Flux", B8Flux)
    model.setParValue("B8NuShift", B8NuShift)
    model.setParValue("JUNO_B12", JUNO_B12)
    model.setParValue("JUNO_Li8", JUNO_Li8)
    model.setParValue("JUNO_C10", JUNO_C10)
    model.setParValue("JUNO_Be11", JUNO_Be11)
    model.setParValue("JUNO_He6", JUNO_He6)
    model.setParValue("JUNO_U238", JUNO_U238)
    model.setParValue("JUNO_Th232", JUNO_Th232)
    model.setParValue("JUNO_ReactorES", JUNO_ReactorES)

    model.setParValue("JUNO_CCBG", JUNO_CCBG)
    model.setParValue("C13CCScale", C13CCScale)
    model.setParValue("C13NCScale", C13NCScale)
    model.setParValue("ESScale", ESScale)
    model.setParValue("sinSqThetaW", sinSqThetaW)
    model.setParValue("JUNODetectorESNCEff", JUNODetectorESNCEff)
    model.setParValue("JUNODetectorESCCEff", JUNODetectorESCCEff)

    if sinSqTheta12 < 0 or sinSqTheta12 > 1 or deltaMSq21 < 0 or B8Flux < 0 or JUNO_B12 < 0 or JUNO_Li8 < 0 or JUNO_C10 < 0 or JUNO_Be11 < 0 or JUNO_He6 < 0 or JUNO_U238 < 0 or JUNO_Th232 < 0 or JUNO_ReactorES < 0:
        return -np.inf  

    return -0.5*model.getChiSquare(), B8Flux*model.getProb(0), B8Flux*model.getProb(1), B8Flux*model.getProb(2), B8Flux*(model.getProb(1)+model.getProb(2))

#### ignoring some useless code

dtype = [("PhiE", float), ("PhiM", float), ("PhiT", float), ("PhiMT", float)]

filename = data+"_Backend.h5"
#np.save(data+"_original_data.npy", sampler.chain)  # emcee_trace)
backend = emcee.backends.HDFBackend(filename)
backend.reset(nwalkers, ndim)

# with Pool() as pool:
sampler = emcee.EnsembleSampler(nwalkers, ndim, log_posterior,  # pool=pool,
                                # sampler = emcee.EnsembleSampler(nwalkers, ndim, log_posterior,
                                #     # )#,
                                #     # moves=[(emcee.moves.WalkMove(), 1),]) # (emcee.moves.DESnookerMove(), 0.2),],)
                                moves=[(emcee.moves.DEMove(), 1), ],
                                blobs_dtype=dtype,
                                backend=backend,
                                )

sampler.run_mcmc(starting_guesses, nsteps)


# sample code goes here...

[andyfaff]

return -0.5*model.getChiSquare(), B8Flux*model.getProb(0), B8Flux*model.getProb(1), B8Flux*model.getProb(2), B8Flux*(model.getProb(1)+model.getProb(2))

This is an issue with your log-posterior. The line above should be returning a single value, not a tuple.

[OneSkyknight]

return -0.5*model.getChiSquare(), B8Flux*model.getProb(0), B8Flux*model.getProb(1), B8Flux*model.getProb(2), B8Flux*(model.getProb(1)+model.getProb(2))

This is an issue with your log-posterior. The line above should be returning a single value, not a tuple.

Thx.
But I think it is not an issue.
Because I am using Blobs function, which can return multi values from log-posterior.
Here is an example from emcee doc. (https://emcee.readthedocs.io/en/latest/user/blobs/#)

<'''
import emcee
import numpy as np

def log_prior(params):
return -0.5 * np.sum(params**2)

def log_like(params):
return -0.5 * np.sum((params / 0.1)**2)

def log_prob(params):
lp = log_prior(params)
if not np.isfinite(lp):
return -np.inf, -np.inf
ll = log_like(params)
if not np.isfinite(ll):
return lp, -np.inf
return lp + ll, lp

coords = np.random.randn(32, 3)
nwalkers, ndim = coords.shape
sampler = emcee.EnsembleSampler(nwalkers, ndim, log_prob)
sampler.run_mcmc(coords, 100)

log_prior_samps = sampler.get_blobs()
flat_log_prior_samps = sampler.get_blobs(flat=True)

print(log_prior_samps.shape) # (100, 32)
print(flat_log_prior_samps.shape) # (3200,)

'''

In another hand, my program ran for a long time before it went wrong.

[dfm]

The problem is with the return -np.inf line. That line needs to return the same number of blobs. I think it would work fine to just return 4 Nones, but it might be better to return values with the right type. Either way, they'll never get saved in the chain, but they are expected to be there.

[OneSkyknight]

The problem is with the return -np.inf line. That line needs to return the same number of blobs. I think it would work fine to just return 4 Nones, but it might be better to return values with the right type. Either way, they'll never get saved in the chain, but they are expected to be there.

Thank you for your meticulous debugging. This is the cause of the error, and now the problem is resolved.
👍🤝