importing functional fails with OSError on MacOS arm64

[shnaqvi]

As I tried using scico and imported functional, I get the error involving bm4d that its mach-o file refers to an incompatible architecture x86_64 whereas arm64 was needed. I tried uninstalling bm4d and reinstalling with architecture specification as suggested here, but that didn't solve the problem either. Is this related to bm4d not having updated their binaries for M1 chipset?

pip uninstall bm4d 
ARCHFLAGS="-arch arm64" pip install bm4d --compile --no-cache-dir

P.S. I am on MacOS M1 running Ventura and on Python 3.10.1.

Below is the error stack upon running the code:

from scico import functional

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Cell In[310], line 1
----> 1 from scico import functional
      3 # im_s = np.zeros((2748, 3840)).astype(float)
      4 # im_jx = jax.device_put(im_s)  # convert to jax type, push
      5 # C = linop.Convolve(h=psf2, input_shape=im_jx.shape)
   (...)
     36 
     37 # plt.imshow(x.reshape(im.shape))

File [~/.pyenv/versions/3.10.1/lib/python3.10/site-packages/scico/functional/__init__.py:25](https://file+.vscode-resource.vscode-cdn.net/Users/salman_naqvi/Documents/Project-Display/t288_display_incubation/playground/~/.pyenv/versions/3.10.1/lib/python3.10/site-packages/scico/functional/__init__.py:25)
     14 from ._norm import (
     15     HuberNorm,
     16     L0Norm,
   (...)
     22     L1MinusL2Norm,
     23 )
     24 from ._indicator import NonNegativeIndicator, L2BallIndicator
---> 25 from ._denoiser import BM3D, BM4D, DnCNN
     26 from ._dist import SetDistance, SquaredSetDistance
     29 __all__ = [
     30     "Functional",
     31     "ScaledFunctional",
   (...)
...
--> 374     self._handle = _dlopen(self._name, mode)
    375 else:
    376     self._handle = handle

OSError: dlopen(/Users/salman_naqvi/.pyenv/versions/3.10.1/lib/python3.10/site-packages/bm4d/libbm4d_mac.so, 0x0006): tried: '/Users/salman_naqvi/.pyenv/versions/3.10.1/lib/python3.10/site-packages/bm4d/libbm4d_mac.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/salman_naqvi/.pyenv/versions/3.10.1/lib/python3.10/site-packages/bm4d/libbm4d_mac.so' (no such file), '/Users/salman_naqvi/.pyenv/versions/3.10.1/lib/python3.10/site-packages/bm4d/libbm4d_mac.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64'))

[bwohlberg]

If I recall correctly, bm3d and bm4d don't work with certain versions of OSX, and possibly also not on the new M1 chipset. If you want to use scico, I recommend just uninstalling bm3d and bm4d, which are optional dependencies for scico. If you really need to use bm4d and don't have access to a Linux host, then your best option would be to contact the author of bm4d for assistance: contact details can be found at https://pypi.org/project/bm4d/ .

[shnaqvi]

Thanks for quick reply @bwohlberg. And regarding MacOS, is there also no GPU acceleration on macOS either?

[bwohlberg]

Not to the best of my knowledge. JAX requires CUDA, and therefore an NVIDIA GPU, while rules out all recent Apple hardware. In principle some older hardware should work, but I don't know how far back you'd have to go, or how difficult it would be to install JAX.

[shnaqvi]

Thanks @bwohlberg. So as a macOS user on M1 GPU, do you see any other advantage of using Scico versus other packages like PyProximal/PyLops or ProxImaL? Is JIT unique to Scico compared to these other packages?

[shnaqvi]

Btw, for posterity, I was able to import scico fine after uninstalling bm4d. Not sure if I'll be needing it soon, in which case, I'll be in trouble. Thanks for unblocking me.

[bwohlberg]

The JIT and autograd capability in SCICO is due to it being based on JAX, which is unique to SCICO within the set of packages you mention. The only other package I'm aware of that is based on JAX and that supports optimization problems is jaxopt.

[bwohlberg]

Closing as resolved. Feel free to re-open if you have further questions.

[shnaqvi]

@bwohlberg , I ran into this bm4d issue again when setting up scico from scratch on a new Mac using pip, because every time you even try upgrading the package with pip, bm4d gets installed as a dependency.

Do you agree that it would be a service to future users to either

1. Have bm3d / bm4d be removed as a dependency when installing from pip? or
2. Have this explanation be added in a list of known / won't-fix issues somewhere on [README](https://github.com/lanl/scico/blob/main/README.rst)?

Thanks! 🙂

P.S. I'm not getting the option to Reopen this issue, possibly because of lack of privileges. Don't know if this is intentional.

[bwohlberg]

Sorry, I assumed you'd be able to reopen. I'll take care of it.

I see your point. I'm not sure if pypi/pip allows for OS-specific dependencies, but I will investigate. Adding a note to the docs is not difficult, and should done.