Dev (#359)

* install current version of suite2p

* moved everything to environment.yml

* added lockfile

* added travis and test that imports work

* added pytest, requirements_dev, and travis config file

* moved to a duplicated environment.yml to simplify usage

* fixed travis

* rebuilt docs

* Added suite2p import in extract.py to fix suite2p not definted error in line 200.

* Add test_pipeline.py which contains several common use cases and fixture for setup/teardown.Add testcases that compare and check F, Fneu, iscell, and spks.

* Remove print statements from test_pipeline and use np.array_equal to avoid edge cases.

* Replaced numpy comparisons with filecmp. Now comparing stat.npy and data.bins in tests.

* updated installation instructions in docs

* changed suite2p import to relative for self-consistency and added numba caching for performance

* deleting docs

* deleted build docs

* added readthedocs config file

* test conda env

* added rtd badge

* try using general environment

* readthedocs not detecting the mixed dependency sources

* test whether rtd workds without pytest and suite2p in environment.yml

* added docs env

* removed mkl and pyqt from doc deps

* removed conda from rtd build

* removed all non-sphinx requirements from rtd build

* added pyqt5 to rtd reqs

* added pyqtgraph to rtd reqs

* integrated build pathways

* fixed travis

* fixing rtd

* fixing classifiers

* change build method with docs

* moved docs/requirements to setup.py

* added gui deps to extras_require

* removed unused mkl_fft imports

* update setuptools in travis for p37

* get more debug info for installer version

* removed pyc files to solve setuptools marchall bug

* more-specific python version to avoid pyc bug

* set zip_safe to try and resolve marshall bug

* remove pyc

* update pycparser

* trying zip_safe True

* changed scanimage version

* remove unnecessary solutions

* changed mkl_fft importerror to warning to support rtd docs

* fixed error type

* fixed error type

* stopped raising warnings

* have rtd get pyqt5

* added notes on necessary dependencies, cleaned up warnings.

* Clean install: travis & rtd compatibility, conda and pypi integration (#343)

* use bash instead of executable

* remove windows, because not supported

* new conda setup setps

* try again

* sudo everywhere

* sudo everywhere

* nosudo

* debug path

* switched to environment variable

* added readthedocs config file

* test conda env

* added rtd badge

* try using general environment

* readthedocs not detecting the mixed dependency sources

* fixed miniconda path

* changed activate command

* changed activate command

* changed activate command

* run single test

* added install command

* added dev environment

* added dev environment

* removed extraneous steps

* run all tests

* added badge

* environment doesn't install suite2p

* changed badge to match dev docs

* Removed unnecessary imports. Added download_url_to_file function from cellpose.

* Made changes to download test_data from suite2p link to .suite2p directory. Using pathlib.Path instead of os.path. Modified test functions to compare to downloaded cache test_files.

* Replaced filecmp with np allclose due to small differences between npy files across OSes.

* Modified atol and rtol parameters to get np.allclose to work

* Circimports (#350)

* removed unused imports

* added dependency chart generation graph

* removed cross-module nested imports

* simplified subpackage inits

* removed unused io imports

* removed last of parent's subpacke introspection

* renamed save/write files to enable package-level usage

* removed built files from version control

* removed inline subpackage importing from run_s2p

* remvobed subpackage introspection

* removed subpackage utils from external interface

* corrected import location

* removed subpackage import from __main__

* moved circle() and boundary() to gui.utils

* organized package-level imports

* moved enhanced_mean_image()

* ran optimize-imports linting refactor

* moved function

* moved unused resample_frames() to utils.py

* removed circular import between gui.classgui and gui.masks

* moved function to registration.utils

* moved roi_stats() to extract.py

* moved functions to utils

* merged reference and register

* moved utils function into using module

* moved sample_means and related functions back into registration.utils

* add cycle graph script

* expose pick_initial_reference()

* removed dependency of sourcery on sparsedetect.

* merged detection and extraction submodules, removed utils.py

* bugfix: fixed imports

* Merge branch 'dev' of D:\ProgrammingProjects\suite2p with conflicts.

* fixed default_ops() import

* added tqdm to test requirements

* fixed imports from package refactor commit

* Docsmerge (#352)

* chenged pick_init() to pick_initial_reference()

* moved in wiki cell detection tutorial page

Co-authored-by: Nicholas A. Del Grosso <[email protected]>
Co-authored-by: Chris <[email protected]>

Author: 3 people

.gitignore

@@ -16,6 +16,9 @@ $RECYCLE.BIN/
 # Windows shortcuts
 *.lnk
 
+# builds
+.eggs
+
 # python and jupyter
 jupyter/.ipynb_checkpoints/
 jupyter/__pycache__/
@@ -41,6 +44,12 @@ doctrees/
 .AppleDouble
 .LSOverride
 
+# visual studio code
+.vscode
+
+# pycharm
+.idea
+
 # Thumbnails
 ._*
 

.readthedocs.yml

@@ -0,0 +1,27 @@
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+  configuration: docs/conf.py
+
+# Build documentation with MkDocs
+#mkdocs:
+#  configuration: mkdocs.yml
+
+# Optionally build your docs in additional formats such as PDF
+formats:
+  - pdf
+
+# specify dependencies
+python:
+   version: 3.7
+   install:
+      - method: pip
+        path: .
+        extra_requirements:
+          - docs
+          - gui

.travis.yml

@@ -0,0 +1,19 @@
+language: python
+os:
+  - linux
+python:
+  - '3.7'
+before_install:
+  # Example from https://github.com/astrofrog/example-travis-conda/
+  - wget http://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
+  - bash ./miniconda.sh -b
+  - export PATH=~/miniconda3/bin:$PATH
+  - conda update --yes conda
+install:
+  - conda env create -f environment.yml
+  - source activate suite2p
+  - python setup.py install
+script:
+  - python setup.py test
+
+

MANIFEST.in

@@ -8,3 +8,4 @@ include suite2p/ops/ops_user.npy
 include suite2p/ops/ops_1P.npy
 include README.md
 include LICENSE
+recursive-exclude * __pycache__

README.md

@@ -1,5 +1,9 @@
 # suite2p <img src="suite2p/logo/logo_unshaded.png" width="250" title="sweet two pea" alt="sweet two pea" align="right" vspace = "50">
 
+[![Documentation Status](https://readthedocs.org/projects/suite2p/badge/?version=dev)](https://suite2p.readthedocs.io/en/dev/?badge=dev)
+
+[![Build Status](https://travis-ci.org/Mouseland/suite2p.svg?branch=dev)](https://travis-ci.org/Mouseland/suite2p)
+
 Pipeline for processing two-photon calcium imaging data.  
 Copyright (C) 2018  Howard Hughes Medical Institute Janelia Research Campus  
 
@@ -28,43 +32,33 @@ Install an [Anaconda](https://www.anaconda.com/download/) distribution of Python
 1. Download the [`environment.yml`](https://github.com/MouseLand/suite2p/blob/master/environment.yml) file from the repository. You can do this by cloning the repository, or copy-pasting the text from the file into a text document on your local computer.
 2. Open an anaconda prompt / command prompt with `conda` for **python 3** in the path
 3. Change directories to where the `environment.yml` is and run `conda env create -f environment.yml`
-4. To activate this new environment, run `conda activate suite2p`
-5. You should see `(suite2p)` on the left side of the terminal line. Now run `pip install suite2p --upgrade`.
+4. Pip install suite2p into the environment: `pip install suite2p`
 6. Now run `python -m suite2p` and you're all set.
 
 If you have an older `suite2p` environment you can remove it with `conda env remove -n suite2p` before creating a new one.
 
 Note you will always have to run **conda activate suite2p** before you run suite2p. Conda ensures mkl_fft and numba run correctly and quickly on your machine. If you want to run jupyter notebooks in this environment, then also `conda install jupyter`.
 
-To upgrade suite2p (package [here](https://pypi.org/project/suite2p/)), run the following in the environment:
+To **upgrade** the suite2p (package [here](https://pypi.org/project/suite2p/)), run the following in the environment:
 ~~~~
-pip install suite2p --upgrade
+pip install git+https://github.com/MouseLand/suite2p
 ~~~~
 
-**Common issues**
-
-If when running `python -m suite2p`, you receive the error: `No module named PyQt5.sip`, then try uninstalling and reinstalling pyqt5
-~~~~
-pip uninstall pyqt5 pyqt5-tools
-pip install pyqt5 pyqt5-tools pyqt5.sip
-~~~~
 
-If when running `python -m suite2p`, you receive an error associated with **matplotlib**, try upgrading it:
-~~~~
-pip install matplotlib --upgrade
-~~~~
+**Common issues**
 
 If you are on Yosemite Mac OS, PyQt doesn't work, and you won't be able to install suite2p. More recent versions of Mac OS are fine.
 
 The software has been heavily tested on Windows 10 and Ubuntu 18.04, and less well tested on Mac OS. Please post an issue if you have installation problems. The registration step runs faster on Ubuntu than Windows, so if you have a choice we recommend using the Ubuntu OS.
 
-## Installation of github version for developers and super-users :)
+## Installation of github version for developers
 
 1. Clone the repository and `cd suite2p` in an anaconda prompt / command prompt with `conda` for **python 3** in the path
 2. Run `conda env create -f environment.yml`
 3. To activate this new environment, run `conda activate suite2p` (you will have to activate every time you want to run suite2p)
-4. You should see `(suite2p)` on the left side of the terminal line. Now run `pip install -e .` to install the code in the github repository.
-5. Run `python -m suite2p` (<-- you can run this command from anywhere now. If you want to go back to the supported version of suite2p, then say `pip install suite2p` and that will grab the pip package of suite2p)
+4. Install the local version of suite2p into this environment in develop mode: `python setup.py develop`
+5. Run tests: `python setup.py test`
+
 
 ## Examples
 

docs/FAQ.rst

@@ -1,8 +1,8 @@
 Frequently Asked Questions
---------------
+--------------------------
 
 Cropped field-of-view
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~
 
 Some issues on this: `#273 <https://github.com/MouseLand/suite2p/issues/273>`_, 
 `#207 <https://github.com/MouseLand/suite2p/issues/207>`_,
@@ -29,7 +29,7 @@ See this page :ref:`inputs`
 for more info.
 
 Deconvolution means what?
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
 There is a lot of misinformation about what deconvolution is and what it isn't. Some issues on this:
 `#267 <https://github.com/MouseLand/suite2p/issues/267>`_,
@@ -83,14 +83,14 @@ know of many papers that make such statements.
 
 
 Multiple functional channels
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 If you have two channels and they both have functional activity, then to 
 process both you need to run suite2p in a jupyter notebook. Here is an example
 notebook for that purpose: `multiple_functional_channels.ipynb <https://github.com/MouseLand/suite2p/blob/master/jupyter/multiple_functional_channels.ipynb>`_
 
 Z-drift
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~
 
 It's not frequently asked about but it should be :)