merge

Author: rmcgibbo

docs/sphinx/installation.rst

@@ -1,87 +1,74 @@
 Installation
 ============
 
-MSMBuilder should run on most modern computers equiped with a scientific python installation. But, in the interest of being explicit, here the requirements
+MSMBuilder is a python package that uses a number of components from the "scientific python" stack. These packages include `numpy and scipy <http://scipy.org/getting-started.html>`_ for array manipulation and numerical linear algebra, `PyTables <http://www.pytables.org/moin>`_ for storing binary data, and others.
 
-- A CPU with SSE3 support, which has been tandard on all x86 processors produce after 2006.
-- A working C compiler, such as GCC 4.2 or later, clang, or MSVC.
-- Python, with some scientific modules installed (see below)
+.. note::
 
-MSMBuilder is written in the python programming language, and uses a
-variety of tools from the wider scientific python ecosystem, which may
-need to be installed separately. They include
+   MSMBuilder currently required python2.7, and is not compatible with the python3.x series.
 
+Easily with ``conda``
+---------------------
 
-Python Prerequisites
---------------------
--  MDTraj
--  Numpy
--  Scipy
--  PyTables
--  numexpr
--  fastcluster (for hierarchical clustering)
--  matplotlib (optional for plotting)
--  ipython (optional for interactive mode)
--  pymol (optional for visualization)
+The easiest way to install MSMBuilder is with the python package manager ``conda``.
+``conda`` is an open-source cross-platform binary package manager integrated with
+the scientific python stack. It's built into the `Anaconda python distribution <http://docs.continuum.io/anaconda/>`_ produced by Continuum Analytics, which is a python installer that comes shipped with many of the python packages needed for science.
+
+.. warning::
 
-Two companies, Enthought and Continuum Analytics, produce python
-distributions which bundle many of these packages in with the python
-interpreter into a single binary installer, available for all major
-operating systems. These are the Enthought Canopy python distribution
-and Continuum’s Anaconda.
+   We **strongly** recommend using Anaconda. Installing the scientific stack by hand can be quite tricky.
 
+If you don't want to get Anaconda, you can also install ``conda`` into an existing python interpreter. Once you have ``conda``, install MSMBuilder with ::
 
-Install Python and Python Packages
-----------------------------------
+   conda config --add channels http://conda.binstar.org/omnia
+   conda install msmbuilder
 
-Rather than individually install the many python dependencies, we
-recommend that you download the Python2.7 version of the Enthought
-Canopy or Continuum Anaconda, which contain almost all python
-dependencies required to run MSMBuilder. If you have a 64 bit platform,
-please use the 64 bit versions, as this will give higher performance.
+Medium With ``pip``
+-------------------
 
-Note for OSX users: Enthought represents the easiest way to obtain a
-working Python installation. The OSX system Python install is broken and
-cannot properly build Python extensions, which are required for
-MSMBuilder installation. Also, see FAQ question 11 for a known issue
-with OSX Lion and OpenMP.
+MSMBuilder can be instaleld with ``pip``, but ``pip`` is not fantastic at installing the dependencies. If you've already got the dependencies installed (see below), then you can download and install MSMBuilder::
 
-Note: if you are unable to use Canopy or Anaconda, there are other
-pre-compiled Python distributions available, although they might not be
-as fast as Enthought. Options include Python(x,y) and the Scipy
-Superpack (OSX). Finally, most Linux users can install most
-prerequisites using their package manager. In Ubuntu, the following will
-install most of the prerequisites:
+    pip install msmbuilder
 
-::
+Hard Way by Hand
+----------------
+
+If you use conda, all of this will be done automatically. If you prefer to do things by hand, keep reading.
+
+You'll need to get the following python packages:
+
+-  `mdtraj >= 0.8 <https://pypi.python.org/pypi/mdtraj>`_
+-  `numpy >= 1.6 <https://pypi.python.org/pypi/numpy>`_
+-  `scipy >= 0.11 <https://pypi.python.org/pypi/scipy>`_
+-  `tables >= 2.4.0 <https://pypi.python.org/pypi/tables>`_
+-  `pyyaml <https://pypi.python.org/pypi/PyYAML>`_
+-  `fastcluster (for hierarchical clustering) <https://pypi.python.org/pypi/fastcluster>`_
+-  matplotlib (optional for plotting)
+-  ipython (optional for interactive mode)
+
+On a debian-based linux, you can get most of them with ::
 
     $ sudo apt-get install libhdf5-serial-dev python-dev python-numpy \
     python-scipy python-setuptools python-nose python-tables \
-    python-matplotlib python-yaml swig ipython
+    python-matplotlib python-yaml swig ipython python-pip
 
-Neither Canopy nor Anaconda include MDTraj nor fastcluster. They can be installed be installed using python’s package manager, ``pip``.
 
-::
+Download MSMBuilder, unzip, move to the msmbuilder directory. Then, use ``pip``
+to install any remaining dependencies ::
 
     $ pip install -r requirements.txt
 
-Download and Install MSMBuilder
--------------------------------
-
-Download MSMBuilder, unzip, move to the msmbuilder directory. Install
-using setup.py:
-
-::
+Then install MSMBuilder with ``setup.py`` ::
 
     $ python setup.py install
 
 You may need root privileges during the install step; alternatively, you
 can specify an alternative install path via ``–prefix=XXX``. If you
 performed the install step with ``–prefix=XXX``, you need to ensure that
 
-#. XXX/bin is included in your PATH
+#. ``XXX/bin`` is included in your ``PATH``
 
-#. XXX/lib/python2.7/site-packages/ is included in your PYTHONPATH
+#. ``XXX/lib/python2.7/site-packages/`` is included in your ``PYTHONPATH``
 
 Step (1) ensures that you can run MSMBuilder scripts without specifying
 their location. Step (2) ensures that your Python can locate the

docs/sphinx/tutorial.rst

@@ -65,19 +65,19 @@ clusters.
 After clustering, one must assign the data to the clusters. For the
 clustering settings used above, this happens automatically, so you will
 not need to run a separate assignment step. For other clustering
-protocols, you may need to run Assign.py or AssignHierarchical.py after
-the clustering phase.
+protocols, you may need to run ``Assign.py`` or ``AssignHierarchical.py``
+after the clustering phase.
 
-The assignments of each conformation are stored as Data/Assignments.h5.
-The cluster centers are stored as Data/Gens.lh5.
+The assignments of each conformation are stored as ``Data/Assignments.h5``.
+The cluster centers are stored as ``Data/Gens.lh5``.
 
 Note that clustering with the RMSD metric requires a list of which atom
 indices to use during RMSD calculation. This file is typically called
-AtomIndices.dat and can typically be created using the script
-CreateAtomIndices.py. Because alanine dipeptide contains non-standard
+``AtomIndices.dat`` and can typically be created using the script
+``CreateAtomIndices.py``. Because alanine dipeptide contains non-standard
 atom names, it cannot be generated automatically; a default
-AtomIndices.dat has already been placed in the Tutorial directory for
-your use. Note that AtomIndices.dat uses *zero* based indexing–the first
+``AtomIndices.dat`` has already been placed in the Tutorial directory for
+your use. Note that ``AtomIndices.dat`` uses *zero* based indexing–the first
 atom in your system has index 0.
 
 Alternative Clustering Protocols
@@ -132,17 +132,17 @@ Assignments.Fixed.h5 contains a “fixed” version of your microstate
 assignments that has removed all data that is trimmed the maximal
 ergodic subgraph of your data.
 
-tCounts.mtx contains the maximum likelihood estimated reversible count
-matrix. This is a symmetric matrix.
+``tCounts.mtx`` contains the maximum likelihood estimated reversible
+count matrix. This is a symmetric matrix.
 
-tProb.mtx contains the maximum likelihood estimated transition
+``tProb.mtx`` contains the maximum likelihood estimated transition
 probability matrix.
 
-Mapping.dat contains a mapping of the original microstate numbering to
+``Mapping.dat`` contains a mapping of the original microstate numbering to
 the “fixed” microstate numbering. This is necessary because some states
 may have been discarded during the ergodic trimming step.
 
-Populations.dat contains the maximum likelihood estimated reversible
+``Populations.dat`` contains the maximum likelihood estimated reversible
 equilibrium populations.
 
 Construct a Macrostate MSM

requirements.txt

@@ -1,4 +1,7 @@
-numpy
-scipy
-mdtraj
-fastcluster
+nose
+pyyaml
+numpy>=1.6
+scipy>=0.11
+mdtraj>=0.8
+fastcluster>=1.1.13
+tables>=2.4.0

setup.py

@@ -11,28 +11,58 @@
 
 import os
 import sys
-import shutil
-import tempfile
 import subprocess
 from glob import glob
-from distutils.ccompiler import new_compiler
-from setuptools import setup, Extension
-
+from distutils.version import StrictVersion
 try:
-    import numpy
-    import scipy
-    if not hasattr(numpy.version, 'full_version') or numpy.version.full_version < '1.6':
-        raise ImportError()
-    if not hasattr(scipy.version, 'full_version') or scipy.version.full_version < '0.11':
-        raise ImportError()
+    from setuptools import setup
 except ImportError:
-    print('numpy>=1.6 and scipy>=0.11 are required for msmbuilder', file=sys.stderr)
-    sys.exit(1)
+    from distutils.core import setup
 
+#########################################
 VERSION = "2.8"
 ISRELEASED = False
 __author__ = "MSMBuilder Team"
 __version__ = VERSION
+########################################
+
+def warn_on_version(module_name, minimum=None, package_name=None, recommend_conda=True):
+    if package_name is None:
+        package_name = module_name
+
+    class VersionError(Exception):
+        pass
+
+    msg = None
+    try:
+        package = __import__(module_name)
+        if minimum is not None:
+            try:
+                v = package.version.short_version
+            except AttributeError:
+                v = package.__version__
+            if StrictVersion(v) < StrictVersion(minimum):
+                raise VersionError
+    except ImportError:
+        if minimum is None:
+            msg = 'MSMBuilder requires the python package "%s", which is not installed.' % package_name
+        else:
+            msg = 'MSMBuilder requires the python package "%s", version %s or later.' % (package_name, minimum)
+    except VersionError:    
+        msg = ('MSMBuilder requires the python package "%s", version %s or '
+               ' later. You have version %s installed. You will need to upgrade.') % (package_name, minimum, v)
+
+    if recommend_conda:
+        install = ('\nTo install %s, we recommend the conda package manger. See http://conda.pydata.org for info on conda.\n'
+                   'Using conda, you can install it with::\n\n    $ conda install %s') % (package_name, package_name)
+        install += '\n\nAlternatively, with pip you can install the package with:\n\n    $ pip install %s' % package_name
+    else:
+        install = '\nWith pip you can install the package with:\n\n    $ pip install %s' % package_name
+    
+    if msg:
+        banner = ('==' * 40)
+        print('\n'.join([banner, banner, "", msg, install, "", banner, banner]))
+
 
 # metadata for setup()
 metadata = {
@@ -44,7 +74,6 @@
     'url': 'https://simtk.org/home/msmbuilder',
     'download_url': 'https://simtk.org/home/msmbuilder',
     'platforms': ["Linux", "Mac OS X"],
-    'zip_safe': False,
     'description': DOCLINES[0],
     'long_description':"\n".join(DOCLINES[2:]),
     'packages': ['msmbuilder', 'msmbuilder.scripts', 'msmbuilder.project',
@@ -57,55 +86,9 @@
     'scripts': ['scripts/msmb'] + [e for e in glob('scripts/*') if not e.endswith('__.py')]
 }
 
+if 'setuptools' in sys.modules:
+    metadata['zip_safe'] = False
 
-def hasfunction(cc, funcname, include=None, extra_postargs=None):
-    # From http://stackoverflow.com/questions/
-    #            7018879/disabling-output-when-compiling-with-distutils
-    tmpdir = tempfile.mkdtemp(prefix='hasfunction-')
-    devnull = oldstderr = None
-    try:
-        try:
-            fname = os.path.join(tmpdir, 'funcname.c')
-            f = open(fname, 'w')
-            if include is not None:
-                f.write('#include %s\n' % include)
-            f.write('int main(void) {\n')
-            f.write('    %s;\n' % funcname)
-            f.write('}\n')
-            f.close()
-            devnull = open(os.devnull, 'w')
-            oldstderr = os.dup(sys.stderr.fileno())
-            os.dup2(devnull.fileno(), sys.stderr.fileno())
-            objects = cc.compile([fname], output_dir=tmpdir,
-                                 extra_postargs=extra_postargs)
-            cc.link_executable(objects, os.path.join(tmpdir, 'a.out'))
-        except Exception as e:
-            return False
-        return True
-    finally:
-        if oldstderr is not None:
-            os.dup2(oldstderr, sys.stderr.fileno())
-        if devnull is not None:
-            devnull.close()
-        shutil.rmtree(tmpdir)
-
-
-def detect_openmp():
-    "Does this compiler support OpenMP parallelization?"
-    compiler = new_compiler()
-    print('Attempting to autodetect OpenMP support...')
-    hasopenmp = hasfunction(compiler, 'omp_get_num_threads()')
-    needs_gomp = hasopenmp
-    if not hasopenmp:
-        compiler.add_library('gomp')
-        hasopenmp = hasfunction(compiler, 'omp_get_num_threads()')
-        needs_gomp = hasopenmp
-    print
-    if hasopenmp:
-        print('Compiler supports OpenMP')
-    else:
-        print('Did not detect OpenMP support; parallel RMSD disabled')
-    return hasopenmp, needs_gomp
 
 # Return the git revision as a string
 # copied from numpy setup.py
@@ -132,6 +115,7 @@ def _minimal_ext_cmd(cmd):
 
     return GIT_REVISION
 
+
 def write_version_py(filename='src/python/version.py'):
     cnt = """
 # THIS FILE IS GENERATED FROM MSMBUILDER SETUP.PY
@@ -164,28 +148,16 @@ def write_version_py(filename='src/python/version.py'):
     finally:
         a.close()
 
-
-def main():
-    compiler_args = ['-O3', '-funroll-loops']
-    if new_compiler().compiler_type == 'msvc':
-        compiler_args.append('/arch:SSE2')
-
-    openmp_enabled, needs_gomp = detect_openmp()
-    if openmp_enabled:
-        compiler_args.append('-fopenmp')
-    compiler_libraries = ['gomp'] if needs_gomp else []
-
-    dist = Extension('msmbuilder._distance_wrap', sources=glob('src/ext/scipy_distance/*.c'),
-                     extra_compile_args=compiler_args,
-                     libraries=compiler_libraries,
-                     include_dirs=[numpy.get_include()])
-    contact = Extension('msmbuilder._contact_wrap', sources=glob('src/ext/contact/*.c'),
-                        extra_compile_args=compiler_args,
-                        libraries=compiler_libraries,
-                        include_dirs = [numpy.get_include()])        
-
-    write_version_py()
-    setup(ext_modules=[dist, contact], **metadata)
-
-if __name__ == '__main__':
-    main()
+write_version_py()
+setup(**metadata)
+
+# running these after setup() ensures that they show
+# at the bottom of the output, since setup() prints
+# a lot to stdout. helps them not get lost
+warn_on_version('numpy', '1.6.0')
+warn_on_version('scipy', '0.11.0')
+warn_on_version('tables', '2.4.0', package_name='pytables')
+warn_on_version('fastcluster', '1.1.13', recommend_conda=False)
+warn_on_version('yaml', package_name='pyyaml')
+# because it requires adding a different channel, the correct instructions are more complex with conda than pip
+warn_on_version('mdtraj', '0.8.0', recommend_conda=False)

src/python/MSMLib.py

@@ -50,7 +50,6 @@
     tarjan
     trim_states
 """
-import warnings
 import scipy.sparse
 import scipy.linalg
 import scipy