Skip to content

Commit c0ccb6d

Browse files
escherbaescherba
authored
update workflow (#17)
* update workflow * version bump
1 parent 7ba3ac0 commit c0ccb6d

File tree

17 files changed

+363
-1853
lines changed

17 files changed

+363
-1853
lines changed

.github/workflows/build-test.yml renamed to .github/workflows/build.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
# This is a partial whorkflow. Use wheels.yml for more
1+
# This is a partial workflow. Use wheels.yml for more
22
# complete workflow.
3-
name: Build and Test
3+
name: build
44

55
on:
66
push:
@@ -17,7 +17,7 @@ jobs:
1717
strategy:
1818
matrix:
1919
os: [windows-latest, macos-latest, ubuntu-latest]
20-
python-version: ["3.7", "3.8", "3.9", "3.10"]
20+
python-version: ["3.9"]
2121

2222
runs-on: ${{ matrix.os }}
2323
steps:

‎.github/workflows/publish.yml‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
name: publish
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
repository:
7+
description: 'The repository to upload the package to'
8+
required: true
9+
default: 'testpypi'
10+
11+
jobs:
12+
build_wheels:
13+
name: Build wheels on ${{ matrix.os }}
14+
runs-on: ${{ matrix.os }}
15+
strategy:
16+
matrix:
17+
os: [ubuntu-20.04, windows-2019, macOS-10.15]
18+
steps:
19+
- name: Checkout
20+
uses: actions/checkout@v2
21+
- name: Set up Python
22+
uses: actions/setup-python@v2
23+
with:
24+
python-version: '3.9'
25+
- name: Set up QEMU
26+
if: runner.os == 'Linux'
27+
uses: docker/setup-qemu-action@v1.0.1
28+
with:
29+
platforms: arm64
30+
- name: Build wheels
31+
uses: joerick/cibuildwheel@v1.9.0
32+
with:
33+
output-dir: wheelhouse
34+
env:
35+
CIBW_BUILD: '{cp36,cp37,cp38,cp39}-{manylinux_x86_64,manylinux_aarch64,win32,win_amd64,macosx_x86_64} cp39-macosx_arm64'
36+
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux2014
37+
CIBW_ARCHS_LINUX: 'auto aarch64'
38+
CIBW_ARCHS_MACOS: 'auto arm64'
39+
CIBW_TEST_REQUIRES: pytest
40+
CIBW_TEST_COMMAND: 'pytest -s {project}/tests'
41+
CIBW_TEST_SKIP: '*-macosx_arm64' # Until the day Apple silicon instances are available on GitHub Actions
42+
- uses: actions/upload-artifact@v2
43+
with:
44+
path: ./wheelhouse/*.whl
45+
build_sdist:
46+
name: Build a source distribution
47+
runs-on: ubuntu-20.04
48+
steps:
49+
- name: Checkout
50+
uses: actions/checkout@v2
51+
- name: Set up Python
52+
uses: actions/setup-python@v2
53+
with:
54+
python-version: '3.9'
55+
- name: Build sdist
56+
run: |
57+
pip install py-cpuinfo
58+
python setup.py build sdist
59+
- uses: actions/upload-artifact@v2
60+
with:
61+
path: dist/*.tar.gz
62+
publish:
63+
name: 'Upload to PyPI/TestPyPI'
64+
runs-on: ubuntu-20.04
65+
needs: [build_wheels, build_sdist]
66+
steps:
67+
- name: Set up Python
68+
uses: actions/setup-python@v2
69+
with:
70+
python-version: '3.9'
71+
- name: Set up built items
72+
uses: actions/download-artifact@v2
73+
with:
74+
name: artifact
75+
path: dist
76+
- name: Install dependencies
77+
run: |
78+
python -m pip install --upgrade pip
79+
pip install setuptools wheel twine
80+
- name: Publish
81+
env:
82+
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
83+
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
84+
run: |
85+
twine upload --verbose --repository ${{ github.event.inputs.repository }} dist/*

‎.gitignore‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ tags
1515
# Exclude these directories
1616
bin/
1717
data/
18+
wheelhouse/
1819

1920
# Distribution / packaging
2021
.Python
@@ -58,9 +59,6 @@ coverage.xml
5859
*.mo
5960
*.pot
6061

61-
# Django stuff:
62-
*.log
63-
6462
# Sphinx documentation
6563
docs/_build/
6664

@@ -71,3 +69,6 @@ target/
7169
.idea/
7270
.vscode/
7371
.ropeproject/
72+
.ipynb_checkpoints/
73+
.pytest_cache/
74+
.mypy_cache/

‎MANIFEST.in‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
include LICENSE
2-
include README.rst
3-
include cpuinfo.py
2+
include README.md
43
include src/*.h
54
include src/*.cc
65
include src/*.cpp

‎README.md‎

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
# MetroHash
2+
3+
Python wrapper for [MetroHash](https://github.com/jandrewrogers/MetroHash), a
4+
fast non-cryptographic hash function.
5+
6+
[![Latest
7+
Version](https://img.shields.io/pypi/v/metrohash.svg)](https://pypi.python.org/pypi/metrohash)
8+
[![Downloads](https://img.shields.io/pypi/dm/metrohash.svg)](https://pypi.python.org/pypi/metrohash)
9+
[![Tests
10+
Status](https://circleci.com/gh/escherba/python-metrohash.png?style=shield)](https://circleci.com/gh/escherba/python-metrohash)
11+
[![Supported Python
12+
versions](https://img.shields.io/pypi/pyversions/cityhash.svg)](https://pypi.python.org/pypi/cityhash)
13+
[![License](https://img.shields.io/pypi/l/cityhash.svg)](https://pypi.python.org/pypi/cityhash)
14+
15+
## Getting Started
16+
17+
To use this package in your program, simply type
18+
19+
``` bash
20+
pip install metrohash
21+
```
22+
23+
After that, you should be able to import the module and do things with
24+
it (see usage example below).
25+
26+
## Usage Examples
27+
28+
### Stateless hashing
29+
30+
This package provides Python interfaces to 64- and 128-bit
31+
implementations of MetroHash algorithm. For stateless hashing, it
32+
exports `metrohash64` and `metrohash128` functions. Both take a value to
33+
be hashed and an optional `seed` parameter:
34+
35+
``` python
36+
>>> import metrohash
37+
...
38+
>>> metrohash.hash64_int("abc", seed=0)
39+
17099979927131455419
40+
>>> metrohash.hash128_int("abc")
41+
182995299641628952910564950850867298725
42+
43+
```
44+
45+
### Incremental hashing
46+
47+
Unlike its cousins CityHash and FarmHash, MetroHash allows incremental
48+
(stateful) hashing. For incremental hashing, use `MetroHash64` and
49+
`MetroHash128` classes. Incremental hashing is associative and
50+
guarantees that any combination of input slices will result in the same
51+
final hash value. This is useful for processing large inputs and stream
52+
data. Example with two slices:
53+
54+
``` python
55+
>>> mh = metrohash.MetroHash64()
56+
>>> mh.update("Nobody inspects")
57+
>>> mh.update(" the spammish repetition")
58+
>>> mh.intdigest()
59+
7851180100622203313
60+
61+
```
62+
63+
The resulting hash value above should be the same as in:
64+
65+
``` python
66+
>>> mh = metrohash.MetroHash64()
67+
>>> mh.update("Nobody inspects the spammish repetition")
68+
>>> mh.intdigest()
69+
7851180100622203313
70+
71+
```
72+
73+
### Fast hashing of NumPy arrays
74+
75+
The Python [Buffer
76+
Protocol](https://docs.python.org/3/c-api/buffer.html) allows Python
77+
objects to expose their data as raw byte arrays to other objects, for
78+
fast access without copying to a separate location in memory. Among
79+
others, NumPy is a major framework that supports this protocol.
80+
81+
All hashing functions in this packege will read byte arrays from objects
82+
that expose them via the buffer protocol. Here is an example showing
83+
hashing of a 4D NumPy array:
84+
85+
``` python
86+
>>> import numpy as np
87+
>>> arr = np.zeros((256, 256, 4))
88+
>>> metrohash.hash64_int(arr)
89+
12125832280816116063
90+
91+
```
92+
93+
The arrays need to be contiguous for this to work. To convert a
94+
non-contiguous array, use NumPy's `ascontiguousarray()` function.
95+
96+
## Development
97+
98+
### Local workflow
99+
100+
For those who want to contribute, here is a quick start using some
101+
makefile commands:
102+
103+
``` bash
104+
git clone https://github.com/escherba/python-metrohash.git
105+
cd python-metrohash
106+
make env # create a Python virtualenv
107+
make test # run Python tests
108+
make cpp-test # run C++ tests
109+
make shell # enter IPython shell
110+
```
111+
112+
The Makefiles provided have self-documenting targets. To find out which
113+
targets are available, type:
114+
115+
``` bash
116+
make help
117+
```
118+
119+
### Distribution
120+
121+
The wheels are built using
122+
[cibuildwheel](https://cibuildwheel.readthedocs.io/) and are distributed
123+
to PyPI using GitHub actions using [this
124+
workflow](.github/workflows/publish.yml). The wheels contain compiled
125+
binaries and are available for the following platforms: windows-amd64,
126+
ubuntu-x86, linux-x86\_64, linux-aarch64, and macosx-x86\_64.
127+
128+
## See Also
129+
130+
For other fast non-cryptographic hash functions available as Python
131+
extensions, see [FarmHash](https://github.com/escherba/python-cityhash)
132+
and [MurmurHash](https://github.com/hajimes/mmh3).
133+
134+
## Authors
135+
136+
The MetroHash algorithm and C++ implementation is due to J. Andrew
137+
Rogers. The Python bindings for it were written by Eugene Scherba.
138+
139+
## License
140+
141+
This software is licensed under the [Apache License,
142+
Version 2.0](https://opensource.org/licenses/Apache-2.0). See the
143+
included LICENSE file for details.

0 commit comments

Comments
 (0)