Automatic decentralised update function using the dht

[chinwendu]

https://github.com/chinwendu/openbay-crawler

I finally released the first 100% independent decentralised automatic update function for the openbay projet. I do not see it as a replacement for an RSS update, but more like a complement. Once you launch it, it will listen to the dht for torrents requests. On torrent request (announce or peer request), it will download the torrent file from the peers found in the dht or try to download it from torcache if none of the peers have responded.
It will then feed the database based on the metadata inside the torrent file (name, list of files, size, creation date ...).

I made a function to distribute the info_hash and torrent url to others running the program to distribute the crawling effort of the dht between the participant.

I personally feed my own database with almost 3.000.000 torrents (starting from scratch) in less than a month.

I love to hear some feedback from openbay users, so feel free to comment.

edit: I forgot to say it can also scrape torrents againts udp://open.demonii.com:1337 and update to db with seeder and leecher number.

[TPBT-OFFICIAL]

Unfortunatly i can't seem to make it work i keep getting some weird python errors if you could help me i would be glad to show you what kind of errors i get.

[chinwendu]

sure no problem, if you can give me a traceback

edit: and the OS you are using

[TPBT-OFFICIAL]

OS: CentOS 6
Errors: When i run "./crawler.py" i get
Traceback (most recent call last):
File "./crawler.py", line 14, in
from btdht import DHT, ID, RoutingTable
File "/usr/lib64/python2.6/site-packages/btdht/init.py", line 13, in
from .dht import DHT, DHT_BASE, ID, Node, Bucket, RoutingTable, NotFound, BucketFull, NoTokenError
File "btdht/krcp.pxd", line 3, in init btdht.dht (btdht/dht.c:48774)
File "btdht/krcp.pyx", line 19, in init btdht.krcp (btdht/krcp.c:19498)
File "btdht/utils.pyx", line 15, in init btdht.utils (btdht/utils.c:12432)
ImportError: cannot import name total_ordering

[chinwendu]

I think you need python 2.7 https://docs.python.org/2/library/functools.html#functools.total_ordering New in version 2.7

edit: I added it as a dependacy in README.md, did think to do it as python 2.7 is the version distributed with the last debian stable (wheezy)