Skip to content
This repository was archived by the owner on Aug 8, 2018. It is now read-only.

Making a User Service: Tutorial

Jump to bottom
vbuterin edited this page Oct 13, 2015 · 14 revisions

WORK IN PROGRESS

Note: this tutorial assumes that you have already installed pyethapp, and running pyethapp run on the command line works

One of pyethapp's most powerful distinguishing features is its ability to easily create built-in user services: scripts written in python that run alongside pyethapp, and run code on startup and every time you receive a new block. This allows you to create "server daemons" for applications that require periodic automated support, such as RANDAO, data feeds, "decentralized dropbox" apps, alarm clocks, decentralized cloud computing services, etc.

User services are stored in the contrib sub-folder of the pyethapp data directory (eg. ~/.config/pyethapp/contrib on Ubuntu); if this folder does not yet exist then create it. To add a service, in that directory create a python script with any name, as long as that name does not conflict with another python module (eg. helloworld.py is fine, math.py is not). Now, let us create our first service. In a file called helloworld.py in the contrib directory, put the following:

import sys

def on_start(app):
    print "Hello, world!"
    sys.exit()

Now, do pyethapp run on the command line. It should start up, and then quit in about five seconds. The last lines of the output should look something like this:

INFO:app	registering service service=factory generated service 0
INFO:app	starting 
Hello world

Now, let's make the service not quit immediately, and instead add an on_block method. Let's put the following into helloworld.py:

count = 0

def on_block(blk):
    global count
    count += 1
    print "Total number of blocks seen this run: %d" % count

When you run pyethapp and start syncing, you should eventually see something like this:

Total number of blocks seen this run: 1
new blk <Block(#375193 17e5591f)>
INFO:eth.chainservice	added txs=2 gas_used=42000 block=<Block(#375193 17e5591f)>
Total number of blocks seen this run: 2
new blk <Block(#375194 d6847603)>
INFO:eth.chainservice	added txs=1 gas_used=21000 block=<Block(#375194 d6847603)>
Total number of blocks seen this run: 3
new blk <Block(#375195 c6061bfe)>

Now, let's use this to do something interesting. What we are going to do is create a centralized data feed service which essentially acts as an HTTP proxy: contracts can call a get(string) function of a particular contract passing any desired URL as an argument, and a successful call triggers a log containing the URL, a callback ID and a callback address. The data feed service listens for logs, and upon seeing a log of the right type it sends an HTTP GET request to the URL, and then sends a callback message to the address containing the HTTP GET response.

First, determine a seed to generate a private key and an address; for this example we will use qwufqhwiufyqwiugxqwqcwrqwrcqr as our seed. The privkey is c7283d309fbac6fd65d2d9664c04dc71bbbdeb945b359501e912ab007da4fd27 and the address is b3cd4c2512402047ef4ddeb1ab9f8df400ce2d3f. Now, we will write a contract as follows:

event GetRequest(url:string, callback:address, responseId:uint256, fee:uint256)
data nextResponseId

def get(url:string):
    if msg.value >= 7 * 10**15:
        self.nextResponseId = sha3(block.prevhash + sha3(url:str) + msg.sender + msg.value * 2**160)
        log(type=GetRequest, url, msg.sender, self.nextResponseId, msg.value)
        send(0xb3cd4c2512402047ef4ddeb1ab9f8df400ce2d3f, self.balance)
        return(self.nextResponseId)

This contract does several things. First, it contains a get method with the right signature, which other contracts can call. It enforces a minimum fee of 7 finney, which covers gas costs for your return transaction plus a bit more to spare. It then generates a callback ID for the request; the complicated sha3 formula for generating the callback ID is used so that if the blockchain forks and a different set of messages is made and/or in a different order then the new IDs will not match up. It is expected that contracts calling get use these IDs as a way of keeping track which function call is a callback to which request; in the future we may well see high-level programming languages for Ethereum that abstract this logic away into something that looks more like nodejs-style callbacks or promises. Finally, it logs the request data (along with the callback ID and the fee paid for the request), passes along the fee to the account that will be returning the callback, and returns the callback ID to the contract that sent the request so that the contract can store it for future use.

Compile the contract, and push it to the blockchain; from here we'll assume that the contract's address is 0xd53096b3cf64d4739bb774e0f055653e7f2cd710. Now, just for our own testing purposes, we'll create a caller contract:

event LogResponse(response:string, fetchId:uint256)
data cbids[]

extern main: [get:[string]:int256]

def callback(response:str, responseId:uint256):
    if self.cbids[responseId] and msg.sender == 0xb3cd4c2512402047ef4ddeb1ab9f8df400ce2d3f:
        log(type=LogResponse, response, self.cbids[responseId])

def call(fetcher:address, url:str, fetchId:uint256):
    log(type=LogResponse, text("cow"), 0)
    x = fetcher.get(url, value=msg.value)
    self.cbids[x] = fetchId

Compile and push this contract; we'll assume the address is 0x6acc9a6876739e9190d06463196e27b6d37405c6. This contract is meant for demonstration purposes, so it does not do much interesting; we call it supplying (i) the address of our proxy contract, (ii) a URL, and (iii) any number, and upon the callback it creates a LogResponse containing the HTTP GET response and the number.

An exercise for the reader is to replace the fetchId argument with a recipientAddress argument, and instead of making a log, parse the HTTP GET response to get the USD price of ether (possibly using a different API that is more convenient), and then divide 5 by that value, and use that result, plus the recipientAddress parameter that got passed through the callback chain, to send the recipientAddress exactly 5 dollars worth of ether (a practical application of this is USD-denominated recurring payments).

Now, let us get to the meat of the task: writing the service that does the HTTP GET fetching.

The entire code is available at https://github.com/ethereum/pyethapp/blob/develop/examples/urlfetcher.py; feel free to simply copy it into your contrib directory, but we will nevertheless go through how the code works. First, the on_start method.

# Called once on startup
def on_start(_app):
    print 'Starting URL translator service'
    global app, my_nonce, chainservice
    app = _app
    chainservice = app.services.chain
    my_nonce = chainservice.chain.head.get_nonce(my_address)

In the code, we had initialized some global variables: app, representing an object for the entire pyethapp application, chainservice, a service that provides access to the chain, and my_nonce, the contract nonce. The on_start method is called with app as an argument; from this we get the app and the chainservice. From the chainservice, we use chainservice.chain to get the blockchain object, and chainservice.chain.head to get the last block in the chain. From there, we can use all of the methods available to blocks (use dir(chainservice.chain.head) to get a listing), including get_nonce to get the nonce of an account.

Now, the start of the on_block method:

# Called every block
def on_block(blk):
    global my_nonce, chainservice
    for receipt in blk.get_receipts():
        for _log in receipt.logs:
            # Get all logs to the proxy contract address of the right type
            if _log.address == my_contract_address:
                log = ct.listen(_log)
                if log and log["_event_type"] == "GetRequest":
                     < ..... continued later ..... >

The on_block method is called every time a block is added to the chain; from that block, we get the list of receipts, from each receipt the list of logs, then filter them just to get logs that are connected to the proxy contract address. Then, we use the ct object, which is of the ContractTranslator type. We initiated it further up in the code, with:

# ContractTranslator object for the main proxy contract
ct = ethereum.abi.ContractTranslator([{"constant": false, "type": "function", "name": "get(string)", "outputs": [{"type": "int256", "name": "out"}], "inputs": [{"type": "string", "name": "url"}]}, {"inputs": [{"indexed": false, "type": "string", "name": "url"}, {"indexed": false, "type": "address", "name": "callback"}, {"indexed": false, "type": "uint256", "name": "responseId"}, {"indexed": false, "type": "uint256", "name": "fee"}], "type": "event", "name": "GetRequest(string,address,uint256,uint256)"}])
# ContractTranslator object for the contract that is used for testing the main contract
ct2 = ethereum.abi.ContractTranslator([{"constant": false, "type": "function", "name": "callback(bytes,uint256)", "outputs": [], "inputs": [{"type": "bytes", "name": "response"}, {"type": "uint256", "name": "responseId"}]}])

A ContractTranslator accepts the ABI declaration of a contract, and returns an object that you can use to "interpret" a log, checking if the log is of the right format and if it is parsing the log topics and data into arguments, as well as encoding function arguments and decoding function return values. It is used in the backend of pyethereum.tester; in general, the methods are:

  • ct.listen(log) - converts a log into human-readable form if possible
  • ct.encode('function_name', [arg1, arg2, arg3]) - converts function arguments into transaction data using the contract ABI
  • ct.decode(bytes) - returns a list of output arguments. In practice, this one is used much less outside of the pyethereum.tester environment because transaction outputs are not logged and are less light-client friendly and so relying on them for applications is discouraged.

The output of ct.listen looks something like this:

{'fetchId': 15, '_event_type': 'LogResponse', 'response': 'test'}

Hence, ContractTranslators are a great way of interpreting log output of contracts inside of pyethereum, and you may want to get acquainted with them.

Now, let's look at the rest of the code, which runs if log and log["_event_type"] == "GetRequest":

print 'fetching: ', log["url"]
# Fetch the response
try:
     response = make_request(log["url"])
except:
     response = ''
print 'response: ', response
# Create the response transaction
txdata = ct2.encode('callback', [response, log["responseId"]])
tx = ethereum.transactions.Transaction(my_nonce, 60 * 10**9, min(100000 + log["fee"] / (60 * 10**9), 2500000), log["callback"], 0, txdata).sign(my_privkey)
print 'txhash: ', tx.hash.encode('hex')
print 'tx: ', rlp.encode(tx).encode('hex')
# Increment the nonce so the next transaction is also valid
my_nonce += 1
# Send it
success = chainservice.add_transaction(tx, broadcast_only=True)
assert success
print 'sent tx'

The most important piece of the service is the single line response = make_request(log["url"]): make an HTTP request containing the URL from the log, and set the response to be the response of the HTTP request. If the request fails, set the response to ''. Then, we use ct2.encode to encode the response, and the callback ID from the request log, into transaction data, and then create a transaction. The transaction's arguments are:

  • The current nonce
  • A standard gas fee of 60 shannon
  • An amount of gas that scales with the fee paid; the more you pay the more gas the service sends along with the transaction
  • The address to send the callback to
  • A value of zero
  • The transaction data that we encoded earlier

Then, we increment the my_nonce value so that the next transaction is sent with a nonce one higher, and use chainservice.add_transaction(tx, broadcast_only=True) to broadcast the transaction. Using broadcast_only=True is required in order for this to work; otherwise, you may get a (silent) error with the transaction failing because the service is running while the blockchain is "syncing" the last block.

Clone this wiki locally