Skip to content

Service for generating unique, time-ordered IDs across distributed worker processes.

License

Notifications You must be signed in to change notification settings

breqdev/snowcloud

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

snowcloud

Snowcloud is a service for generating unique, time-ordered, 64-bit IDs across distributed worker processes. This format is commonly called "snowflake" IDs, and is used by Twitter and Discord.

Snowflake ID Format

Timestamp Worker ID Increment
42 bits 10 bits 12 bits
milliseconds since Jan 1, 2020 allocated by delegator incremented by generator

Snowcloud Deployment Structure

A Snowcloud deployment consists of a delegator and many generators.

Delegator

The delegator (located in snowcloud/delegator.py, run with e.g. python3 -m snowcloud.delegator) keeps track of assigning worker IDs to different generator processes. It uses Redis to keep track of a pool of worker IDs. It then allows clients to register for a worker ID or renew their existing worker ID.

Generators

Generators obtain a worker ID from the delegator and use it to generate their own Snowflake IDs. These can be integrated into Flask applications using the snowcloud.flask_ext module.

An example Flask server is located in snowcloud/server.py. However, more commonly, the Flask application would use the Snowflake ID internally instead. For instance, it could generate a Snowflake ID to use as the primary key column whenever it inserts a new row into the database.

About

Service for generating unique, time-ordered IDs across distributed worker processes.

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published