why? instagram is the biggest vangav backend template designed for big scale and covering all the core features/utilities: service oriented architecture, worker service, multi-entry-point api, multi-keysapce database, oauth 2, facebook authentication, facebook graph api, rest jobs, periodic jobs, vangav mighty, push notifications, logging, analytics, client generator, test service, bots service, ...
- signup/login using email/facebook
- request authentication using oauth 2
- fetching/updating users' information
- follow/unfollow and get followers/following lists for users
- posting photos
- get posts by feed and by user
- interacting with posts (like, comment, ...)
- get top users/posts
- searching for users
- get server time
- is a helper service for instagram; responsible for distributing posted photos to followers (because on photo can be posted to tens of millions of followers; such an expensive operation should be distributable and shouldn't overload the main service)
- crawls for failed jobs in the database to retry executing them (e.g.: a dispense job, ...)
- uses vangav mighty to update top users (per geo grid, per country and world wide) every week
- uses vangav mighty to update top posts (per geo grid, per country and world wide) every day
- writes logs and analytics in the database
- sends push notifications (ios and android)
- handles fetching logs and analytics for system admin
- has a full-coverage test for all the services above
- simulates incremental usage behavior from many cities around the world with some celebrity users
- this service is based on vangav backend's instagram template
- this service has the 90+% of the vangav backend's generated code + the 10-% of the logic code needed to complete the service
- for first timers - follow the steps in the system requirements tutorial
- for first timers - follow the steps in the workspace initialization tutorial
- download instagram, instagram dispense, instagram jobs, instagram worker and instagram dash board projects (from the green
clone or download button) inside the workspace directory created previously (my_services) and unzip them
- rename unzipped directories, remove the
-master from their names
- in the terminal
cd to vos_instagram/cassandra/cql/
- execute
./_start_cassandra.sh to start cassandra
cd to vos_instagram/cassandra/cql/drop_and_create/- execute the commands
./_execute_cql.sh ig_....cql to initialize the services' database tables; repeat this step for all the .cql files
cd to vos_instagram_worker and execute ./_run.sh to start the instagram worker service on port 8000cd to vos_instagram_dispense and execute ./_run.sh 3000 to start the instagram dispense service on port 3000cd to vos_instagram_jobs and execute ./_run.sh 4000 to start the instagram jobs service on port 4000cd to vos_instagram_dash_board and execute ./_run.sh 5000 to start the instagram dash board service on port 5000cd to vos_instagram and execute ./_run.sh to start the instagram service on port 9000- from your prefered client (we recommned postman) start trying the service; refer to the features and service references sections for reference
- at the end to stop the services: press
control + d in the terminal session where each service was started in (9, 10, 11, 12 and 13)
- to stop cassandra: execute
ps auwx | grep cassandra to get cassandra's (pid) then kill -9 (pid) to stop cassandra
- follow the following steps to import the downloaded backend service in eclipse
cd to the service's directory and execute the ./_eclipsify.sh script- file > import > general > existing projects into workspace > next > set "select root directory" to my_services > under projects make sure that vos_calculate_sum is selected > finish
- double check the java version used for compiling the project: right click the project > properties > java compiler > enable project specific settings > compiler compliance level > 1.7 or 1.8
- generate multiple services (main + dispense + jobs + worker + dash board) to work together in a service oriented architecture
- generate a multi-keyspace database
- oauth 2 and facebook authentication
- facebook graph api
- rest jobs
- periodic jobs
- vangav mighty
- push notifications
- logging
- analytics
- client generator
- test and bots services
| controller(s) |
feature |
| signup email, login email, login facebook, get access token, refresh access token and logout |
handle user signup, loging and requests' authentication |
| update facebook info, update last active time, update location, update profile picture, get user info and get profile picture |
handle setting/updating users' info and getting it |
| follow, unfollow, get followers and get following |
handle building the social graph and exploring it |
| post photo |
handles posting a new photo to a user's followers |
| get feed posts, get user posts, get post info, get post photo id, get photo, get photo thumbnail, get post likes and get post comments |
handle getting feed/users' posts and posts' info/content/likes/comments |
| like post, unlike post, comment and delete comment |
handle interacting with posts |
| get top users and get top posts |
handle exploring top users/posts |
| search by email |
handles searching for users by their e-mail |
| get server time |
getting server's time is important for various reasons like enabling the client to set how old a post is (2h, 1w, ...) |
| controller |
feature |
| start jobs |
stars the periodic job for retrying failed rest jobs and the periodic jobs for ranking users and posts |
| controller |
feature |
| test all controllers |
runs a full-coverage test for the instagram services above and returns a full report (success, failure, request-to-response time, ...) |
| controller |
feature |
| start bots |
starts simulating incremental usage behavior from many cities around the world with some celebrity users |
| reference |
explanation |
| routes |
api routes |
| prop |
in addition to vangav backend's properties files, this directory contains instagram properties files: constants and dispense |
| reverse_geo_coding |
reverse geo coding data |
| controllers.json |
api request/response's elements |
| ig_auth.keyspace |
ig_auth is the keyspace used for all authentication-related tables |
| ig_app_data.keyspace |
ig_app_data is the keyspace used for all user-info-related tables which are directly relevant to the app's functionality (e.g.: posts, followers, ...) |
| ig_blobs.keyspace |
ig_blobs is the keyspace used for all blob-related tables (photos, thumbnails, ...) |
| ig_jobs.keyspace |
ig_jobs is the keyspace used for all jobs-related tables -- a job is an operation that's either executed at a later time or issued by one service and executed by another where a jobs table keeps track of unsuccessful jobs which need to be reexecuted |
| ig_logging.keyspace |
ig_logging is the keyspace used for all logging-related tables |
| ig_analytics.keyspace |
ig_analytics is the keyspace used for all analytics-related tables |
| Global.java |
initializes reverse geo coding data |
| common |
handles common operations like authentication, cassandra's frequent-operations, omitting blobs from requests/responses for logging, ... |
| controllers |
api implementation |
| ig_auth |
ig_auth cassandra's keyspace client |
| ig_app_data |
ig_app_data cassandra's keyspace client |
| ig_blobs |
ig_blobs cassandra's keyspace client |
| ig_jobs |
ig_jobs cassandra's keyspace client |
| ig_logging |
ig_logging cassandra's keyspace client |
| ig_analytics |
ig_analytics cassandra's keyspace client |
| reference |
explanation |
| routes |
api routes |
| controllers.json |
api request/response's elements |
| ig_auth.keyspace |
ig_auth is the keyspace used for all authentication-related tables |
| ig_app_data.keyspace |
ig_app_data is the keyspace used for all user-info-related tables which are directly relevant to the app's functionality (e.g.: posts, followers, ...) |
| ig_jobs.keyspace |
ig_jobs is the keyspace used for all jobs-related tables -- a job is an operation that's either executed at a later time or issued by one service and executed by another where a jobs table keeps track of unsuccessful jobs which need to be reexecuted |
| ig_logging.keyspace |
ig_logging is the keyspace used for all logging-related tables |
| ig_analytics.keyspace |
ig_analytics is the keyspace used for all analytics-related tables |
| controllers |
api implementation |
| ig_auth |
ig_auth cassandra's keyspace client |
| ig_app_data |
ig_app_data cassandra's keyspace client |
| ig_jobs |
ig_jobs cassandra's keyspace client |
| ig_logging |
ig_logging cassandra's keyspace client |
| ig_analytics |
ig_analytics cassandra's keyspace client |
| reference |
explanation |
| routes |
api routes |
| prop |
in addition to vangav backend's properties files, this directory contains instagram jobs properties files: constants, users rank and posts rank |
| reverse_geo_coding |
reverse geo coding data |
| controllers.json |
api request/response's elements |
| ig_app_data.keyspace |
ig_app_data is the keyspace used for all user-info-related tables which are directly relevant to the app's functionality (e.g.: posts, followers, ...) |
| ig_jobs.keyspace |
ig_jobs is the keyspace used for all jobs-related tables -- a job is an operation that's either executed at a later time or issued by one service and executed by another where a jobs table keeps track of unsuccessful jobs which need to be reexecuted |
| ig_logging.keyspace |
ig_logging is the keyspace used for all logging-related tables |
| Global.java |
initializes reverse geo coding data and periodic jobs |
| common |
handles common operations like instagram-jobs-service's properties |
| periodic_jobs |
handles retrying failed jobs as well as ranking users and posts |
| controllers |
api implementation |
| ig_app_data |
ig_app_data cassandra's keyspace client |
| ig_jobs |
ig_jobs cassandra's keyspace client |
| ig_logging |
ig_logging cassandra's keyspace client |
| reference |
explanation |
| routes |
api routes |
| controllers.json |
api request/response's elements |
| ig_auth.keyspace |
ig_auth is the keyspace used for all authentication-related tables |
| ig_app_data.keyspace |
ig_app_data is the keyspace used for all user-info-related tables which are directly relevant to the app's functionality (e.g.: posts, followers, ...) |
| ig_logging.keyspace |
ig_logging is the keyspace used for all logging-related tables |
| ig_analytics.keyspace |
ig_analytics is the keyspace used for all analytics-related tables |
| common |
handles common operations like service-wide constants |
| controllers |
api implementation |
| ig_auth |
ig_auth cassandra's keyspace client |
| ig_app_data |
ig_app_data cassandra's keyspace client |
| ig_logging |
ig_logging cassandra's keyspace client |
| ig_analytics |
ig_analytics cassandra's keyspace client |
| reference |
explanation |
| routes |
api routes |
| setup_data |
contains a one-time setup data with all the cities (id, location, population, ...) where the bots are created |
| prop |
in addition to vangav backend's properties files, this directory contains the clients properties file clients |
| controllers.json |
api request/response's elements |
| vos_instagram.client_java |
instagram client's api request/response's elements |
| ig_bots.keyspace |
ig_bots is the keyspace used for all bots-related vos_instagream tables |
| Global.java |
initializes cities and bots |
| cities |
has the cities' json representation and loader |
| bots |
has the bots' periodic jobs and the bots creator |
| clients |
contains the generated clients for instagram service |
| controllers |
api implementation |
| ig_bots |
ig_bots cassandra's keyspace client |
- this section lists the 10-% code added after vangav backend generated 90+% of the code
| file/dir |
change |
| prop |
added instagram properties files: constants and dispense |
| reverse_geo_coding |
added reverse geo coding data |
| Global.java |
added the initialization of reverse geo coding data |
| common |
added common operations like authentication, cassandra's frequent-operations, omitting blobs from requests/responses for logging, ... |
| controllers |
added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages |
| file/dir |
change |
| controllers |
added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages |
| file/dir |
change |
| prop |
added instagram jobs properties files: constants, users rank and posts rank |
| reverse_geo_coding |
added reverse geo coding data |
| Global.java |
added the initialization of reverse geo coding data and periodic jobs |
| common |
added common operations like instagram-jobs-service's properties |
| periodic_jobs |
added the periodic jobs for retrying failed jobs as well as ranking users and posts |
| controllers |
added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages |
| file/dir |
change |
| common |
added common operations like service-wide constants |
| controllers |
added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages |
| file/dir |
change |
| common |
added common operations like representing instagram users |
| controllers |
added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages |
| file/dir |
change |
| setup_data |
added a one-time setup data with all the cities (id, location, population, ...) where the bots are created |
| Global.java |
added the initialization of cities and bots |
| cities |
added the cities' json representation and loader |
| bots |
added the bots' periodic jobs and the bots creator |
| controllers |
added the implementation of request processing logic under controller_name/HandlerControllerName.java classes and nested response json structures under controller_name/response_json packages |
- following are the error codes of instagram services
| class |
code : sub_code |
explanation |
| Constants |
|
|
|
300 : 1 |
couldn't initialize kAuthCodeLifeTime |
|
300 : 2 |
couldn't initialize kAccessTokenLifeTime |
|
300 : 3 |
couldn't initialize kFacebookAppId |
|
300 : 4 |
couldn't initialize kDefaultRegion |
|
300 : 5 |
couldn't initialize kWorldRegion |
|
300 : 6 |
couldn't initialize kDefaultGridId |
|
300 : 7 |
couldn't initialize kGeoGridDimensionMetres |
|
300 : 8 |
couldn't initialize kGeoGridsConfig |
|
300 : 9 |
couldn't initialize kThumbnailDimension |
| CommonPlayHandler |
|
|
|
400 : 1 |
no access token for user id and device token |
|
400 : 2 |
wrong access token |
|
400 : 3 |
already expired access token |
|
400 : 4 |
just expired access token |
| HandlerComment |
|
|
|
401 : 1 |
can't comment on a post that doesn't exist |
|
401 : 2 |
non-fatal: invalid device type |
| DeleteComment |
|
|
|
402 : 1 |
post/comment doesn't exist |
| HandlerFollow |
|
|
|
403 : 1 |
can't follow self |
|
403 : 2 |
can't follow an non-existing user |
|
403 : 3 |
already following user |
|
403 : 4 |
non-fatal: invalid device type |
| HandlerGetAccessToken |
|
|
|
404 : 1 |
auth code doesn't exist |
|
404 : 2 |
wrong auth code |
| HandlerGetFollowers |
|
|
|
406 : 1 |
can't get followers for non-existing user |
| HandlerGetFollowing |
|
|
|
407 : 1 |
can't get following for non-existing user |
| HandlerGetPhoto |
|
|
|
408 : 1 |
photo doesn't exist |
| HandlerGetPhotoThumbnail |
|
|
|
409 : 1 |
photo doesn't exist |
| HandlerGetPostComments |
|
|
|
410 : 1 |
post doesn't exist |
| HandlerGetPostInfo |
|
|
|
411 : 1 |
post doesn't exist |
| HandlerGetPostLikes |
|
|
|
412 : 1 |
post doesn't exist |
| HandlerGetPostPhotoId |
|
|
|
413 : 1 |
post doesn't exist |
| HandlerGetProfilePicture |
|
|
|
414 : 1 |
user doesn't exist |
| HandlerGetUserInfo |
|
|
|
418 : 1 |
user doesn't exist |
| HandlerGetUserPosts |
|
|
|
419 : 1 |
user doesn't exist |
| HandlerLikePost |
|
|
|
420 : 1 |
post doesn't exist |
|
420 : 2 |
already liked |
|
420 : 3 |
non-fatal: invalid device type |
| HandlerLoginEmail |
|
|
|
421 : 1 |
e-mail isn't signed up |
|
421 : 2 |
wrong password |
|
421 : 3 |
e-mail isn't signed up - triggers in case authentication is disabled |
| HandlerLoginFacebook |
|
|
|
422 : 1 |
couldn't get user's name from facebook graph api |
| HandlerLogout |
|
|
|
423 : 1 |
no refresh token for this user/device_token |
|
423 : 2 |
wrong refresh token |
| HandlerRefreshAccessToken |
|
|
|
425 : 1 |
no access token for user |
|
425 : 2 |
wrong access token |
|
425 : 3 |
no refresh token |
|
425 : 4 |
wrong refresh token |
| HandlerSignupEmail |
|
|
|
427 : 1 |
e-mail already signed up |
| HandlerUnfollow |
|
|
|
428 : 1 |
can't unfollow a user that doesn't exist |
|
428 : 2 |
a user can't unfollow another unless she/he is following that user already |
| HandlerUnlikePost |
|
|
|
429 : 1 |
can't unlike a post that doesn't exist |
|
429 : 2 |
user didn't like this post before to unlike it |
| class |
code : sub_code |
explanation |
| Constants |
|
|
|
1000 : 1 |
couldn't initialize kVosInstagramUrl |
|
1000 : 2 |
couldn't initialize kVosInstagramDashBoardUrl |
| class |
code : sub_code |
explanation |
| Constants |
|
|
|
1000 : 1 |
couldn't initialize kVosInstagramUrl |
on YouTube
