Skip to content

Commit

Permalink
Export players table
Browse files Browse the repository at this point in the history
Contains only the *current* user_id to name mapping for players
that ever played in any public match.
  • Loading branch information
p2004a committed Sep 21, 2024
1 parent ae236aa commit 2acda02
Show file tree
Hide file tree
Showing 6 changed files with 35 additions and 0 deletions.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,13 @@ under:

- https://data-marts.beyondallreason.dev/matches.parquet
- https://data-marts.beyondallreason.dev/match_players.parquet
- https://data-marts.beyondallreason.dev/players.parquet

and Compressed CSV file under:

- https://data-marts.beyondallreason.dev/matches.csv.gz
- https://data-marts.beyondallreason.dev/match_players.csv.gz
- https://data-marts.beyondallreason.dev/players.csv.gz

More documentation is available at
https://beyond-all-reason.github.io/data-processing/.
Expand Down
1 change: 1 addition & 0 deletions models/intermediate/schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,3 +114,4 @@ sources:
- name: teiserver_battle_match_memberships
- name: teiserver_battle_matches
- name: teiserver_game_rating_logs
- name: teiserver_users
4 changes: 4 additions & 0 deletions models/intermediate/tei_players.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
SELECT
id AS user_id,
name
FROM {{ source('pgdumps', 'teiserver_users') }}
14 changes: 14 additions & 0 deletions models/marts/players.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{{ config(location='data_export/players.parquet') }}

WITH
active_players AS (
SELECT DISTINCT user_id
FROM {{ ref("match_players") }}
)
SELECT
user_id,
name
FROM active_players
INNER JOIN {{ ref("tei_players")}}
USING (user_id)
ORDER BY user_id
13 changes: 13 additions & 0 deletions models/marts/schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,3 +88,16 @@ models:
Faction that was used, important note is that it can be null, and is
also the faction selected in lobby. In additon, faction like Legion
are not recognized here. Use with caution, not high quality field.
- name: players
description: List of active players
columns:
- name: user_id
description: Teiserver account id of the user
data_tests:
- unique
- not_null
- name: name
description: Current user selected name
- unique
- not_null
1 change: 1 addition & 0 deletions scripts/build_more_formats.sql
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@

COPY 'data_export/matches.parquet' TO 'data_export/matches.csv.gz';
COPY 'data_export/match_players.parquet' TO 'data_export/match_players.csv.gz';
COPY 'data_export/players.parquet' TO 'data_export/players.csv.gz';

0 comments on commit 2acda02

Please sign in to comment.