Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new approach in public-adventures #5431

Draft
wants to merge 57 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
fbf7399
WIP use indexes and filters tables
hasan-sh Apr 17, 2024
526ac0d
temporary fiels for creating new tables
hasan-sh Apr 17, 2024
f043f03
- remove filters
hasan-sh Apr 18, 2024
a7eacfb
needed changes
hasan-sh Apr 18, 2024
997a7ad
always filter by own language
hasan-sh Apr 22, 2024
5ab56fe
sort levels and filter by search field too
hasan-sh Apr 22, 2024
b36c08b
remove duplicates cloned adventures
hasan-sh Apr 22, 2024
f16c649
Merge branch 'main' into new-PA-approach
hasan-sh Apr 22, 2024
e53ac4b
bundle file
hasan-sh Apr 22, 2024
d3bff69
change field#value to field_value such that it works with dynamo's st…
hasan-sh Apr 22, 2024
42ae51d
fix tests
hasan-sh Apr 22, 2024
897f1e9
remove old files
hasan-sh Apr 22, 2024
912c199
🤖 Automatically update generated files
hasan-sh Apr 22, 2024
971e464
migrate scripts
hasan-sh Apr 23, 2024
7cf47b7
remove filters and indexes when removing an adventure
hasan-sh Apr 23, 2024
28f1c4a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 23, 2024
dbab774
- added pagination token
hasan-sh May 6, 2024
b96a187
Merge branch 'main' into new-PA-approach
hasan-sh May 6, 2024
a834e1f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 6, 2024
0e56cb3
🤖 Automatically update generated files
pre-commit-ci[bot] May 6, 2024
17b7eec
Merge branch 'main' into new-PA-approach
hasan-sh May 8, 2024
35977bd
minor fixes
hasan-sh May 8, 2024
1622054
🤖 Automatically update generated files
hasan-sh May 8, 2024
5d324f2
fix tests 1
hasan-sh May 13, 2024
e29f999
Merge branch 'main' into new-PA-approach
hasan-sh May 13, 2024
4c125f4
🤖 Automatically update generated files
hasan-sh May 13, 2024
d9d8c91
fix flake
hasan-sh May 13, 2024
3c5c670
Merge branch 'new-PA-approach' of github.com:hedyorg/hedy into new-PA…
hasan-sh May 13, 2024
e792dc5
remove key field (id) when updating
hasan-sh May 13, 2024
07fe6ac
Merge branch 'main' into new-PA-approach
hasan-sh May 21, 2024
3d18927
sort tags when creating them
hasan-sh May 22, 2024
c17729d
🤖 Automatically update generated files
hasan-sh May 22, 2024
8dfa0b3
use get_page method, add prev page button and sort adventures by name
hasan-sh May 23, 2024
f87090b
Merge branch 'new-PA-approach' of github.com:hedyorg/hedy into new-PA…
hasan-sh May 23, 2024
706635c
Merge branch 'main' into new-PA-approach
hasan-sh May 29, 2024
5c5deb0
🤖 Automatically update generated files
hasan-sh May 29, 2024
283fc33
remove unneeded code
hasan-sh May 29, 2024
45bba2c
Merge branch 'new-PA-approach' of github.com:hedyorg/hedy into new-PA…
hasan-sh May 29, 2024
a2c1183
remove race conditions
hasan-sh May 29, 2024
7cf0004
Merge branch 'main' into new-PA-approach
hasan-sh May 29, 2024
2245704
update filters/indexes logic in update_adventures + docstring
hasan-sh May 30, 2024
2cc8121
move logic to db and remove old code
hasan-sh May 30, 2024
5459e58
Merge branch 'main' into new-PA-approach
hasan-sh May 30, 2024
32b10b2
remove htmx
hasan-sh May 30, 2024
01cc74b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 30, 2024
aeca1c1
1 endpoint
hasan-sh May 30, 2024
2e03ff6
data
hasan-sh May 30, 2024
d3c521b
Merge branch 'new-PA-approach' of github.com:hedyorg/hedy into new-PA…
hasan-sh May 30, 2024
4d93779
🤖 Automatically update generated files
hasan-sh May 30, 2024
a9a0f46
Merge branch 'main' into new-PA-approach
hasan-sh Jun 3, 2024
a7671cf
rename tables and small fixes
hasan-sh Jun 3, 2024
b7079f9
add filter to path
hasan-sh Jun 3, 2024
158b8ac
Merge branch 'new-PA-approach' of github.com:hedyorg/hedy into new-PA…
hasan-sh Jun 3, 2024
1822686
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 3, 2024
ced1c09
return body.html not index
hasan-sh Jun 3, 2024
abf0554
fix search field
hasan-sh Jun 3, 2024
4209b7e
Merge branch 'main' into new-PA-approach
hasan-sh Jun 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions create-public-adventures-filters.py
hasan-sh marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
from website.database import PUBLIC_ADVENTURES_FILTERS, ADVENTURES


def create_filters_table(record):
lang_value = record.get("language", "en")
levels = record.get("levels")
if not levels:
levels = [record.get("level")]
for level in levels:
value = f"{lang_value}#{level}"
PUBLIC_ADVENTURES_FILTERS.put({"field": "lang#level", "value": value})
tags = record.get("tags", [])
for tag in tags:
PUBLIC_ADVENTURES_FILTERS.put({"field": "tag", "value": tag})


def get_filters():
filters = PUBLIC_ADVENTURES_FILTERS.get_all({"field": "lang#level"})
for f in filters:
print(f)


def main():
for record in ADVENTURES.get_all({"public": 1}):
create_filters_table(record)
get_filters()


if __name__ == "__main__":
main()
42 changes: 42 additions & 0 deletions create-public-adventures-index.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
from website.database import PUBLIC_ADVENTURES_INDEXES, ADVENTURES


def create_multi_index_table(record):
lang_value = record.get("language", "en")
id = record.get("id")
date = record.get("date")
PUBLIC_ADVENTURES_INDEXES.put({"field#value": f"lang#{lang_value}", "date#adventure_id": f"{date}#{id}"})
levels = record.get("levels")
if not levels:
levels = [record.get("level")]
for level in levels:
PUBLIC_ADVENTURES_INDEXES.put({"field#value": f"level#{level}", "date#adventure_id": f"{date}#{id}"})
tags = record.get("tags", [])
for tag in tags:
PUBLIC_ADVENTURES_INDEXES.put({"field#value": f"tag#{tag}", "date#adventure_id": f"{date}#{id}"})


def get_indexes():
indexes = PUBLIC_ADVENTURES_INDEXES.get_all({"field#value": "lang#en"})
filter_fields = {}
for i in indexes:
field, value = i.get("field#value").split("#")
if filter_fields.get(field):
filter_fields[field].append(value)
else:
filter_fields[field] = [value]

time_id = i.get("date#adventure_id")

print(field, value)
print(time_id)


def main():
for record in ADVENTURES.get_all({"public": 1}):
create_multi_index_table(record)
# get_indexes()


if __name__ == "__main__":
main()
2 changes: 1 addition & 1 deletion templates/public-adventures/body.html
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@
add .rotate-180 to arrow
end">
<div class="option default"
hx-post="/public-adventures/filter?lang=reset&level={{selectedLevel}}&tag={{selectedTag}}&search={{currentSearch}}"
hx-post="/public-adventures/filter?lang=&level={{selectedLevel}}&tag={{selectedTag}}&search={{currentSearch}}"
hx-target="#public-adventures"
value="">{{_('select_lang')}}</div>
{% for lang in available_languages %}
Expand Down
10 changes: 10 additions & 0 deletions website/database.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@
ADVENTURES = dynamo.Table(storage, "adventures", "id", indexes=[
dynamo.Index("creator"), dynamo.Index("public"),
dynamo.Index("name", sort_key="creator", index_name="name-creator-index")])
PUBLIC_ADVENTURES_INDEXES = dynamo.Table(storage, "public-adventures-indexes",
"field#value", sort_key="date#adventure_id")
PUBLIC_ADVENTURES_FILTERS = dynamo.Table(storage, "public-adventures-filters", "field", sort_key="value")
INVITATIONS = dynamo.Table(
storage, "invitations", partition_key="username#class_id",
indexes=[dynamo.Index("username"), dynamo.Index("class_id")],
Expand Down Expand Up @@ -608,6 +611,13 @@ def batch_get_adventures(self, adventure_ids):
keys = {id: {"id": id} for id in adventure_ids}
return ADVENTURES.batch_get(keys) if keys else {}

def get_public_adventures_filters(self, key):
return PUBLIC_ADVENTURES_FILTERS.get_all(key)
hasan-sh marked this conversation as resolved.
Show resolved Hide resolved

def get_public_adventures_indexes(self, key):
print("\n\n get indexes", key)
return PUBLIC_ADVENTURES_INDEXES.get_all(key)

def get_public_adventures(self):
return ADVENTURES.get_many({"public": 1})

Expand Down
203 changes: 152 additions & 51 deletions website/public_adventures.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,67 +31,168 @@ def __init__(self, db: Database, achievements: Achievements):
self.available_languages = set()
self.available_tags = set()

def init(self, user):
included = {}
self.adventures = {}
self.available_languages = set()
self.available_tags = set()

public_adventures = self.db.get_public_adventures()
public_adventures = sorted(public_adventures, key=lambda a: a["creator"] == user["username"], reverse=True)
for adventure in public_adventures:
adv_lang = adventure.get("language", g.lang)
adv_tags = adventure.get("tags", [])
self.available_languages.update([adv_lang])
self.available_tags.update(adv_tags)
# NOTE: what if another author has an adventure with the same name?
# Perhaps we could make this name#creator!
if included.get(adventure["name"]):
continue
public_profile = self.db.get_public_profile_settings(adventure.get('creator'))
included[adventure["name"]] = True

content = safe_format(adventure.get('formatted_content', adventure['content']),
**hedy_content.KEYWORDS.get(g.keyword_lang))
current_adventure = {
"id": adventure.get("id"),
"name": adventure.get("name"),
"short_name": adventure.get("name"),
"author": adventure.get("author", adventure["creator"]),
"creator": adventure.get("creator"),
"creator_public_profile": public_profile,
"date": utils.localized_date_format(adventure.get("date")),
"level": adventure.get("level"),
"levels": adventure.get("levels"),
"language": adv_lang,
"cloned_times": adventure.get("cloned_times"),
"tags": adv_tags,
"text": content,
"is_teacher_adventure": True,
}

# save adventures for later usage.
for _level in adventure.get("levels", [adventure.get("level")]):
_level = int(_level)
if self.adventures.get(_level):
self.adventures[_level].append(current_adventure)
else:
self.adventures[_level] = [current_adventure]

available_levels = adventure["levels"] if adventure.get("levels") else [adventure["level"]]
self.customizations["available_levels"].update([int(adv_level) for adv_level in available_levels])

@route("/", methods=["GET"])
@route("/filter", methods=["POST"])
@requires_teacher
def filtering(self, user, index_page=False):
def filtering(self, user):
level = request.args["level"] if request.args.get("level") else "1"
language = request.args.get("lang", g.lang)
tag = request.args.get("tag", "")
search = request.form.get("search", request.args.get("search", ""))
hasan-sh marked this conversation as resolved.
Show resolved Hide resolved
print('\n\n\n', level, language, tag, search, '\n\n\n')

available_languages = set()
available_levels = set()
available_tags = set()

# Get all possible filters
lang_level_filters = self.db.get_public_adventures_filters({"field": "lang#level", })
for record in lang_level_filters:
lang, _level = record.get("value", "#").split("#")
available_levels.update([_level])
hasan-sh marked this conversation as resolved.
Show resolved Hide resolved
if level == _level:
available_languages.update([lang])
customizations = {"available_levels": available_levels}

tag_filters = self.db.get_public_adventures_filters({"field": "tag"})
hasan-sh marked this conversation as resolved.
Show resolved Hide resolved
for record in tag_filters:
_tag = record.get("value")
available_tags.update([_tag])

customizations = {"available_levels": available_levels}

tags = []
if tag:
toReset = request.args.get("reset")
if toReset:
# then it's the current selected tags, so remove current's tag
tags = [_tag for _tag in toReset.split(",") if _tag != tag]
else:
tags = tag.split(",")
tags = [t for t in tags if t]

# Get indexes
level_adventure_ids = self.db.get_public_adventures_indexes({"field#value": f"level#{level}"})
lang_adventure_ids = self.db.get_public_adventures_indexes({"field#value": f"lang#{language}"})
hasan-sh marked this conversation as resolved.
Show resolved Hide resolved
tag_adventure_ids = []
for _t in tags:
tag_adventure_ids += self.db.get_public_adventures_indexes({"field#value": f"tag#{_t}"})

level_adventure_ids = [record["date#adventure_id"].split("#")[1] for record in level_adventure_ids]
hasan-sh marked this conversation as resolved.
Show resolved Hide resolved
lang_adventure_ids = [record["date#adventure_id"].split("#")[1] for record in lang_adventure_ids]
tag_adventure_ids = [record["date#adventure_id"].split("#")[1] for record in tag_adventure_ids]

# Filter out based on level
lang_adventure_ids = [_id for _id in lang_adventure_ids if _id in level_adventure_ids]
tag_adventure_ids = [_id for _id in tag_adventure_ids if _id in level_adventure_ids]
hasan-sh marked this conversation as resolved.
Show resolved Hide resolved

adventure_ids = level_adventure_ids + lang_adventure_ids + tag_adventure_ids

print("ids all", adventure_ids)
adventures = self.db.batch_get_adventures(adventure_ids)

initial_tab = None
initial_adventure = None
commands = {}
prev_level = None
next_level = None

if adventures:
adventures = list(adventures.values())
initial_tab = adventures[0]["name"]
initial_adventure = adventures[-1]

# Add the commands to enable the language switcher dropdown
commands = hedy.commands_per_level.get(level)
print("\n\ncustomiz", customizations)
if customizations["available_levels"]:
prev_level, next_level = utils.find_prev_next_levels(list(customizations["available_levels"]), level)

customized_adventures = []
for adventure in adventures:
if language and adventure.get("language", g.lang) != language:
print("\n\n\nherererere", language, adventure.get("language"))
continue
if tags and not any(_t in adventure.get("tags", []) for _t in tags):
print("\n\n\nherererere tags", tag, tags, adventure.get("tags"))
continue

content = safe_format(adventure.get('formatted_content', adventure['content']),
**hedy_content.KEYWORDS.get(g.keyword_lang))
current_adventure = {
"id": adventure.get("id"),
"name": adventure.get("name"),
"short_name": adventure.get("name"),
"author": adventure.get("author", adventure["creator"]),
"creator": adventure.get("creator"),
"date": utils.localized_date_format(adventure.get("date")),
"level": adventure.get("level"),
"levels": adventure.get("levels"),
"language": adventure.get("language", g.lang),
"cloned_times": adventure.get("cloned_times"),
"tags": adventure.get("tags", []),
"text": content,
"is_teacher_adventure": True,
}
customized_adventures.append(current_adventure)
adventures = customized_adventures

print("\n\n\nADVENTURES", adventures)
js = dict(
page='code',
lang=g.lang,
level=level,
adventures=adventures,
initial_tab='',
current_user_name=user['username'],
)

temp = render_template(
f"public-adventures/{'index' if request.method == 'GET' else 'body'}.html",
adventures=adventures,
teacher_adventures=adventures,
available_languages=available_languages,
available_tags=available_tags,
selectedLevel=level,
selectedLang=language,
# selectedTag=",".join(self.selectedTag),
selectedTag=",".join(tags),
currentSearch=search,

user=user,
current_page="public-adventures",
page_title=gettext("title_public-adventures"),

initial_adventure=initial_adventure,
initial_tab=initial_tab,
commands=commands,
level=level,
max_level=18,
level_nr=str(level),
prev_level=prev_level,
next_level=next_level,

customizations=customizations,

public_adventures_page=True,
javascript_page_options=js,
)

response = make_response(temp)
response.headers["HX-Trigger"] = json.dumps({"updateTSCode": js})
return response

@route("/filter-old", methods=["POST"])
@requires_teacher
def filtering_old(self, user, index_page=False):
index_page = request.method == "GET"

level = int(request.args["level"]) if request.args.get("level") else 1
language = request.args.get("lang", "")
tag = request.args.get("tag", "")
search = request.form.get("search", request.args.get("search", ""))
if index_page or not self.adventures or not self.adventures.get(level):
print("\n\n\n GET IT MORE TIME!!\n\n", level, language, tag)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops?

self.init(user)

adventures = self.adventures.get(level, [])
Expand Down
Loading