Integrate with addon that supports yomitan frequency

[weirdalsuperfan]

Can you integrate with this new addon (or copy its concept)? https://discord.com/channels/617136488840429598/778430038159655012/1380666667117445281

It allows you to sort your cards using yomitan-formatted frequency lists, either individually or in combination (you can choose how they're combined, with harmonic mean, lowest number, etc.).

Direct link to addon is here, but more info is in the discord link b/c I guess they haven't added much description on Ankiweb yet.

The advantages of this would be:

• ability to sort based on multiple lists weighted together
• future-proofing when yomitan frequency lists are refined or new ones come out
• allowing people easier access to frequency lists besides the default ones without everyone having to convert them from yomitan individually

[mortii]

i don't have discord, can you give a summary instead?

[weirdalsuperfan]

This is a copy-and-paste of the person's discord post:

Hello I wrote an addon that integrates Yomitan frequency dictionaries directly within Anki.

• Compatible with any Yomitan dictionary (only tested Japanese ones though), individually toggleable
• Different average calculation methods (harmonic mean, minimum value, etc.)
• Doesn't touch the mined frequency field, but I can add the option to backfill if people want
• Only requires word field and optionally reading field
• Runs in 2-3 seconds
• Option to sort on startup (remember to disable autoreorder)

https://ankiweb.net/shared/info/295700265

[mortii]

Can you provide an example yomitan-formatted frequency list?

[weirdalsuperfan]

Here are a few.

1. JPDB v2.2 Frequency Kana: JPDB_v2.2_Frequency_Kana_2024-10-13.zip

"Displays the frequency of the word and the frequency of the word being written in kana (denoted by ㋕). Matches the format of the original JPDB frequency dictionary.

Preview: 読む -> JPDBv2㋕ 395, 19886㋕, よむ -> JPDBv2㋕ 19886㋕, 仮名 -> JPDBv2㋕ 117㋕, 34102"

More info here (source):
https://github.com/Kuuuube/yomitan-dictionaries

2. "Corpus of Everyday Japanese Conversation - The Corpus of Everyday Japanese Conversation (CEJC) is a vocabulary and word count table based on 200 hours of recorded data (approximately from April 2016 to 2020).": Corpus of Everyday Japanese Conversation.zip

More info here (source):
https://github.com/forsakeninfinity/CEJC_yomichan_freq_dict

3. BCCWJ SUW LUW Combined - "Frequency list made from the Balanced Corpus of Contemporary Written Japanese. ": BCCWJ_SUW_LUW_combined.zip

LUW = Long Unit Word
SUW = Short Unit Word

"BCCWJ SUW and BCCWJ LUW combined into one dictionary to save space on the Yomitan popup."

More info here (source):
https://github.com/Kuuuube/yomitan-dictionaries?tab=readme-ov-file#bccwj-suw-luw-combined

4. Youtube frequency: [JA Freq] YoutubeFreqV3.zip

"New frequency dictionary from YouTube video"s

More info here (source):
https://github.com/MarvNC/yomitan-dictionaries?tab=readme-ov-file#youtube-frequency-dictionaries

5. Youtube frequency - Vlogs only: YouTubeVlogs.zip

"Using data from 40k manually transcribed YouTube videos we have created 16 domain specific frequency lists for YomiChan. Enjoy and feel free to share around. Created by @ Zetta @ Vexxed @ Anonymous"

More info here (source):
https://discord.com/channels/617136488840429598/778430038159655012/933392362472865832

[NaabZer]

I actually just wrote a small jupyter-notebook that extracted all sentences in my anki decks, which I then run through the priority generator, and then sorted them/gave priority based on the average of above mentioned frequency dictionaries. It was originally inspired by MarvNC/JP-Resources which contains a CLI-script to backfill frequencies in to an Anki field.

Here's my repo if you wanna have a look how I extracted the frequencies from the dictionaries (relevants part are in cell 13 & 14 of the jupyter notebook). They have slightly different formatting, but generally pretty simple:
NaabZer/anki-morph-frequency-prio

I'd be down to look into/help adding this functionality, but could use some guidance for where to start looking in that case (completely new to Anki addons, very comfortable in python)

For some more info about what the dictionaries looks like, they're all JSON encoded lists, where each item looks like this
[word, 'freq', freq_obj], where word is the word, 'freq' is just a string that's common for position two (probably to differentiate different vocabularies), and freq_obj is a bit different for different dictionaries, containing the actual frequencies. Here's some examples of the four types of freq_obj i saw in the dictionaries (these examples are the entire entry):

• ["要る", "freq", 1]
• ["の","freq",{"reading":"の","frequency":1}]
• ["の","freq",{"value":1,"displayValue":"1㋕"}]
• ['思う', 'freq', {'reading': 'おもう', 'frequency': {'value': 20, 'displayValue': '20, 9493㋕'}}]

[mortii]

I actually just wrote a small jupyter-notebook that extracted all sentences in my anki decks, which I then run through the priority generator, and then sorted them/gave priority based on the average of above mentioned frequency dictionaries.

very cool!

I'm pretty much done with implementing new features myself, but I'm still open to merging pull requests if someone else wants to do it (given it doesn't add much technical debt).

There are some dev guides that might be useful for anyone that wants to contribute: https://mortii.github.io/anki-morphs/developer_guide/setup.html

This is the probably the most relevant starting point for this particular issue:

anki-morphs/ankimorphs/morph_priority_utils.py

Lines 69 to 93 in 4f5ae23

	def _load_morph_priorities_from_file(
	priority_file_name: str, only_lemma_priorities: bool
	) -> dict[tuple[str, str], int]:
	assert mw is not None
	priority_file_path = Path(
	mw.pm.profileFolder(),
	am_globals.PRIORITY_FILES_DIR_NAME,
	priority_file_name,
	)
	try:
	with open(priority_file_path, encoding="utf-8") as csvfile:
	morph_reader = csv.reader(csvfile, delimiter=",")
	headers: list[str] | None = next(morph_reader, None)
	priority_file: PriorityFile = _get_file_type_and_format(
	priority_file_path, headers
	)
	return _get_morph_priorities_from_file(
	priority_file_path=priority_file_path,
	morph_reader=morph_reader,
	priority_file=priority_file,
	only_lemma_priorities=only_lemma_priorities,
	)
	except FileNotFoundError as exc:
	raise PriorityFileNotFoundException(str(priority_file_path)) from exc