Skip to content
This repository was archived by the owner on Mar 9, 2023. It is now read-only.

Commit 6276fa9

Browse files
authored
Fix connection cost lookup, fix #129 (#131)
* Fix connection cost lookup, fix #129 The connection costs lookup was backwards. There was a comment in the pre-cython code that the call to the cost lookup function looked backwards, but was actually correct. It was calling with (l_node.right_id, r_node.left_id). I kept this order when I replaced the function call with a memoryview access, but that was wrong; I hadn't noticed that the access function was actually reversing the order of its arguments. This fix was verified by checking the tokenization of a short document vs v0.4.5 and making sure there were no changes. * Remove old comment
1 parent 1c4ba98 commit 6276fa9

File tree

1 file changed

+1
-2
lines changed

1 file changed

+1
-2
lines changed

sudachipy/lattice.pyx

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -94,8 +94,7 @@ cdef class Lattice:
9494
for l_node in self.end_lists[begin]:
9595
if not l_node.is_connected_to_bos:
9696
continue
97-
# right_id and left_id look reversed, but it works ...
98-
connect_cost = self.connect_costs[l_node.right_id, r_node.left_id]
97+
connect_cost = self.connect_costs[r_node.left_id, l_node.right_id]
9998

10099
# 0x7fff == Grammar.INHIBITED_CONNECTION:
101100
if connect_cost == 0x7fff:

0 commit comments

Comments
 (0)