Use lambda instead of functools.partial to create single-host config maker#1152
Use lambda instead of functools.partial to create single-host config maker#1152yhtang wants to merge 1 commit intoapple:mainfrom
Conversation
|
I have not encountered this issue. How can I reproduce it? |
| # Make single-host config | ||
| make_single_host_config_func = functools.partial(make_single_host_config, config_name) | ||
| config_map[f"{config_name}-single-host"] = make_single_host_config_func | ||
| config_map[f"{config_name}-single-host"] = lambda: make_single_host_config(config_name) |
There was a problem hiding this comment.
IIUC, this won't work as is due to how closures work in Python?
| config_map[f"{config_name}-single-host"] = lambda: make_single_host_config(config_name) | |
| config_map[f"{config_name}-single-host"] = lambda config_name=config_name: make_single_host_config(config_name) |
There was a problem hiding this comment.
That should work just fine, like how we captured things in the idiomatic way of defining decorators.
Here is a simple test that captures the pattern:
def f():
x = 1
return lambda: print(x)
g = f()
g()
which correctly prints 1 when we called g().
There was a problem hiding this comment.
Maybe I missed something, but why doesn't the code in the PR exhibit the same problem as the following code:
fns = []
for i in range(3):
fns.append(lambda: i)
print( fns[0]() ) # Incorrectly prints 2 instead of 0.
I agree your snippet doesn't have this issue because the value being closed over is not modified.
This is only triggered if using one of the *-single-host configs, e.g. |
Yeah I have been using single-host config for my testing on TPU recently. |
|
This pull request has been automatically marked as stale because it has been inactive for 60 days. It will be closed in 7 days if no further activity occurs. If you would like to continue working on this, please remove the |
|
This pull request has been automatically marked as stale because it has been inactive for 60 days. It will be closed in 7 days if no further activity occurs. If you would like to continue working on this, please remove the |
|
This pull request was closed because it has been inactive for more than 7 days since being marked as stale. Please feel free to reopen it if you would like to continue. |
4 participants
Fixes #1151
An alternative solution is to use
functools.wrapsbut that requires a custom config validator: