-
Notifications
You must be signed in to change notification settings - Fork 141
SNOW-2866776: add private var in numeric type #4022
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
src/snowflake/snowpark/types.py
Outdated
| def __init__(self, **kwargs) -> None: | ||
| self._precision = kwargs.get("precision", None) | ||
| self._scale = kwargs.get("scale", None) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| def __init__(self, **kwargs) -> None: | |
| self._precision = kwargs.get("precision", None) | |
| self._scale = kwargs.get("scale", None) | |
| def __init__(self, precision: int | None = None, scale: int | None = None) -> None: | |
| self._precision = precision | |
| self._scale = scale |
Can we do this style instead to make it more explicit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is a internal only feature, I think we don't want to expose this information to customer?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, we can just keep it as kwargs then.
What do you think about moving the arguments onto _IntegralType instead of _NumericType? It looks like Snowflake only has a single precision for FLOAT and DECIMAL (docs), so we should never have to expose precision information for those types.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 on using _IntegralType, I don't think we need _scale as it would always be 0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i see, will fix this
|
I modified eq function to exclude _precision and _scale |
src/snowflake/snowpark/modin/plugin/_internal/snowpark_pandas_types.py
Outdated
Show resolved
Hide resolved
src/snowflake/snowpark/types.py
Outdated
| def __eq__(self, other): | ||
| def filtered(d: dict) -> dict: | ||
| return {k: v for k, v in d.items() if k not in ("_precision", "_scale")} | ||
|
|
||
| return isinstance(other, self.__class__) and filtered( | ||
| self.__dict__ | ||
| ) == filtered(other.__dict__) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why we overwrite the logic to exclude the new prop?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
here is a example test:
expected_fields = [
StructField("COL1", LongType(), nullable=True),
StructField("COL2", LongType(), nullable=True),
]
assert df.schema.fields == expected_fields
we exclude the private var so that these existing test does not fail
src/snowflake/snowpark/modin/plugin/_internal/snowpark_pandas_types.py
Outdated
Show resolved
Hide resolved
| ) == filtered(other.__dict__) | ||
|
|
||
| def __hash__(self): | ||
| return hash(repr(self)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need a custom hash implementation for this class?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TL:DR: this is to fix some test failure
when you defined eq, you need to redefine hash otherwise it is considered None
src/snowflake/snowpark/types.py
Outdated
| def filtered(d: dict) -> dict: | ||
| return {k: v for k, v in d.items() if k != "_precision"} | ||
|
|
||
| return isinstance(other, self.__class__) and filtered( | ||
| self.__dict__ | ||
| ) == filtered(other.__dict__) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think whether excluding precision or not should be based upon the context._is_snowpark_connect_mode.
scos they might want to distinguish LongType by precision
|
|
||
| if kwargs != {}: | ||
| raise TypeError( | ||
| f"__init__() takes 0 argument but {len(kwargs.keys())} were given" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Grammar Error in Exception Message
The error message uses singular "argument" but will be grammatically incorrect when multiple arguments are passed.
# Current: "takes 0 argument but 2 were given" (incorrect grammar)Fix: Use proper singular/plural form:
arg_word = "argument" if len(kwargs) == 1 else "arguments"
raise TypeError(
f"__init__() takes 0 arguments but {len(kwargs)} {arg_word if len(kwargs) != 1 else 'was'} given"
)Or simply use plural consistently:
raise TypeError(
f"__init__() takes 0 arguments but {len(kwargs)} were given"
)Spotted by Graphite Agent
Is this helpful? React 👍 or 👎 to let us know.
|
I still kept the eq function in TimedeltaType class. |
Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-2866776
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
https://snowpark-python-001.jenkinsdev1.us-west-2.aws-dev.app.snowflake.com/job/SnowparkConnectRegressionRunner/220/
scos test