[FLINK-36245][FOLLOWUP] Remove the @Deprecated for ArrowSourceFunction and SinkFunction #26297

beliefer · 2025-03-14T13:20:33Z

What is the purpose of the change

This PR follows up #25331
It seems missing the two classes.

Brief change log

Remove the @deprecated for ArrowSourceFunction and SinkFunction

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)

flinkbot · 2025-03-14T13:35:56Z

CI report:

c3bcbda Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

beliefer · 2025-03-14T13:52:43Z

ping @lvyanquan @1996fanrui @davidradl cc @twalthr @JunRuiLee @afedulov

beliefer · 2025-03-15T02:13:09Z

flink-runtime/src/main/java/org/apache/flink/streaming/api/datastream/DataStream.java

     */
+    @Deprecated


To fix the issue.

2025-03-14T14:31:28.5664450Z Mar 14 14:31:28 14:31:28.565 [ERROR] Failures: 2025-03-14T14:31:28.5666940Z Mar 14 14:31:28 14:31:28.565 [ERROR] Architecture Violation [Priority: MEDIUM] - Rule 'Return and argument types of methods annotated with @Public must be annotated with @Public.' was violated (2 times): 2025-03-14T14:31:28.5669896Z Mar 14 14:31:28 org.apache.flink.streaming.api.datastream.DataStream.addSink(org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction): Argument leaf type org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction does not satisfy: reside outside of package 'org.apache.flink..' or reside in any package ['..shaded..'] or annotated with @Public or annotated with @Deprecated 2025-03-14T14:31:28.5673341Z Mar 14 14:31:28 org.apache.flink.streaming.api.datastream.KeyedStream.addSink(org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction): Argument leaf type org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction does not satisfy: reside outside of package 'org.apache.flink..' or reside in any package ['..shaded..'] or annotated with @Public or annotated with @Deprecated

…n and SinkFunction

beliefer · 2025-03-15T06:38:59Z

flink-runtime/src/main/java/org/apache/flink/streaming/api/datastream/KeyedStream.java

@@ -301,6 +301,7 @@ protected <R> SingleOutputStreamOperator<R> doTransform(
    }

    @Override
+    @Deprecated


To fix the issue.

2025-03-15T03:40:51.6629822Z Mar 15 03:40:51 03:40:51.661 [ERROR] Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.56 s <<< FAILURE! -- in org.apache.flink.architecture.rules.ApiAnnotationRules 2025-03-15T03:40:51.6631130Z Mar 15 03:40:51 03:40:51.662 [ERROR] ApiAnnotationRules.PUBLIC_API_METHODS_USE_ONLY_PUBLIC_API_TYPES -- Time elapsed: 0.118 s <<< FAILURE! 2025-03-15T03:40:51.6631774Z Mar 15 03:40:51 java.lang.AssertionError: 2025-03-15T03:40:51.6632704Z Mar 15 03:40:51 Architecture Violation [Priority: MEDIUM] - Rule 'Return and argument types of methods annotated with @Public must be annotated with @Public.' was violated (1 times): 2025-03-15T03:40:51.6634529Z Mar 15 03:40:51 org.apache.flink.streaming.api.datastream.KeyedStream.addSink(org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction): Argument leaf type org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction does not satisfy: reside outside of package 'org.apache.flink..' or reside in any package ['..shaded..'] or annotated with @Public or annotated with @Deprecated

Poorvankbhatia

Hey @beliefer . We are in the process of Rewriting the Arrow Source Function, i.e moving it to the new Source API. So should we continue this deprecated for now. Here is the ticket
cc: @afedulov wdyt?

beliefer · 2025-03-17T06:29:34Z

@Poorvankbhatia After the rewrite, will the behavior changed? If yes, I think we should continue this deprecated pr.

Poorvankbhatia · 2025-03-17T09:04:52Z

@Poorvankbhatia After the rewrite, will the behavior changed? If yes, I think we should continue this deprecated pr.

I think the core functionality of the source should remain the same, but the design is still WIP. So IMO we can keep the @deprecated annotation on ArrowSourceFunction

beliefer · 2025-03-17T09:48:36Z

IMO, I think this PR is not related to the rewrite work.
This PR is just to fix bug that both deprecated with annotation and comment.

davidradl · 2025-03-20T14:30:46Z

flink-runtime/src/main/java/org/apache/flink/streaming/api/datastream/DataStream.java

@@ -864,7 +864,11 @@ protected DataStream<T> setConnectionType(StreamPartitioner<T> partitioner) {
     *
     * @param sinkFunction The object containing the sink's invoke function.
     * @return The closed DataStream.
+     * @deprecated This method relies on the {@link SinkFunction} API, which is due to be removed.


I see text "due to be removed." It will not be removed until at least Flink v3.
Idoes not look right to have a deprecated method referencing parameters are annotated with internal

Yes. Because the SinkFunction will be removed in future versions, this function depends it.

ApiAnnotationRules lead to the error https://github.com/apache/flink/pull/26297/files#r1996509218
It means we should add @Deprecated

davidradl · 2025-03-20T14:45:50Z

...runtime/src/main/java/org/apache/flink/streaming/api/functions/sink/legacy/SinkFunction.java

@@ -32,7 +32,6 @@
 *     org.apache.flink.api.connector.sink2.Sink} interface instead.
 */
 @Internal
-@Deprecated


why are we changing this tag?
It is was part of the legacy deprecated API - should this not be deprecated?

It should still be deprecated. I remove it since the annotation @deprecated duplicate with the comments @deprecated. And I follow up the #25331

flinkbot added the component=Connectors/Common label Mar 14, 2025

beliefer force-pushed the FLINK-36245_followup branch from 2c2840a to 14433b6 Compare March 15, 2025 02:11

beliefer commented Mar 15, 2025

View reviewed changes

[FLINK-36245][FOLLOWUP] Remove the @deprecated for ArrowSourceFunctio…

c3bcbda

…n and SinkFunction

beliefer force-pushed the FLINK-36245_followup branch from 14433b6 to c3bcbda Compare March 15, 2025 06:38

beliefer commented Mar 15, 2025

View reviewed changes

Poorvankbhatia suggested changes Mar 15, 2025

View reviewed changes

davidradl reviewed Mar 20, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FLINK-36245][FOLLOWUP] Remove the @Deprecated for ArrowSourceFunction and SinkFunction #26297

[FLINK-36245][FOLLOWUP] Remove the @Deprecated for ArrowSourceFunction and SinkFunction #26297

beliefer commented Mar 14, 2025

flinkbot commented Mar 14, 2025 •

edited

Loading

beliefer commented Mar 14, 2025

beliefer Mar 15, 2025

beliefer Mar 15, 2025

Poorvankbhatia left a comment •

edited

Loading

beliefer commented Mar 17, 2025

Poorvankbhatia commented Mar 17, 2025

beliefer commented Mar 17, 2025

davidradl Mar 20, 2025 •

edited

Loading

beliefer Mar 21, 2025 •

edited

Loading

davidradl Mar 20, 2025

beliefer Mar 21, 2025

[FLINK-36245][FOLLOWUP] Remove the @Deprecated for ArrowSourceFunction and SinkFunction #26297

Are you sure you want to change the base?

[FLINK-36245][FOLLOWUP] Remove the @Deprecated for ArrowSourceFunction and SinkFunction #26297

Conversation

beliefer commented Mar 14, 2025

What is the purpose of the change

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

flinkbot commented Mar 14, 2025 • edited Loading

CI report:

beliefer commented Mar 14, 2025

beliefer Mar 15, 2025

Choose a reason for hiding this comment

beliefer Mar 15, 2025

Choose a reason for hiding this comment

Poorvankbhatia left a comment • edited Loading

Choose a reason for hiding this comment

beliefer commented Mar 17, 2025

Poorvankbhatia commented Mar 17, 2025

beliefer commented Mar 17, 2025

davidradl Mar 20, 2025 • edited Loading

Choose a reason for hiding this comment

beliefer Mar 21, 2025 • edited Loading

Choose a reason for hiding this comment

davidradl Mar 20, 2025

Choose a reason for hiding this comment

beliefer Mar 21, 2025

Choose a reason for hiding this comment

flinkbot commented Mar 14, 2025 •

edited

Loading

Poorvankbhatia left a comment •

edited

Loading

davidradl Mar 20, 2025 •

edited

Loading

beliefer Mar 21, 2025 •

edited

Loading