ETA for TensorFlow Runtime Integration?

[jeongukjae]

Feature Request

May I know the expected release date of TensorFlow Serving with TensorFlow Runtime?

For the background, recently, I found that tensorflow runtime is integrated into tensorflow/serving codes. (2.15.0-rc0 release)

serving/RELEASE.md

Line 42 in 0bfaf6a

* Moves model server TFRT integration code oss (commit: 50ebab4ca601b5243b7aac674628954bef2d734b)

I'm really interested in this feature, so I tried to patch platform_config_utils.cc & server_init.cc, I could successfully build tensorflow serving with TFRT and inference arbitrary saved models (usually in CPU, transformer-based models).

Following the blog posts and GCP Vertex AI's guide, I expected that TFRT could show better performance.

• https://blog.tensorflow.org/2020/04/tfrt-new-tensorflow-runtime.html
• https://blog.tensorflow.org/2022/02/tfrt-progress-update.html
• https://cloud.google.com/vertex-ai/docs/predictions/optimized-tensorflow-runtime

But I think maybe due to plenty of tuning points, I couldn't get the great performance til now.
So it would be great if I could know when TF Serving with TFRT is released.