Enable remote inference non streaming response

[srinarayan-srikanthan]

Description

Enable remote inference non streaming response

Issues

Na

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)
• Others (enhancement, documentation, validation, etc.)

Dependencies

na

Tests

na

[codecov]

Codecov Report

Attention: Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
comps/cores/mega/micro_service.py	0.00%	1 Missing ⚠️

Files with missing lines	Coverage Δ
comps/cores/mega/orchestrator.py	90.43% <100.00%> (ø)
comps/cores/mega/micro_service.py	79.43% <0.00%> (+0.73%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[letonghan]

LGTM, please check the failed CI test, thanks.

[srinarayan-srikanthan]

@letonghan @louie-tsai @chensuyue and @lvliang-intel , need help with this PR to be merged. The test is failing, timing out after 1hr. Not sure what the issue is. Not related to the changes in this PR.

[chensuyue]

@letonghan @louie-tsai @chensuyue and @lvliang-intel , need help with this PR to be merged. The test is failing, timing out after 1hr. Not sure what the issue is. Not related to the changes in this PR.

Yes, the CI issue is known issue, #1725

[srinarayan-srikanthan]

So can this PR be merged, two other PRs are dependent on this. @chensuyue @lvliang-intel @ftian1 @letonghan