NEW: Bndtools 7.1/Eclipse Plugin: gRPC Remote Services Tooling Feature

This feature in the p2 repository in this repo here: https://raw.githubusercontent.com/ECF/grpc-RemoteServicesProvider/master/build/ is a Eclipse feature with name: ECF Remote Services Tooling Feature and id: org.eclipse.ecf.provider.grpc.tooling.feature.

To install this feature into Bndtools 7.1/Eclipse using p2 goto Help -> Install New Software...->Add...

Name: ECF Remote Services Tooling Feature

URL: https://raw.githubusercontent.com/ECF/grpc-RemoteServicesProvider/master/build/

Once added, select the added repository and uncheck the Group items by category checkbox in lower left. Then add the feature named: ECF Remote Services Tooling Feature.

NOTE: Once this Bndtools/Eclipse plugin is installed, to use the wizard it's necessary to create a new bndtools workspace using the ECF Remote Services Bndtools Workspace Template.

gRPC-RemoteServicesProvider

This project provides an ECF Remote Services Distribution Provider based upon Google RPC for Java (grpc-java). See http://www.grpc.io, https://github.com/grpc, and https://github.com/grpc/grpc-java for information on grpc-java, and https://wiki.eclipse.org/ECF for information about ECF's implentation of OSGi Remote Services.

This distribution provider uses grpc-java to export (server) and import (clients) OSGi Remote Services and make them available for remote access as full OSGi services; with all of the service dynamics, versioning, async, security, management, extensibility, and other features that come with the OSGi Remote Services and OSGi Remote Service Admin.

NEW: Videos Tutorials. This is a video tutorial showing how to use this workspace for gRPC development. In 4 parts: Part 1 - API Generation, Part 2 - Remote Service Implementation, Part 3 - Remote Service Consumer, Part 4 - Remote Service Debugging

New and Noteworthy

1. Support use of either ReactiveX version 2 (2.2.19), OR ReactiveX version 3 (3.1.1) in code generation (new). For an example of how to generate ReactiveX 3 java code from proto file via maven plugin, see the pom.xml in thie (new) example project: org.eclipse.ecf.examples.provider.grpc.rx3.health.api
2. Example ReactiveX 3 Impl and Consumer projects for Health API. See the Impl project here, and the consumer project here.
3. Use of gRPC version 1.39.0. The version of grpc being used by the distribution provider is now 1.39.0.
4. Support for gRPC ProtoReflectionService. gRPC 1.36 introduced a new service that allows clients to reflect on other services exposed by a server. This is very helpful for testing and debugging. There's a grpc tutorial for the proto reflection service here. The proto reflection service has been built in to the grpc remote services provider, and can be dynamically added/removed from an exported remote service via gogo console commands. See this wiki page for documentation.
5. Example Non-OSGi and Non-Java clients. One of the features of gRPC is the ability to use multiple languages for accessing a service. This capability also exists with this distribution provider, so that it's possible for OSGi servers to be accessed by Java-only clients (not OSGi)...or other languages supported by grpc. There is now an example project showing a Java-only (not OSGi) client for the HealthCheck service. This client works with the health check service impl example, which runs as a remote service in an OSGi environment (such as Karaf). See also this issue for other language example.
6. Bndtools Workspace - This distribution provider and examples are included as part of the ECF bndtools workspace template. The workspace template also includes 3 project templates: the gRPC HealthCheck API template, the gRPC HealthCheck Impl template and the gRPC HealthCheck Consumer template. The API project template has only an example proto file, from which bndtools will generate the java source code as part of the background compile sequence in Eclipse/bndtools. This makes it very easy to create an API and quickly modify proto files in an editor...and Eclipse/bndtools will immediately re-generate (and then compile) the java source code from the proto file. A video of using bndtools and these project templates is available here.

OSGi Remote Service API

OSGi Remote Services are typically declared by one or more Java interfaces representing the service contract(s). This service interface (and any classes referenced by this interface) is usually created by the service programmer to match the desired semantics of the service. The service interface and referenced classes represent the service api.

An implementation of this interface is typically created by the programmer and exported at runtime via one or more distribution providers.

See here for a short tutorial describing how to Create the Remote Service and Implement the Service.

In OSGi the service api is often separated into a distinct bundle from both the implementation classes (i.e. the service 'host' that actually implements and exports the service) and the service consumer that discovers, imports, and uses the service (i.e. calls methods). This separation between contract (service api) and implementation is a very strong approach to remote service development as it weakends the contract between the service implementer and the service consumer.

Generating the Protobuf/Grpc Service API with protoc and protoc plugins: grpc-java, reactive-grpc, grpc-osgi-generator

NOTE: Below describes how to use maven to run the protoc code generation with the grpc-java, reactive-grpc, ang grpc-osgi-generator. Another simpler and easier way to do code generation is to use the ECF Bndtools Remote Service Workspace Template that uses Bndtools. There is a video showing this method here.

For Protobuf + grpc-java, a service is defined by a .proto file with a service entry. For example, here is a proto file for a simple 'HealthCheck' service

// Copyright 2015 The gRPC Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// The canonical version of this proto can be found at
// https://github.com/grpc/grpc-proto/blob/master/grpc/health/v1/health.proto

syntax = "proto3";

package grpc.health.v1;

option java_multiple_files = true;
option java_outer_classname = "HealthProto";
option java_package = "io.grpc.health.v1";

message HealthCheckRequest {
  string message = 1;
}

message HealthCheckResponse {
  enum ServingStatus {
    UNKNOWN = 0;
    SERVING = 1;
    NOT_SERVING = 2;
    SERVICE_UNKNOWN = 3;  // Used only by the Watch method.
  }
  ServingStatus status = 1;
}

service HealthCheck {
  // Unary method
  rpc Check(HealthCheckRequest) returns (HealthCheckResponse);
  // Server streaming method
  rpc WatchServer(HealthCheckRequest) returns (stream HealthCheckResponse);
  // Client streaming method
  rpc WatchClient(stream HealthCheckRequest) returns (HealthCheckResponse);
  // bidi streaming method
  rpc WatchBidi(stream HealthCheckRequest) returns (stream HealthCheckResponse);
}

Service API Generation

NEW: The service generation now supports generating code that uses either the ReactiveX 2 OR ReactiveX 3 APIs. See the org.eclipse.ecf.examples.provider.grpc.rx3.health.api, .impl, and .consumer projects in the examples directory to see the ReactiveX 3 generator output. Using protoc and the three following protoc plugins: grpc-java compiler, reactive-grpc, and the grpc-osgi-generator plugin protoc will generate the complete service api in a single maven generate-sources phase. See the docs for grpc-osgi-generator for explanation of how to invoke protoc and these protoc plugins via maven.

The directory here has Java classes generated by using the protobuf-maven-plugin with protoc, grpc-java, and grpc-osgi-generator. Note that there are classes generated by protoc (HealthCheckRequest, HealthCheckResponse), classes generated by grpc-java (HealthCheckGrpc), classes generated by reactive-grpc (RxHealthCheckGrpc), and classes generated by grpc-osgi-generator (HealthCheckService interface). All of these classes are generated via the protoc and plugins compile/generation against src/main/proto/health.proto.

For a full understanding of how to setup maven to support service code generation see the grpc-osgi-generator README.md and the maven configuration defined for building the healthcheck api example bundle here.

Service Implementation and Service Consumer Creation

Running (via maven) the protobuf+grpc-java+grpc-osgi-generator creates the Java service api. Once done, all that needs to complete the api bundle is to export the appropriate package in the bundle manifest.mf so that the implementation bundle and the consumer bundle can import the needed classes. The completed example health check service api bundle is here. Note that the HealthCheckService class is the service interface for this example and that this class was generated by the grpc-osgi-generator protobuf plugin.

The service implementation can now be created in an additional bundle that will run only on the service host. For reference, a completed healthcheck implementation bundle is here. Note that the HealthServiceImpl class implements the HealthCheckService class, and extends the RxHealthCheckGrpc, which was generated by the reactive-grpc protoc plugin.

The health check service consumer is here and it uses an injected instance of the HealthCheckService (the RSA-created proxy) to access the service/call it's check method which has io.reactivex.Single types for both the HealthCheckRequest and the HealthCheckResponse. For reference, the completed healthcheck consumer bundle is here.

Installing and Running the Example on Apache Karaf

Install the gRPC Remote Services Distribution Provider

1. Start Karaf
2. At prompt type

karaf@root()> repo-add https://raw.githubusercontent.com/ECF/grpc-RemoteServicesProvider/master/build/karaf-features.xml

3. Install the ECF gRPC Distribution Provider type

karaf@root()> feature:install -v ecf-rs-distribution-grpc

This will result in install of all of the necessary gRPC Remote Service Provider plugins (and dependencies) to support running either/both the HealthCheck example impl or the remote service consumer.

Install and Start the HealthCheck Example Implementation (Server)

In console type:

karaf@root()> feature:install -v ecf-rs-examples-grpc-healthcheck-impl

The source code for this HealthCheckService implementation is here in this project

Note that in this example, do network-based discovery provider is installed, so for automatic network discovery one of the ECF Discovery Providers will need to be installed. Another option is to used the RSA importservice console command to import the service in the consumer without automatic network discovery.

Install and Start the HealthCheck Example Consumer (Client)

In console type:

karaf@root()> feature:install -v ecf-rs-examples-grpc-healthcheck-consumer

The consumer implementation class here will be activated by calling the activate method and the consumer will start making remote calls to the healthService.