Interface PredictionServiceGrpc.AsyncService (3.49.0) 

public static interface PredictionServiceGrpc.AsyncService

A service for online predictions and explanations.

Methods

directPredict(DirectPredictRequest request, StreamObserver responseObserver)

public default void directPredict(DirectPredictRequest request, StreamObserver responseObserver)

Perform an unary online prediction request to a gRPC model server for Vertex first-party products and frameworks.

Parameters
Name Description
request DirectPredictRequest
responseObserver io.grpc.stub.StreamObserver<DirectPredictResponse>

directRawPredict(DirectRawPredictRequest request, StreamObserver responseObserver)

public default void directRawPredict(DirectRawPredictRequest request, StreamObserver responseObserver)

Perform an unary online prediction request to a gRPC model server for custom containers.

Parameters
Name Description
request DirectRawPredictRequest
responseObserver io.grpc.stub.StreamObserver<DirectRawPredictResponse>

explain(ExplainRequest request, StreamObserver responseObserver)

public default void explain(ExplainRequest request, StreamObserver responseObserver)

Perform an online explanation. If deployed_model_id is specified, the corresponding DeployModel must have explanation_spec populated. If deployed_model_id is not specified, all DeployedModels must have explanation_spec populated.

Parameters
Name Description
request ExplainRequest
responseObserver io.grpc.stub.StreamObserver<ExplainResponse>

generateContent(GenerateContentRequest request, StreamObserver responseObserver)

public default void generateContent(GenerateContentRequest request, StreamObserver responseObserver)

Generate content with multimodal inputs.

Parameters
Name Description
request GenerateContentRequest
responseObserver io.grpc.stub.StreamObserver<GenerateContentResponse>

predict(PredictRequest request, StreamObserver responseObserver)

public default void predict(PredictRequest request, StreamObserver responseObserver)

Perform an online prediction.

Parameters
Name Description
request PredictRequest
responseObserver io.grpc.stub.StreamObserver<PredictResponse>

rawPredict(RawPredictRequest request, StreamObserver responseObserver)

public default void rawPredict(RawPredictRequest request, StreamObserver responseObserver)

Perform an online prediction with an arbitrary HTTP payload. The response includes the following HTTP headers:

  • X-Vertex-AI-Endpoint-Id: ID of the Endpoint that served this prediction.
  • X-Vertex-AI-Deployed-Model-Id: ID of the Endpoint's DeployedModel that served this prediction.
Parameters
Name Description
request RawPredictRequest
responseObserver io.grpc.stub.StreamObserver<com.google.api.HttpBody>

serverStreamingPredict(StreamingPredictRequest request, StreamObserver responseObserver)

public default void serverStreamingPredict(StreamingPredictRequest request, StreamObserver responseObserver)

Perform a server-side streaming online prediction request for Vertex LLM streaming.

Parameters
Name Description
request StreamingPredictRequest
responseObserver io.grpc.stub.StreamObserver<StreamingPredictResponse>

streamDirectPredict(StreamObserver responseObserver)

public default StreamObserver streamDirectPredict(StreamObserver responseObserver)

Perform a streaming online prediction request to a gRPC model server for Vertex first-party products and frameworks.

Parameter
Name Description
responseObserver io.grpc.stub.StreamObserver<StreamDirectPredictResponse>
Returns
Type Description
io.grpc.stub.StreamObserver<StreamDirectPredictRequest>

streamDirectRawPredict(StreamObserver responseObserver)

public default StreamObserver streamDirectRawPredict(StreamObserver responseObserver)

Perform a streaming online prediction request to a gRPC model server for custom containers.

Parameter
Name Description
responseObserver io.grpc.stub.StreamObserver<StreamDirectRawPredictResponse>
Returns
Type Description
io.grpc.stub.StreamObserver<StreamDirectRawPredictRequest>

streamGenerateContent(GenerateContentRequest request, StreamObserver responseObserver)

public default void streamGenerateContent(GenerateContentRequest request, StreamObserver responseObserver)

Generate content with multimodal inputs with streaming support.

Parameters
Name Description
request GenerateContentRequest
responseObserver io.grpc.stub.StreamObserver<GenerateContentResponse>

streamRawPredict(StreamRawPredictRequest request, StreamObserver responseObserver)

public default void streamRawPredict(StreamRawPredictRequest request, StreamObserver responseObserver)

Perform a streaming online prediction with an arbitrary HTTP payload.

Parameters
Name Description
request StreamRawPredictRequest
responseObserver io.grpc.stub.StreamObserver<com.google.api.HttpBody>

streamingPredict(StreamObserver responseObserver)

public default StreamObserver streamingPredict(StreamObserver responseObserver)

Perform a streaming online prediction request for Vertex first-party products and frameworks.

Parameter
Name Description
responseObserver io.grpc.stub.StreamObserver<StreamingPredictResponse>
Returns
Type Description
io.grpc.stub.StreamObserver<StreamingPredictRequest>

streamingRawPredict(StreamObserver responseObserver)

public default StreamObserver streamingRawPredict(StreamObserver responseObserver)

Perform a streaming online prediction request through gRPC.

Parameter
Name Description
responseObserver io.grpc.stub.StreamObserver<StreamingRawPredictResponse>
Returns
Type Description
io.grpc.stub.StreamObserver<StreamingRawPredictRequest>