Interface GoogleAIBaseLLMInput<AuthOptions>

Parameters needed to setup the client connection. AuthOptions are something like GoogleAuthOptions (from google-auth-library) or WebGoogleAuthOptions.

interface GoogleAIBaseLLMInput<AuthOptions> {
    apiKey?: string;
    apiVersion?: string;
    authOptions?: AuthOptions;
    endpoint?: string;
    location?: string;
    maxOutputTokens?: number;
    model?: string;
    modelName?: string;
    platformType?: "gai" | "gcp";
    safetyHandler?: GoogleAISafetyHandler;
    safetySettings?: GoogleAISafetySetting[];
    stopSequences?: string[];
    temperature?: number;
    topK?: number;
    topP?: number;
}

Type Parameters

  • AuthOptions

Hierarchy (view full)

Implemented by

Properties

apiKey? apiVersion? authOptions? endpoint? location? maxOutputTokens? model? modelName? platformType? safetyHandler? safetySettings? stopSequences? temperature? topK? topP?

Properties

Optional apiKey

apiKey?: string

Some APIs allow an API key instead

Optional apiVersion

apiVersion?: string

The version of the API functions. Part of the path.

Optional authOptions

authOptions?: AuthOptions

Optional endpoint

endpoint?: string

Hostname for the API call (if this is running on GCP)

Optional location

location?: string

Region where the LLM is stored (if this is running on GCP)

Optional maxOutputTokens

maxOutputTokens?: number

Maximum number of tokens to generate in the completion.

Optional model

model?: string

Model to use

Optional modelName

modelName?: string

Model to use Alias for model

Optional platformType

platformType?: "gai" | "gcp"

What platform to run the service on. If not specified, the class should determine this from other means. Either way, the platform actually used will be in the "platform" getter.

Optional safetyHandler

safetyHandler?: GoogleAISafetyHandler

Optional safetySettings

safetySettings?: GoogleAISafetySetting[]

Optional stopSequences

stopSequences?: string[]

Optional temperature

temperature?: number

Sampling temperature to use

Optional topK

topK?: number

Top-k changes how the model selects tokens for output.

A top-k of 1 means the selected token is the most probable among all tokens in the model’s vocabulary (also called greedy decoding), while a top-k of 3 means that the next token is selected from among the 3 most probable tokens (using temperature).

Optional topP

topP?: number

Top-p changes how the model selects tokens for output.

Tokens are selected from most probable to least until the sum of their probabilities equals the top-p value.

For example, if tokens A, B, and C have a probability of .3, .2, and .1 and the top-p value is .5, then the model will select either A or B as the next token (using temperature).

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc