LLM

class sycamore.llms.llms.LLM(model_name, default_mode: LLMMode, cache: Cache | None = None)[source]

Abstract representation of an LLM instance. and should be subclassed to implement specific LLM providers.

default_mode() → LLMMode[source]

Returns the default execution mode for the llm

format_image(image: Image) → dict[str, Any][source]

Returns a dictionary containing the specified image suitable for use in an LLM message.

abstract generate(*, prompt: RenderedPrompt, llm_kwargs: dict | None = None) → str[source]

Generates a response from the LLM for the given prompt and LLM parameters.

async generate_async(*, prompt: RenderedPrompt, llm_kwargs: dict | None = None) → str[source]

Generates a response from the LLM for the given prompt and LLM parameters asynchronously.

generate_batch(*, prompts: list[RenderedPrompt], llm_kwargs: dict | None = None) → list[str][source]

Generates a series of responses from the LLM for the given series of prompts. Order is preserved.

get_metadata(kwargs, response_text, wall_latency, in_tokens, out_tokens) → dict[source]

Generate metadata for the LLM response.

abstract is_chat_mode() → bool[source]

Returns True if the LLM is in chat mode, False otherwise.

enum sycamore.llms.llms.LLMMode(value)[source]

Valid values are as follows:

SYNC = <LLMMode.SYNC: 1>
ASYNC = <LLMMode.ASYNC: 2>
BATCH = <LLMMode.BATCH: 3>
class sycamore.llms.openai.OpenAI(model_name: OpenAIModels | OpenAIModel | str, api_key: str | None = None, client_wrapper: OpenAIClientWrapper | None = None, params: OpenAIClientWrapper | None = None, default_mode: LLMMode = LLMMode.ASYNC, cache: Cache | None = None, **kwargs)[source]

Bases: LLM

An LLM interface to OpenAI models.

Parameters:

  • model_name -- The name of the OpenAI model to use. This can be an instance of OpenAIModels, an instance of OpenAIModel, or a string. If a string is provided, it must be the name of the model.

  • api_key -- The API key to use for the OpenAI client. If not provided, the key will be read from the OPENAI_API_KEY environment variable.

  • client_wrapper -- An instance of OpenAIClientWrapper to use for the OpenAI client. If not provided, a new instance will be created using the provided parameters.

  • params -- An instance of OpenAIClientParameters to use for the OpenAI client. If not provided, a new instance will be created using the provided parameters.

  • cache -- An instance of Cache to use for caching responses. If not provided, no caching will be used.

  • default_mode -- Default execution mode for the llm

  • **kwargs -- Additional parameters to pass to the OpenAI client.

format_image(image: Image) → dict[str, Any][source]

Returns a dictionary containing the specified image suitable for use in an LLM message.

generate(*, prompt: RenderedPrompt, llm_kwargs: dict | None = None) → str[source]

Generates a response from the LLM for the given prompt and LLM parameters.

async generate_async(*, prompt: RenderedPrompt, llm_kwargs: dict | None = None) → str[source]

Generates a response from the LLM for the given prompt and LLM parameters asynchronously.

generate_batch(*, prompts: list[RenderedPrompt], llm_kwargs: dict | None = None) → list[str][source]

Generates a series of responses from the LLM for the given series of prompts. Order is preserved.

is_chat_mode()[source]

Returns True if the LLM is in chat mode, False otherwise.

enum sycamore.llms.openai.OpenAIClientType(value)[source]

Bases: Enum

Valid values are as follows:

OPENAI = <OpenAIClientType.OPENAI: 0>
AZURE = <OpenAIClientType.AZURE: 1>
class sycamore.llms.openai.OpenAIModel(name: str, is_chat: bool = False)[source]

Bases: object

enum sycamore.llms.openai.OpenAIModels(value)[source]

Bases: Enum

Valid values are as follows:

TEXT_DAVINCI = <OpenAIModels.TEXT_DAVINCI: OpenAIModel(name='text-davinci-003', is_chat=True)>
GPT_3_5_TURBO = <OpenAIModels.GPT_3_5_TURBO: OpenAIModel(name='gpt-3.5-turbo', is_chat=True)>
GPT_4_TURBO = <OpenAIModels.GPT_4_TURBO: OpenAIModel(name='gpt-4-turbo', is_chat=True)>
GPT_4O = <OpenAIModels.GPT_4O: OpenAIModel(name='gpt-4o', is_chat=True)>
GPT_4O_STRUCTURED = <OpenAIModels.GPT_4O_STRUCTURED: OpenAIModel(name='gpt-4o-2024-08-06', is_chat=True)>
GPT_4O_MINI = <OpenAIModels.GPT_4O_MINI: OpenAIModel(name='gpt-4o-mini', is_chat=True)>
GPT_3_5_TURBO_INSTRUCT = <OpenAIModels.GPT_3_5_TURBO_INSTRUCT: OpenAIModel(name='gpt-3.5-turbo-instruct', is_chat=False)>
class sycamore.llms.anthropic.Anthropic(model_name: AnthropicModels | str, default_mode: LLMMode = LLMMode.ASYNC, cache: Cache | None = None)[source]

Bases: LLM

This is an LLM implementation that uses the AWS Claude API to generate text.

Parameters:

  • model_name -- The name of the Claude model to use.

  • cache -- A cache object to use for caching results.

default_mode() → LLMMode[source]

Returns the default execution mode for the llm

format_image(image: Image) → dict[str, Any][source]

Returns a dictionary containing the specified image suitable for use in an LLM message.

generate(*, prompt: RenderedPrompt, llm_kwargs: dict | None = None) → str[source]

Generates a response from the LLM for the given prompt and LLM parameters.

async generate_async(*, prompt: RenderedPrompt, llm_kwargs: dict | None = None) → str[source]

Generates a response from the LLM for the given prompt and LLM parameters asynchronously.

generate_batch(*, prompts: list[RenderedPrompt], llm_kwargs: dict | None = None) → list[str][source]

Generates a series of responses from the LLM for the given series of prompts. Order is preserved.

is_chat_mode() → bool[source]

Returns True if the LLM is in chat mode, False otherwise.

enum sycamore.llms.anthropic.AnthropicModels(value)[source]

Bases: Enum

Represents available Claude models.

Valid values are as follows:

CLAUDE_3_7_SONNET = <AnthropicModels.CLAUDE_3_7_SONNET: 'claude-3-7-sonnet-latest'>
CLAUDE_3_5_SONNET = <AnthropicModels.CLAUDE_3_5_SONNET: 'claude-3-5-sonnet-latest'>
CLAUDE_3_5_HAIKU = <AnthropicModels.CLAUDE_3_5_HAIKU: 'claude-3-5-haiku-latest'>
CLAUDE_3_OPUS = <AnthropicModels.CLAUDE_3_OPUS: 'claude-3-opus-latest'>
CLAUDE_3_SONNET = <AnthropicModels.CLAUDE_3_SONNET: 'claude-3-sonnet-20240229'>
CLAUDE_3_HAIKU = <AnthropicModels.CLAUDE_3_HAIKU: 'claude-3-haiku-20240307'>
class sycamore.llms.gemini.Gemini(model_name: GeminiModels | str, default_mode: LLMMode = LLMMode.ASYNC, cache: Cache | None = None, api_key: str | None = None)[source]

Bases: LLM

This is an LLM implementation that uses the Google Gemini API to generate text.

Parameters:

  • model_name -- The name of the Gemini model to use.

  • cache -- A cache object to use for caching results.

default_mode() → LLMMode[source]

Returns the default execution mode for the llm

generate(*, prompt: RenderedPrompt, llm_kwargs: dict | None = None) → str[source]

Generates a response from the LLM for the given prompt and LLM parameters.

async generate_async(*, prompt: RenderedPrompt, llm_kwargs: dict | None = None) → str[source]

Generates a response from the LLM for the given prompt and LLM parameters asynchronously.

is_chat_mode() → bool[source]

Returns True if the LLM is in chat mode, False otherwise.

class sycamore.llms.gemini.GeminiModel(name: str, is_chat: bool = False)[source]

Bases: object

enum sycamore.llms.gemini.GeminiModels(value)[source]

Bases: Enum

Represents available Gemini models. More info: https://googleapis.github.io/python-genai/

Valid values are as follows:

GEMINI_2_FLASH = <GeminiModels.GEMINI_2_FLASH: GeminiModel(name='gemini-2.0-flash', is_chat=True)>
GEMINI_2_FLASH_LITE = <GeminiModels.GEMINI_2_FLASH_LITE: GeminiModel(name='gemini-2.0-flash-lite', is_chat=True)>
GEMINI_2_FLASH_THINKING = <GeminiModels.GEMINI_2_FLASH_THINKING: GeminiModel(name='gemini-2.0-flash-thinking-exp', is_chat=True)>
GEMINI_2_PRO = <GeminiModels.GEMINI_2_PRO: GeminiModel(name='gemini-2.0-pro-exp-02-05', is_chat=True)>
GEMINI_1_5_PRO = <GeminiModels.GEMINI_1_5_PRO: GeminiModel(name='gemini-1.5-pro', is_chat=True)>
class sycamore.llms.bedrock.Bedrock(model_name: BedrockModels | str, cache: Cache | None = None)[source]

Bases: LLM

This is an LLM implementation that uses the AWS Bedrock API to generate text.

Parameters:

  • model_name -- The name of the Bedrock model to use.

  • cache -- A cache object to use for caching results.

format_image(image: Image) → dict[str, Any][source]

Returns a dictionary containing the specified image suitable for use in an LLM message.

generate(*, prompt: RenderedPrompt, llm_kwargs: dict | None = None) → str[source]

Generates a response from the LLM for the given prompt and LLM parameters.

is_chat_mode() → bool[source]

Returns True if the LLM is in chat mode, False otherwise.

class sycamore.llms.bedrock.BedrockModel(name: str, is_chat: bool = False)[source]

Bases: object

enum sycamore.llms.bedrock.BedrockModels(value)[source]

Bases: Enum

Represents available Bedrock models.

Valid values are as follows:

CLAUDE_3_HAIKU = <BedrockModels.CLAUDE_3_HAIKU: BedrockModel(name='anthropic.claude-3-haiku-20240307-v1:0', is_chat=True)>
CLAUDE_3_SONNET = <BedrockModels.CLAUDE_3_SONNET: BedrockModel(name='anthropic.claude-3-sonnet-20240229-v1:0', is_chat=True)>
CLAUDE_3_OPUS = <BedrockModels.CLAUDE_3_OPUS: BedrockModel(name='anthropic.claude-3-opus-20240229-v1:0', is_chat=True)>
CLAUDE_3_5_SONNET = <BedrockModels.CLAUDE_3_5_SONNET: BedrockModel(name='anthropic.claude-3-5-sonnet-20240620-v1:0', is_chat=True)>