How to use Amazon Bedrock as LLM/Embeddings after LangChain Wrappers are removed

[licux]

[ x ] I checked the documentation and related resources and couldn't find an answer to my question.

Your Question
In pull request #2253, LangChainLLMWrapper and LangchainEmbeddingsWrapper have been deprecated, but could you tell me about alternative approaches when using Amazon Bedrock? It seems that instructor_llm_factory and embedding_factory don't support Bedrock yet. Will Bedrock support be added to these factories in the future?

Code Examples
None

Additional context

• This question is related to belows:
  • Deprecation warnings for LLMs and Prompts #2253

[jjmachan]

thank you so much for raising this @licux - this is something new we are exploring and we would highly appreciate your feedback on this 🙌🏽 .

so currently you can leverage InstructorLLM base class an manually create the instructor client as so

# create the instructor client
import boto3
import instructor

bedrock_client = boto3.client('bedrock-runtime')
client = instructor.from_bedrock(bedrock_client)

# use that to create InstructorLLM from ragas
from ragas.llms.base import InstructorLLM

llm = InstructorLLM(
    model="anthropic.claude-3-sonnet-20240229-v1:0",
    provider="bedrock",
    client=client,
)

llm.generate("Extract: Jason is 25 years old name", response_model=User)

# use that to create metrics
from ragas.metrics import DiscreteMetric, NumericMetric

summary_accuracy = DiscreteMetric(
    name="summary_accuracy",
    prompt="Evaluate if given summary is accurate for the user's query: {user_input}? {response}",
    allowed_values=(0, 1),
)

summary_accuracy.score(**test_data, llm=llm)
# MetricResult(value=1, reason='The given summary accurately captures the key points from the text, including the 8% growth in Q3 2024, the strong performance in the Asian market as a driving factor, the role of strategic marketing and product localization in this success, and the positive outlook for continued growth in the next quarter.')

we should also add "bedrock" as one of the supported providers @anistark maybe?

[jjmachan]

the main change we are trying to do is depend on the clients the provider gives directly instead of using langchain and since we need structured outputs we use Instructor to do this

[minorun365]

Thank you!

As below LangChainLLMWrapper will be deprecated so Ragas users using AWS must migrate to this feature. We strongly hope this to support Amazon Bedrock LLMs & embeddings!
#2253

[jjmachan]

hey @minorun365 we'll definitely be adding first class support for it too 🙂