合成器

AbstractQuerySynthesizer dataclass

AbstractQuerySynthesizer(
    name: str = "",
    llm: BaseRagasLLM = llm_factory(),
    critic_query_prompt: PydanticPrompt = CriticUserInput(),
    query_modification_prompt: PydanticPrompt = ModifyUserInput(),
    generate_reference_prompt: PydanticPrompt = GenerateReference(),
    generate_user_input_prompt: PydanticPrompt = AbstractQueryFromTheme(),
)

Bases: QuerySynthesizer

Synthesizes abstract queries which generate a theme and a set of summaries from a cluster of chunks and then generate queries based on that.

Attributes:

Name	Type	Description
generate_user_input_prompt	PydanticPrompt	The prompt used for generating the user input.

ComparativeAbstractQuerySynthesizer dataclass

ComparativeAbstractQuerySynthesizer(
    name: str = "",
    llm: BaseRagasLLM = llm_factory(),
    critic_query_prompt: PydanticPrompt = CriticUserInput(),
    query_modification_prompt: PydanticPrompt = ModifyUserInput(),
    generate_reference_prompt: PydanticPrompt = GenerateReference(),
    common_concepts_prompt: PydanticPrompt = CommonConceptsFromKeyphrases(),
    generate_query_prompt: PydanticPrompt = ComparativeAbstractQuery(),
)

Bases: QuerySynthesizer

Synthesizes comparative abstract queries which generate a common concept and a set of keyphrases and summaries and then generate queries based on that.

Attributes:

Name	Type	Description
common_concepts_prompt	PydanticPrompt	The prompt used for generating common concepts.
generate_query_prompt	PydanticPrompt	The prompt used for generating the query.

BaseSynthesizer dataclass

BaseSynthesizer(
    name: str = "", llm: BaseRagasLLM = llm_factory()
)

Bases: ABC, Generic[Scenario], PromptMixin

Base class for synthesizing scenarios and samples.

QuerySynthesizer dataclass

QuerySynthesizer(
    name: str = "",
    llm: BaseRagasLLM = llm_factory(),
    critic_query_prompt: PydanticPrompt = CriticUserInput(),
    query_modification_prompt: PydanticPrompt = ModifyUserInput(),
    generate_reference_prompt: PydanticPrompt = GenerateReference(),
)

Bases: BaseSynthesizer[Scenario]

Synthesizes Question-Answer pairs. Used as a base class for other query synthesizers.

Attributes:

Name	Type	Description
critic_query_prompt	PydanticPrompt	The prompt used for criticizing the query.
query_modification_prompt	PydanticPrompt	The prompt used for modifying the query.
generate_reference_prompt	PydanticPrompt	The prompt used for generating the reference.

SpecificQuerySynthesizer dataclass

SpecificQuerySynthesizer(
    name: str = "",
    llm: BaseRagasLLM = llm_factory(),
    critic_query_prompt: PydanticPrompt = CriticUserInput(),
    query_modification_prompt: PydanticPrompt = ModifyUserInput(),
    generate_reference_prompt: PydanticPrompt = GenerateReference(),
    generate_query_prompt: PydanticPrompt = SpecificQuery(),
)

Bases: QuerySynthesizer

Synthesizes specific queries by choosing specific chunks and generating a keyphrase from them and then generating queries based on that.

default_query_distribution

default_query_distribution(
    llm: BaseRagasLLM,
) -> QueryDistribution

Default query distribution for the test set.

By default, 25% of the queries are generated using AbstractQuerySynthesizer, 25% are generated using ComparativeAbstractQuerySynthesizer, and 50% are generated using SpecificQuerySynthesizer.

Source code in src/ragas/testset/synthesizers/__init__.py

def default_query_distribution(llm: BaseRagasLLM) -> QueryDistribution:
    """
    Default query distribution for the test set.

    By default, 25% of the queries are generated using `AbstractQuerySynthesizer`,
    25% are generated using `ComparativeAbstractQuerySynthesizer`, and 50% are
    generated using `SpecificQuerySynthesizer`.
    """
    return [
        (AbstractQuerySynthesizer(llm=llm), 0.25),
        (ComparativeAbstractQuerySynthesizer(llm=llm), 0.25),
        (SpecificQuerySynthesizer(llm=llm), 0.5),
    ]