DedocPDFLoader#

class langchain_community.document_loaders.pdf.DedocPDFLoader(file_path: str, *, split: str = 'document', with_tables: bool = True, with_attachments: str | bool = False, recursion_deep_attachments: int = 10, pdf_with_text_layer: str = 'auto_tabby', language: str = 'rus+eng', pages: str = ':', is_one_column_document: str = 'auto', document_orientation: str = 'auto', need_header_footer_analysis: str | bool = False, need_binarization: str | bool = False, need_pdf_table_analysis: str | bool = True, delimiter: str | None = None, encoding: str | None = None)[source]#

DedocPDFLoader 文档加载器集成，用于使用 dedoc 加载 PDF 文件。文件加载器可以自动检测文本层的正确性。

PDF文档。

Note that __init__ method supports parameters that differ from ones of

DedocBaseLoader。

Setup:

安装 dedoc 包。

pip install -U dedoc

Instantiate:

from langchain_community.document_loaders import DedocPDFLoader

loader = DedocPDFLoader(
    file_path="example.pdf",
    # split=...,
    # with_tables=...,
    # pdf_with_text_layer=...,
    # pages=...,
    # ...
)

Load:

docs = loader.load()
print(docs[0].page_content[:100])
print(docs[0].metadata)

Some text
{
    'file_name': 'example.pdf',
    'file_type': 'application/pdf',
    # ...
}

Lazy load:

docs = []
docs_lazy = loader.lazy_load()

for doc in docs_lazy:
    docs.append(doc)
print(docs[0].page_content[:100])
print(docs[0].metadata)

Some text
{
    'file_name': 'example.pdf',
    'file_type': 'application/pdf',
    # ...
}

Parameters used for document parsing via dedoc

(https://dedoc.readthedocs.io/en/latest/parameters/pdf_handling.html):

with_attachments: 启用附件文件提取 recursion_deep_attachments: 附件文件提取的递归层级,

仅在 with_attachments==True 时有效

pdf_with_text_layer: type of handler for parsing, available options: [“true”, “false”, “tabby”, “auto”, “auto_tabby” (默认)]
language: language of the document for PDF without a textual layer,: 可用选项 [“eng”, “rus”, “rus+eng” (默认)]，语言列表可以扩展，请参阅 https://dedoc.readthedocs.io/en/latest/tutorials/add_new_language.html

页面：定义解析读取范围的页面切片 is_one_column_document：检测没有文本的PDF的列数

层，可用选项 [“true”, “false”, “auto” (默认)]

document_orientation: fix document orientation (90, 180, 270 degrees) for PDF: 没有文本层时，可用选项 [“auto”（默认）， “no_change”]

need_header_footer_analysis: 从输出结果中移除页眉和页脚 need_binarization: 为没有文本的PDF清理页面背景（二值化）

层

need_pdf_table_analysis: 解析没有文本层的PDF表格

使用文件路径和解析参数进行初始化。

Parameters:

file_path (str) – 用于处理的文件路径
split (str) –
文档分割成部分的类型（每个部分单独返回），默认值为“document” “document”：文档文本作为单个langchain文档返回

对象（不分割）

”page”：将文档文本分割成页面（适用于PDF、DJVU、PPTX、PPT、
ODP）

”node”：将文档文本分割成树节点（标题节点、列表项
节点、原始文本节点）

”line”：将文档文本分割成行
with_tables (bool) – 将表格添加到结果中 - 每个表格作为一个单独的 langchain Document 对象返回
dedoc (Parameters used for document parsing via) –
(https://dedoc.readthedocs.io/en/latest/parameters/parameters.html):

with_attachments: enable attached files extraction recursion_deep_attachments: recursion level for attached files

extraction, works only when with_attachments==True

pdf_with_text_layer: type of handler for parsing PDF documents,
available options [“true”, “false”, “tabby”, “auto”, “auto_tabby” (default)]

language: language of the document for PDF without a textual layer and
images, available options [“eng”, “rus”, “rus+eng” (default)], the list of languages can be extended, please see https://dedoc.readthedocs.io/en/latest/tutorials/add_new_language.html

pages: page slice to define the reading range for parsing PDF documents is_one_column_document: detect number of columns for PDF without

a textual layer and images, available options [“true”, “false”, “auto” (default)]

document_orientation: fix document orientation (90, 180, 270 degrees)
for PDF without a textual layer and images, available options [“auto” (default), “no_change”]

need_header_footer_analysis: remove headers and footers from the output
result for parsing PDF and images

need_binarization: clean pages background (binarize) for PDF without a
textual layer and images

need_pdf_table_analysis: parse tables for PDF without a textual layer
and images

delimiter: column separator for CSV, TSV files encoding: encoding of TXT, CSV, TSV
with_attachments (str | bool)
recursion_deep_attachments (int)
pdf_with_text_layer (str)
language (str)
pages (str)
is_one_column_document (str)
document_orientation (str)
need_header_footer_analysis (str | bool)
need_binarization (str | bool)
need_pdf_table_analysis (str | bool)
delimiter (str | None)
encoding (str | None)

方法

`__init__`(file_path, *[, split, with_tables, ...])	使用文件路径和解析参数进行初始化。
`alazy_load`()	一个用于文档的懒加载器。
`aload`()	将数据加载到Document对象中。
`lazy_load`()	懒加载文档。
`load`()	将数据加载到Document对象中。
`load_and_split`([text_splitter])	加载文档并将其分割成块。

__init__(file_path: str, *, split: str = 'document', with_tables: bool = True, with_attachments: str | bool = False, recursion_deep_attachments: int = 10, pdf_with_text_layer: str = 'auto_tabby', language: str = 'rus+eng', pages: str = ':', is_one_column_document: str = 'auto', document_orientation: str = 'auto', need_header_footer_analysis: str | bool = False, need_binarization: str | bool = False, need_pdf_table_analysis: str | bool = True, delimiter: str | None = None, encoding: str | None = None) → None#

使用文件路径和解析参数进行初始化。

Parameters:

file_path (str) – 用于处理的文件路径
split (str) –
文档分割成部分的类型（每个部分单独返回），默认值为“document” “document”：文档文本作为单个langchain文档返回

对象（不分割）

”page”：将文档文本分割成页面（适用于PDF、DJVU、PPTX、PPT、
ODP）

”node”：将文档文本分割成树节点（标题节点、列表项
节点、原始文本节点）

”line”：将文档文本分割成行
with_tables (bool) – 将表格添加到结果中 - 每个表格作为一个单独的 langchain Document 对象返回
dedoc (Parameters used for document parsing via) –
(https://dedoc.readthedocs.io/en/latest/parameters/parameters.html):

with_attachments: enable attached files extraction recursion_deep_attachments: recursion level for attached files

extraction, works only when with_attachments==True

pdf_with_text_layer: type of handler for parsing PDF documents,
available options [“true”, “false”, “tabby”, “auto”, “auto_tabby” (default)]

language: language of the document for PDF without a textual layer and
images, available options [“eng”, “rus”, “rus+eng” (default)], the list of languages can be extended, please see https://dedoc.readthedocs.io/en/latest/tutorials/add_new_language.html

pages: page slice to define the reading range for parsing PDF documents is_one_column_document: detect number of columns for PDF without

a textual layer and images, available options [“true”, “false”, “auto” (default)]

document_orientation: fix document orientation (90, 180, 270 degrees)
for PDF without a textual layer and images, available options [“auto” (default), “no_change”]

need_header_footer_analysis: remove headers and footers from the output
result for parsing PDF and images

need_binarization: clean pages background (binarize) for PDF without a
textual layer and images

need_pdf_table_analysis: parse tables for PDF without a textual layer
and images

delimiter: column separator for CSV, TSV files encoding: encoding of TXT, CSV, TSV
with_attachments (str | bool)
recursion_deep_attachments (int)
pdf_with_text_layer (str)
language (str)
pages (str)
is_one_column_document (str)
document_orientation (str)
need_header_footer_analysis (str | bool)
need_binarization (str | bool)
need_pdf_table_analysis (str | bool)
delimiter (str | None)
encoding (str | None)

Return type:

无

async alazy_load() → AsyncIterator[Document]#

文档的懒加载器。

Return type:: AsyncIterator[Document]

async aload() → list[Document]#

将数据加载到Document对象中。

Return type:: 列表[Document]

lazy_load() → Iterator[Document]#

懒加载文档。

Return type:: 迭代器[文档]

load() → list[Document]#

将数据加载到Document对象中。

Return type:: 列表[Document]

load_and_split(text_splitter: TextSplitter | None = None) → list[Document]#

加载文档并将其分割成块。块以文档形式返回。

不要重写此方法。它应该被视为已弃用！

Parameters:: text_splitter (可选[TextSplitter]) – 用于分割文档的TextSplitter实例。默认为RecursiveCharacterTextSplitter。
Returns:: 文档列表。
Return type:: 列表[Document]

使用 DedocPDFLoader 的示例

Dedoc