Document AI Architecture
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
| ┌─────────────────────────────────────────────────────────────┐
│ Document Input │
│ (PDF, Image, Scan, Word, Email) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Pre-processing │
│ (OCR, deskew, noise removal, enhancement) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Layout Analysis │
│ (Sections, tables, headers, paragraphs, images) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Classification │
│ (Document type → extraction schema) │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────┼─────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌─────────────────┐ ┌───────────────┐
│ Table Extract │ │ Form Extract │ │ LLM Extract │
│ (Structure) │ │ (Key-Value) │ │ (Reasoning) │
└───────────────┘ └─────────────────┘ └───────────────┘
│ │ │
└─────────────────────┼─────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Validation & Confidence │
│ (Schema validation, cross-check, scoring) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Structured Output │
│ (JSON, database, downstream API) │
└─────────────────────────────────────────────────────────────┘
|
Document Processing Pipeline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
| from pdf2image import convert_from_bytes
from layoutparser import Detectron2LayoutModel
class IntelligentDocumentProcessor:
def __init__(self):
self.ocr = TesseractOCR()
self.layout_model = Detectron2LayoutModel('lp://PubLayNet')
self.llm = get_llm()
self.classifier = DocumentClassifier()
async def process(self, document: bytes) -> ProcessedDocument:
# Convert to images for layout analysis
images = convert_from_bytes(document)
pages = []
for image in images:
# Layout detection
layout = self.layout_model.detect(image)
# OCR text extraction
text_blocks = []
for block in layout:
text = self.ocr.extract(image, block.coordinates)
text_blocks.append(TextBlock(
text=text,
type=block.type, # title, paragraph, table, etc.
bbox=block.coordinates
))
pages.append(Page(blocks=text_blocks, layout=layout))
# Classify document
doc_type = await self.classifier.classify(pages)
# Extract based on type
extracted = await self.extract(pages, doc_type)
return ProcessedDocument(
pages=pages,
doc_type=doc_type,
extracted_data=extracted
)
async def extract(self, pages: list[Page], doc_type: str) -> dict:
schema = self.get_schema(doc_type)
# Combine extraction methods
extracted = {}
# Rule-based for known patterns
extracted.update(self.rule_based_extract(pages, schema))
# LLM for complex reasoning
llm_extract = await self.llm_extract(pages, schema)
extracted.update(llm_extract)
# Validate
return self.validate(extracted, schema)
|
| Document Type | Extraction Method | Key Fields |
|---|
| Invoices | Template + LLM | Vendor, amount, line items |
| Contracts | LLM + NER | Parties, dates, clauses |
| Patents | Structure + LLM | Claims, citations, inventors |
| Forms | OCR + Template | Field values |
| Reports | Layout + LLM | Tables, figures, summaries |
Technologies for Document AI
- OCR: Tesseract, AWS Textract, Google Cloud Vision
- Layout: LayoutParser, Detectron2
- LLMs: GPT-4 Vision, Gemini, Claude
- PDF: PyMuPDF, pdf2image
- Tables: Camelot, Tabula
- NER: spaCy, custom models
Patent Document Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
| class PatentProcessor:
async def extract_patent(self, pdf: bytes) -> PatentData:
doc = await self.process(pdf)
return PatentData(
# Bibliographic
patent_number=doc.get_field("patent_number"),
filing_date=doc.get_field("filing_date"),
inventors=doc.get_entities("PERSON"),
assignee=doc.get_entities("ORGANIZATION")[0],
# Claims
claims=self.extract_claims(doc),
independent_claims=self.filter_independent(doc.claims),
# Citations
prior_art=self.extract_citations(doc, type="prior_art"),
cited_patents=self.extract_citations(doc, type="patent"),
# Full text sections
abstract=doc.get_section("abstract"),
description=doc.get_section("description"),
# Confidence
confidence=doc.overall_confidence
)
|
Frequently Asked Questions
What is Document AI?
Document AI uses machine learning to extract, classify, and understand information from documents. This includes: OCR, key-value extraction, document classification, summarization, and converting unstructured documents into structured data.
How much does Document AI development cost?
Document AI development typically costs $110-160 per hour. A basic document extraction pipeline starts around $20,000-40,000, while enterprise systems with custom models and complex document types range from $75,000-200,000+.
What document types can be processed?
I work with: invoices, contracts, forms, receipts, IDs, medical records, legal documents, and technical manuals. Each document type may need specific extraction logic, but LLMs have made general document understanding much more accessible.
Use OCR + templates for: high-volume, consistent document formats. Use LLM-based extraction for: varied layouts, complex documents, or when you need understanding (not just text extraction). Many solutions combine both approaches.
How accurate is Document AI?
Accuracy depends on document quality, complexity, and requirements. For structured forms, 95%+ accuracy is achievable. For complex documents with varied layouts, 85-95% with human review for exceptions. I implement confidence scoring and exception handling.
Experience:
Case Studies:
Related Technologies: RAG Systems, LangChain, AI Agents, Python, AI Workflow Automation