OCR text is easier to use when its original structure survives extraction. This checklist explains how to use OCR bounding boxes and coordinates to preserve headings, columns, reading order, and form fields when converting pages into searchable text or structured data.
Overview
An OCR engine typically returns more than recognized words. Depending on the OCR API, app, or SDK, each page may include words, lines, paragraphs, blocks, confidence values, and bounding boxes. A bounding box describes where a text item appears on the page, usually with an x-coordinate, y-coordinate, width, and height. Those coordinates let your parser rebuild a document’s visual relationships instead of treating the response as one undifferentiated string.
This matters because the order in which an OCR service returns text is not always the order a person reads it. A two-column article, for example, may be returned across the page from left to right, line by line, or in an order influenced by the model’s internal layout detection. A form may contain labels beside values, while a receipt may place totals in a visually separate region. If you discard coordinates, these relationships become difficult to recover.
A reliable workflow has five stages:
- Capture the source image or PDF page and preserve its dimensions.
- Send the page to your OCR API or OCR SDK with layout information enabled when available.
- Normalize the returned coordinates into one consistent page coordinate system.
- Group words into lines, blocks, columns, and fields using geometry and text metadata.
- Validate the reconstructed output against the original page or a rendered overlay.
Coordinate data should support text extraction, not replace semantic validation. A nearby word is not automatically the correct value for a label, and a higher-confidence result is not necessarily correct in a complex layout.
Checklist by scenario
For headings and paragraphs
- Confirm that each word or line includes a bounding box and, if available, confidence and block identifiers.
- Group words into the same line when their vertical centers are close and their horizontal ranges overlap or nearly touch.
- Group lines into paragraphs when their left alignment, indentation, and vertical spacing are consistent.
- Detect headings using layout signals such as larger text height, distinct spacing, capitalization, or a separate block. Treat these as clues rather than universal rules.
- Keep the original coordinates in your output so downstream users can trace extracted text back to the page.
For ordinary prose, sort lines by their top coordinate and then their left coordinate after grouping. Do not sort individual words globally before identifying lines; this can mix words from adjacent lines or columns.
For multi-column pages
- Measure the horizontal distribution of text boxes across the page.
- Look for a sustained vertical gap between clusters of text rather than relying on a single blank space.
- Assign lines to columns before sorting them vertically.
- Sort columns in the intended reading direction, usually left to right for left-to-right scripts, while allowing for document-specific rules.
- Process each column from top to bottom, then move to the next column.
A practical column detector can begin with the left and right edges of line boxes. If most lines fall into two stable horizontal groups with little overlap, treat those groups as candidate columns. Headers, footers, sidebars, and pull quotes can complicate this approach, so use block types or minimum text density where your OCR response provides them.
For forms and key-value fields
- Identify labels, values, checkboxes, tables, and handwritten regions as separate object types when possible.
- Search for a value to the right of a label first, then below it if the form uses stacked labels.
- Use the label’s vertical range to limit candidate values and prevent matches from nearby rows.
- Preserve field coordinates, the extracted value, confidence, and validation status together.
- Require review when a field has multiple plausible candidates or overlaps another region.
Forms benefit from combining geometry with known field names, expected formats, and page templates. For example, a date field can be checked against an expected date pattern, but the parser should still retain the original text when the pattern fails.
For receipts and invoices
- Separate the header, line-item table, totals area, tax area, and payment details into regions.
- Group line-item words by their baseline or vertical center before assigning them to columns such as description, quantity, and amount.
- Align numeric values by their right edge when the document uses a right-aligned amount column.
- Use nearby labels such as “total” or “tax” as semantic anchors, but verify that the amount belongs to the same visual region.
- Retain page numbers and coordinates for every extracted amount so a reviewer can resolve ambiguity quickly.
For a broader field-validation approach, see OCR for Forms: A Field Validation Workflow for Reliable Data Capture. If the input is multilingual or uses mixed scripts, test the layout rules against each script direction and line shape; the guidance in Testing OCR on Multilingual and Mixed-Script Documents can help structure that test set.
Coordinate-system reference
Before writing layout logic, document exactly how your OCR response represents position. Coordinate assumptions are a frequent source of apparently random parsing errors.
| Item | Questions to answer | Why it matters |
|---|---|---|
| Origin | Does (0,0) start at the top-left, bottom-left, or another corner? | Determines whether larger y-values mean lower or higher on the page. |
| Units | Are coordinates pixels, points, normalized values, or another unit? | Controls scaling and comparison across pages. |
| Box format | Is the box x, y, width, height; left, top, right, bottom; or a polygon? | Prevents incorrect edge calculations. |
| Rotation | Are coordinates reported before or after page rotation? | Affects sorting, overlays, and orientation detection. |
| Page size | What width and height are associated with each page? | Allows coordinates to be mapped back to the source image. |
| Precision | Are values rounded, normalized, or integer-based? | Influences overlap thresholds and line grouping. |
Store the page width, page height, orientation, and coordinate convention alongside the OCR result. If you convert coordinates into a common internal format, retain the original values as well. This makes debugging possible when an API response schema changes or a page is rendered at a different resolution.
What to double-check
Line grouping
Use a tolerance rather than requiring identical y-values. Scans can introduce small vertical differences even when words belong to one line. A useful starting point is to compare vertical-center distance with the median character or box height on that page. Adjust the tolerance for small text, skewed scans, and handwriting. Check the result visually instead of treating one threshold as universal.
Overlapping boxes
Overlaps can occur when an OCR response includes both a word and its parent line or block, or when text is detected twice. Identify the hierarchy if the API provides it. Otherwise, remove or separate near-duplicate boxes using text similarity, intersection-over-union, and size relationships. Do not remove every overlap: table cells, superscripts, stamps, and annotations may legitimately intersect.
Rotated pages and text
Determine whether the OCR service automatically deskews or rotates pages. A page-level rotation is different from rotated text inside an otherwise upright page. For rotated content, axis-aligned rectangles may be insufficient; use polygons or angle metadata when available. Test portrait, landscape, and mixed-orientation documents separately.
Visual-text alignment
Render the source page and draw each returned box over it using a consistent scale. An overlay quickly reveals flipped axes, incorrect page dimensions, offset origins, and rotation mistakes. Include overlays in development tests for representative pages: a single-column page, a two-column page, a form, a table, and a low-quality scan.
Reading order
Reading order should be an explicit output property, not an accidental result of array order. Record a sequence number after layout analysis, and retain the source block, line, and word identifiers where available. This makes it easier to explain why a parser selected a value and to revise ordering rules without re-running OCR.
Common mistakes
- Sorting all words by y and x: This often interleaves columns. Group into regions first.
- Assuming every page uses the same dimensions: Normalize per page and never reuse one page’s scale blindly.
- Ignoring rotation: A correct box in the wrong orientation can produce incorrect reading order.
- Discarding low-confidence text immediately: It may be essential for layout detection or review. Flag it instead.
- Using coordinates without text rules: Geometry alone cannot distinguish a subtotal from a nearby total.
- Hard-coding one template: Small changes in logos, spacing, or page design can move fields outside fixed regions.
- Failing to keep raw OCR responses: Preserve raw and normalized data so a parsing issue can be reproduced.
- Skipping contract tests: Verify field names, types, coordinate formats, and page metadata whenever an OCR provider or SDK changes. See OCR API Contract Testing: A Release Checklist for Stable Document Extraction.
When the final destination is JSON, separate layout reconstruction from field extraction. First create a trustworthy page model containing regions, lines, and reading order. Then map that model into application fields. The workflow described in How to Convert OCR Text into Reliable Structured JSON is a useful companion for that second stage.
When to revisit
Revisit your bounding-box and layout rules before a seasonal planning cycle, after changing OCR providers or SDK versions, or whenever the document sources change. New templates, revised invoices, updated forms, different scanners, and mobile photographs can all alter spacing and coordinate behavior without changing the business meaning of the document.
Use this short maintenance checklist:
- Collect a fresh sample that represents current page designs and orientations.
- Run the existing parser without changing its rules.
- Compare reading order, field placement, confidence flags, and visual overlays with the expected result.
- Inspect failures by category: coordinate conversion, rotation, grouping, column detection, or semantic mapping.
- Update thresholds or rules narrowly, then rerun regression tests on older samples.
- Keep a small set of difficult pages for manual review and future release testing.
For sensitive documents, include privacy controls in the same review. Confirm what is stored, where raw images and OCR responses are retained, and whether redaction belongs before or after extraction for your use case. The guide How to Build a Privacy-First OCR Workflow for Sensitive Documents provides a starting framework.
The goal is not to reproduce every visual detail. It is to preserve the relationships that users and downstream systems need: which text belongs together, what should be read first, and where each extracted value came from. Keep coordinates, make ordering explicit, validate against rendered pages, and revisit the rules whenever the inputs or OCR contract change.