<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[StatementXL Engineering]]></title><description><![CDATA[StatementXL Engineering]]></description><link>https://statementxl.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Fri, 30 Oct 2026 05:32:40 GMT</lastBuildDate><atom:link href="https://statementxl.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Which rows of a converted bank statement can you trust?]]></title><description><![CDATA[Why I label every extracted row instead of promising an accurate spreadsheet]]></description><link>https://statementxl.hashnode.dev/which-rows-of-a-converted-bank-statement-can-you-trust</link><guid isPermaLink="true">https://statementxl.hashnode.dev/which-rows-of-a-converted-bank-statement-can-you-trust</guid><category><![CDATA[pdf]]></category><category><![CDATA[data-quality]]></category><category><![CDATA[fintech]]></category><category><![CDATA[BookKeeping]]></category><category><![CDATA[Python]]></category><dc:creator><![CDATA[Reavid]]></dc:creator><pubDate>Wed, 21 Oct 2026 07:46:43 GMT</pubDate><content:encoded><![CDATA[<p>Most bookkeepers know the email: "Here are the statements," with a stack of PDFs attached. Sometimes the months are older than the bank's online export window, the account is already closed, or the client doesn't know there's another way. Getting rows and columns out of those PDFs isn't the hard part anymore. The hard part comes right after: which of these rows can I trust?</p>
<p>Disclosure: I build StatementXL, a bank statement PDF-to-Excel converter, and this story was written with the assistance of an AI writing program. I reviewed and edited it. The examples are generic and contain no customer data.</p>
<h2>A clean spreadsheet is not evidence</h2>
<p>PDF was designed to present a document the same way everywhere (that's the job of the <a href="https://pdfa.org/resource/iso-32000-pdf/">ISO 32000 specification</a>). It wasn't designed to carry data. A statement page is mostly characters placed at coordinates, and a "table" is usually just words that happen to line up.</p>
<p>So every converter makes a chain of guesses. Which line is the header? Where does one column end and the next begin? Is this line a new transaction, or the second line of the previous description? Most of the guesses are right and some aren't. The output looks equally tidy either way.</p>
<p>That's the real cost for a bookkeeper. If you can't tell which rows are wrong, you check all of them. The converter has saved you typing, but not review time, and review is the expensive part.</p>
<h2>What "wrong" usually looks like</h2>
<p>The failures I see fall into a handful of patterns:</p>
<ul>
<li><strong>Shifted columns.</strong> A deposit lands under "Balance", or a date and an amount end up in one cell.</li>
<li><strong>Broken descriptions.</strong> A wrapped payee name becomes two transactions, or a watermark word ("COPY") gets mixed into the description.</li>
<li><strong>Missing pieces.</strong> A row has an amount but no recognisable date, or a date but no amount.</li>
<li><strong>Things that aren't transactions.</strong> Repeated page headers, page totals and opening-balance lines get treated as real entries.</li>
</ul>
<p>None of these are exotic. Statements vary by bank, by year and sometimes by branch, and these errors come with that.</p>
<h2>Flag the row, not the file</h2>
<p>The approach I settled on is to check each extracted row against what its columns claim to contain, and then give it a label:</p>
<ul>
<li>Is the value in the date column a real calendar date, or just digits in a date-like shape?</li>
<li>Is there a description?</li>
<li>Is there an amount-shaped value in an amount column? If not, is there one sitting in a <em>different</em> column? "Amount found in the wrong column" is a far more useful message than "amount missing".</li>
</ul>
<p>Each row then gets one of three labels: <strong>OK</strong> (no issue detected), <strong>DISCREPANCY</strong> (the evidence conflicts) or <strong>UNVERIFIABLE</strong> (key evidence is missing). Each row also keeps the page it came from, so the reviewer can go straight to the right place in the PDF.</p>
<p>Data-quality tools work along similar lines. Libraries such as <a href="https://docs.greatexpectations.io/docs/">Great Expectations</a> state explicit, checkable expectations about the data instead of giving one overall pass/fail. Showing uncertainty row by row is usually cheaper than chasing a perfect extractor.</p>
<h2>Two rules that matter as much as the checks</h2>
<ol>
<li><strong>"OK" means "no issue detected", nothing more.</strong> A row can have a valid date, a description and an amount and still be the wrong transaction. If the label claimed more than the checks can prove, people would stop looking, and in bookkeeping the mistakes nobody looks for are the expensive ones.</li>
<li><strong>Never silently drop a suspicious row.</strong> If a line has text and an amount but doesn't fit the table, keep it and flag it. A truncated statement is the worst failure, because nobody goes looking for a transaction they don't know exists.</li>
</ol>
<h2>Scans, AI and where the file goes</h2>
<p>Scanned statements are a different problem. There's no text layer, so coordinate-based parsing has nothing to read. The common answer is to send the file to an AI model.</p>
<p>That can work, but it changes the privacy picture, and I think tools should say so plainly. In StatementXL the regular parser runs first for everyone. Only on the Professional and Enterprise plans, and only when regular parsing fails, can an AI fallback be used. In that case the complete PDF goes to a third-party AI parsing provider. The site itself doesn't keep a stored history of PDFs, extracted transactions or spreadsheets, but that promise doesn't cover the AI provider, which has its own processing terms. A bank statement holds names, account numbers and a month of transactions, so users deserve to know exactly when it leaves the system.</p>
<h2>If you're building or choosing a converter</h2>
<ul>
<li><strong>Measure review time, not extraction rate.</strong> A table that is almost entirely right but hides its errors can cost more than a less accurate one that flags them.</li>
<li><strong>Keep the source location on every row.</strong> A page reference is cheap and saves the reviewer real minutes.</li>
<li><strong>Prefer specific errors.</strong> "Amount in the wrong column" tells you what to do. "Low confidence" doesn't.</li>
<li><strong>Be precise about privacy.</strong> "We don't store your files" and "a third party may process your file" can both be true. Say both.</li>
</ul>
<h2>Further reading</h2>
<ul>
<li><a href="https://github.com/jsvine/pdfplumber">pdfplumber</a>, the open-source library I use for word-level PDF extraction.</li>
<li>The <a href="https://github.com/story-has-you/bank-statement-converter">StatementXL source code</a> on GitHub (MIT licensed).</li>
</ul>
<p>If you'd like to see the row labels on one of your own statements, <a href="https://bankstatementconverter.xyz/?ref=hashnode">StatementXL</a> has a small free daily quota. I'm especially interested in statement layouts that break it.</p>
]]></content:encoded></item><item><title><![CDATA[Rebuilding table columns from word coordinates in bank statement PDFs]]></title><description><![CDATA[Most bank statement PDFs don't contain a "table" in any structural sense. They contain words placed at x/y positions that look like a table to a human. If…]]></description><link>https://statementxl.hashnode.dev/rebuilding-table-columns-from-word-coordinates-in-bank-statement-pdfs</link><guid isPermaLink="true">https://statementxl.hashnode.dev/rebuilding-table-columns-from-word-coordinates-in-bank-statement-pdfs</guid><category><![CDATA[Python]]></category><category><![CDATA[pdf]]></category><category><![CDATA[data extraction]]></category><category><![CDATA[fintech]]></category><dc:creator><![CDATA[Reavid]]></dc:creator><pubDate>Mon, 19 Oct 2026 08:23:19 GMT</pubDate><content:encoded><![CDATA[<p>Most bank statement PDFs don't contain a "table" in any structural sense. They contain words placed at x/y positions that <em>look</em> like a table to a human. If the statement draws cell borders, a line-based table detector gets you most of the way. If it doesn't, you have to rebuild the columns yourself.</p>
<p>This post walks through the coordinate-based fallback I use in a bank statement → Excel converter, and the checks that decide when to use it.</p>
<h2>When to fall back</h2>
<p>I always try border/line-based detection first, because when it works it's precise. But I don't trust its output blindly. The trigger for the coordinate fallback is a simple sanity check on the result: if the header claims separate date and amount columns, but the detected cells show dates and currency values collapsed together, the table is rejected and rebuilt.</p>
<p>That one check catches most of the "it parsed, but it's wrong" cases.</p>
<h2>Step 1: words → lines</h2>
<p>Using <a href="https://github.com/jsvine/pdfplumber">pdfplumber</a>'s word extraction, each word comes with <code>x0</code>, <code>x1</code>, <code>top</code> and <code>bottom</code>. Words whose vertical positions are close enough are grouped into one visual line, then sorted left to right.</p>
<p>Before this, watermark glyphs are removed. Diagonal "COPY"-style stamps are often real text and would otherwise land in the middle of a line. They're detected by character style, relative size, rotation and page coverage rather than by matching specific strings.</p>
<h2>Step 2: find the header line</h2>
<p>A header candidate is a line whose words score well against known header vocabulary — date, description/details/memo, debit/credit/amount, balance — in several languages (the current list includes English, German and Chinese terms). The best-scoring line becomes the header; lines above it (account details, addresses, statement period) are skipped until data starts.</p>
<h2>Step 3: anchors</h2>
<p>Each header word (or cluster of nearby header words, e.g. "Transaction" + "Date") becomes a <strong>column anchor</strong> with a horizontal span. Clustering matters: without it, a two-word header becomes two columns and every row shifts by one.</p>
<h2>Step 4: assign words to anchors</h2>
<p>For every data line, each word is assigned to the anchor it overlaps most, or the nearest one. Amounts are usually right-aligned under their header, descriptions left-aligned, so "nearest" has to consider the word's span, not just its left edge.</p>
<h2>Step 5: clean up the rows</h2>
<p>The reconstructed rows still contain noise:</p>
<ul>
<li><strong>Repeated header lines</strong> on later pages are dropped.</li>
<li><strong>Page numbers</strong> and <strong>section-end markers</strong> stop or skip parsing.</li>
<li><strong>Summary and opening-balance rows</strong> are excluded from transactions.</li>
<li><strong>Continuation lines</strong> — rows with text only in the description column — are appended to the previous transaction's description.</li>
</ul>
<p>Every surviving row keeps its source page and row number, so it can be traced back to the PDF.</p>
<h2>Step 6: don't hide what didn't fit</h2>
<p>Some lines contain text and an amount but don't align with any sensible column assignment. It's tempting to drop them. I keep them and mark them as unverifiable instead, because silently truncating a statement is the worst possible failure for anyone doing bookkeeping.</p>
<p>After reconstruction, every row goes through validation (real calendar date? non-empty description? amount-shaped value in an amount column, or in the <em>wrong</em> column?) and gets one of three labels: <strong>OK</strong>, <strong>DISCREPANCY</strong> or <strong>UNVERIFIABLE</strong>. The person reviewing the spreadsheet only needs to look at the last two.</p>
<h2>Trade-offs</h2>
<ul>
<li>Coordinate reconstruction is more forgiving of borderless layouts but more sensitive to uneven column alignment.</li>
<li>Header vocabulary is a maintenance cost; new layouts sometimes need new terms.</li>
<li>None of this works on scans — there are no word coordinates without a text layer.</li>
</ul>
<p>If you'd like to see the output on a real statement, the converter is live at <a href="https://bankstatementconverter.xyz/?ref=hashnode">bankstatementconverter.xyz</a>. I'd love to hear about layouts where this approach falls apart. The code is open source (MIT) on <a href="https://github.com/story-has-you/bank-statement-converter">GitHub</a>.</p>
]]></content:encoded></item></channel></rss>