Artsyl Transaction Parser Package

Artsyl Transaction Parser (ATP) is a solution aimed at facilitating automatic capture from documents containing transactional-structured data, such as Explanation of Benefits (EOB) forms, student transcripts, and all kinds of reports and statements. It is capable of processing transactions with any number of nesting levels and any quantity of fields per level. It includes all necessary components for the job – image pre-processing, page identification and document separation, a top-of-the-line OCR engine, a Transactional Analysis module that finds and classifies the transactions inside a document, and a special verification module optimized for visual verification of transactional documents. The default output data format for processed documents is XML; release scripts to other file formats and popular back-end systems are available as add-on modules.

Transactional Challenges and How ATP Works to Resolve Them

The biggest challenge of transactional documents is the complex transaction structure. Many of them look like nested tables, for example one line of patient information can be followed with three lines of event details and then with another four lines of historical reference information, and each of those lines has to be captured by the rules of the correspondent level (patient, detail, history) even though the fields are not marked in any way inside the transaction. Some kinds of lines can be repeated numerous times, others can be present only once per transaction, and still, others are optional and can be missing in any transaction without making it incorrect in any way. Above this challenge, transactional documents usually span multiple pages (some of them have thousands of pages), and the information can freely flow from one page to another, even right in the middle of the transaction.

ATP answers this challenge by treating the document as a logical structure consisting of transactions, as opposed to a sequence of separate pages. That approach allows to “smoothly” process multi-page documents with transactions flowing from page to page. Internally ATP keeps track of physical pages from which the data comes, as this can be required for visual verification of the document by an operator (comparing recognized results with an enlarged segment of the scanned image). But for transaction integrity checks, automatic cross-validation rules and other transaction-level operations, the transaction printed on the same page and the transaction that spans five pages are processed by the same rules and do not require any additional efforts from the system administrator to set up the processing.

The processing with ATP starts with acquiring the image (by operator or automatically from a Watched Folder). Then image pre-processing is performed according with the requirements set in the document definition file, such as auto-detection of page orientation and auto-rotation of pages where needed, image de-skew, de-speckle, etc. After that, the recognition phase starts. ATP uses the extremely accurate OCR engine licensed from ABBYY Software House to retrieve all “raw OCR” results from the page. During the “raw OCR” run, it monitors the confidence of the OCR conversion at the character level. If it goes below a set threshold, several more “fine-tuned OCR” runs are performed, for which the default document analysis and recognition parameters are modified based on the information retrieved on the previous runs. After that, all characters, lines, spaces between fields, etc. are classified by their type and placement on the page. Those become the “raw objects” used in the next analysis step.

The next step is Transaction Analysis. It takes the “raw objects” detected during the previous step and attempts to assemble from them the transactions of different levels based on the allowed structures described in the document definition (DD) file. The DD file is simply an XML file with a tree-like structure explaining which levels are allowed in transactions, how each level is related to other levels (parent-to-child relations and same-level relations), and what fields are required and optional for each level. At the field level the DD file contains explanation of what data types can be used in the field, how many characters it can contain, whether it can contain spaces, and many more features that help to locate the field by its structure and to more accurately retrieve the value of the field by fine-tuning the recognition settings at the zone level. The result of Transaction Analysis is a tree-like document structure that best fits the DD file description and covers all transaction lines found in the document.

The step that follows is field-level recognition. To speed up the processing, the administrator can select to skip this step and simply use the value of the fields detected during the last “raw OCR” run for the field-level values. For good-quality documents, this doesn’t influence the quality much and does increase the speed. But for low-quality images it is recommended to include the zonal-level OCR run in the processing workflow because it uses very specialized settings for each individual field inside the transaction; this makes for fewer possible choices for the OCR engine and therefore commands a higher accuracy since there are fewer chances for errors. For example, a character can look like a perfect zero, but knowing that the field is the customer’s last name, the OCR engine will not consider numerical characters as possible for the field, and will come to the right conclusion that the character is a capital “O”. All field-level settings and business rules are automatically applied to the recognized zones to increase recognition accuracy and to auto-detect suspicious sections requiring the verification operator’s attention on the next step.

The next step is visual verification of results. The DD file contains the threshold of character-level confidence that marks the character as suspicious. As with most settings, the threshold can be defined at any level on the definition tree – for example, it can be set separately for the field, or for all fields of the same transaction level, or for all levels of the transaction, or even for all types of transactions defined for the document. The system always looks first for the most “nested” value – i.e. it will see first if the field has the setting; if not, it will look at the level line; if the setting is not found there, it checks at the transaction level, and then at the document level. If the setting is not mentioned at any of these levels, the system default threshold will be used. A similar approach also works for all other settings, which allows keeping the DD files neat and small and only mentioning the settings that are unique to the specific field or transaction level.

A field can be marked for verification for several reasons – for example, for low character recognition confidence as described above, or for not belonging to the data type described in the XML DD file, or for breaking the logic of a business rule associated with the field. Criteria that can trigger the verification can be selected by the system administrator during the setup; or, as usual, the default system settings will be applied.

Even visual verification of these documents presents a special requirement: to be able to confirm or change a field value, the operator quite often needs to see other fields of the same transaction, but NOT fields belonging to unrelated transactions. To facilitate this, ATP provides several different verification modes, including the one that is the most popular among our customers – the Transaction Level mode that allows seeing all fields from that level of the transaction, but not any other fields that belong to different levels or different transactions. This mode allows the operator to go through verification of such documents with the least possible number of keystrokes, which translates into saving time and resources during the verification, the only human labor-intensive step in processing documents with ATP.

After the verification, there can be additional final check-ups performed to make sure that the captured data is correct. For example, for EOBs such check-up usually is the Reconciliation step that compares the total amount accumulated across all processed pages with the known value in the back-end accounting system. After all those checks are done, the final step is exporting data to the final destination – an XML data stream, database, or back-end application or workflow. Export can be initiated by the operator or performed automatically. In the latter case the criteria is a completion of all automatic validation checks and manual verification required.

Learn more and try it out for yourself!

If you process EOB forms, student transcripts, delivery slips, any type of reports or statements with many lines of repeating information arranged like nested tables or transactions, let our team share our experience in automating this work with you. You can contact us to see a web-based presentation or sample run of a multi-page document through ATP; we can help you calculate the expected time to automate the process in your environment and the ROI for it. ATP is available in several versions suited to a wide range of project sizes and budgets. ATP can be used as a complete turn-key solution or be plugged in a workflow already implemented at the customer’s site as a technological component to enable automatic processing of transactional documents.