AI for Handwritten Order Data Processing

In today’s fast-paced business environment, speed, efficiency and accuracy are paramount to maintaining a competitive edge. This is particularly true in the traditional wholesale distribution industry (in India), where processing large volumes of handwritten orders can be both time-consuming and error-prone.

The Business Challenge

As a solutions architect at Teqnius, my team and I had just begun working on the digital transformation of a client in the footwear distribution business when an urgent issue surfaced. Following a major sales event, they were overwhelmed with handwritten orders that needed immediate processing. These orders included:

• Brand Name
• Model Number
• Colors
• Sizes
• Quantities

Manually processing this data presented several challenges including:

• Different handwriting styles: Many were difficult to read and interpret accurately.
• Lack of context awareness: Data entry operators who would need to be hired might not understand the nuances of the data.
• Slow processing speed: Manual entry was too slow to meet the distributor’s urgent needs.
• Error-prone processes: Mistakes in transcription could result in incorrect orders and delays.

With a competitor’s event and the festive season just around the corner, The client needed a fast, reliable solution that would help them analyze and process the orders.

The AI-Driven Solution

To tackle this challenge, we developed an AI-powered data entry solution using both Gemini 1.5 Pro and ChatGPT, training each model on the client’s dataset of brand names, colors, sizes, and model information extracted from their ERP system. While both models showed promise, Gemini 1.5 Pro emerged as the superior tool for this use case. A human-in-the-loop interface was also implemented for manual review and corrections. Here’s how the project unfolded:

1. Training the AI Models

We extracted a dataset from the client’s ERP system containing valid entries for brands, colors, sizes, and models.

Both ChatGPT and Gemini 1.5 Pro were trained on this dataset, enabling them to recognize domain-specific details and handle handwritten data effectively.

We evaluated each model’s accuracy and efficiency in extracting key data points. Gemini 1.5 Pro outperformed ChatGPT due to its advanced contextual understanding and superior accuracy in handling structured data.

2. Image Preprocessing

Handwritten orders were digitized using mobile photography. To address concerns about data leakage while using Large Language Models (LLMs), we anonymized sensitive information in the images before processing.

3. Handwritten Text Recognition (HTR)

We leveraged Gemini’s advanced handwriting recognition capabilities to convert handwritten text into machine-readable format with high accuracy.

4. Data Extraction and Human Review

Gemini’s superior training and advanced NLP capabilities enabled it to extract data fields like brand, color, size, and quantity with high accuracy.

A custom interface was designed to display the extracted data alongside the corresponding image of the handwritten order. Users could review the data, compare it with the image, and make corrections as needed.

This interface ensured that even edge cases—such as unclear handwriting or missing data—could be resolved efficiently.

5. Data Validation

The extracted data was validated against the ERP system’s master dataset, ensuring accuracy and consistency.

Any discrepancies or unrecognized entries flagged during the review were updated through the interface, maintaining high data integrity.

6. Export to Structured Format

The validated data was exported into structured formats like Excel and CSV for easy analysis and integration into inventory systems.

The final output seamlessly fed into the distributor’s sales and inventory management systems, accelerating their workflows.

Key Takeaways: Gemini vs. ChatGPT

While both models were trained on the same dataset, Gemini 1.5 Pro demonstrated significant advantages:

1. Higher Accuracy: Gemini consistently delivered better results in recognizing handwritten data, especially for complex fields.
2. Contextual Understanding: Its ability to understand the relationships between fields (e.g., brand and size ranges) reduced errors.
3. Efficiency: Gemini required fewer iterations and less manual correction, making it the more time-effective solution.

Business Impact and Results

1. Significant Time Savings: Tasks that previously required days of manual data entry were completed in hours, allowing the client to focus on sales and inventory planning.
2. Improved Accuracy: The combination of AI and human review ensured consistently high accuracy in data extraction and validation.
3. Competitive Advantage: By rapidly processing orders, the client maintained a competitive edge, meeting customer demand even as competitors vied for market share.

Why This Matters

This project underscores the value of combining AI and human oversight to optimize manual processes. The integration of a user-friendly review interface ensured high data quality while maintaining the speed and efficiency of automated systems. By leveraging AI tailored to their specific needs, businesses can optimize workflows, reduce costs, and improve decision-making—all while ensuring accuracy and trust in their data.