OCR-Based Identifier Extraction in Desktop Applications

This rule extracts structured text-based identifiers (e.g., purchase order numbers, case IDs) from desktop applications where data access via COM (Component Object Model) or DOM (Document Object Model) is unavailable. OCR is performed locally before transmitting extracted text to Cloud Analysis.

Why Use OCR Extraction?

• Some enterprise applications (e.g., legacy SAP GUI) lack structured UI data access.

• OCR captures key identifiers displayed visually but not exposed via APIs.

• Local processing ensures security, with only extracted text transmitted.

JSON Code

Expand to see example code

sap_ocr_identification.json

{
  "id": "e0029cc2-7ed3-4d9b-9fb2-1a7b03b6a095",
  "tags": [
    {
      "key": "content-category",
      "value": "Create Purchase Order"
    }
  ],
  "allow_tracking": true,
  "salvage_fields": ["text_events", "interacted_elements"],
  "extract_identifiers": [
    {
      "key": "SAP_PO_number",
      "identifier_name": "SAP_PO_number_creation",
      "id": "21cf3053-0318-4af5-a3c7-5e1aa67e8a82",
      "hash_identifier": false
    },
    {
      "key": "SAP_PO_number",
      "identifier_name": "SAP_PO_number",
      "id": "b5220460-753a-465e-b194-8b34fbced1c0",
      "hash_identifier": false
    }
  ],
  "reactions": [
    {
      "on": ["LeftClick", "SwitchWindow"],
      "react": [
        {
          "ExtractTextFromApplicationZone": {
            "X": 7,
            "Y": 7,
            "Width": 555,
            "Height": 33,
            "As": "SAP_PO_number",
            "Origin": "LeftBottom",
            "MatchingRe": "((?:45|44|47)\\d{8})"
          }
        }
      ]
    },
    {
      "on": ["SwitchWindow"],
      "react": [
        { "Go": "Foreground" },
        { "Go": { "Child": 3 } },
        {
          "Extract": {
            "Property": "Name",
            "As": "SAP_PO_number",
            "MatchingRe": "((?:45|44|47)\\d{8})"
          }
        }
      ]
    }
  ],
  "matching_criteria": {
    "rule_engine_rule": "active_process_name == context.process and title=~~context.regex",
    "context": {
      "process": "saplogon.exe",
      "regex": "Create Purchase Order"
    }
  }
}

2KB

sap_ocr_identifier.json

Explanation

1. OCR Usage

   • Used when applications do not expose UI data via COM or DOM.

   • Captures text from graphical elements instead of structured APIs.

2. Local OCR Processing

   • Extraction occurs locally on Windows machines using the Process Intelligence Plugin.

   • Text is extracted from predefined application zones (coordinates-based extraction).

3. Secure Cloud Processing

   • Only extracted text is sent to Cloud Analysis—not raw screen captures.

   • Ensures privacy compliance while enabling process insights.

Expected Outcome

• Extracts key identifiers (e.g., PO numbers, case numbers) from desktop applications.

• Local OCR processing ensures security before transmitting text-based results.

• Dynamically tracks user interactions to extract structured information for process intelligence.