Examples menu

Examples / Fill in forms

Fill an online form from a scanned paper form

A person uploads a scan of a paper form they filled in by hand, and the online version of the form fills itself in from it.

Paper forms do not go away: people print them, fill them in by hand, and send a scan or a photo. This example reads that scan into the online version of the same form, in two steps, each of which the person can see and correct:

  1. An AI Document field reads the scan. It sends the upload to an AI model that can read images, with instructions the form designer wrote in its instructions setting: copy each handwritten field as Section, Label: value. That transcript goes into the form's What the scan says field, the AI Document's target.
  2. Agent Input, part of the UAG's browser library @formio/uag-js, fills the form's fields from the transcript, as in Fill a form from a description. What the scan says is the field it reads (uagField: "input"), and every field it should fill carries the same task name in uag (here fill).

You can upload a PNG, a JPEG or a scanned PDF; the UAG turns images into a PDF before the AI reads them. Nothing is saved until the person checks the form and submits it, and the form's own validation applies to every value. If a line of the transcript is wrong, the person can correct it and click outside the box: the fields they have not changed themselves update to match.

A few lines of page code wait for the AI Document's result, then ask Agent Input to read it. Open the blank paper form to see the original, and the sample scan to see a filled-in copy as a scanner captured it.

Try it: click Use the sample scan, or upload your own scan of the blank form (up to 3 MB), and watch the transcript appear and then the form fill in.

To run this on your own UAG, you need the AI Document add-on (formio/uag-documents) and an AI model that can read images, such as Claude.

Turn it on

json
{
  "title": "Example: Fill an online form from a scanned paper form",
  "name": "exampleFillFromScannedForm",
  "path": "example-fill-from-scanned-form",
  "type": "form",
  "display": "form",
  "tags": [
    "uag"
  ],
  "properties": {
    "description": "A change of address: the person's details, their current and new address, and how to contact them, filled in from a scan of the paper form."
  },
  "components": [
    {
      "type": "uagdocument",
      "key": "scan",
      "label": "Upload the scanned paper form",
      "input": true,
      "tree": true,
      "tableView": false,
      "description": "A scan or photo of the filled-in paper form: PDF, PNG or JPEG, up to 3 MB.",
      "uag": {
        "plugin": "analyze",
        "settings": {},
        "instructions": "This is a scan of a Change of Address form, filled in by hand. Transcribe what the person wrote in every field, one line per field, as \"Section, Label: value\" with the form's own printed section names and labels (for example \"New address, City: Portland\"). Copy the handwriting exactly as written; do not correct, complete or reformat it. Write (blank) for an empty field and (unclear) for writing you cannot read. For the contact section, name the box that is ticked. Write nothing else: no heading, no summary.",
        "instructionsComponent": "",
        "accept": [],
        "converters": {},
        "engine": "",
        "auto": true,
        "target": "transcript"
      },
      "components": [
        {
          "type": "file",
          "key": "files",
          "label": "Scanned form",
          "input": true,
          "storage": "base64",
          "multiple": false,
          "hideLabel": true,
          "filePattern": ".pdf,.png,.jpg,.jpeg",
          "fileMaxSize": "3MB",
          "tableView": false
        },
        {
          "type": "hidden",
          "key": "result",
          "input": true
        }
      ]
    },
    {
      "type": "textarea",
      "key": "transcript",
      "label": "What the scan says",
      "input": true,
      "rows": 8,
      "description": "Read from the scan by the AI. Check it against the paper and correct it if needed: the fields below are filled in from it.",
      "properties": {
        "uag": "fill",
        "uagField": "input"
      }
    },
    {
      "type": "panel",
      "key": "yourDetails",
      "title": "Your details",
      "input": false,
      "properties": {
        "uag": "fill"
      },
      "components": [
        {
          "type": "columns",
          "key": "columnsFullName",
          "input": false,
          "columns": [
            {
              "components": [
                {
                  "type": "textfield",
                  "key": "fullName",
                  "label": "Full name",
                  "input": true,
                  "validate": {
                    "required": true
                  }
                }
              ],
              "width": 8,
              "size": "md"
            },
            {
              "components": [
                {
                  "type": "textfield",
                  "key": "reference",
                  "label": "Reference number",
                  "input": true
                }
              ],
              "width": 4,
              "size": "md"
            }
          ]
        },
        {
          "type": "columns",
          "key": "columnsEmail",
          "input": false,
          "columns": [
            {
              "components": [
                {
                  "type": "email",
                  "key": "email",
                  "label": "Email",
                  "input": true,
                  "validate": {
                    "required": true
                  }
                }
              ],
              "width": 8,
              "size": "md"
            },
            {
              "components": [
                {
                  "type": "phoneNumber",
                  "key": "phone",
                  "label": "Phone",
                  "input": true
                }
              ],
              "width": 4,
              "size": "md"
            }
          ]
        }
      ]
    },
    {
      "type": "panel",
      "key": "currentAddress",
      "title": "Current address",
      "input": false,
      "properties": {
        "uag": "fill"
      },
      "components": [
        {
          "type": "textfield",
          "key": "currentStreet",
          "label": "Street address",
          "input": true,
          "validate": {
            "required": true
          }
        },
        {
          "type": "columns",
          "key": "columnsCurrentCity",
          "input": false,
          "columns": [
            {
              "components": [
                {
                  "type": "textfield",
                  "key": "currentCity",
                  "label": "City",
                  "input": true,
                  "validate": {
                    "required": true
                  }
                }
              ],
              "width": 6,
              "size": "md"
            },
            {
              "components": [
                {
                  "type": "textfield",
                  "key": "currentState",
                  "label": "State",
                  "input": true
                }
              ],
              "width": 2,
              "size": "md"
            },
            {
              "components": [
                {
                  "type": "textfield",
                  "key": "currentZip",
                  "label": "ZIP code",
                  "input": true
                }
              ],
              "width": 4,
              "size": "md"
            }
          ]
        }
      ]
    },
    {
      "type": "panel",
      "key": "newAddress",
      "title": "New address",
      "input": false,
      "properties": {
        "uag": "fill"
      },
      "components": [
        {
          "type": "textfield",
          "key": "newStreet",
          "label": "Street address",
          "input": true,
          "validate": {
            "required": true
          }
        },
        {
          "type": "columns",
          "key": "columnsNewCity",
          "input": false,
          "columns": [
            {
              "components": [
                {
                  "type": "textfield",
                  "key": "newCity",
                  "label": "City",
                  "input": true,
                  "validate": {
                    "required": true
                  }
                }
              ],
              "width": 6,
              "size": "md"
            },
            {
              "components": [
                {
                  "type": "textfield",
                  "key": "newState",
                  "label": "State",
                  "input": true
                }
              ],
              "width": 2,
              "size": "md"
            },
            {
              "components": [
                {
                  "type": "textfield",
                  "key": "newZip",
                  "label": "ZIP code",
                  "input": true
                }
              ],
              "width": 4,
              "size": "md"
            }
          ]
        },
        {
          "type": "day",
          "key": "moveDate",
          "label": "Date you move",
          "input": true
        }
      ]
    },
    {
      "type": "radio",
      "key": "contactBy",
      "label": "How should we contact you?",
      "input": true,
      "values": [
        {
          "label": "Email",
          "value": "email"
        },
        {
          "label": "Phone",
          "value": "phone"
        },
        {
          "label": "Mail",
          "value": "mail"
        }
      ],
      "properties": {
        "uag": "fill"
      }
    },
    {
      "type": "day",
      "key": "signedOn",
      "label": "Date signed",
      "input": true,
      "properties": {
        "uag": "fill"
      }
    },
    {
      "type": "button",
      "key": "submit",
      "label": "Submit",
      "action": "submit",
      "input": true
    }
  ]
}

Try it

Example: Fill an online form from a scanned paper form

Read about it in the developer docs