Skip to content

fix(tools): keep docx paragraphs whole at tags that start with the paragraph tag name - #8178

Open
SoroushRF wants to merge 1 commit into
google:mainfrom
SoroushRF:fix/docx-paragraph-split
Open

SoroushRF wants to merge 1 commit into
google:mainfrom
SoroushRF:fix/docx-paragraph-split

Conversation

@SoroushRF

Copy link
Copy Markdown
Contributor

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

Problem:

_try_extract_docx_text splits word/document.xml into paragraphs with <w:p(?:[^>]*)>. Nothing requires the tag name to end after w:p, so the pattern also matches <w:proofErr>, <w:pPr>, <w:permStart>, <w:pgSz> and every other tag whose name starts with w:p. Word writes a <w:proofErr> pair around each word its spell checker flags, so load_artifacts hands the model a paragraph broken into several lines at each flagged word.

Solution:

Require the tag name to end where the paragraph tag ends: <w:p(?=[\s/>])[^>]*>. This still matches <w:p>, <w:p/>, <w:p w:rsidR="..."> and a tag name followed by a newline or tab, and no longer matches longer tag names. It is a one-line change; the namespace-prefix handling and the text extraction are untouched.

Known limitation, left out on purpose: text separated only by <w:ptab/> used to land on separate lines as a side effect of this bug and is now joined, the same way text around <w:tab/> and <w:br/> already is today. Turning tabs and breaks inside a run into whitespace is a separate change.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass in the hosted tox matrix on the PR commit.

New test test_load_artifacts_keeps_docx_paragraph_with_proofing_marks_on_one_line goes through load_artifacts_tool.process_llm_request like the neighbouring docx tests. It fails on main and passes with the fix:

main:      1 failed, 61 passed   (tests/unittests/tools/test_load_artifacts_tool.py)
this PR:   62 passed

Full tox matrix on Linux for commit 8cd1b4fc6494bf39db34058cb0c68277efd09a9b, isolated environments, pytest tests/unittests -n 2:

Python Passed Skipped Xfailed Xpassed Failed
3.11 24,823 97 39 2 0
3.12 24,814 98 39 2 0
3.13 24,814 98 39 2 0
3.14 24,816 96 39 2 0

The workflow and the dependency lock it uses live on a separate fork branch and are not part of this PR.

For transparency: a local tox run on Windows 11 has 70 to 104 failures per Python version in unrelated areas (BigQuery plugin, telemetry, Cloud Run deploy, shell and path tests). None are in the docx tests, and the files that failed on 3.13 fail the same way on untouched main.

pyink 25.12.0, isort 8.0.1, ruff and codespell are clean on both changed files.

Manual End-to-End (E2E) Tests:

Setup: a real Runner with InMemoryArtifactService, an LlmAgent with the load_artifacts tool, and a report.docx saved from Microsoft Word 16 containing two paragraphs and three misspelled words (six <w:proofErr> tags). The model is a local stub that calls load_artifacts once and prints the artifact text it is then sent, so no external model is called.

main (d05e482):
TEXT SENT TO MODEL FOR report.docx:
'The quick \nbrwn\n fox jumps \novr\n the lazy dog.\nSecond paragraph is spelt \ncorectly\n.'

this PR:
TEXT SENT TO MODEL FOR report.docx:
'The quick brwn fox jumps ovr the lazy dog.\nSecond paragraph is spelt corectly.'

I also built the wheel with uv build, installed it into a clean virtual environment, and reran the same script from outside the checkout with the same passing result.

Runner script
"""Runner E2E for the docx paragraph split: real Runner, real LoadArtifactsTool,
a .docx saved by Microsoft Word, and a stub model that prints what it is sent."""
import asyncio
import sys

from google.adk.agents import LlmAgent
from google.adk.artifacts import InMemoryArtifactService
from google.adk.models.base_llm import BaseLlm
from google.adk.models.llm_response import LlmResponse
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.adk.tools import load_artifacts
from google.genai import types

DOCX_MIME = 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'


class EchoModel(BaseLlm):
  """Calls load_artifacts once, then prints the artifact text it receives."""

  model: str = 'echo'

  async def generate_content_async(self, llm_request, stream=False):
    texts = [
        p.text
        for c in llm_request.contents
        for p in c.parts or []
        if p.text and not p.text.startswith('Artifact ')
        and c.role == 'user' and p.text != 'read report.docx'
    ]
    has_response = any(
        p.function_response for c in llm_request.contents for p in c.parts or []
    )
    if not has_response:
      yield LlmResponse(
          content=types.Content(
              role='model',
              parts=[
                  types.Part(
                      function_call=types.FunctionCall(
                          name='load_artifacts',
                          args={'artifact_names': ['report.docx']},
                      )
                  )
              ],
          )
      )
      return
    print('TEXT SENT TO MODEL FOR report.docx:')
    for t in texts:
      print(repr(t))
    yield LlmResponse(
        content=types.Content(role='model', parts=[types.Part(text='done')])
    )


async def main(path):
  artifacts = InMemoryArtifactService()
  sessions = InMemorySessionService()
  agent = LlmAgent(name='reader', model=EchoModel(), tools=[load_artifacts])
  runner = Runner(
      app_name='app', agent=agent, artifact_service=artifacts,
      session_service=sessions,
  )
  session = await sessions.create_session(app_name='app', user_id='u')
  await artifacts.save_artifact(
      app_name='app', user_id='u', session_id=session.id,
      filename='report.docx',
      artifact=types.Part(
          inline_data=types.Blob(data=open(path, 'rb').read(), mime_type=DOCX_MIME)
      ),
  )
  async for _ in runner.run_async(
      user_id='u', session_id=session.id,
      new_message=types.Content(
          role='user', parts=[types.Part(text='read report.docx')]
      ),
  ):
    pass


asyncio.run(main(sys.argv[1]))

Run as python run_e2e.py report.docx.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass with my changes (hosted tox matrix above).
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules. N/A: no dependent changes.

Additional context

#7088 edits the next line of the same function (entity decoding) and does not change the split. The two changes merge without a textual conflict.

Prepared with AI assistance.

🤖 Generated with Claude Code

…ragraph tag name

_try_extract_docx_text split paragraphs on any tag whose name begins with
"w:p", so <w:proofErr>, which Word writes around each flagged misspelling,
broke a paragraph into several lines. Match only the paragraph tag itself.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

load_artifacts splits a Word (.docx) paragraph into several lines at spell-check marks

2 participants