Repository navigation
Commit 10b7162
fix(plugins): answer failed tool calls in eval traces and skip HITL tools
- A TOOL_ERROR row, which the plugin logs instead of TOOL_COMPLETED when a
tool raises, becomes a `function_response` of `{"error": ...}` that holds
the error message, or `Tool execution failed` when the row has none. Every
tool call that completes or raises now has a response, which comes after
the calls that started with it. The transfer and HITL request tools still
add no response, as their calls reach the trace as AGENT_TRANSFER and HITL
request rows. `tool_error_count` no longer counts the errors of the
transfer and HITL request tools. With `eval_view_mode="full"`, TOOL_ERROR
joins the events that the eval views require, so denying it or leaving it
out of `event_allowlist` is rejected at construction.
- A TOOL_STARTING, TOOL_COMPLETED or TOOL_ERROR row that names no tool, as
when a custom `content_formatter` fails, adds no event to the trace, and
such a TOOL_STARTING is not a tool call.
- `adk_request_credential`, `adk_request_confirmation` and
`adk_request_input`, the tools that pause a turn for a credential, a
confirmation or the user's input, are no longer tool calls:
`tool_call_count`, `distinct_tools_used` and the consecutive-call columns
skip them, so asking the user the same question three times is not a tool
loop. The trace still shows each request as a `function_call`. Calls on
either side of a HITL request are not consecutive, even when the turn
resumes with the same `invocation_id`.
Co-authored-by: Haiyuan Cao <haiyuan@google.com>
PiperOrigin-RevId: 10140926771 parent 02db0ff commit 10b7162
4 files changed
Lines changed: 635 additions & 71 deletions
File tree
- src/google/adk/plugins
- bigquery_agent_analytics_sql
- tests/unittests/plugins
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2658 | 2658 | | |
2659 | 2659 | | |
2660 | 2660 | | |
2661 | | - | |
| 2661 | + | |
2662 | 2662 | | |
2663 | 2663 | | |
2664 | 2664 | | |
| |||
5466 | 5466 | | |
5467 | 5467 | | |
5468 | 5468 | | |
5469 | | - | |
5470 | | - | |
5471 | | - | |
| 5469 | + | |
| 5470 | + | |
| 5471 | + | |
| 5472 | + | |
| 5473 | + | |
| 5474 | + | |
| 5475 | + | |
| 5476 | + | |
| 5477 | + | |
| 5478 | + | |
| 5479 | + | |
| 5480 | + | |
| 5481 | + | |
| 5482 | + | |
5472 | 5483 | | |
5473 | 5484 | | |
5474 | 5485 | | |
| |||
5497 | 5508 | | |
5498 | 5509 | | |
5499 | 5510 | | |
5500 | | - | |
5501 | | - | |
5502 | | - | |
| 5511 | + | |
| 5512 | + | |
| 5513 | + | |
| 5514 | + | |
| 5515 | + | |
| 5516 | + | |
| 5517 | + | |
| 5518 | + | |
5503 | 5519 | | |
5504 | 5520 | | |
5505 | | - | |
| 5521 | + | |
| 5522 | + | |
5506 | 5523 | | |
5507 | 5524 | | |
5508 | 5525 | | |
| |||
5524 | 5541 | | |
5525 | 5542 | | |
5526 | 5543 | | |
| 5544 | + | |
5527 | 5545 | | |
5528 | 5546 | | |
5529 | 5547 | | |
| |||
7372 | 7390 | | |
7373 | 7391 | | |
7374 | 7392 | | |
7375 | | - | |
7376 | | - | |
7377 | | - | |
7378 | | - | |
7379 | | - | |
7380 | | - | |
7381 | | - | |
7382 | | - | |
| 7393 | + | |
| 7394 | + | |
| 7395 | + | |
| 7396 | + | |
| 7397 | + | |
| 7398 | + | |
| 7399 | + | |
| 7400 | + | |
7383 | 7401 | | |
7384 | 7402 | | |
7385 | 7403 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
74 | | - | |
75 | | - | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
76 | 77 | | |
77 | 78 | | |
78 | 79 | | |
79 | 80 | | |
80 | 81 | | |
81 | 82 | | |
82 | 83 | | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
83 | 94 | | |
84 | 95 | | |
85 | 96 | | |
| |||
167 | 178 | | |
168 | 179 | | |
169 | 180 | | |
170 | | - | |
171 | | - | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
172 | 184 | | |
173 | 185 | | |
174 | 186 | | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
175 | 190 | | |
176 | 191 | | |
177 | | - | |
| 192 | + | |
| 193 | + | |
178 | 194 | | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
179 | 209 | | |
180 | 210 | | |
181 | 211 | | |
| |||
210 | 240 | | |
211 | 241 | | |
212 | 242 | | |
213 | | - | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
214 | 246 | | |
215 | 247 | | |
216 | 248 | | |
| |||
222 | 254 | | |
223 | 255 | | |
224 | 256 | | |
225 | | - | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
226 | 263 | | |
227 | 264 | | |
228 | 265 | | |
| |||
248 | 285 | | |
249 | 286 | | |
250 | 287 | | |
251 | | - | |
| 288 | + | |
| 289 | + | |
252 | 290 | | |
253 | 291 | | |
254 | 292 | | |
| |||
286 | 324 | | |
287 | 325 | | |
288 | 326 | | |
| 327 | + | |
289 | 328 | | |
290 | 329 | | |
291 | 330 | | |
| |||
306 | 345 | | |
307 | 346 | | |
308 | 347 | | |
309 | | - | |
310 | | - | |
311 | | - | |
312 | | - | |
313 | | - | |
314 | | - | |
315 | | - | |
316 | | - | |
| 348 | + | |
317 | 349 | | |
318 | 350 | | |
319 | 351 | | |
| |||
352 | 384 | | |
353 | 385 | | |
354 | 386 | | |
355 | | - | |
356 | | - | |
357 | | - | |
358 | | - | |
359 | | - | |
360 | | - | |
| 387 | + | |
| 388 | + | |
361 | 389 | | |
362 | 390 | | |
363 | 391 | | |
| |||
379 | 407 | | |
380 | 408 | | |
381 | 409 | | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
382 | 428 | | |
383 | 429 | | |
384 | 430 | | |
| |||
408 | 454 | | |
409 | 455 | | |
410 | 456 | | |
411 | | - | |
| 457 | + | |
| 458 | + | |
412 | 459 | | |
413 | 460 | | |
414 | 461 | | |
| |||
0 commit comments