You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug] explain data change on operatorMem, mpp22698 test failed #295
SET gp_resqueue_print_operator_memory_limits=on;
SET gp_resqueue_memory_policy_auto_fixed_mem=100;
create table mpp22698(c1 int, c2 int) partition by range(c2) (start(1) end(6) every(1));
select get_operator_mem('explain select * from mpp22698;');
You will get the following:
NOTICE: One or more columns in the following table(s) do not have statistics: mpp22698
HINT: For non-partitioned tables, run analyze <table_name>(<column_list>). For partitioned tables, run analyze rootpartition <table_name>(<column_list>). See log for columns missing statistics.
ERROR: IndexError: list index out of range
CONTEXT: Traceback (most recent call last):
PL/Python function "get_operator_mem", line 8, in
list_opermem.add(operatorMemStr.split(':')[1])
PL/Python function "get_operator_mem"
Hi, I try to repeoduce but failed, get_operator_mem doesn't exist in CBDB,
gpadmin=# select get_operator_mem('explain select * from mpp22698;');
ERROR: function get_operator_mem(unknown) does not exist
LINE1: select get_operator_mem('explain select * from mpp22698;');
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
NOTICE: One or more columns in the following table(s) do not have statistics: mpp22698
HINT: For non-partitioned tables, run analyze <table_name>(<column_list>). For partitioned tables, run analyze rootpartition <table_name>(<column_list>). See log for columns missing statistics.
ERROR: IndexError: list index out of range
CONTEXT: Traceback (most recent call last):
PL/Python function "get_operator_mem", line 8, in
list_opermem.add(operatorMemStr.split(':')[1])
PL/Python function "get_operator_mem"
According to the message: it's not a statistic issue(just a notice and hint).
It shows the real errors are inside PL/Python function "get_operator_mem", a related pr #292 has fixed sth about operatorMem in explain output.
I think this is not a CBDB bug, change out-side-of CBDB's functionget_operator_mem according the the explain output will resolve your issue.
The following is DDL of this function. I believe something changed caused this failure.
create or replace function get_operator_mem(query text) returns text as
$$
rv = plpy.execute(query)
list_opermem = set()
for i in range(len(rv)):
cur_line = rv[i]['QUERY PLAN']
if 'operatorMem' in cur_line or 'Function Scan on gp_partition_expansion' in cur_line or 'Function Scan on gp_partition_inverse' in cur_line:
operatorMemStr = cur_line[cur_line.rindex(')'):]
list_opermem.add(operatorMemStr.split(':')[1])
return list(list_opermem)
$$
language plpython3u;
@avamingli Oh, ok. Let me look into the function. Will get back to you for confirmation. If it is an incompatible change, I would update test case accordingly.
changed the title [-][Bug] Unexpected missing statistics, mpp22698 test failed[/-][+][Bug] explain data change on operatorMem, mpp22698 test failed[/+]on Nov 10, 2023
Activity
avamingli commentedon Nov 10, 2023
Hi, I try to repeoduce but failed,
get_operator_memdoesn't exist in CBDB,Could you provide a complete case?
my-ship-it commentedon Nov 10, 2023
@congxuebin
avamingli commentedon Nov 10, 2023
According to the message: it's not a statistic issue(just a notice and hint).
It shows the real errors are inside PL/Python function "get_operator_mem", a related pr #292 has fixed sth about operatorMem in explain output.
I think this is not a CBDB bug, change out-side-of CBDB's function
get_operator_memaccording the the explain output will resolve your issue.congxuebin commentedon Nov 10, 2023
The following is DDL of this function. I believe something changed caused this failure.
create or replace function get_operator_mem(query text) returns text as
$$
rv = plpy.execute(query)
list_opermem = set()
for i in range(len(rv)):
cur_line = rv[i]['QUERY PLAN']
if 'operatorMem' in cur_line or 'Function Scan on gp_partition_expansion' in cur_line or 'Function Scan on gp_partition_inverse' in cur_line:
operatorMemStr = cur_line[cur_line.rindex(')'):]
list_opermem.add(operatorMemStr.split(':')[1])
return list(list_opermem)
$$
language plpython3u;
congxuebin commentedon Nov 10, 2023
@avamingli Oh, ok. Let me look into the function. Will get back to you for confirmation. If it is an incompatible change, I would update test case accordingly.
congxuebin commentedon Nov 10, 2023
I spoke with Mingli and confirmed this is as designed.
The following is more details I identified:
New behavior:
Gather Motion 3:1 (slice1; segments: 3) (cost=0.00..431.00 rows=1 width=8) (operatorMem: 100kB)...
Old behavior:
Gather Motion 3:1 (slice1; segments: 3) (cost=0.00..431.00 rows=1 width=8) operatorMem: 100kBThat is why Python PL failed.
operatorMemStr = cur_line[cur_line.rindex(')'):]
list_opermem.add(operatorMemStr.split(':')[1])
Gonna change test case to fix it. Closing this issue.
[-][Bug] Unexpected missing statistics, mpp22698 test failed[/-][+][Bug] explain data change on operatorMem, mpp22698 test failed[/+]Fix flaky test test_rejectmap_mul_db (#295)
Fix flaky test test_rejectmap_mul_db (#295)
Fix flaky test test_rejectmap_mul_db (#295)
Apache main rebase (#28)
Fix flaky test test_rejectmap_mul_db (#295)
Fix flaky test test_rejectmap_mul_db (apache#295)
Fix flaky test test_rejectmap_mul_db (apache#295)
Sync with apache main (#29)
Fix flaky test test_rejectmap_mul_db (apache#295)
Fix flaky test test_rejectmap_mul_db (apache#295)
Fix flaky test test_rejectmap_mul_db (apache#295)