Skip to content

任务适配输出异常 #61

Open
Open

Description

@xjtupy

想用作者的代码训练我自己的任务,任务大致如下:

### 任务目标
你是一个智能决策模型,负责根据用户搜索的query列表与点击的poi列表,找出数据库中缺失的query集合。

### 任务执行方式
1、找出不相关query集合:分析每一个搜索query与点击poi列表中每个poi的相关性,如果搜索query与点击poi列表中每个poi都不相关,那么这个query就是不相关query,你的分析过程需要用<no_relate_think>和</no_relate_think>包裹起来。最终找出的不相关的query集合以json格式保存,并用<no_relate_query>和</no_relate_query>包裹起来,例如:<no_relate_query>[{{"query":"xxx", "lng":"xxx", "lat":"xxx", "country_code":"xxx"}}, ...]</no_relate_query>
2、生成调用接口请求:如果存在不相关的query集合,你可以通过<search>[{{"query":"xxx", "lng":"xxx", "lat":"xxx", "country_code":"xxx"}}, ...]</search>调用检索接口,它会返回每个不相关query检索到的poi数据列表,并用<poi_recall_informations>和</poi_recall_informations>包裹起来。
3、找出数据库中缺失的query集合:分析每个不相关query和它检索到的poi数据列表中每个poi数据的相关性,如果不相关query与它检索到的poi数据列表中每个poi都不相关,那么这个query就是数据库中缺失的query,你的分析过程用<miss_query_think>和</miss_query_think>包裹起来。最终找出的数据库中缺失的query集合以json格式保存,并用<answer>和</answer>包裹起来,例如<answer>[{{"query":"xxx", "lng":"xxx", "lat":"xxx", "country_code":"xxx"}}, ...]</answer>

### 输出格式
你的输出格式必须是以下两种的一种:
1、存在不相关的query集合的输出格式:
<no_relate_think>你分析不相关query的思考过程</no_relate_think>
<no_relate_query>不相关的query集合</no_relate_query>
<search>调用搜索工具的query集合</search>
<poi_recall_informations>召回的poi列表</poi_recall_informations>
<miss_query_think>你分析缺失query的思考过程</miss_query_think>
<answer>最终答案</answer>
2、不存在不相关的query集合的输出格式:
<no_relate_think>你分析不相关query的思考过程</no_relate_think>
<no_relate_query>[]</no_relate_query>
<miss_query_think>你分析缺失query的思考过程</miss_query_think>
<answer>最终答案</answer>

用户搜索的query列表:{search_querys}
用户点击的poi列表:{click_pois}\n

出现一下问题:
1、训练之前的校验阶段,输出内容和格式正常,但调用完工具后不在继续执行

Image 2、训练阶段,除了上述错误外,还会输出异常字符,格式也是混乱的 Image Image

我的训练脚本:

#model_type=$1

export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
export GPU_NUMS=8
export WANDB_API_KEY=''
export RAY_TMPDIR='/nfs/dataset-ofs-search-v1/ray_tmpdir'
export DATA_DIR='/nfs/dataset-ofs-search-v1/ddmpeng/miss_mining_agent_v1/miss_mining_agent/data'

WAND_PROJECT='miss_mining_agent_v2'

BASE_MODEL=""
EXPERIMENT_NAME=""

#if [ "$model_type" = "llama3.1_8b_instruct" ]; then
#  BASE_MODEL='/nfs/dataset-ofs-search-v1/ddmpeng/miss_mining_agent_v2/miss_mining_agent/models/Meta-Llama-3.1-8B-Instruct'
#  EXPERIMENT_NAME='miss_mining_agent_v2-grpo-llama3.1-8b'
#elif [ "$model_type" = "llama3.1_8b_instruct_sft" ]; then
#  BASE_MODEL='/nfs/dataset-ofs-search-v1/ddmpeng/product_auto/model/merge/merge_miss_query_agent_lora_sft_ds3_llama3.1_8b'
#  EXPERIMENT_NAME='miss_mining_agent_v2-grpo-llama3.1-8b_sft'
#elif [ "$model_type" = "llama3.2_1b_instruct" ]; then
#  BASE_MODEL='/nfs/dataset-ofs-search-v1/ddmpeng/miss_mining_agent_v2/miss_mining_agent/models/Llama-3.2-1B-Instruct'
#  EXPERIMENT_NAME='miss_mining_agent_v2-grpo-llama3.2-1b'
#elif [ "$model_type" = "llama3.2_3b_instruct" ]; then
#  BASE_MODEL='/nfs/dataset-ofs-search-v1/ddmpeng/miss_mining_agent_v2/miss_mining_agent/models/Llama-3.2-3B-Instruct'
#  EXPERIMENT_NAME='miss_mining_agent_v2-grpo-llama3.2-3b'
#elif [ "$model_type" = "qwen2.5_7b_instruct_sft" ]; then
#  BASE_MODEL='/nfs/dataset-ofs-search-v1/ddmpeng/product_auto/model/merge/merge_miss_query_agent_lora_sft_ds3_qwen2.5_7b_2'
#  EXPERIMENT_NAME='miss_mining_agent_v2-grpo-qwen2.5_7b'
#else
#  BASE_MODEL=""
#  EXPERIMENT_NAME=""
#fi

#export BASE_MODEL='/nfs/dataset-ofs-search-v1/ddmpeng/LLaMA-Factory/pretrain_model/Qwen2.5-7B-Instruct'
export BASE_MODEL='/nfs/dataset-ofs-search-v1/ddmpeng/product_auto/model/merge/merge_miss_query_agent_lora_sft_ds3_qwen2.5_7b'
export EXPERIMENT_NAME='miss_mining_agent-grpo-qwen2.5_7b_lora'

HYDRA_FULL_ERROR=1 python3 -m agent_r1.src.main_agent \
    algorithm.adv_estimator=grpo \
    data.train_files=["$DATA_DIR/train.parquet"] \
    data.val_files=["$DATA_DIR/test.parquet"] \
    data.train_batch_size=64 \
    data.val_batch_size=64 \
    data.max_prompt_length=4096 \
    data.max_response_length=4096 \
    data.max_response_length_single_turn=1024 \
    data.use_default_tool_template=False \
    actor_rollout_ref.model.path=$BASE_MODEL \
    actor_rollout_ref.actor.optim.lr=1e-6 \
    actor_rollout_ref.actor.optim.lr_warmup_steps_ratio=0.285 \
    actor_rollout_ref.model.use_remove_padding=True \
    actor_rollout_ref.actor.ppo_mini_batch_size=32 \
    actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=1 \
    actor_rollout_ref.actor.use_kl_loss=True \
    actor_rollout_ref.actor.kl_loss_coef=0.001 \
    actor_rollout_ref.actor.kl_loss_type=low_var_kl \
    actor_rollout_ref.model.enable_gradient_checkpointing=True \
    actor_rollout_ref.actor.fsdp_config.param_offload=False \
    actor_rollout_ref.actor.fsdp_config.optimizer_offload=False \
    actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=1 \
    actor_rollout_ref.rollout.tensor_model_parallel_size=2 \
    actor_rollout_ref.rollout.name=vllm \
    actor_rollout_ref.rollout.gpu_memory_utilization=0.6 \
    actor_rollout_ref.rollout.n_repeat=5 \
    actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=1 \
    actor_rollout_ref.ref.fsdp_config.param_offload=True \
    algorithm.kl_ctrl.kl_coef=0.001 \
    trainer.logger=['console','wandb'] \
    trainer.project_name=$PROJECT_NAME \
    trainer.experiment_name=$EXPERIMENT_NAME \
    trainer.n_gpus_per_node=$GPU_NUMS \
    trainer.nnodes=1 \
    trainer.save_freq=20 \
    trainer.test_freq=10 \
    trainer.total_epochs=10 \
    trainer.val_before_train=True \
    trainer.log_val_generations=0 \
    tool.max_turns=2 \
    tool.tools=['dd_search'] \
    tool.env=dd_search \
    tool.max_tool_response_length=2048 \
    2>&1 | tee $EXPERIMENT_NAME.log

看了源代码,感觉没啥问题,希望作者帮忙解答下

Activity

  1. 0russwest0 commented on Aug 2, 2025

    @0russwest0
    Collaborator

    理想中的格式是什么样的,有例子吗?

  2. xjtupy commented on Aug 2, 2025

    @xjtupy
    Author

    理想中的格式是什么样的,有例子吗?
    感谢回复

    格式如下图红框所示:

    Image

    目前问题就是上面总结2种:
    1、调用工具返回结果后,不在继续执行
    2、训练输出格式混乱且带有很多奇怪输出

  3. 0russwest0 commented on Aug 2, 2025

    @0russwest0
    Collaborator

    初始权重是sft后的,能够正常按照期望的格式运行,能够正常调用工具并继续执行,但是初始的eval阶段调工具后无法继续执行,是这样吗?

    如果是这样的话可能是你环境实现的有问题,比如eos没处理好?初始阶段的eval表现应该和你sft后的初始权重一致才对。

  4. xjtupy commented on Aug 2, 2025

    @xjtupy
    Author

    初始权重是sft后的,能够正常按照期望的格式运行,能够正常调用工具并继续执行,但是初始的eval阶段调工具后无法继续执行,是这样吗?

    如果是这样的话可能是你环境实现的有问题,比如eos没处理好?初始阶段的eval表现应该和你sft后的初始权重一致才对。

    Tip

    初始权重是sft后的,能够正常按照期望的格式运行,能够正常调用工具并继续执行,但是初始的eval阶段调工具后无法继续执行,是这样吗?

    是这样的,并且我用相同的model,跑其他论文的代码是能正常调工具后继续执行

    在你的代码上复现我的任务的整个过程,参考你的retool的代码,并设置工具环境如下:

    import re
    
    from agent_r1.tool.base import BaseToolEnv, BaseTool
    from typing import List, Tuple, Any
    
    
    class XXSearchEnv(BaseToolEnv):
        def __init__(self, tools: List[BaseTool], max_tool_response_length: int):
            self.tool = tools[0]
            assert self.tool.name == "xx_search"
            self.max_tool_response_length = max_tool_response_length
    
        def step(self, raw_response: str) -> Tuple[str, List[bool], bool]:
            queries = self.extract_tool_calls(raw_response)
            if len(queries) == 0:
                return "", [], False
            queries = queries[0]
            result = self.tool.execute({"queries": queries})
            tool_response = self.format_tool_response([result['content']])
            return tool_response, [result['success']], True
    
        def batch_step(self, raw_responses: List[str]) -> Tuple[List[str], List[List[bool]], List[bool]]:
            batch_tool_response = [""] * len(raw_responses)
            batch_tool_successes = [[]] * len(raw_responses)
            batch_active = [True] * len(raw_responses)
            codes = []
            for i, raw_response in enumerate(raw_responses):
                code = self.extract_tool_calls(raw_response)
                if len(code) == 0:
                    batch_tool_response[i] = ""
                    batch_tool_successes[i] = []
                    batch_active[i] = False
                    continue
                codes.append({"queries": code[0]})
            results = self.tool.batch_execute(codes)
            i = 0
            for j in range(len(raw_responses)):
                if batch_active[j]:
                    result = results[i]
                    batch_tool_response[j] = self.format_tool_response([result["content"]])
                    batch_tool_successes[j] = [result["success"]]
                    i += 1
            return batch_tool_response, batch_tool_successes, batch_active
    
        def stop(self, raw_response: str) -> bool:
            tool_calls = self.extract_tool_calls(raw_response)
            if len(tool_calls) == 0:
                return True
            else:
                return False
            
        def extract_tool_calls(self, raw_response: str) -> List[Any]:
            """
            extract the request query after "<search>", and before "</search>"
            """
            match = re.finditer(r'<search>(.*?)</search>', raw_response, re.DOTALL)
            matches = list(match)
    
            if len(matches) <= 0:
                return []
    
            return [matches[-1].group(1).strip()]
            
        def format_tool_response(self, tool_responses: List[str]) -> str:
            if len(tool_responses) == 0:
                return ""
            if len(tool_responses[0]) > self.max_tool_response_length:
                tool_responses[0] = tool_responses[0][:self.max_tool_response_length] + "..."
            return "<poi_recall_informations>\n" + tool_responses[0] + "\n</poi_recall_informations>"

    如果是eos的问题,那还需要如何处理呢,或者还有其他原因什么原因吗

  5. 0russwest0 commented on Aug 3, 2025

    @0russwest0
    Collaborator

    可以试试在 agent_r1/llm_agent/generation.py 里把每一轮的输入输出找个例子打印出来看看

  6. xjtupy commented on Aug 7, 2025

    @xjtupy
    Author

    可以试试在 agent_r1/llm_agent/generation.py 里把每一轮的输入输出找个例子打印出来看看

    @0russwest0 给了一个训练之前校验例子的输入输出,发现在工具调用结果返回后,模型就不再输出了
    输出代码

        def run_llm_loop(self, gen_batch, env: Union[BaseToolEnv, BaseImageToolEnv]) -> Tuple[Dict, Dict]:
            """Run main LLM generation loop."""
    
            batch_size = gen_batch.batch['input_ids'].shape[0]
            
            active_mask = torch.ones(batch_size, dtype=torch.bool)
            turns = torch.zeros(batch_size, dtype=torch.int32)
            active_num_list = [active_mask.sum().item()]
            rollings = gen_batch
            meta_info = gen_batch.meta_info
            prompts = gen_batch.batch['input_ids'][:, -self.config.max_prompt_length:].clone()
    
            # Main generation loop
            for turn in range(self.config.max_turns):
                if not active_mask.sum():
                    break
    
                # Check if any sequence exceeds max length
                effective_len = rollings.batch['attention_mask'].sum(dim=1)
                length_exceeded = effective_len > self.config.max_prompt_length
    
                if length_exceeded.sum() > 0:
                    print("[WARNING] SEQUENCE LENGTH EXCEEDED MAX PROMPT LENGTH")
                    active_mask[length_exceeded] = 0
    
                raw_prompt_ids = rollings.non_tensor_batch['raw_prompt_ids']
                length_exceeded = [len(prompt_id) > self.config.max_prompt_length for prompt_id in raw_prompt_ids]
                if any(length_exceeded):
                    print("[WARNING] SEQUENCE LENGTH EXCEEDED MAX PROMPT LENGTH")
                    for prompt_id, length_exceeded_ in zip(raw_prompt_ids, length_exceeded):
                        if length_exceeded_:
                            print(f"[DEBUG] LENGTH {len(prompt_id)}: {self.tokenizer.decode(prompt_id)}")
                    active_mask[length_exceeded] = 0
                
                if not active_mask.sum():
                    print("[WARNING] NO ACTIVE SEQUENCES")
                    break
                
                if hasattr(rollings, 'non_tensor_batch') and rollings.non_tensor_batch:
                    # Create active batch with tensor data
                    rollings_active = DataProto.from_dict(
                        tensors={
                            k: v[active_mask] for k, v in rollings.batch.items()
                        },
                        non_tensors={
                            k: v[active_mask.numpy()] for k, v in rollings.non_tensor_batch.items()
                        },
                        meta_info=meta_info
                    )
                else:
                    rollings_active = DataProto.from_dict(
                        tensors={
                            k: v[active_mask] for k, v in rollings.batch.items()
                        },
                        meta_info=meta_info
                    )
    
                inputs = self.tokenizer.batch_decode(rollings_active.batch['input_ids'], skip_special_tokens=True)
                print('------------------turn:{0},输入-------------------------'.format(turn))
                print(inputs[0])
    
                rollings_active, pad_size = pad_dataproto_to_divisor(rollings_active, self.actor_rollout_wg.world_size)
                gen_output = self.actor_rollout_wg.generate_sequences(rollings_active)
                gen_output = unpad_dataproto(gen_output, pad_size=pad_size)
    
                raw_responses_ids = gen_output.batch['responses']
                responses_ids = env.process_responses_ids(self.tokenizer, raw_responses_ids)
                raw_responses = self.tokenizer.batch_decode(responses_ids, skip_special_tokens=True)
    
                print('------------------turn:{0},模型响应-------------------------'.format(turn))
                print(raw_responses[0])
    
                if isinstance(env, BaseToolEnv):
                    if self.config.use_batch_tool_calls:
                        tool_responses, _, new_active_masks = env.batch_step(raw_responses)
    
                        print('------------------turn:{0},工具调用输出-------------------------'.format(turn))
                        print(tool_responses[0])
                    else:
                        tool_responses = []
                        new_active_masks = []
                        for raw_response in raw_responses:
                            tool_response, _, active = env.step(raw_response)
                            tool_responses.append(tool_response)
                            new_active_masks.append(active)
                    tool_images = [[]] * len(raw_responses)
                elif isinstance(env, BaseImageToolEnv):
                    if self.config.use_batch_tool_calls:
                        tool_responses, tool_images, _, new_active_masks = env.batch_step(raw_responses)
                    else:
                        tool_responses = []
                        tool_images = []
                        new_active_masks = []
                        for raw_response in raw_responses:
                            assistant_message, tool_message, tool_image, success, stop = env.step(raw_response)
                            tool_responses.append(tool_message)
                            tool_images.append(tool_image)
                            new_active_masks.append(stop)
    
                responses_ids = self._example_level_pad(responses_ids, active_mask)
                tool_responses = self._example_level_pad(tool_responses, active_mask, pad_value="")
                tool_images = self._example_level_pad(tool_images, active_mask, pad_value=[])
    
                active_mask[active_mask.clone()] = torch.tensor(new_active_masks, dtype=torch.bool)
    
                turns[active_mask] += 1
    
                active_num_list.append(active_mask.sum().item())
    
                # Update states
                rollings = self._update_rolling_state(
                    rollings,
                    responses_ids,
                    tool_responses,
                    tool_images
                )
    
                outputs = self.tokenizer.batch_decode(rollings.batch['responses'], skip_special_tokens=True)
                inputs2 = self.tokenizer.batch_decode(rollings.batch['input_ids'], skip_special_tokens=True)
                print('------------------turn:{0},当前轮输出-------------------------'.format(turn))
                print(outputs[0])
                print('------------------turn:{0},下轮输入-------------------------'.format(turn))
                print(inputs2[0])
     
            print("ACTIVE_TRAJ_NUM:", active_num_list)
    
            # Compose final output
            final_output = {}
            final_output['turns'] = turns
            final_output['prompts'] = prompts
            final_output['responses'] = rollings.batch['responses'].long()
            final_output['input_ids'] = torch.cat([
                prompts,
                rollings.batch['responses'].long()
            ], dim=1)
            final_output['attention_mask'] = self.tensor_fn.create_attention_mask(final_output['input_ids'])
            if "multi_modal_data" in rollings.non_tensor_batch.keys():
                from verl.models.transformers.qwen2_vl import get_rope_index
                position_ids = []
                for i in range(len(rollings.non_tensor_batch['multi_modal_data'])):
                    position_ids.append(get_rope_index(
                        processor=self.processor,
                        input_ids=final_output['input_ids'][i],
                        image_grid_thw=rollings.non_tensor_batch['multi_modal_inputs'][i]['image_grid_thw'],
                        attention_mask=final_output['attention_mask'][i],
                    ))
    
                position_ids = torch.stack(position_ids, dim=0)
                final_output['position_ids'] = position_ids
            else:
                final_output['position_ids'] = self.tensor_fn.create_position_ids(final_output['attention_mask'])
    
            response_length = final_output['responses'].shape[-1]
            response_mask = final_output['attention_mask'][:, -response_length:]
    
            final_output['action_mask'] = response_mask.clone()
    
            for i, action_mask in enumerate(rollings.non_tensor_batch['action_mask']):
                mask_len = min(len(action_mask), response_mask.shape[1])
                final_output['action_mask'][i, :mask_len] = torch.tensor(action_mask[:mask_len]) * response_mask[i, :mask_len]
            
            final_output = DataProto.from_dict(final_output)
            final_output.non_tensor_batch = rollings.non_tensor_batch
    
            print('---------输出-----------')
            print(self.tokenizer.batch_decode(final_output['input_ids'], skip_special_tokens=True))
            return final_output

    第一轮模型的输入输出
    Image

    Image

    Image

    第二轮模型的输入输出

    Image
    Image
    第二轮无输出
    Image

    是那个参数没设置正确吗

  7. 0russwest0 commented on Aug 7, 2025

    @0russwest0
    Collaborator

    skip_special_tokens=False再打印一下试试,我怀疑是第一轮生成完在末尾加了eos

  8. xjtupy commented on Aug 7, 2025

    @xjtupy
    Author

    skip_special_tokens=False再打印一下试试,我怀疑是第一轮生成完在末尾加了eos

    第一轮生成完并没有在末尾加了eos,而是在第二轮模型直接输出了<|im_end|>,
    Image

  9. xjtupy commented on Aug 8, 2025

    @xjtupy
    Author

    skip_special_tokens=False再打印一下试试,我怀疑是第一轮生成完在末尾加了eos

    第一轮生成完并没有在末尾加了eos,而是在第二轮模型直接输出了<|im_end|>, Image

    @0russwest0 你知道这个问题产生的原因吗

  10. 0russwest0 commented on Aug 8, 2025

    @0russwest0
    Collaborator

    感觉是SFT的有问题?没理解错的话,你的理想格式应该是在一个assistant message内完成多轮工具调用并返回最终结果吧。是不是SFT的时候拆成了多个单轮训的导致模型学会了在每轮结束之后加eos?

  11. xjtupy commented on Aug 8, 2025

    @xjtupy
    Author

    感觉是SFT的有问题?没理解错的话,你的理想格式应该是在一个assistant message内完成多轮工具调用并返回最终结果吧。是不是SFT的时候拆成了多个单轮训的导致模型学会了在每轮结束之后加eos?

    确实中间有拆成单轮训练,但并没有在工具返回结果之后就停止输出的样本构造方式,大致如下几类:

    样本1:
    输入:
    prompt+输入+以下内容
    <no_relate_think>你分析不相关query的思考过程</no_relate_think>
    <no_relate_query>不相关的query集合</no_relate_query>
    <search>调用搜索工具的query集合</search>
    <poi_recall_informations>召回的poi列表</poi_recall_informations>
    <miss_query_think>你分析缺失query的思考过程</miss_query_think>
    输出:
    <answer>最终答案</answer>
    
    
    样本2:
    输入:
    prompt+输入+以下内容
    <no_relate_think>你分析不相关query的思考过程</no_relate_think>
    <no_relate_query>[]</no_relate_query>
    <miss_query_think>你分析缺失query的思考过程</miss_query_think>
    输出:
    <answer>最终答案</answer>
    
    样本3:
    输入:
    prompt+输入+以下内容
    输出:
    <no_relate_think>你分析不相关query的思考过程</no_relate_think>
    <no_relate_query>不相关的query集合</no_relate_query>
    
    
    样本4:
    输入:
    prompt+输入+以下内容
    <no_relate_think>你分析不相关query的思考过程</no_relate_think>
    <no_relate_query>不相关的query集合</no_relate_query>
    输出:
    <search>调用搜索工具的query集合</search>

    现在模型是在第一轮正确调用工具,将输出结果拼接作为下一轮输入,就直接输出<|im_end|>

    输入:
    prompt+输入+以下内容
    <no_relate_think>你分析不相关query的思考过程</no_relate_think>
    <no_relate_query>不相关的query集合</no_relate_query>
    <search>调用搜索工具的query集合</search>
    <poi_recall_informations>召回的poi列表</poi_recall_informations>
    输出:
    <|im_end|>
  12. 0russwest0 commented on Aug 8, 2025

    @0russwest0
    Collaborator
    样本1:
    输入:
    prompt+输入+以下内容
    <no_relate_think>你分析不相关query的思考过程</no_relate_think>
    <no_relate_query>不相关的query集合</no_relate_query>
    <search>调用搜索工具的query集合</search>
    <poi_recall_informations>召回的poi列表</poi_recall_informations>
    <miss_query_think>你分析缺失query的思考过程</miss_query_think>
    输出:
    <answer>最终答案</answer>
    

    样本1是不是应该改成下面这样?

    样本1:
    输入:
    prompt+输入+以下内容
    <no_relate_think>你分析不相关query的思考过程</no_relate_think>
    <no_relate_query>不相关的query集合</no_relate_query>
    <search>调用搜索工具的query集合</search>
    <poi_recall_informations>召回的poi列表</poi_recall_informations>
    输出:
    <miss_query_think>你分析缺失query的思考过程</miss_query_think>
    <answer>最终答案</answer>
    
  13. xjtupy commented on Aug 8, 2025

    @xjtupy
    Author
    样本1:
    输入:
    prompt+输入+以下内容
    <no_relate_think>你分析不相关query的思考过程</no_relate_think>
    <no_relate_query>不相关的query集合</no_relate_query>
    <search>调用搜索工具的query集合</search>
    <poi_recall_informations>召回的poi列表</poi_recall_informations>
    <miss_query_think>你分析缺失query的思考过程</miss_query_think>
    输出:
    <answer>最终答案</answer>
    

    样本1是不是应该改成下面这样?

    样本1:
    输入:
    prompt+输入+以下内容
    <no_relate_think>你分析不相关query的思考过程</no_relate_think>
    <no_relate_query>不相关的query集合</no_relate_query>
    <search>调用搜索工具的query集合</search>
    <poi_recall_informations>召回的poi列表</poi_recall_informations>
    输出:
    <miss_query_think>你分析缺失query的思考过程</miss_query_think>
    <answer>最终答案</answer>
    

    @0russwest0 写错了,就是这个

    样本1:
    输入:
    prompt+输入+以下内容
    <no_relate_think>你分析不相关query的思考过程</no_relate_think>
    <no_relate_query>不相关的query集合</no_relate_query>
    <search>调用搜索工具的query集合</search>
    <poi_recall_informations>召回的poi列表</poi_recall_informations>
    输出:
    <miss_query_think>你分析缺失query的思考过程</miss_query_think>
    <answer>最终答案</answer>
  14. 0russwest0 commented on Aug 8, 2025

    @0russwest0
    Collaborator

    那感觉没啥问题呀。按理说最开始eval的模型表现应该和训练框架无关,其实就是拿vllm做了个推理,应该只能是模型本身的问题。

  15. xjtupy commented on Aug 8, 2025

    @xjtupy
    Author

    那感觉没啥问题呀。按理说最开始eval的模型表现应该和训练框架无关,其实就是拿vllm做了个推理,应该只能是模型本身的问题。

    @0russwest0 我也觉得没啥问题,但是就是找不出原因了。先不跑你这个代码,我直接拿vllm推理试试

  16. xjtupy commented on Aug 11, 2025

    @xjtupy
    Author

    那感觉没啥问题呀。按理说最开始eval的模型表现应该和训练框架无关,其实就是拿vllm做了个推理,应该只能是模型本身的问题。

    @0russwest0 我解决了这两个问题
    1、对工具调用结果添加tag,之前模仿retool的代码未添加
    Image
    2、设置enable_prefix_caching=False可解决胡乱输出

    我还有个问题,大佬这套代码怎么适配到llama或其他模型啊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions