QWED

Type to search pages, engines and advisories.

    Looking for API details?Search the docs

    Security·Advisories·GHSA-83v6-6h23-pvp4

    LlamaIndex and CrewAI integrations report verified=True for responses, retrieved nodes and agent outputs they never verified

    highFixed in 8.0.0GHSA-83v6-6h23-pvp4 · Published 26 October 2026

    Summary

    The LlamaIndex integration (qwed_sdk/llamaindex.py) and the CrewAI integration (qwed_sdk/crewai.py) label content as verified without running any verification:

    1. QWEDQueryEngine returns VerifiedResponse(verified=True, status="PASSED") for every response that does not match one of five arithmetic regexes. No QWED engine is called. With verify_facts=True (the default), a fact-style query (what is, who is, when did, where is, how many) takes a branch commented "For now, pass through" and also returns verified=True.
    2. QWEDVerificationTransform (a node postprocessor) marks retrieved nodes qwed_verified=True when verification raises. _verify_content ends with except Exception: return True # Pass through on error. The verify_code call it wraps reads result.verified, which qwed_sdk.VerificationResult does not have (the field is is_verified). So every node that contains a fenced code block raises, is caught, and is marked verified.
    3. QWEDVerifiedAgent.verify_output (CrewAI) starts from {"verified": True}, then reads verification.verified. That raises AttributeError, the except only records the error, and the result keeps verified: True. verification_summary() then counts the output as verified. The same happens when VerificationConfig(enabled=False), where no check runs at all.

    Affected components

    Component Location Behaviour
    QWEDQueryEngine._verify_response qwed_sdk/llamaindex.py L148-169 Non-math and fact-style responses return {"verified": True, "status": "PASSED"} without a QWED call
    QWEDVerificationTransform._verify_content qwed_sdk/llamaindex.py L250-264 Any exception returns True; result.verified always raises AttributeError
    QWEDVerifiedAgent.verify_output (CrewAI) qwed_sdk/crewai.py L265-288 Default verified: True survives any exception; verification_summary() reports a 100% rate
    QWEDQueryEngine math path qwed_sdk/llamaindex.py L151-157 Also reads result.verified; this path happens to fail closed (ERROR) because of the same AttributeError

    Root cause

    • The default branch of _verify_response is a pass-through, not a refusal. Content the integration cannot verify is reported as verified.
    • _verify_content (LlamaIndex) and verify_output (CrewAI) treat an error as success.
    • The module reads result.verified from QWEDClient, whose result type exposes is_verified. This turns every code check into an exception.

    Proof of concept (local, synthetic)

    from qwed_sdk.llamaindex import QWEDQueryEngine
    
    class FakeEngine:
        def query(self, q, **kw):
            return "The Eiffel Tower is in Berlin."
    
    engine = QWEDQueryEngine(FakeEngine(), api_key="x")
    r = engine.query("Where is the Eiffel Tower?")
    print(r.verified, r.status)   # True PASSED  (no network call is made)
    

    CrewAI, with the client returning an UNVERIFIABLE result (is_verified=False):

    agent.verify_output("2 + 2 = 5")   # {'verified': True, 'error': "... has no attribute 'verified'", ...}
    agent.verification_summary()       # {'total_outputs': 1, 'verified': 1, 'failed': 0, 'verification_rate': 1.0}
    

    For the LlamaIndex postprocessor, any node whose text contains a fenced code block gets metadata["qwed_verified"] = True, whatever the code does, because self._client.verify_code(code).verified raises and the exception is swallowed as success.

    Run against 7.3.0 with network access blocked: the query engine prints True PASSED, and _verify_content returns True for a node containing os.system(...) in a code block. The CrewAI case was also run against 7.3.0 with network access blocked (output above). main @ 4989490 has identical files.

    Impact

    Integrity of the verification signal. An application that uses VerifiedResponse.verified, the qwed_verified node metadata, or the CrewAI verify_output / verification_summary results to decide whether to show, store or act on a model answer will treat unverified, and possibly false, content as verified by QWED. Attacker capability: influence over the model output or the retrieved documents. No credentials are needed.

    Affected versions

    • The pass-through branches exist in tag v2.0.0 (the first PyPI release that ships qwed_sdk/llamaindex.py and qwed_sdk/crewai.py) through 7.3.0. Introduced in 3ff1b4b.
    • qwed 0.1.0 does not contain either module.
    • The second copy of the integrations, qwed_sdk/integrations/llamaindex.py and qwed_sdk/integrations/crewai.py, has the same pass-through and attribute errors.

    Patches

    Fixed in 8.0.0 (ef6b5df):

    • QWEDQueryEngine returns verified=False, status UNVERIFIABLE, with a reason, for any response no QWED engine verified. The pass-through branches are removed.
    • QWEDVerificationTransform._verify_content is verified only when a check ran and admitted; an error fails closed, and a node with nothing to check is marked qwed_verified: False, qwed_checked: False.
    • QWEDVerifiedAgent.verify_output (CrewAI) starts from verified: False; a disabled config reports SKIPPED and an error ERROR, both unverified.
    • The LlamaIndex, CrewAI and LangChain integrations read is_verified, which follows the server's admission (GHSA-jppc-7r58-7pjh), instead of the missing .verified.
    • The same fixes apply to qwed_sdk/integrations/.

    Upgrade with pip install -U qwed (8.0.0 or later).

    Workarounds

    Do not act on VerifiedResponse.verified, qwed_verified node metadata or verify_output results. Call QWEDClient directly and gate on the server's admission == "ADMIT".

    All advisories