Skip to content

Llama.close() doesn't clean up chat_handler causing null pointer on second load of a vision model #2342

Description

@WarmneoN

In ComfyUI's ToriiGate Llama.cpp Vision Generate node, when keep_model_alive is set to false, the first generation works, but the second inference crashes.

Root cause: when unloading the model, Llama.close() does not clean up the chat_handler. On second load, the C++ layer is polluted and mtmd_ctx becomes a null pointer.

Temporary fix: in nodes_api.py _drop_local_llama, before del llm:

ch = getattr(llm, "chat_handler", None)
if ch is not None and hasattr(ch, "_exit_stack"):
    try:
        ch._exit_stack.close()
    except Exception:
        pass

Environment: Python 3.13 · PyTorch 2.9.1+cu130 · ComfyUI v0.28.0 · llama-cpp-python 0.3.34

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions