In ComfyUI's ToriiGate Llama.cpp Vision Generate node, when keep_model_alive is set to false, the first generation works, but the second inference crashes.
Root cause: when unloading the model, Llama.close() does not clean up the chat_handler. On second load, the C++ layer is polluted and mtmd_ctx becomes a null pointer.
Temporary fix: in nodes_api.py _drop_local_llama, before del llm:
ch = getattr(llm, "chat_handler", None)
if ch is not None and hasattr(ch, "_exit_stack"):
try:
ch._exit_stack.close()
except Exception:
pass
Environment: Python 3.13 · PyTorch 2.9.1+cu130 · ComfyUI v0.28.0 · llama-cpp-python 0.3.34
In ComfyUI's ToriiGate Llama.cpp Vision Generate node, when
keep_model_aliveis set tofalse, the first generation works, but the second inference crashes.Root cause: when unloading the model,
Llama.close()does not clean up thechat_handler. On second load, the C++ layer is polluted andmtmd_ctxbecomes a null pointer.Temporary fix: in
nodes_api.py_drop_local_llama, beforedel llm:Environment: Python 3.13 · PyTorch 2.9.1+cu130 · ComfyUI v0.28.0 · llama-cpp-python 0.3.34