halcmd: add -p option to generate PlantUML diagram of HAL pins/signals#4214
halcmd: add -p option to generate PlantUML diagram of HAL pins/signals#4214petterreinholdtsen wants to merge 1 commit into
Conversation
Emit bracket-style component boxes grouped by instance, with the component type name (loadrt/loadusr module name) in parentheses. Signals are rendered as queue entities so one writer can fan out to multiple readers via a single node. Components with all unconnected pins are filtered out. Also documents the new -p option in the halcmd man page. This patch was created with help from OpenCode using local llama.cpp server with Qwen 3.6.
petterreinholdtsen
force-pushed
the
halcmd-plantuml-output
branch
from
3d1fab3 to
cb8b95f
Compare
|
Why can't you just use script-mode for output (halcmd -s) and post-process with your own script? |
Who say I can not? I just believe it is more convenient and available for a large audience if the program do this directly and on its own, instead of having to track down a separate tool. This patch is actually a byproduct of an experiment adding a graph tab to halshow. Have not yet found a way to make that graph pretty and useful, so in the mean time I have been using plantuml to get an overview of the HAL setup and changes on machines. :) |
|
Do you have a sample of the plantuml output as SVG? |
|
But still, the correct way then is to make the script that is part of LinuxCNC for that "large audience". You still need plantuml for it to function. So, you make a script that calls The problem I'm addressing here is that you can add N+1 output interfaces to halcmd and still not be able to capture them all for any audience's liking. However, it makes halcmd unnecessarily complex and difficult to maintain. Therefore, using one standard script-friendly output format, which we have, should suffice and everything else is external or encapsulated post-processing. |
Sure. Attached. |
While it of course is true and sound very dramatic, I suspect that in reality, there are two major formats that are relevant, the graphviz dot format and plantuml. So N=0 will make a lot of people happy and N=1 will provide a useful format for the wast majority. I would be happy to add a -g option for graphviz dot, if you believe it is vital to increase the audience approval. |
|
What about PGF/TikZ? Or R? Or Vym?. Then there are numerous other packages. Just looking at the very long list of available UML tools should give you an indication how many different ways there are to tell a similar story. There is the whole category of argument/concept mapping software and associated languages. You may not be able to surface once you down that rabbit hole and forever stay in the Red Queen's prison with your head chopped off. FWIW, we already have N=1you know the saying, use N=0, N=1 or N=∞. With N=1 we have the script-friendly output. As said, if it is insufficient or incomplete, then it should be fixed to be generic. But I'd strongly resist adding any specific target language. |
|
[BsAtHome]
What about PGF/TikZ? Or R? Or Vym?. Then there are numerous other
packages. Just looking at the very long list of available UML tools
should give you an indication how many different ways there are to
tell a similar story. There is the whole category of argument/concept
mapping software and associated languages. You may not be able to
surface once you down that rabbit hole and forever stay in the Red
Queen's prison with your head chopped off.
I can probably even come up with my own special graph format, to
increase the number of packages and formats. But doing this do not
really affect the popularity of the most used formats, and thus not my
observation on how many formats are needed to make the majority
happy. :)
A more relevant measurement might be to see how many competing LinuxCNC
extentions exist for drawing a graph from the HAL structure, and which
format they are using. I am thinking about projects like
<URL: https://github.com/JTrantow/LCNC_HAL_to_graphviz > and the many
mentioned in
<URL: https://www.forum.linuxcnc.org/24-hal-components/37821-graphing-a-hal-configuration >
FWIW, we already have N=1<sub>you know the saying, use N=0, N=1 or
N=∞</sub>. With N=1 we have the script-friendly output. As said, if it
is insufficient or incomplete, then it should be fixed to be generic.
But I'd strongly resist adding any specific target language.
Always fun with some math, and while induction is a powerful tool, it is
unlikely to apply if the goal is to provide for the majority, not track
down every potential user and cater for every need. :)
…--
Happy hacking
Petter Reinholdtsen
|
|
The current script-friendly output isn't something that any apt-installable package can parse into a diagram. Ideally we would have a version of Halshow where you could drag the blocks around to untangle lines, and the values were displayed live on the nets. But at that point we would be re-inventing Simulink or LabView. It's worth remembering, also, that most HAL files are fit-and-forget. after initial configuration they are not looked at again for years at a time. For most users there are more valuable places to invest developer effort. This PR was prompted by trying to get to the bottom of the spindle control on Petter's Mazak, which after a few years was a mystery to both of us. Using this script (converted to Python3) was very helpful when we were trying to work out how the spindle was controlled |
|
I agree that However, having ways to generate a graph from the connections is for sure nice and looking at the above repo, parsing the output of Options:
Adding a few python graph generators into the repo in an appropriate location would probably also be fine. Together with python hal -> graph generator -> render tool, this can be even integrated into an UI. The drag mode is also realizable by improving python hal so all info can be gathered and then a python tool that does this. As much as I know, there are nice library's to do that. |
|
In principle, you need a "dump" of all modules/threads/functions/components/signal/pins/parameters with all the retained information in a structured format. Although halmodule may be interesting, it is not necessarily the right place. However, there may be a case for it (see below). |
|
Hmm, is this an issue if you have to create a temporary halmodule? Many components do this like halshow / halcmd and so on. Of course, having better separation would help and there is really no need for a component just to access shm. A small python script that shows what is already available in hal python. For the get_info_pins() a DRIVER field is missing and there is also no get_info_threads(). But this could be done and then all should be there needed to create a graph in python. import hal
import os
comp_name = f"halpy{os.getpid()}"
if not hal.is_initialized():
comp = hal.component(comp_name)
print("pins-----------")
for pin in hal.get_info_pins():
for k, v in pin.items():
print(k, v)
print()
print("signals-----------")
for sig in hal.get_info_signals():
for k, v in sig.items():
print(k, v)
print()
print("params-----------")
for par in hal.get_info_params():
for k, v in par.items():
print(k, v)
print() |
|
A small change in halmodule.cc and I think everything needed to generate a diagram from python is there. If hal.get_info_components() / hal.get_info_threads() is needed, this should also be possible. It would just expand an already existing pattern. @BsAtHome Do you think this could be a way? Yes, it needs a component but for now, all modules accessing the HAL need one. If your changes gets in, the component creation could be dropped, done. diff --git a/src/hal/halmodule.cc b/src/hal/halmodule.cc
index b0a132fd0c..8be61a160d 100644
--- a/src/hal/halmodule.cc
+++ b/src/hal/halmodule.cc
@@ -1554,10 +1554,12 @@ PyObject *get_info_pins(PyObject * /*self*/, PyObject * /*args*/) {
static const char str_v[] = "VALUE";
static const char str_t[] = "TYPE";
static const char str_d[] = "DIRECTION";
+ static const char str_s[] = "SIGNAL";
hal_data_u *d_ptr;
hal_pin_t *pin;
hal_sig_t *sig;
+ char* sig_name;
PyObject* python_list = PyList_New(0);
PyObject *obj;
@@ -1573,61 +1575,69 @@ PyObject *get_info_pins(PyObject * /*self*/, PyObject * /*args*/) {
if (pin->signal != 0) {
sig = (hal_sig_t*)SHMPTR(pin->signal);
d_ptr = reinterpret_cast<hal_data_u *>(SHMPTR(sig->data_ptr));
+ sig_name = sig->name;
} else {
sig = NULL;
d_ptr = &(pin->dummysig);
+ sig_name = (char*)"UNCONNECTED";
}
-
/* convert to dict of python values */
switch(type) {
case HAL_BIT:
- obj = Py_BuildValue("{s:s,s:N,s:N,s:N}",
+ obj = Py_BuildValue("{s:s,s:N,s:N,s:N,s:s}",
str_n, pin->name,
str_v, PyBool_FromLong((long)d_ptr->b),
str_d, PyLong_FromLong(pin->dir),
- str_t, PyLong_FromLong(HAL_BIT));
+ str_t, PyLong_FromLong(HAL_BIT),
+ str_s, sig_name);
break;
case HAL_U32:
- obj = Py_BuildValue("{s:s,s:k,s:N,s:N}",
+ obj = Py_BuildValue("{s:s,s:k,s:N,s:N,s:s}",
str_n, pin->name,
str_v, (unsigned long)d_ptr->u,
str_d, PyLong_FromLong(pin->dir),
- str_t, PyLong_FromLong(HAL_U32));
+ str_t, PyLong_FromLong(HAL_U32),
+ str_s, sig_name);
break;
case HAL_S32:
- obj = Py_BuildValue("{s:s,s:l,s:N,s:N}",
+ obj = Py_BuildValue("{s:s,s:l,s:N,s:N,s:s}",
str_n, pin->name,
str_v, (long)d_ptr->s,
str_d, PyLong_FromLong(pin->dir),
- str_t, PyLong_FromLong(HAL_S32));
+ str_t, PyLong_FromLong(HAL_S32),
+ str_s, sig_name);
break;
case HAL_U64:
- obj = Py_BuildValue("{s:s,s:K,s:N,s:N}",
+ obj = Py_BuildValue("{s:s,s:K,s:N,s:N,s:s}",
str_n, pin->name,
str_v, (unsigned long long)d_ptr->lu,
str_d, PyLong_FromLong(pin->dir),
- str_t, PyLong_FromLong(HAL_S64));
+ str_t, PyLong_FromLong(HAL_S64),
+ str_s, sig_name);
break;
case HAL_S64:
- obj = Py_BuildValue("{s:s,s:L,s:N,s:N}",
+ obj = Py_BuildValue("{s:s,s:L,s:N,s:N,s:s}",
str_n, pin->name,
str_v, (long long)d_ptr->ls,
str_d, PyLong_FromLong(pin->dir),
- str_t, PyLong_FromLong(HAL_S64));
+ str_t, PyLong_FromLong(HAL_S64),
+ str_s, sig_name);
break;
case HAL_FLOAT:
- obj = Py_BuildValue("{s:s,s:d,s:N,s:N}",
+ obj = Py_BuildValue("{s:s,s:d,s:N,s:N,s:s}",
str_n, pin->name,
str_v, (double)d_ptr->f,
str_d, PyLong_FromLong(pin->dir),
- str_t, PyLong_FromLong(HAL_FLOAT));
+ str_t, PyLong_FromLong(HAL_FLOAT),
+ str_s, sig_name);
break;
case HAL_PORT:
- obj = Py_BuildValue("{s:s,s:l,s:N,s:N}",
+ obj = Py_BuildValue("{s:s,s:l,s:N,s:N,s:s}",
str_n, pin->name,
str_v, (long)d_ptr->p,
str_d, PyLong_FromLong(pin->dir),
- str_t, PyLong_FromLong(HAL_PORT));
+ str_t, PyLong_FromLong(HAL_PORT),
+ str_s, sig_name);
break;
case HAL_TYPE_UNSPECIFIED: /* fallthrough */ ;
case HAL_TYPE_UNINITIALIZED: /* fallthrough */ ;
@@ -1636,7 +1646,8 @@ PyObject *get_info_pins(PyObject * /*self*/, PyObject * /*args*/) {
str_n, pin->name,
str_v, NULL,
str_d, PyLong_FromLong(pin->dir),
- str_t, NULL);
+ str_t, NULL,
+ str_s, sig_name);
break;
}
Python script: import hal
import os
comp_name = f"halpy{os.getpid()}"
if not hal.is_initialized():
comp = hal.component(comp_name)
print("pins-----------")
for pin in hal.get_info_pins():
for k, v in pin.items():
print(k, v, end='; ')
print()
print("signals-----------")
for sig in hal.get_info_signals():
for k, v in sig.items():
print(k, v, end='; ')
print()
print("paramteters-----------")
for par in hal.get_info_params():
for k, v in par.items():
print(k, v, end='; ')
print()Output: |
|
here my one-script solution: https://www.youtube.com/watch?v=isgavyVCYJc it using graphviz for the initial layout and QGraphicsScene for display and editing |
So if you're going to use an export format, Graphviz / dot is the better choice |
|
Halmodule does not provide any relation to the owner component of the pins and parameters. You also need to implement component listing. And while you are at it, you want all the other info too. That is simply not implemented in halmodule and is a bit off-topic for halmodule. It would then be better to create a And, about the format, there is no point in having a "well, most use XYZ" format implemented. You need a generic interface so that the question of "who can interface" is answered with "anyone, use your own parser/converter to mangle the generic output". FWIW, I already created a #!/usr/bin/env python3
import halquery
print("# Pins")
for k, v in halquery.pins().items():
print(k, v)
print("# Params")
for k, v in halquery.params().items():
print(k, v)
print("# Signals")
for k, v in halquery.signals().items():
print(k, v)
print("# Components")
for k, v in halquery.comps().items():
print(k, v)
print("# Functions")
for k, v in halquery.functs().items():
print(k, v)
print("# Threads")
for k, v in halquery.threads().items():
fc = v['functions']
v['functions'] = '...'
print(k, v)
for f in fc:
print(" ", f)It reveals the complete internal structure in python dictionaries which you can use in any way you want and mangle into whatever you like. |
That's why I think we should not export a specific format at all as @BsAtHome pointed out to. This is a discussion that can never be ended... ;-) The better way is good support for everyone that wants to write a graph exporter or a real time graph. Based on all scripts I have seen so far, having this as a python module probably would do it. That would involve the following steps:
This would result in the end: This way, all can be integrated without one integration breaking an other one and without having a single line of change in halcmd. Reasoning behind:
That said, this is just my opinion and a short concept. Other opinions are welcome. @multigcs I really like you viewer. The only thing I don't really like is that you have to use halcmd and decode the text to get the graph. Not your fault, I think there is just now other way to do this but we should change that. |
Looks nice! I was not aware you are already working on it. One thing: Having type / dir as the C enum value doesn't look python style to me: https://docs.python.org/3/library/enum.html @grandixximo seams you know python better than me, any insight? I know it is also done this way in halmodule.cc but I think there are better ways. Either directly in the C Python library or also on the python side by using the C constants. Do you have a branch to share? |
Thanks ! I would have liked to do it entirely using the HAL module, but as you already mentioned, I was missing some information:
But the advantage of the halcmd parser is that you can use a dump to view other people's configurations. :) |
Looks like @BsAtHome is working on it.
The disadvantage is, it will break as soon someone decides to change the way the values are shown on the command line... ;-) The scripting mode You can still dump the config. Just dump the python structure as json. Then you can read it in again on your side to check. https://docs.python.org/3/library/json.html should do the trick. |
|
The values are raw C enum ints, and I think it's worth noting the module already exports matching constants ( I'd lean toward the The nice part, I think, is that since both are int subclasses this stays fully backward compatible: My preference would be to define these enums once on the python side (in hal.py, or a shared module that both halmodule and your new halquery could import) and alias |
|
Yes, the enum question is a bit harder because they are slightly disjoint between C/C++ and Python. An enum is part of a class in python. That is in itself is fine, but you need to set it from C, which is involved afaik. Secondly, you need to split the enum definition out into a separate module (like f.ex. haldefs) because both halmodule and halquery would need them. Creating the a halquery module was very easy because I already have separated out all access to HAL in a query API. It also was a good test-case for the query API. There is no code uploaded in a branch, yet. This is my working tree that builds on #4099 (and that seems to be in limbo). There is, in my version, a very clear separation between hal_lib and everything else. No code can any longer include hal_priv.h (only halrmt still needs fixing). I'd rather not start backporting all that because it means distributing more mess to cleanup again later. Let me see if I can fork off a branch so you can see what is living in it. Will take a some time... @multigcs for dumping running configs we just add a command |
Oops, I didn't know about that option—I've changed it. Thanks! |
This is the local tree pushed: Beware: this is WIP. Adjustments and fixing some ordering/naming issues are expected before the query API is finalized and fixed (have a look at the bottom of hal.h). Nearly all old hal types are gone and replaced, still working on the remaining few. Tests pass, except... Warnings are not an error at the moment in the build. Rtai has several deprecation warnings that are exposed but I don't fully understand why they trigger. Uspace RIP builds should be warning free. |
6 participants
Emit bracket-style component boxes grouped by instance, with the component type name (loadrt/loadusr module name) in parentheses. Signals are rendered as queue entities so one writer can fan out to multiple readers via a single node.
Components with all unconnected pins are filtered out.
Also documents the new -p option in the halcmd man page.
This patch was created with help from OpenCode using local llama.cpp server with Qwen 3.6.