Proposal: An AI-assisted UltraPlot skill for scientific plotting

[lwq-star]

Hi UltraPlot community,

First of all, thank you for building and maintaining UltraPlot. I have been using it for scientific plotting in my research for about two years, and I really enjoy working with it. It has helped me focus more on the data, the message, and the figure design, instead of repeatedly writing low-level Matplotlib boilerplate.

With the rapid rise of AI agents, I am thinking about creating an AI-assisted scientific plotting skill based on UltraPlot.

By “skill,” I mean a reusable guidance package for an AI assistant or agent: a structured set of plotting principles, UltraPlot best practices, code templates, examples, and quality-check rules that can help users generate clear, publication-ready scientific figures with UltraPlot.

My current idea is roughly:

1. Review classic scientific/data visualization principles
   I would like to start by collecting and summarizing well-known principles for scientific and data visualization, including visual perception, color choice, annotation, layout, accessibility, and graphical integrity.

2. Combine these principles with practical UltraPlot workflows
   The skill could include recommended UltraPlot patterns for common scientific plotting tasks, such as multi-panel layouts, figure-level formatting, colorbars, legends, colormaps, geographic plots, and publication-style figures.

3. Create an AI-agent workflow for scientific plotting
   The agent could guide users through questions such as: What is the scientific message of the figure? Which variables and uncertainties should be shown? What plot type and layout are appropriate? Which UltraPlot APIs should be used?

I would love to hear your thoughts:

• Would this be useful for the UltraPlot community?
• Which UltraPlot features or workflows should such a skill emphasize?
• Are there any concerns about AI agents generating plotting code, such as hallucinated APIs, version compatibility, or reproducibility?

Thanks again for the great library!

[cvanelteren]

hmm not really sure what you are asking here. I think coding is rather personal and I am not enforcing any particular pattern necessarily on people -- at least this is not my intention. I think coding agents can in principle access the source and therefore see how some of the boilerplate code could be replaced with higher level UltraPlot functions (such as format).

I tend to prefer neater (read shorter) code but some of the plots I produce are still hundreds of lines long. Over my time with scientific plotting, I stopped optimizing and started producing. That is, initially I thought I could somehow take short cuts by cutting through and putting repeated calls in functions over my plotting career. Later, I noticed that this is moot and just started considering each plot as a piece of unique art.

You are of course free to make a skill that would produce such a thing (and happy to take a look at it), but as of now I don't really now what those best types of practices would look like.

[lwq-star]

I fully agree with the point that coding is personal.

I probably did not describe the idea clearly enough. I do not intend to make scientific plotting more template-driven, or to define a preferred coding style for UltraPlot users. I also agree that each scientific figure is often quite specific to the data, the argument, and the paper, so it cannot be fully standardized.

What I have in mind is more of an AI-agent skill that helps generate plotting code from the user’s scientific goal and data context. The purpose would be to reduce the amount of manual coding where possible, while still leaving room for the user to inspect, refine, and customize the details. For fine-grained figure adjustments, I agree that the user will almost always need to modify the code manually.

So the skill would not be a set of rules saying “this is how people should write UltraPlot code.” Instead, it would provide the AI agent with useful context, examples, and guardrails, such as:

• how to recognize when UltraPlot provides higher-level functions that can replace repeated Matplotlib boilerplate;
• how to use UltraPlot features such as format(), layouts, colorbars, legends, colormaps, and figure-level formatting more naturally;
• how to ask the user about the scientific message of the figure before generating code;
• how to avoid hallucinated or outdated APIs;
• how to review the generated figure for readability, reproducibility, and publication quality.

The kind of skill I have in mind is similar in spirit to existing scientific visualization / figure-generation skill projects such as:

• scientific-agent-skills/skills/scientific-visualization
• scipilot-figure-skill
• nature-skills/skills/nature-figure

These projects are not necessarily trying to force one plotting style either. They are more like guidance systems for AI agents: helping the agent understand the task, choose an appropriate visualization strategy, generate code, and then review the output.

For an UltraPlot-based version, the focus would be on helping AI agents understand the UltraPlot API and use its higher-level abstractions effectively, rather than falling back to verbose low-level Matplotlib code by default.

Once there is something concrete, I would be very happy to share it here and get your feedback.

[cvanelteren]

My question to this would be is is there any interaction with an existing AI agent that goes off the rails where this would require such a skill. I need to read more into these systems to know what is going on for it to work, but from what I have seen these "skills" seem rather unnecessary -- but I am not backing this up with any numbers just more from passing interactions.

[lwq-star]

I’m very sorry for the delayed response. Last week, I was waiting for the release of GPT-5.6, and I also conducted multiple rounds of testing and optimization on the skill.

It has now been open-sourced on GitHub. You can find the project here:
https://github.com/lwq-star/ultraplot-figures

Regarding your question about whether the skill is truly necessary in real-world use cases, I have prepared a comparison example here:
https://github.com/lwq-star/ultraplot-figures/tree/main/examples/earthquake-skill-comparison

I would greatly appreciate any guidance or feedback you could provide, as it would be extremely helpful in further improving the skill.

[cvanelteren]

No worries at all! Thanks for the visual example it was very useful. It is clear that the skill produces a more scientific look over the right image default image.

I do wonder, did you try to repeat the example by just stating for the example to use ultraplot in the prompt without the skill itself?

This makes me wonder if I should add a skill section to the docs and put this on there. Keep me posted on how this develops!

[lwq-star]

Hello, I have added a new scatter plot example. I also found that the delivery script used with the skill contained unnecessary content such as EXPECTED_CONTRACT. This issue has now been fixed, and the version has been updated.

As described in the example documentation at https://github.com/lwq-star/ultraplot-figures/tree/main/examples/correlation-scatter-plot, for the branch that does not use the skill, I instructed the AI to prioritize the UltraPlot library, while allowing it to use other standard Python libraries when appropriate. The AI did follow this instruction, as can be seen in the corresponding script:

https://github.com/lwq-star/ultraplot-figures/blob/main/examples/correlation-scatter-plot/without_skill/plot_prediction_vs_observed.py

However, it is also worth noting that even when the skill is used, the AI still chooses Matplotlib in some places where UltraPlot could be used instead. This is because Matplotlib has a much longer development history and far more plotting examples available online. Unless the AI is explicitly told that it can use the UltraPlot package, it will not consider UltraPlot at all. This is precisely why I created this skill: to provide the AI with a set of guiding rules without making those rules overly restrictive.

Even the most capable model currently available, GPT 5.6 sol, still cannot consistently produce a scientific figure that fully meets the specified requirements without a highly detailed prompt—which can also be regarded as a kind of skill. For somewhat less capable models, the improvement provided by the skill may be even more significant.

As AI models continue to advance, it remains unclear whether producing publication-quality scientific figures will become much easier in the future—perhaps as soon as next year, given how rapidly AI is developing. Nevertheless, I will continue testing. I am also looking forward to the possible release of GPT 6 next month. Claude is simply too expensive, so I have not tested it yet.

[lwq-star]

For this new example, I also identified an area where UltraPlot could be improved.

When a spanning y-axis label is used together with categorical row labels, UltraPlot places the shared y-axis label between the row labels and the subplot grid. This pushes labels such as “Cropland,” “Forest,” “Grassland,” and “Savanna” farther away from their corresponding rows, creating an unclear visual hierarchy. The shared y-axis label should preferably be outermost, while the categorical row labels should remain closer to their panels. This behavior is reproducible in plain UltraPlot and is unaffected by the order of the formatting calls.

Minimal Reproducible Example

import matplotlib.pyplot as plt
import ultraplot as uplt

fig, axs = uplt.subplots(
    nrows=4,
    ncols=4,
    sharex=True,
    sharey=True,
)

axs.format(
    xlabel="Observed value",
    ylabel="Predicted value",
)

fig.format(
    rowlabels=[
        "Cropland",
        "Forest",
        "Grassland",
        "Savanna",
    ],
)

plt.show()

Although the final figure still requires some refinement—for example, the label “Paired observations per hexagon” should be replaced with wording that is more intuitive and easier for readers to understand in a scientific figure—AI can help reduce the amount of manual editing required. Alternatively, we can simply tell the AI what else needs to be changed. In any case, the underlying Python script is fully reproducible.

[cvanelteren]

The first issue, is a good one. The second I don't understand. Could you elaborate a bit more?

[cvanelteren]

First issue is fixed an on main.

[lwq-star]

The first issue, is a good one. The second I don't understand. Could you elaborate a bit more?

By the second point, I was not referring to another UltraPlot issue. I only meant that the AI-generated figure still needs some semantic refinement. For example, the colorbar label “Paired observations per hexagon” is technically correct but may not be immediately intuitive to readers. This wording comes from the generated plotting script rather than UltraPlot itself. The only UltraPlot issue I intended to report was the placement of the spanning y-axis label relative to the row labels.

[cvanelteren]

I added my take here: https://github.com/cvanelteren/ultraplot-figures