Tkinter - _tkinter.TclError: invalid boolean operator in tag search expression

[ThingamabobsCoding]

Bug report

Bug description:

Due to the naming convention in tkinter, which adds an prefix "!" to the widget name, there is an error dropped when using the name as a tag.
This would be easy to fix by just not adding signs that can be interpreted as logical expressions.
It would work more intuitively and I can't think of a scenario where a change in the naming convention would ruin someones day.

Thread on StackOverflow

import tkinter as tk

root = tk.Tk()
cnvs = tk.Canvas(root)
cnvs.pack()
labl = tk.Label(cnvs, text='hello world')
laid = cnvs.create_window(0,0,window=labl,anchor='nw', tags=str(labl))
tags = cnvs.gettags(laid)
cnvs.find_withtag(tags)
root.mainloop()

CPython versions tested on:

3.11

Operating systems tested on:

Windows

[terryjreedy]

A prefix was added by @serhiy-storchaka to generated widget names, about a decade ago, to solve an issue (which I have forgotten). Git blame would help find the issue. I helped pick '!' from among candidates as the most visually appealing. The problem of using widget names in tag expressions was either not considered or not thought to be a blocker since not necessary (see below). To make a change, the possibility of breaking code would have to be considered. (I added a similar comment to the SO question.)

The answers on SO gave 2 immediate solutions.

1. Explicitly name the widgets without '!' when creating them by adding an arg such as name="canvas" to the call.
2. Remove the '!'s when creating the tag, such as with ntag = str(labl).replace('!',"")

[ronaldoussoren]

The relevant issue is #71212, with discussion about the character used kicking of at #71212 (comment).

[ThingamabobsCoding]

Thank you guys for your insights.
I see the change was from previously generated numeric identifier to a more human readable approach.
I'm just not convinced the prefix "!" has to be put in place, since there is already a suffix with increasing numbers per class.name.

[TheLizzard]

I believe that adding validation to tag names would be the best solution in terms of backwards compatibility. There's also the option of replacing exclamation marks with another character when in the context of a tag.

I think that removing the exclamation mark from widget names might break existing code. I think that I might have a script that assumes there is an exclamation mark in the widget name. But I can also make a strong argument that that's just bad code practice.

[ThingamabobsCoding]

I believe that adding validation to tag names would be the best solution in terms of backwards compatibility. There's also the option of replacing exclamation marks with another character when in the context of a tag.

I think that removing the exclamation mark from widget names might break existing code. I think that I might have a script that assumes there is an exclamation mark in the widget name. But I can also make a strong argument that that's just bad code practice.

I can't think of any code that is dependent on the prefix "!" in a name and if you are dependent on the "!", it would be best to have to define the name for yourself. The issue here is that the convenient part of tkinter, that is to generate the names automatically, has a drawback in utility when working with these names.

In addition, tkinter is designed as thin wrapper around tcl/tk, so that tkinter can point to tcl/tk manpage.
It wouldn't be wise to filter user input on the python-site of things. The maintainers end up looking on every patch-note if their code is still in sync with what tcl/tk offers.

The valid concern of the #71212 is

there is large chance that generated "button2" or "button-2" will conflict with explicit names.

However, it is stated that

We can change this in any time if it looks bad.

So no one guaranteed to hold on to that either. A leading underscore would be best practice and concluded with python private variables or a regular expression implemented as a gate keeper for duplicates.

A fair question would be, why this issue needs to be solved.
Well I guess, you can leave the issue for another 15 years and barely someone will notice it.
But I think to have the widget name as a tag in a canvas is a reasonable thing to do for instance, when you want to design your own geometry manager.
And it seems quite possible, that one day a new geometry manager is added in a pure tcl script in that fashion.

[serhiy-storchaka]

We tried to avoid possible conflicts in any imaginary scenarios, but this case just didn't come to mind. You are the first person who reported this in 9 years, so it is not particularly common case. Well, I think we have a reason to choose a different character now. The change perhaps will not be backported, due to non-zero risk of occurring other conflict, so you have to live with a workaround.

The remaining candidates in #71212 were '!', '&', '+', '/', ':', '<', '=', '>', '@', '^', '_', '|', '~'.

• '!', '&', '^' and '|' are used in the tagOrId expression.
• '@' starts coordinates or image path in some commands. It can also have special meaning when post the code on internet.
• '~' is expanded to home directory in paths.

[TheLizzard]

I can't think of any code that is dependent on the prefix "!" in a name and if you are dependent on the "!", it would be best to have to define the name for yourself.

I can't remember the context in which I assumed that there would be an exclamation mark in the widget name, I think it might have been when my class subclassed tk.Frame and needed to know some stuff about it's children which are created by the caller. My bad code shouldn't change how tkinter works but I am just pointing out that other people's code might also break. Only if enough tkinter code is to be broken do we need to consider if there are better alternatives to this problem.

In addition, tkinter is designed as thin wrapper around tcl/tk, so that tkinter can point to tcl/tk manpage.
It wouldn't be wise to filter user input on the python-site of things. The maintainers end up looking on every patch-note if their code is still in sync with what tcl/tk offers.

As I see it, while tcl might allow ! in the tag name in the create_window, it's invalid in the find_withtag. While I don't know what the tcl/tk community think about exclamation marks in tag names, I would imagine it's not advised or even discouraged.

At the very least adding validation to places like .find_withtag where it raises an error if there is a ! in the tag name, would be the same as python catching that indecipherable error message and transforming it to something closer to what python programmers expect.

A fair question would be, why this issue needs to be solved. Well I guess, you can leave the issue for another 15 years and barely someone will notice it.

While it's an edge case that isn't very common, it should be reasonable to at least provide a useful error message since anyone that encounters this bug who doesn't know tcl, would need quite a bit of time to understand where the problem is coming from. This is especially the case since ! isn't even mentioned in the error message.

[ThingamabobsCoding]

I strongly advocate for the underscore, it might don't look appealing but it follows a logic and how many times you really look at a widgets name, especially with python. In my opinion we only have generated names because we store the widgets in variable names and only reference the widgets with the name in context of the tcl interface. E.g iterating over children, name_to_widget etc which we all store again in variables that we can adjust visually appealing as we like.

For python developer the underscore already is used to avoid naming conflicts and also signals it's private "do not touch or be responsible".
From the truly remaining signs '+', '/', ':', '<', '=', '>','_', are most semantically already connoted which will leave you just with the underscore, which is semantically commonly used as placeholder. It might be worth thinking about naming them tkbutton1 or pybutton (like we already do with images) which may gives the eyecandy for everybody.

[serhiy-storchaka]

A serious argument against an undercore is that it is a common symbol, there is a large chance of conflicts with user specified names. If you create a megawidget, chances are that you use underscored names for non-public subwidgets. It is even used in Tk itself: ._notify, ._tray, ._print, $w._balloon, $w._bg, $top._txt, $data(-parent).__tk__messagebox.

We need something more unusual, which will break minimal amount of user code.

[ThingamabobsCoding]

@serhiy-storchaka Ok, but what about pybutton this concept holds stable for pyimage since forever and there seems to be no need for two separate naming conventions.

EDIT: After thinking about it. a class named Image would break this... then what about py_button clearly indicating a python created widget and sperated from pyimage.
In addition to that, it would be possible in the future change to have a warning if name[0:3] == "py_":.
Since there is no back port, developer with that will be warned by upgrading to python 3.xx