X fonts are indexed by 16-bit quantities, so even Unicode-capable
fonts can only have characters in the Basic Multilingual Plane (BMP).
PuTTY, however, tried to look up all Unicode characters in X fonts,
and did so by effectively ignoring all but the low-order 16 bits of
the character code. This meant that trying to display a non-BMP
character could get you the corresponding character from the BMP
instead, if that character was in the font.
Now, x11font_has_glyph() always returns false for glyphs outside the
BMP, which should mean that x11font_draw_text() never gets asked to
draw them.
Xlib and the X protocol provide text handling calls in versions that
take both 8-bit and 16-bit character indices. They are mostly
interchangable, except that of course the 8-bit calls can only access
characters with codes up to 255.
Heretofore, PuTTY used the 16-bit X calls when using a font in the
"iso10646-1" encoding and 8-bit calls otherwise. This led to a lot of
duplicate code, and in particular two large and almost identical
implementations of x11font_cairo_draw*().
Now, PuTTY uses 16-bit calls throughout, even when using an 8-bit font
encoding. This simplifies the code at the expense of needing an
extra bit of conversion to expand the char array that we get from
put_wc_to_mb() into an array of XChar2b when using an 8-bit font.
In 687efc3a5da1fb85c86a3c871b00a7234b97c2e9, Simon noted that when PuTTY
was running under X and using an RGB24 image surface as its backing
surface, it would fail to draw on its window. Changing the backing
image to ARGB32 caused the problem to go away.
If you set GDK_BACKEND=x11 and GDK_RENDERING=image, then PuTTY's
gdk_window_create_similar_surface() returns an RGB24 image surface, and
it appears to have precisely the same problem. Dumping the surfaces to
PNG files revealed that Cairo thought they had the right context. But
xtruss didn't show any actual requests to write to the window.
So on a hunch approximately as well-informed as Simon's, I added a call
to cairo_flush(), to explicitly ask Cairo to flush its changes to the
underlying surface. I would have hoped that GTK would do something like
this for us, but adding that call seems to have made things work
properly. Like Simon, I have no idea if this is the correct fix, but it
seems like a reasonable one and the problem is no longer occurring for
me.
Now that we use gdk_window_create_similar_surface() to make our backing
surface, most of the code to handle scale factors is unnecessary and
indeed slightly harmful. gdk_window_create_similar_surface() creates a
surface suitable for use as backing for a window, so it already has the
scale factor applied. This means that it should be sized in nominal
pixels, we can paint to it in nominal pixels, and the fact that it has
the same extra resolution as the actual window is entirely transparent
to us.
Now the only reason we pay attention to the scale factor at all is to
detect changes and use them as a prompt to re-create the backing
surface.
CreatePixmap returns a Pixmap with undefined contents, and ImageText16
doesn't quite erase the whole rectangle covered by the text (and hence
the whole Pixmap. So to be on the safe side we should make sure to
erase the entire Pixmap before drawing the text.
Conveniently, ImageText16 ignores the function specified in the GC, so
we can set that to GXclear and avoid needing to change the GC
thereafter.
This is a fairly radical change of how X bitmap fonts are handled when
using Cairo for rendering. Before, we would download each glyph to the
client on first use and then composite those glyphs into the terminal's
backing surface. This worked pretty well when we were keeping an image
of the whole screen on the client anyway, but once I'd pushed all the
other Cairo rendering onto the X server, it meant that the character
bitmaps had to be repeatedly pushed to the X server.
The new arrangement just renders each string into a temporary Pixmap
using the usual X text-drawing calls and then asks Cairo to paste it
into the main backing Pixmap. It's tempting to draw the text straight
into the backing Pixmap, but that would require dealing directly with
X colour management. This way, we get to leave colours in the hands
of Cairo (and hence the Render extension).
There are still fragments of the old system around. Those should go
in the next commit.
If gdk_window_create_similar_surface() isn't available, we now fall back
to cairo_surface_create_similar(). This is relevant only on GTK between
2.00 and 2.22 with deprecated calls disabled.
By default, when we're asked to draw a GTK widget, GTK creates a
temporary surface (a Pixmap under X) and redirects our rendering into
that. Then it blits that into the actual window. This is silly for
PuTTY because all that PuTTY does to render its drawing area is to
blit into it from _another_ surface. So now PuTTY asks GTK not to do
that. According to the GTK documentation, GTK as of 3.10 has
completely restructured its drawing routines so that turning off
double-buffering actually makes things worse and slower, so we turn it
off only in GTK 2
Still, this now means that painting text on the screen in GTK 2 causes
precisely one CopyArea operation, which is what we want.
This removes the case where we draw into a Cairo surface and then copy
the results into a GdkPixmap. Now, if we've got a GdkPixmap, we just
draw into it directly using Cairo. This vastly reduces the number of
CopyArea operations needed to draw on the screen.
I just found cairo_paint() while wandering the Cairo documentation. It
just fills the entire clip region with data from the current source,
which is precisely what draw_area() wants to do. This is simpler for us
than requesting the bounding rectangle of the clipping region and then
filling it, and as far as I can tell the clipping rectangle generally
covers the whole window anyway.
Rather than constructing a transformation matrix piece by piece (with
very branchy code), draw_stretch_before now just calls cairo_translate()
and cairo_scale() with values that are almost-obviously correct.
Also, rather than stashing and restoring the transformation matrix
ourselves, it seems simpler to use cairo_save() and cairo_restore().
That requires that draw_stretch_before() and draw_stretch_after() be
called strictly in pairs, but they are so that's OK.
According to the X specs, WhitePixel and BlackPixel refer to permanent
entries in the default colourmap. This means that they're not
necessarily appropriate for use with a Drawable with a different depth
than the root window. When drawing to a Pixmap that will be used as a
1-bit alpha mask by Cairo, the correct values are simply 0
(transparent) and 1 (opaque).
On Windows, when a blinking cursor is enabled, PuTTY uses the system
default blink time from GetCaretBlinkTime(), which can be configured in
Control Panel.
Control Panel allows caret blinking to be disabled entirely, in which
case GetCaretBlinkTime() returns INFINITE. PuTTY wasn't handling this
case; if cursor blinking was enabled in PuTTY but disabled at the system
level, the terminal window would hang, blinking the cursor madly.
This commit fixes a problem that Simon observed when using an X bitmap
font with Cairo and making a line double-width or double-size. When
using Cairo, PuTTY implements double-width and double-size by just
asking Cairo to scale all its drawing operations. This works fine
with outline fonts, but when using a bitmap font the results are a bit
fuzzy. This appears to be because Cairo's default is to use bilinear
interpolation when scaling an image, which is fine for photos but not
so good for fonts.
In this commit, I decompose PuTTY's cairo_mask_surface() call into its
component parts so that I can set the mask pattern's filter to
CAIRO_FILTER_NEAREST before using it. That solves the problem, but it
suggests that maybe we should be caching the pattern rather then the
surface.
When using an X server-side font with Cairo rendering, PuTTY takes the
rather horrible approach of rendering each glyph it uses into a depth-1
Pixmap and then copying the result into a Cairo surface that it uses
every time it wants to display that glyph.
Heretofore, the conversion of the Pixmap into a Cairo surface was done
by downloading it using XGetImage() and then manually re-arringing the
bits into a suitable form for Cairo. But Cairo has a way of turning an
X Drawable (including a Pixmap) into a surface, and then it's just a
case of copying one surface to another using cairo_paint(). So that's
what PuTTY does now and the process is a little less unpleasant than it
was.
The Cairo documentation is clear that cairo_set_antialias() only
affects shape drawing and not text rendering. To change
anti-aliasing settings for font rendering you need
cairo_font_options_set_antialias() instead. Therefore the comment
can be a bit more certain than just describing what Cairo "appears"
to do.
A user reports that our top-level .gitignore ignores several files
that are actually part of the real git repository. This is
inconvenient if you start from a downloaded tarball or zip file, and
try to make it _back_ into a git repository to work with it.
The blanket rule to ignore files called "Makefile" (on the theory that
they're autogenerated by cmake, or in the pre-cmake days, by
autotools) was also excluding two handwritten Makefiles, in 'icons'
and in 'contrib/cygtermd'. And the rule about doc/*.txt, intended to
exclude Halibut's plain-text output, also excluded doc/CMakeLists.txt.
With these exclusions in place, if you download a PuTTY source
.tar.gz, unpack it, change into the unpacked subdirectory, and run
'git init', 'git add .' and 'git commit', then 'git status --ignored'
to see what files in the tarball weren't added to the repo, you'll
find that the remaining ones are all in the 'doc' directory, and
really _are_ Halibut outputs: all the man pages (putty.1 etc), the
Windows help file putty.chm, and the plain text puttydoc.txt.
Currently, we display the cursor at the right-hand end of the pre-edit
text. That looks good for block and underline cursors, but it's a bit
weird for the vertical line, which naturally appears at the left edge
of the rightmost character. Setting ATTR_RIGHTCURS fixes this and
means that the vertical-line cursor appears at the right edge of the
rightmost character.
That also corrects a weirdness where ATTR_RIGHTCURS was leaking
through from the underlying pending-wrap state of the terminal, which
was definitely wrong.
That's the more logical location in a string more than one character
long. GTK does actually tell us where it thinks the cursor should be,
but we don't yet pay attention to that.
This involves repeatedly resizing it as we decode characters. That's a
bit inefficient (at least with the current implementation of
resizeline()), but it makes it much easier to be certain that the line
is actually the right length.
I think supporting combining characters in pre-edit text will be simpler
if I can use add_cc, which operated on termlines. Also we have code for
resizing termlines, which means I might not need to count the width of
the pre-edit string accurately before allocating it.
If a character cell under the pre-edit text has a combining character,
it shouldn't be combined with a character from the pre-edit text, but
should be hidden instead. This also means that the pre-edit text
could contain combining characters if I implemented a way to put them
into it.
Now the pre-edit text is converted into a dynamically-allocated array of
termchars in term_set_preedit_text(), which slightly simplifies
do_paint(). This means that the long pre-edit generated by Ctrl+Shift+U
in GNOME now displays more or less properly. I may need a better plan
for what to do about cursor positioning, though.
Now we can cope with a single wide or narrow pre-edit character, which
is good enough for the input methods that I use. When rendering the
line that contains the cursor we set up a little array of termchars
that contains the pre-edit text and work out where it should be
displayed. Then when rendering the screen we switch between
displaying text from the real terminal and from the pre-edit string as
necessary.
Ideally, we should support longer strings, combining characters, and
setting attributes. I think the current architecture should make all
of those possible, but not entirely easy.
This is approximately how it should work: term_set_preedit_text stashes
data in the terminal structure and then do_paint() renders it in place
of what's in the terminal buffer. Currently this only works for a
single narrow character, and it copies the existing attributes under the
cursor, but this might actually be enough for the UK keyboard layout in
GNOME.
We simply pass each character to term_display_graphic_char and then
put the cursor back where we found it. This works in simple cases,
but is fundamentally wrong. Really we should do this in a way that
doesn't touch the terminal state and just gets rendered on top of it
somehow.
The terminal code doesn't yet do anything with the text other than feed
it to a debugging printf. The call uses UTF-8 and expects the terminal
to copy the string because that's compatible with
gtk_im_context_get_preedit_string().
There was a bytes / array elements confusion in the code that prints
out the input and output Unicode strings when a test fails. It was
using a loop with index variable 'pos', which was used as an array
index, but incremented by sizeof(a character) each time, leading to
only every fourth character actually being printed.
I assume this is leftover confusion from when I hadn't quite decided
whether to abuse the char-based strbuf for these Unicode character
buffers, or make a specialist type.
Discovered when I reached for this test program just now in order to
manually decompose a Unicode string. It doesn't have a convenient CLI
for that, but it was a thing I already knew where to find!
When telling front ends to paint the screen, the terminal code treats
the cursor as an attribute applied to the character cell(s) it appears
in. do_paint() detects changes to most such attributes by storing what
it last sent to the front end in term->disptext and comparing that with
what it thinks should be displayed in the window. However, before this
commit the cursor was special. Its last-drawn position was recorded in
special structure members and invalidated parts of the display based on
those. The cursor attributes were treated as "temporary attributes" and
were not saved in term->disptext.
This commit regularizes this and turns the cursor attributes into normal
attributes that are stored in term->disptext. This removes a bunch of
special-case code in do_paint() because now the normal update code
handles the cursor properly, and also removes some members from the
Terminal structure. I hope it will also make future cursor-handling
changes (for instance for input method pre-editing) simpler.
This commit makes the required semantic changes but doesn't make the
rather more pervasive change of actually renaming the attributes from
TATTR_ to ATTR_. That will be in the next commit.
Up-to-date trunk clang has introduced a built-in operator called
_Countof, which is like the 'lenof' macro in this code (returns the
number of elements in a statically-declared array object) but with the
safety advantage that it provokes a compile error if you accidentally
use it on a pointer. In this commit I add a cmake-time check for it,
and conditional on that, switch over the definition of lenof.
This should add a safety check for accidental uses of lenof(pointer).
When I tested it with new clang, this whole code base compiled cleanly
with the new setting, so there aren't currently any such accidents.
clang cites C2y as the source for _Countof: WG14 document N3369
initially proposed it under a different name, and then there was a big
internet survey about naming (in which of course I voted for lenof!),
and document N3469 summarises the results, which show that the name
_Countof and/or countof won. Links:
https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3369.pdfhttps://www.open-std.org/jtc1/sc22/wg14/www/docs/n3469.htm
My reading of N3469 seems to say that there will _either_ be _Countof
by itself, _or_ lowercase 'countof' as a new keyword, but they don't
say which. They say they _don't_ intend to do the same equivocation we
had with _Complex and _Bool, where you have a _Countof keyword and an
optional header file defining a lowercase non-underscore macro
wrapping it. But there hasn't been a new whole draft published since
N3469 yet, so I don't know what will end up in it when there is.
However, as of now, _Countof exists in at least one compiler, and that
seems like enough reason to implement it here. If it becomes 'countof'
in the real standard, then we can always change over later. (And in
that case it would probably make sense to rename the macro throughout
the code base to align with what will become the new standard usage.)
GtkIMContext has focus_in and focus_out methods for telling it when the
corresponding widget gains or loses keyboard focus. It's not obvious to
me why these are necessary, but PuTTY now calls them when it sees
focus-in and focus-out events for the terminal window. Somehow, this
has caused Hangul input to start working in PuTTY. I can't yet
see what I'm typing for lack of proper preedit support, though.
[ECMA-48] section 8.3.27 specifies the format of Device Control String
(DCS) commands which are used for XTGETTCAP and other sequences.
We don't parse DCS commands. This causes this command to wrongly
output some characters:
printf '\033P+q616d\033\\'
Fix that by parsing DCS commands just like other OSC-like commands.
(Apart from the initial characters, DCS has the same format as OSC.)
We also allow 0x07 as sequence terminator which does not seem specified
but a lot of people use it with OSC; it's fine because 0x07 is not
allowed in the OSC/DCS payload.
[ECMA-48]: https://www.ecma-international.org/wp-content/uploads/ECMA-48_2nd_edition_august_1979.pdf
It was a bit far to the right, looking at risk of falling off. Now
moved it as far left as it will go without the top right corner of the
computer monitor peeking out from behind it.
