Half the release checklist has changed recently, what with me
completely reworking the website and also writing all this release
automation. I think these are all the checklist changes needed now the
dust has settled, though of course when I do the next actual release I
expect there'll turn out to be something I missed...
(cherry picked from commit 3e811b3dff)
I've added extra modes to release.pl which should automate the more
tedious parts of the deployment phase: uploading the release build to
all the places it needs to go, checking its integrity once it gets
there, verifying that everything can be downloaded again usefully,
checking content-types etc.
The new version should check more thoroughly (it checks the whole FTP
and HTTP download directories, so it will spot errors like failing to
update the FTP 'latest' symlink), and take fewer commands to run.
(cherry picked from commit f08e2de078)
The length coming back from ber_read_id_len might have overflowed, so
treat it as potentially negative. Also, while I'm here, accumulate it
inside ber_read_id_len as an unsigned, so as to avoid undefined
behaviour on integer overflow, and toint() it before return.
Thanks to Hanno Böck for spotting this, with the aid of AFL.
(cherry picked from commit 5b7833cd47)
Conflicts:
import.c
(cherry-picker's note: resolving the conflict involved removing an
entire section of the original commit which fixed ECDSA code not
present on this branch)
Forgot that a zero-length string might have come back from fgets.
Thanks to Hanno Böck for spotting this, with the aid of AFL.
(cherry picked from commit 5815d6a65a)
The initial test for a line ending with "PRIVATE KEY-----" failed to
take into account the possibility that the line might be shorter than
that. Fixed by introducing a new library function strendswith(), and
strstartswith() for good measure, and using that.
Thanks to Hanno Böck for spotting this, with the aid of AFL.
(cherry picked from commit fa7b23ce90)
Conflicts:
misc.c
misc.h
(cherry-picker's note: the conflicts were only due to other functions
introduced on trunk just next to the ones introduced by this commit)
TOOLTYPE_NONNETWORK (i.e. pterm) already has "-log" (as does Unix
PuTTY), so there's no sense suppressing the synonym "-sessionlog".
Undocumented lacunae that remain:
plink accepts -sessionlog, but does nothing with it. Arguably it should.
puttytel accepts -sshlog/-sshrawlog (and happily logs e.g. Telnet
negotiation, as does PuTTY proper).
(cherry picked from commit a454399ec8)
Conflicts:
unix/uxplink.c
windows/winplink.c
(cherry-picker's notes: the conflict was only contextual, in the Plink
help output)
I've added a few sample shell commands in the upload procedure (mostly
so that I don't have to faff about remembering how rsync trailing
slashes work every time), and also written a script called
'release.pl', which automates the updating of the version number in
all the various places it needs to be done and also ensures the PSCP
and Plink transcripts in the docs will match the release itself.
(cherry picked from commit f3230c8545)
I spotted that I've been checking that old-style Windows Help files
were delivered with content-type "application/octet-stream", but not
also checking the same thing about the marginally newer .CHM ones. (Or
at least not writing it down in the wishlist; I think I did actually
check on at least one occasion.)
(cherry picked from commit 3552f37ba5)
GCC 6 emits strict-aliasing warnings here, so use the existing
sockaddr_union arrangements to avoid those. As a prerequisite for being
able to express sk_tcp_peer_info in terms of sockaddr_union, I fixed up
the union elements to be a bit less odd in the NO_IPV6 case.
GCC 6 warns about potentially misleading indentation, such as:
if (condition) stmt1; stmt2;
Chaining multiple ifs on a single line runs into this warning, even if
it's probably not actually misleading to a human eye, so just add a
couple of newlines to pacify the compiler.
Not that anyone actually needs to use that conditioned-out main(),
since it only generates the table already present in the same source
file, but since @ch3root's unused-variable patch touched it I tried
compiling it and noticed in passing that I'd also got the wrong printf
format directive for an unsigned long.
Thanks to @ch3root on Twitter for spotting it, and thanks to Chris
Emerson for bothering to let me know. I must have missed this when I
code-reviewed the ECC contribution.
Now we have licence.pl, it seems to me to make very good sense to have
it generate the Halibut form(s) of the licence and copyright year as
well as the source-code forms.
As a result, I believe _no_ copies of the licence text or copyright
date exist any more except for the master one in LICENCE - so I can
completely remove the checklist section about all the places to update
it, because there's only one. Hooray!
Now all the uses of the licence text or the short copyright notice get
it from a new header "licence.h", which in turn is built by a Perl
script licence.pl invoked by mkfiles.pl, using LICENCE itself as the
source.
Hence, I can completely remove a whole section from the list of
licence locations in CHECKLST.txt :-)
This makes the About and Licence boxes copy-and-pasteable, similarly
to what I've just done on Unix.
(But unlike on the Unix side, here I haven't touched the host key
prompt dialog, because that's a standard Windows MessageBox and not
easy to mess around with. Plus, in any case, you can already hit ^C to
copy the whole text out of a MessageBox. Same goes for the PGP
fingerprints dialog.)
As a side effect, several copies of the copyright notice and licence
text have moved from .rc files into C source. I've updated
CHECKLST.txt, but they won't stay there for long.
I've made the licence text, the About box, and the host key dialog
into GTK selectable edit controls. (The former because it contains a
lot of text; the About box because pasting version numbers into bug
reports is obviously useful; the host key because of the fingerprint.)
Proper snprintf is finally supported as of the latest Visual Studio,
and has better semantics for my purposes than the old MS-specific
_snprintf. (Specifically, if its output doesn't fit the buffer, it
returns the full size it _would_ have wanted, so that you can then
immediately allocate that much space, and don't have to keep going
round a loop increasing the buffer size until you find the answer.)
Occurred as a side effect of commit 198bca233, in which I wrote a Perl
loop of the form 'foreach $srcdir (@srcdirs)' inside which I modified
$srcdir - forgetting the Perl gotcha that if you do that, $srcdir
temporarily aliases the actual array element, so you end up modifying
the array you iterated over. Hence, a set of transformations intended
to convert the source directory list into a special form for the nmake
batch-mode inference rule syntax in particular ended up back in
@srcdirs to be reflected in unrelated makefiles output later in the
run.
Now you can run a command like "nmake /f Makefile.vc BUILDDIR=foo\",
which will cause all the generated files to appear in a subdirectory
of putty\windows. This is immediately useful for testing multiple
build configurations against each other by hand; later on I hope it
will also be a convenient way to run multiple build configurations in
the proper bob build.
This enables it to combine the compilation of multiple source files
into a single 'cl' command with multiple input file arguments, which
speeds up the build noticeably.
(I think nmake could be doing a lot more to improve this - for a
start, I haven't found any way to let it aggregate compilations of
source files in more than one directory, and also, it seems to me that
it really ought to be able to reduce down to just _one_ invocation of
cl by choosing the best topological sort of its build operations,
whereas in fact it looks as if it's sorting the operations _before_
doing the aggregation. But even so, it's a big improvement on the
previous build time.)
This is noticeably faster than a sequence of 'echo' commands, because
the file gets created all in one go. The most natural approach to this
job would also hide the file's contents, but doing it this way with a
'type' command lets me see the file on nmake's standard output, so
that the build log should still contain everything useful for
debugging build problems.
I've found in the last day or two that the first thing I want to do
after any successful run of testbn is to check whether I was running
it with the right compile settings - so I should have made it easier
to find that out to begin with! Better late than never.
This makes it easier to compile in multiple debugging modes, or on
Windows, without having to constantly paste annoying test-compile
commands out of comments in sshbn.c.
The new binary is compiled into the build directory, but not shipped
by 'make install', just like fuzzterm. Unlike fuzzterm, though, testbn
is also compiled on Windows, for which we didn't already have a
mechanism for building unshipped binaries; I've done the very simplest
thing for the moment, of providing a target in Makefile.vc to delete
them.
In order to comply with the PuTTY makefile system's constraint of
never compiling the same object multiple times with different ifdefs,
I've also moved testbn's main() out into its own source file.
This commit fulfills the promise of the previous one: now one of the
branches of sshbn.h's big ifdef _doesn't_ define a BignumDblInt, and
instead provides implementations of the primitive arithmetic macros in
terms of Visual Studio's x86-64 compiler intrinsics. So now, when this
codebase is compiled with 64-bit VS, it can use a 64-bit BignumInt and
everything still seems to work.
As I mentioned in the previous commit, I'm going to want PuTTY to be
able to run sensibly when compiled with 64-bit Visual Studio,
including handling bignums in 64-bit chunks for speed. Unfortunately,
64-bit VS does not provide any type we can use as BignumDblInt in that
situation (unlike 64-bit gcc and clang, which give us __uint128_t).
The only facilities it provides are compiler intrinsics to access an
add-with-carry operation and a 64x64->128 multiplication (the latter
delivering its product in two separate 64-bit output chunks).
Hence, here's a substantial rework of the bignum code to make it
implement everything in terms of _those_ primitives, rather than
depending throughout on having BignumDblInt available to use ad-hoc.
BignumDblInt does still exist, for the moment, but now it's an
internal implementation detail of sshbn.h, only declared inside a new
set of macros implementing arithmetic primitives, and not accessible
to any code outside sshbn.h (which confirms that I really did catch
all uses of it and remove them).
The resulting code is surprisingly nice-looking, actually. You'd
expect more hassle and roundabout circumlocutions when you drop down
to using a more basic set of primitive operations, but actually, in
many cases it's turned out shorter to write things in terms of the new
BignumADC and BignumMUL macros - because almost all my uses of
BignumDblInt were implementing those operations anyway, taking several
lines at a time, and now they can do each thing in just one line.
The biggest headache was Poly1305: I wasn't able to find any sensible
way to adapt the existing Python script that generates the various
per-int-size implementations of arithmetic mod 2^130-5, and so I had
to rewrite it from scratch instead, with nothing in common with the
old version beyond a handful of comments. But even that seems to have
worked out nicely: the new version has much more legible descriptions
of the high-level algorithms, by virtue of having a 'Multiprecision'
type which wraps up the division into words, and yet Multiprecision's
range analysis allows it to automatically drop out special cases such
as multiplication by 5 being much easier than multiplication by
another multi-word integer.
DIVMOD_WORD is a portability hazard, because implementing it requires
either a way to get direct access to the x86 DIV instruction or
equivalent (be it inline assembler or a compiler intrinsic), or else
an integer type we can use as BignumDblInt. But I'm starting to think
about porting to 64-bit Visual Studio with a 64-bit BignumInt, and in
that situation neither of those options will be available.
I could write a piece of _out_-of-line x86-64 assembler in a separate
source file and put a function call in DIVMOD_WORD, but instead I've
decided to solve the problem in a more futureproof way: remove
DIVMOD_WORD totally and write a division function that doesn't need it
at all, solving not only today's porting headache but all future ones
in this area.
The new implementation works by precomputing (a good enough
approximation to) the leading word of the reciprocal of the modulus,
and then getting each word of quotient by multiplying by that
reciprocal, where we previously used DIVMOD_WORD to divide by the
leading word of the actual modulus. The reciprocal itself is computed
outside internal_mod() and passed in as a parameter, allowing me to
save time by only computing it once when I'm about to do a modpow.
To some extent this complicates the implementation: the advantage of
DIVMOD_WORD was that it yielded a full word q of quotient every time
it was used, so the subtraction of q*m from the input could be done in
a nicely word-aligned way. But the reciprocal multiply approach yields
_almost_ a full word of quotient, because you have to make the
reciprocal a bit short to avoid overflow at multiplication time. For a
start, this means we have to do fractionally more iterations of the
main loop; but more painfully, we can no longer depend on the
subtraction of q*m at every step being word-aligned, and instead we
have to be prepared to do it at any bit shift.
But the flip side is that once we've implemented that, the rest of the
algorithm becomes a lot less full of horrible special cases: in
particular, we can now completely throw away the horribleness at all
the call sites where we shift the modulus up by a fractional word to
set its top bit, and then have to do a little dance to get the last
few bits of quotient involving a second call to internal_mod.
So there are points both for and against the new implementation in
simplicity terms; but I think on balance it's more comprehensible than
the old one, and a quick timing test suggests it also ends up a touch
faster overall - the new testbn gets through the output of
testdata/bignum.py in 4.034s where the old one took 4.392s.
I'm about to rewrite the division code, so it'll be useful to have a
way to test it directly, particularly one which exercises difficult
cases such as extreme values of the leading word and remainders just
above and below zero.
Or, at least, potentially do so. The build script now has a slot into
which code-signing can be dropped by setting a variable in the bob
configuration to specify an appropriate command line.
The variable will typically need to point at a script wrapping the
actual signing tool, since there are lots of fiddly details
(timestamping countersignature, certificate, private key, etc) not
given on the command lines in this build script, on the basis that
they're local configuration questions for whoever is _running_ this
build script.
When we provide an editable text box with a drop-down list of useful
preset values, such as the one full of character sets in the
Translation panel, we implement it on GTK 2.4+ as a GtkComboBox
pointing at a two-column GtkListStore, in which the second column is
the actual text (the first being a numeric id). Therefore, we need to
set the "entry-text-column" property to tell GtkComboBox which of
those columns to look in for the value corresponding to the edit-box
text.
Thanks to Robert de Bath for spotting the problem and tracing it as
far as commit 5fa22495c. That commit replaced a widget construction
call via gtk_combo_box_entry_new_with_model() with one using the newer
gtk_combo_box_new_with_model_and_entry(), overlooking the fact that
the former provided the text column number as a parameter, and the
latter didn't.
logevent() doesn't do printf-style formatting (though the logeventf
wrapper in ssh.c does), so if you need to format a message, it has to
be done separately with dupprintf.
I'd missed out an if statement in the Unix proxy stderr code
introduced by commit 297efff30, causing ret->cmd_err to be passed to
uxsel_set even when it was -1 (which happened in the non-GUI tools).
Unfortunately, putting a negative fd into the uxsel tree has really
bad effects, because the first_fd / next_fd interface returns a
negative number to signal end-of-list - and since the uxsel tree is
sorted by fd, that happens _immediately_.
Added the missing if statement, and also an assertion to make sure we
never pass -1 to uxsel_set by mistake again!
By default Windows processes have wide open ACLs which allow interference
by other processes running as the same user. Adjust our ACL to make this
a bit harder.
Because it's useful to protect PuTTYtel as well, carve winsecur.c into
advapi functions and wincapi.c for crypt32 functions.
gtk_misc_set_alignment was deprecated in GTK 3.14. But my replacement
code using gtk_label_set_xalign doesn't work there, because that
function wasn't introduced until GTK 3.16, so there are two minor
versions in the middle where a third strategy is needed.
Thanks to Colin Harrison for spotting it very quickly. No thanks to
Visual Studio for only giving me a _warning_ when I prototyped a
function with four parameters and called it with five!
It has three settings: on, off, and 'only until session starts'. The
idea of the last one is that if you use something like 'ssh -v' as
your proxy command, you probably wanted to see the initial SSH
connection-setup messages while you were waiting to see if the
connection would be set up successfully at all, but probably _didn't_
want a slew of diagnostics from rekeys disrupting your terminal in
mid-emacs once the session had got properly under way.
Default is off, to avoid startling people used to the old behaviour. I
wonder if I should have set it more aggressively, though.
On both Unix and Windows, we now redirect the local proxy command's
standard error into a third pipe; data received from that pipe is
broken up at newlines and logged in the Event Log. So if the proxy
command emits any error messages in the course of failing to connect
to something, you now have a fighting chance of finding out what went
wrong.
This feature is disabled in command-line tools like PSFTP and Plink,
on the basis that in that situation it seems more likely that the user
would expect standard-error output to go to the ordinary standard
error in the ordinary way. Only GUI PuTTY catches it and logs it like
this, because it either doesn't have a standard error at all (on
Windows) or is likely to be pointing it at some completely unhelpful
session log file (under X).
