nhyatt/putty-source
1
0
Fork 0
mirror of https://git.tartarus.org/simon/putty.git synced 2025-07-01 11:32:48 -05:00
Code Issues Projects Releases Wiki Activity

Replace mkfiles.pl with a CMake build system.

Browse Source 
This brings various concrete advantages over the previous system:

 - consistent support for out-of-tree builds on all platforms

 - more thorough support for Visual Studio IDE project files

 - support for Ninja-based builds, which is particularly useful on
   Windows where the alternative nmake has no parallel option

 - a really simple set of build instructions that work the same way on
   all the major platforms (look how much shorter README is!)

 - better decoupling of the project configuration from the toolchain
   configuration, so that my Windows cross-building doesn't need
   (much) special treatment in CMakeLists.txt

 - configure-time tests on Windows as well as Linux, so that a lot of
   ad-hoc #ifdefs second-guessing a particular feature's presence from
   the compiler version can now be replaced by tests of the feature
   itself

Also some longer-term software-engineering advantages:

 - other people have actually heard of CMake, so they'll be able to
   produce patches to the new build setup more easily

 - unlike the old mkfiles.pl, CMake is not my personal problem to
   maintain

 - most importantly, mkfiles.pl was just a horrible pile of
   unmaintainable cruft, which even I found it painful to make changes
   to or to use, and desperately needed throwing in the bin. I've
   already thrown away all the variants of it I had in other projects
   of mine, and was only delaying this one so we could make the 0.75
   release branch first.

This change comes with a noticeable build-level restructuring. The
previous Recipe worked by compiling every object file exactly once,
and then making each executable by linking a precisely specified
subset of the same object files. But in CMake, that's not the natural
way to work - if you write the obvious command that puts the same
source file into two executable targets, CMake generates a makefile
that compiles it once per target. That can be an advantage, because it
gives you the freedom to compile it differently in each case (e.g.
with a #define telling it which program it's part of). But in a
project that has many executable targets and had carefully contrived
to _never_ need to build any module more than once, all it does is
bloat the build time pointlessly!

To avoid slowing down the build by a large factor, I've put most of
the modules of the code base into a collection of static libraries
organised vaguely thematically (SSH, other backends, crypto, network,
...). That means all those modules can still be compiled just once
each, because once each library is built it's reused unchanged for all
the executable targets.

One upside of this library-based structure is that now I don't have to
manually specify exactly which objects go into which programs any more
- it's enough to specify which libraries are needed, and the linker
will figure out the fine detail automatically. So there's less
maintenance to do in CMakeLists.txt when the source code changes.

But that reorganisation also adds fragility, because of the trad Unix
linker semantics of walking along the library list once each, so that
cyclic references between your libraries will provoke link errors. The
current setup builds successfully, but I suspect it only just manages
it.

(In particular, I've found that MinGW is the most finicky on this
score of the Windows compilers I've tried building with. So I've
included a MinGW test build in the new-look Buildscr, because
otherwise I think there'd be a significant risk of introducing
MinGW-only build failures due to library search order, which wasn't a
risk in the previous library-free build organisation.)

In the longer term I hope to be able to reduce the risk of that, via
gradual reorganisation (in particular, breaking up too-monolithic
modules, to reduce the risk of knock-on references when you included a
module for function A and it also contains function B with an
unsatisfied dependency you didn't really need). Ideally I want to
reach a state in which the libraries all have sensibly described
purposes, a clearly documented (partial) order in which they're
permitted to depend on each other, and a specification of what stubs
you have to put where if you're leaving one of them out (e.g.
nocrypto) and what callbacks you have to define in your non-library
objects to satisfy dependencies from things low in the stack (e.g.
out_of_memory()).

One thing that's gone completely missing in this migration,
unfortunately, is the unfinished MacOS port linked against Quartz GTK.
That's because it turned out that I can't currently build it myself,
on my own Mac: my previous installation of GTK had bit-rotted as a
side effect of an Xcode upgrade, and I haven't yet been able to
persuade jhbuild to make me a new one. So I can't even build the MacOS
port with the _old_ makefiles, and hence, I have no way of checking
that the new ones also work. I hope to bring that port back to life at
some point, but I don't want it to block the rest of this change.
This commit is contained in:
Simon Tatham
2021-04-10 15:21:11 +01:00
parent 97f7a7cb4d
commit c19e7215dd
44 changed files with 1272 additions and 3176 deletions
Download Patch File Download Diff File Expand all files Collapse all files

140
Buildscr
View File

@ -74,12 +74,6 @@ ifneq "$(PRERELEASE)" "" set Uxarcsuffix -$(PRERELEASE)~pre$(Ndate).$(vcsid)
ifneq "$(SNAPSHOT)" "" set Uxarcsuffix -$(Lastver)-$(Date).$(vcsid)
ifeq "$(RELEASE)$(PRERELEASE)$(SNAPSHOT)" "" set Uxarcsuffix -custom-$(Date).$(vcsid)
# Set up the version number for the autoconf system.
ifneq "$(RELEASE)" "" set Autoconfver $(RELEASE)
ifneq "$(PRERELEASE)" "" set Autoconfver $(PRERELEASE)~pre$(Ndate).$(vcsid)
ifneq "$(SNAPSHOT)" "" set Autoconfver $(Lastver)-$(Date).$(vcsid)
ifeq "$(RELEASE)$(PRERELEASE)$(SNAPSHOT)" "" set Autoconfver Custom.$(Date).$(vcsid)
# Set up the filenames for the Windows installers (minus extension,
# which goes on later).
ifneq "$(RELEASE)" "" set Isuffix $(RELEASE)-installer
@ -122,31 +116,37 @@ ifneq "$(SNAPSHOT)" "" in putty do echo '$#define SNAPSHOT' >> version.h
in putty do echo '$#define TEXTVER "$(Textver)"' >> version.h
in putty do echo '$#define SSHVER "$(Sshver)"' >> version.h
in putty do echo '$#define BINARY_VERSION $(Winvercommas)' >> version.h
in putty do echo '$#define SOURCE_COMMIT "$(vcsfullid)"' >> version.h
# Set up the extra arguments for the main Windows nmake command. The
# user can define XFLAGS and MAKEARGS on the bob command line, to pass
# in extra compile and make options respectively (e.g. to do a
# debugging or Minefield build).
set Makeargs
ifneq "$(XFLAGS)" "" set Makeargs $(Makeargs) XFLAGS="$(XFLAGS)"
ifneq "$(MAKEARGS)" "" set Makeargs $(Makeargs) $(MAKEARGS)
# In cmake/gitcommit.cmake, replace the default output "unavailable"
# with the commit string generated by bob, so that people rebuilding
# the source archive will still get a useful value.
in putty do sed -i '/set(DEFAULT_COMMIT/s/unavailable/$(vcsfullid)/' cmake/gitcommit.cmake
in putty do ./mksrcarc.sh
in putty do ./mkunxarc.sh '$(Autoconfver)' '$(Uxarcsuffix)' $(Docmakever)
in putty do perl mkfiles.pl
in putty do ./mkunxarc.sh '$(Uxarcsuffix)' $(Docmakever)
in putty/doc do make $(Docmakever) putty.chm -j$(nproc)
delegate -
# Run the test suite, under self-delegation so that we don't leave any
# cruft lying around. This involves doing a build of the Unix tools
# (which is a useful double-check anyway to pick up any build failures)
in putty do ./mkauto.sh
in putty do ./configure CC=clang CFLAGS="-fsanitize=address -fsanitize=leak"
in putty do make -j$(nproc)
in putty do cmake . -DCMAKE_C_COMPILER=clang -DCMAKE_C_FLAGS="-fsanitize=address -fsanitize=leak" -DSTRICT=ON
in putty do make -j$(nproc) VERBOSE=1
in putty do python3 test/cryptsuite.py
enddelegate
delegate -
# Also, test-build the Windows tools using MinGW. This is important if
# we want the MinGW build to carry on working, partly because of the
# chance of compiler compatibility issues, but mostly because MinGW's
# linker uses Unix-style library search semantics (once down the
# library list), and no other Windows toolchain we build with is that
# picky. So this ensures the Windows library structure continues to
# work in the most difficult circumstance we expect it to encounter.
in putty do cmake . -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain-mingw.cmake
in putty do make -j$(nproc) VERBOSE=1
enddelegate
# Windowsify LICENCE, since it's going in the Windows installers.
in putty do perl -i~ -pe 'y/\015//d;s/$$/\015/' LICENCE
@ -165,20 +165,20 @@ mkdir putty/windows/abuild64
#
# For the 32-bit ones, we set a subsystem version of 5.01, which
# allows the resulting files to still run on Windows XP.
in putty/windows with clangcl32 do Platform=x86 make -f Makefile.clangcl BUILDDIR=build32/ SUBSYSVER=,5.01 $(Makeargs) all -j$(nproc)
in putty/windows with clangcl64 do Platform=x64 make -f Makefile.clangcl BUILDDIR=build64/ $(Makeargs) all -j$(nproc)
in putty/windows/build32 with cmake_at_least_3.20 do cmake ../.. -DCMAKE_TOOLCHAIN_FILE=$(cmake_toolchain_clangcl32) -DCMAKE_BUILD_TYPE=Release -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DPUTTY_LINK_MAPS=ON
in putty/windows/build32 with cmake_at_least_3.20 do make -j$(nproc) VERBOSE=1
in putty/windows/build64 with cmake_at_least_3.20 do cmake ../.. -DCMAKE_TOOLCHAIN_FILE=$(cmake_toolchain_clangcl64) -DCMAKE_BUILD_TYPE=Release -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DPUTTY_LINK_MAPS=ON
in putty/windows/build64 with cmake_at_least_3.20 do make -j$(nproc) VERBOSE=1
# Build experimental Arm Windows binaries.
in putty/windows with clangcl_a32 do Platform=arm make -f Makefile.clangcl BUILDDIR=abuild32/ SUBSYSVER=,5.01 $(Makeargs) all -j$(nproc)
in putty/windows with clangcl_a64 do Platform=arm64 make -f Makefile.clangcl BUILDDIR=abuild64/ $(Makeargs) all -j$(nproc)
in putty/windows/abuild32 with cmake_at_least_3.20 do cmake ../.. -DCMAKE_TOOLCHAIN_FILE=$(cmake_toolchain_clangcl_a32) -DCMAKE_BUILD_TYPE=Release -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DPUTTY_LINK_MAPS=ON
in putty/windows/abuild32 with cmake_at_least_3.20 do make -j$(nproc) VERBOSE=1
in putty/windows/abuild64 with cmake_at_least_3.20 do cmake ../.. -DCMAKE_TOOLCHAIN_FILE=$(cmake_toolchain_clangcl_a64) -DCMAKE_BUILD_TYPE=Release -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DPUTTY_LINK_MAPS=ON
in putty/windows/abuild64 with cmake_at_least_3.20 do make -j$(nproc) VERBOSE=1
# Remove Windows binaries for the test programs we don't want to ship,
# like testcrypt.exe. (But we still _built_ them, to ensure the build
# worked.)
in putty/windows do make -f Makefile.clangcl BUILDDIR=build32/ cleantestprogs
in putty/windows do make -f Makefile.clangcl BUILDDIR=build64/ cleantestprogs
in putty/windows do make -f Makefile.clangcl BUILDDIR=abuild32/ cleantestprogs
in putty/windows do make -f Makefile.clangcl BUILDDIR=abuild64/ cleantestprogs
# Make a list of the Windows binaries we're going to ship, so that we
# can sign them.
in putty/windows do for subdir in build32 abuild32 build64 abuild64; do sed "s!^!$$subdir/!" $$subdir/shipped.txt; done > to-sign.txt
# Code-sign the Windows binaries, if the local bob config provides a
# script to do so in a cross-compiling way. We assume here that the
@ -187,7 +187,7 @@ in putty/windows do make -f Makefile.clangcl BUILDDIR=abuild64/ cleantestprogs
# take the program name from an .exe's version resource, and that it
# can accept multiple .exe or .msi filename arguments and sign them
# all in place.
ifneq "$(cross_winsigncode)" "" in putty/windows do $(cross_winsigncode) -N -i https://www.chiark.greenend.org.uk/~sgtatham/putty/ build*/*.exe abuild*/*.exe
ifneq "$(cross_winsigncode)" "" in putty/windows do $(cross_winsigncode) -N -i https://www.chiark.greenend.org.uk/~sgtatham/putty/ $$(cat to-sign.txt)
# Make a preliminary set of cryptographic checksums giving the hashes
# of these versions of the binaries. We'll make the rest below.
@ -217,19 +217,16 @@ in putty/windows do ./msifixup.py installera64.msi --dialog-bmp-width=123 --plat
# Sign the Windows installers.
ifneq "$(cross_winsigncode)" "" in putty/windows do $(cross_winsigncode) -i https://www.chiark.greenend.org.uk/~sgtatham/putty/ -n "PuTTY Installer" installer32.msi installer64.msi installera32.msi installera64.msi
# Delete the binaries and resource files from the build directories,
# so that we can rebuild them differently.
in putty/windows/build32 do rm -f *.exe *.res *.rcpp
in putty/windows/build64 do rm -f *.exe *.res *.rcpp
in putty/windows/abuild32 do rm -f *.exe *.res *.rcpp
in putty/windows/abuild64 do rm -f *.exe *.res *.rcpp
# Build the standalone binaries, in both 32- and 64-bit flavours.
# These differ from the previous set in that they embed the help file.
in putty/windows with clangcl32 do Platform=x86 make -f Makefile.clangcl BUILDDIR=build32/ RCFL=-DEMBED_CHM SUBSYSVER=,5.01 $(Makeargs) all -j$(nproc)
in putty/windows with clangcl64 do Platform=x64 make -f Makefile.clangcl BUILDDIR=build64/ RCFL=-DEMBED_CHM $(Makeargs) all -j$(nproc)
in putty/windows with clangcl_a32 do Platform=arm make -f Makefile.clangcl BUILDDIR=abuild32/ RCFL=-DEMBED_CHM SUBSYSVER=,5.01 $(Makeargs) all -j$(nproc)
in putty/windows with clangcl_a64 do Platform=arm64 make -f Makefile.clangcl BUILDDIR=abuild64/ RCFL=-DEMBED_CHM $(Makeargs) all -j$(nproc)
in putty/windows/build32 with cmake_at_least_3.20 do cmake . -DPUTTY_EMBEDDED_CHM_FILE=$$(realpath ../../doc/putty.chm)
in putty/windows/build32 with cmake_at_least_3.20 do make -j$(nproc) VERBOSE=1
in putty/windows/build64 with cmake_at_least_3.20 do cmake . -DPUTTY_EMBEDDED_CHM_FILE=$$(realpath ../../doc/putty.chm)
in putty/windows/build64 with cmake_at_least_3.20 do make -j$(nproc) VERBOSE=1
in putty/windows/abuild32 with cmake_at_least_3.20 do cmake . -DPUTTY_EMBEDDED_CHM_FILE=$$(realpath ../../doc/putty.chm)
in putty/windows/abuild32 with cmake_at_least_3.20 do make -j$(nproc) VERBOSE=1
in putty/windows/abuild64 with cmake_at_least_3.20 do cmake . -DPUTTY_EMBEDDED_CHM_FILE=$$(realpath ../../doc/putty.chm)
in putty/windows/abuild64 with cmake_at_least_3.20 do make -j$(nproc) VERBOSE=1
# Build the 'old' binaries, which should still run on all 32-bit
# versions of Windows back to Win95 (but not Win32s). These link
@ -241,42 +238,45 @@ in putty/windows with clangcl_a64 do Platform=arm64 make -f Makefile.clangcl BUI
#
# There's no installer to go with these, so they must also embed the
# help file.
in putty/windows with clangcl32_2003 do Platform=x86 make -f Makefile.clangcl BUILDDIR=buildold/ RCFL=-DEMBED_CHM $(Makeargs) CCTARGET=i386-pc-windows-msvc13.0.0 SUBSYSVER=,4.0 EXTRA_windows=wincrt0.obj EXTRA_console=crt0.obj EXTRA_libs=libcpmt.lib XFLAGS="/arch:IA32 -Wno-pragma-pack" all -j$(nproc)
in putty/windows/buildold with cmake_at_least_3.20 do cmake ../.. -DCMAKE_TOOLCHAIN_FILE=$(cmake_toolchain_clangcl32_2003) -DCMAKE_BUILD_TYPE=Release -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DPUTTY_LINK_MAPS=ON
in putty/windows/buildold with cmake_at_least_3.20 do make -j$(nproc) VERBOSE=1
# Remove test programs again.
in putty/windows do make -f Makefile.clangcl BUILDDIR=build32/ cleantestprogs
in putty/windows do make -f Makefile.clangcl BUILDDIR=build64/ cleantestprogs
in putty/windows do make -f Makefile.clangcl BUILDDIR=abuild32/ cleantestprogs
in putty/windows do make -f Makefile.clangcl BUILDDIR=abuild64/ cleantestprogs
in putty/windows do make -f Makefile.clangcl BUILDDIR=buildold/ cleantestprogs
# Regenerate to-sign.txt with the 'old' binaries included.
in putty/windows do for subdir in build32 abuild32 build64 abuild64 buildold; do sed "s!^!$$subdir/!" $$subdir/shipped.txt; done > to-sign.txt
# Code-sign the standalone versions of the binaries.
ifneq "$(cross_winsigncode)" "" in putty/windows do $(cross_winsigncode) -N -i https://www.chiark.greenend.org.uk/~sgtatham/putty/ build*/*.exe abuild*/*.exe
ifneq "$(cross_winsigncode)" "" in putty/windows do $(cross_winsigncode) -N -i https://www.chiark.greenend.org.uk/~sgtatham/putty/ $$(cat to-sign.txt)
# Move the shipped (and signed) binaries into another directory to
# deliver them from, so that we omit testcrypt and its ilk.
in putty/windows do mkdir deliver
in putty/windows do for subdir in build32 abuild32 build64 abuild64 buildold; do mkdir deliver/$$subdir; done
in putty/windows do while read x; do mv $$x deliver/$$x; mv $$x.map deliver/$$x.map; done < to-sign.txt
in putty/doc do make mostlyclean
in putty/doc do make $(Docmakever) -j$(nproc)
in putty/windows/buildold do zip -k -j putty.zip `ls *.exe | grep -v puttytel` ../../doc/putty.chm
in putty/windows/build32 do zip -k -j putty.zip `ls *.exe | grep -v puttytel` ../../doc/putty.chm
in putty/windows/build64 do zip -k -j putty.zip `ls *.exe | grep -v puttytel` ../../doc/putty.chm
in putty/windows/abuild32 do zip -k -j putty.zip `ls *.exe | grep -v puttytel` ../../doc/putty.chm
in putty/windows/abuild64 do zip -k -j putty.zip `ls *.exe | grep -v puttytel` ../../doc/putty.chm
in putty/windows/deliver/buildold do zip -k -j putty.zip `ls *.exe | grep -v puttytel` ../../doc/putty.chm
in putty/windows/deliver/build32 do zip -k -j putty.zip `ls *.exe | grep -v puttytel` ../../doc/putty.chm
in putty/windows/deliver/build64 do zip -k -j putty.zip `ls *.exe | grep -v puttytel` ../../doc/putty.chm
in putty/windows/deliver/abuild32 do zip -k -j putty.zip `ls *.exe | grep -v puttytel` ../../doc/putty.chm
in putty/windows/deliver/abuild64 do zip -k -j putty.zip `ls *.exe | grep -v puttytel` ../../doc/putty.chm
in putty/doc do zip puttydoc.zip *.html
# Deliver the actual PuTTY release directory into a subdir `putty'.
deliver putty/windows/buildold/*.exe putty/w32old/$@
deliver putty/windows/buildold/putty.zip putty/w32old/$@
deliver putty/windows/build32/*.exe putty/w32/$@
deliver putty/windows/build32/putty.zip putty/w32/$@
deliver putty/windows/build64/*.exe putty/w64/$@
deliver putty/windows/build64/putty.zip putty/w64/$@
deliver putty/windows/deliver/buildold/*.exe putty/w32old/$@
deliver putty/windows/deliver/buildold/putty.zip putty/w32old/$@
deliver putty/windows/deliver/build32/*.exe putty/w32/$@
deliver putty/windows/deliver/build32/putty.zip putty/w32/$@
deliver putty/windows/deliver/build64/*.exe putty/w64/$@
deliver putty/windows/deliver/build64/putty.zip putty/w64/$@
deliver putty/windows/installer32.msi putty/w32/$(Ifilename32).msi
deliver putty/windows/installer64.msi putty/w64/$(Ifilename64).msi
deliver putty/windows/installera32.msi putty/wa32/$(Ifilenamea32).msi
deliver putty/windows/installera64.msi putty/wa64/$(Ifilenamea64).msi
deliver putty/windows/abuild32/*.exe putty/wa32/$@
deliver putty/windows/abuild32/putty.zip putty/wa32/$@
deliver putty/windows/abuild64/*.exe putty/wa64/$@
deliver putty/windows/abuild64/putty.zip putty/wa64/$@
deliver putty/windows/deliver/abuild32/*.exe putty/wa32/$@
deliver putty/windows/deliver/abuild32/putty.zip putty/wa32/$@
deliver putty/windows/deliver/abuild64/*.exe putty/wa64/$@
deliver putty/windows/deliver/abuild64/putty.zip putty/wa64/$@
deliver putty/doc/puttydoc.zip putty/$@
deliver putty/doc/putty.chm putty/$@
deliver putty/doc/puttydoc.txt putty/$@
@ -285,11 +285,11 @@ deliver putty/putty-src.zip putty/$@
deliver putty/*.tar.gz putty/$@
# Deliver the map files alongside the `proper' release deliverables.
deliver putty/windows/buildold/*.map maps/w32old/$@
deliver putty/windows/build32/*.map maps/w32/$@
deliver putty/windows/build64/*.map maps/w64/$@
deliver putty/windows/abuild32/*.map maps/wa32/$@
deliver putty/windows/abuild64/*.map maps/wa64/$@
deliver putty/windows/deliver/buildold/*.map maps/w32old/$@
deliver putty/windows/deliver/build32/*.map maps/w32/$@
deliver putty/windows/deliver/build64/*.map maps/w64/$@
deliver putty/windows/deliver/abuild32/*.map maps/wa32/$@
deliver putty/windows/deliver/abuild64/*.map maps/wa64/$@
# Deliver sign.sh, so that whoever has just built PuTTY (the
# snapshot scripts or me, depending) can conveniently sign it with