nhyatt/putty-source
1
0
Fork 0
mirror of https://git.tartarus.org/simon/putty.git synced 2025-01-10 09:58:01 +00:00
Code Issues Projects Releases Wiki Activity
0a877e9df5
putty-source/test/testcrypt.c

1482 lines
46 KiB
C
Raw Normal View History

New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
/*
* testcrypt: a standalone test program that provides direct access to
* PuTTY's cryptography and mp_int code.
*/
/*
* This program speaks a line-oriented protocol on standard input and
* standard output. It's a half-duplex protocol: it expects to read
* one line of command, and then produce a fixed amount of output
* (namely a line containing a decimal integer, followed by that many
* lines each containing one return value).
*
* The protocol is human-readable enough to make it debuggable, but
* verbose enough that you probably wouldn't want to speak it by hand
* at any great length. The Python program test/testcrypt.py wraps it
* to give a more useful user-facing API, by invoking this binary as a
* subprocess.
*
* (I decided that was a better idea than making this program an
* actual Python module, partly because you can rewrap the same binary
* in another scripting language if you prefer, but mostly because
* it's easy to attach a debugger to testcrypt or to run it under
* sanitisers or valgrind or what have you.)
*/
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include "defs.h"
#include "ssh.h"
Refactor generation of candidate integers in primegen. I've replaced the random number generation and small delta-finding loop in primegen() with a much more elaborate system in its own source file, with unit tests and everything. Immediate benefits: - fixes a theoretical possibility of overflowing the target number of bits, if the random number was so close to the top of the range that the addition of delta * factor pushed it over. However, this only happened with negligible probability. - fixes a directional bias in delta-finding. The previous code incremented the number repeatedly until it found a value coprime to all the right things, which meant that a prime preceded by a particularly long sequence of numbers with tiny factors was more likely to be chosen. Now we select candidate delta values at random, that bias should be eliminated. - changes the semantics of the outermost primegen() function to make them easier to use, because now the caller specifies the 'bits' and 'firstbits' values for the actual returned prime, rather than having to account for the factor you're multiplying it by in DSA. DSA client code is correspondingly adjusted. Future benefits: - having the candidate generation in a separate function makes it easy to reuse in alternative prime generation strategies - the available constraints support applications such as Maurer's algorithm for generating provable primes, or strong primes for RSA in which both p-1 and p+1 have a large factor. So those become things we could experiment with in future.
2020-02-23 14:30:03 +00:00
#include "sshkeygen.h"
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
#include "misc.h"
#include "mpint.h"
Move a few stray header files into the crypto subdir. sshblowf.h (as was) is 100% internal to that directory. And mpint_i.h and ecc.h are specialist enough that it's reasonable to ask clients outside the crypto directory to include them with a subdirectory path, to hint that it's an unusual thing to be doing.
2021-04-22 16:57:56 +00:00
#include "crypto/ecc.h"
HTTP proxy: implement Digest authentication. In http.c, this drops in reasonably neatly alongside the existing support for Basic, now that we're waiting for an initial 407 response from the proxy to tell us which auth mechanism it would prefer to use. The rest of this patch is mostly contriving to add testcrypt support for the function in cproxy.c that generates the complicated output header to go in the HTTP request: you need about a dozen assorted parameters, the actual response hash has two more hashes in its preimage, and there's even an option to hash the username as well if necessary. Much more complicated than CHAP (which is just plain HMAC-MD5), so it needs testing! Happily, RFC 7616 comes with some reasonably useful test cases, and I've managed to transcribe them directly into cryptsuite.py and demonstrate that my response-generator agrees with them. End-to-end testing of the whole system was done against Squid 4.13 (specifically, the squid package in Debian bullseye, version 4.13-10).
2021-11-20 14:56:32 +00:00
#include "proxy/cproxy.h"
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
Greatly improve printf format-string checking. I've added the gcc-style attribute("printf") to a lot of printf-shaped functions in this code base that didn't have it. To make that easier, I moved the wrapping macro into defs.h, and also enabled it if we detect the __clang__ macro as well as __GNU__ (hence, it will be used when building for Windows using clang-cl). The result is that a great many format strings in the code are now checked by the compiler, where they were previously not. This causes build failures, which I'll fix in the next commit.
2020-01-26 14:49:31 +00:00
static NORETURN PRINTF_LIKE(1, 2) void fatal_error(const char *p, ...)
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
{
va_list ap;
fprintf(stderr, "testcrypt: ");
va_start(ap, p);
vfprintf(stderr, p, ap);
va_end(ap);
fputc('\n', stderr);
exit(1);
}
void out_of_memory(void) { fatal_error("out of memory"); }
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
static bool old_keyfile_warning_given;
void old_keyfile_warning(void) { old_keyfile_warning_given = true; }
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
static bufchain random_data_queue;
testcrypt: allow random_read() to use a full PRNG. This still isn't the true random generator used in the live tools: it's deterministic, for repeatable testing. The Python side of testcrypt can now call random_make_prng(), which will instantiate a PRNG with the given seed. random_clear() still gets rid of it. So I can still have some tests control the precise random numbers received by the function under test, but for others (especially key generation, with its uncertainty about how much randomness it will actually use) I can just say 'here, have a seed, generate as much stuff from that seed as you need'.
2020-02-19 19:12:59 +00:00
static prng *test_prng;
Replace random_byte() with random_read(). This is in preparation for a PRNG revamp which will want to have a well defined boundary for any given request-for-randomness, so that it can destroy the evidence afterwards. So no more looping round calling random_byte() and then stopping when we feel like it: now you say up front how many random bytes you want, and call random_read() which gives you that many in one go. Most of the call sites that had to be fixed are fairly mechanical, and quite a few ended up more concise afterwards. A few became more cumbersome, such as mp_random_bits, in which the new API doesn't let me load the random bytes directly into the target integer without triggering undefined behaviour, so instead I have to allocate a separate temporary buffer. The _most_ interesting call site was in the PKCS#1 v1.5 padding code in sshrsa.c (used in SSH-1), in which you need a stream of _nonzero_ random bytes. The previous code just looped on random_byte, retrying if it got a zero. Now I'm doing a much more interesting thing with an mpint, essentially scaling a binary fraction repeatedly to extract a number in the range [0,255) and then adding 1 to it.
2019-01-22 19:43:27 +00:00
void random_read(void *buf, size_t size)
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
{
testcrypt: allow random_read() to use a full PRNG. This still isn't the true random generator used in the live tools: it's deterministic, for repeatable testing. The Python side of testcrypt can now call random_make_prng(), which will instantiate a PRNG with the given seed. random_clear() still gets rid of it. So I can still have some tests control the precise random numbers received by the function under test, but for others (especially key generation, with its uncertainty about how much randomness it will actually use) I can just say 'here, have a seed, generate as much stuff from that seed as you need'.
2020-02-19 19:12:59 +00:00
if (test_prng) {
prng_read(test_prng, buf, size);
} else {
if (!bufchain_try_fetch_consume(&random_data_queue, buf, size))
fatal_error("No random data in queue");
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
}
Replace PuTTY's PRNG with a Fortuna-like system. This tears out the entire previous random-pool system in sshrand.c. In its place is a system pretty close to Ferguson and Schneier's 'Fortuna' generator, with the main difference being that I use SHA-256 instead of AES for the generation side of the system (rationale given in comment). The PRNG implementation lives in sshprng.c, and defines a self- contained data type with no state stored outside the object, so you can instantiate however many of them you like. The old sshrand.c still exists, but in place of the previous random pool system, it's just become a client of sshprng.c, whose job is to hold a single global instance of the PRNG type, and manage its reference count, save file, noise-collection timers and similar administrative business. Advantages of this change include: - Fortuna is designed with a more varied threat model in mind than my old home-grown random pool. For example, after any request for random numbers, it automatically re-seeds itself, so that if the state of the PRNG should be leaked, it won't give enough information to find out what past outputs _were_. - The PRNG type can be instantiated with any hash function; the instance used by the main tools is based on SHA-256, an improvement on the old pool's use of SHA-1. - The new PRNG only uses the completely standard interface to the hash function API, instead of having to have privileged access to the internal SHA-1 block transform function. This will make it easier to revamp the hash code in general, and also it means that hardware-accelerated versions of SHA-256 will automatically be used for the PRNG as well as for everything else. - The new PRNG can be _tested_! Because it has an actual (if not quite explicit) specification for exactly what the output numbers _ought_ to be derived from the hashes of, I can (and have) put tests in cryptsuite that ensure the output really is being derived in the way I think it is. The old pool could have been returning any old nonsense and it would have been very hard to tell for sure.
2019-01-22 22:42:41 +00:00
uint64_t prng_reseed_time_ms(void)
{
static uint64_t previous_time = 0;
return previous_time += 200;
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
#define VALUE_TYPES(X) \
X(string, strbuf *, strbuf_free(v)) \
X(mpint, mp_int *, mp_free(v)) \
X(modsqrt, ModsqrtContext *, modsqrt_free(v)) \
X(monty, MontyContext *, monty_free(v)) \
X(wcurve, WeierstrassCurve *, ecc_weierstrass_curve_free(v)) \
X(wpoint, WeierstrassPoint *, ecc_weierstrass_point_free(v)) \
X(mcurve, MontgomeryCurve *, ecc_montgomery_curve_free(v)) \
X(mpoint, MontgomeryPoint *, ecc_montgomery_point_free(v)) \
X(ecurve, EdwardsCurve *, ecc_edwards_curve_free(v)) \
X(epoint, EdwardsPoint *, ecc_edwards_point_free(v)) \
X(hash, ssh_hash *, ssh_hash_free(v)) \
X(key, ssh_key *, ssh_key_free(v)) \
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
X(cipher, ssh_cipher *, ssh_cipher_free(v)) \
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
X(mac, ssh2_mac *, ssh2_mac_free(v)) \
X(dh, dh_ctx *, dh_cleanup(v)) \
X(ecdh, ecdh_key *, ssh_ecdhkex_freekey(v)) \
X(rsakex, RSAKey *, ssh_rsakex_freekey(v)) \
X(rsa, RSAKey *, rsa_free(v)) \
Replace PuTTY's PRNG with a Fortuna-like system. This tears out the entire previous random-pool system in sshrand.c. In its place is a system pretty close to Ferguson and Schneier's 'Fortuna' generator, with the main difference being that I use SHA-256 instead of AES for the generation side of the system (rationale given in comment). The PRNG implementation lives in sshprng.c, and defines a self- contained data type with no state stored outside the object, so you can instantiate however many of them you like. The old sshrand.c still exists, but in place of the previous random pool system, it's just become a client of sshprng.c, whose job is to hold a single global instance of the PRNG type, and manage its reference count, save file, noise-collection timers and similar administrative business. Advantages of this change include: - Fortuna is designed with a more varied threat model in mind than my old home-grown random pool. For example, after any request for random numbers, it automatically re-seeds itself, so that if the state of the PRNG should be leaked, it won't give enough information to find out what past outputs _were_. - The PRNG type can be instantiated with any hash function; the instance used by the main tools is based on SHA-256, an improvement on the old pool's use of SHA-1. - The new PRNG only uses the completely standard interface to the hash function API, instead of having to have privileged access to the internal SHA-1 block transform function. This will make it easier to revamp the hash code in general, and also it means that hardware-accelerated versions of SHA-256 will automatically be used for the PRNG as well as for everything else. - The new PRNG can be _tested_! Because it has an actual (if not quite explicit) specification for exactly what the output numbers _ought_ to be derived from the hashes of, I can (and have) put tests in cryptsuite that ensure the output really is being derived in the way I think it is. The old pool could have been returning any old nonsense and it would have been very hard to tell for sure.
2019-01-22 22:42:41 +00:00
X(prng, prng *, prng_free(v)) \
cmdgen: add a --dump option. Also spelled '-O text', this takes a public or private key as input, and produces on standard output a dump of all the actual numbers involved in the key: the exponent and modulus for RSA, the p,q,g,y parameters for DSA, the affine x and y coordinates of the public elliptic curve point for ECC keys, and all the extra bits and pieces in the private keys too. Partly I expect this to be useful to me for debugging: I've had to paste key files a few too many times through base64 decoders and hex dump tools, then manually decode SSH marshalling and paste the result into the Python REPL to get an integer object. Now I should be able to get _straight_ to text I can paste into Python. But also, it's a way that other applications can use the key generator: if you need to generate, say, an RSA key in some format I don't support (I've recently heard of an XML-based one, for example), then you can run 'puttygen -t rsa --dump' and have it print the elements of a freshly generated keypair on standard output, and then all you have to do is understand the output format.
2020-02-17 19:53:19 +00:00
X(keycomponents, key_components *, key_components_free(v)) \
Refactor generation of candidate integers in primegen. I've replaced the random number generation and small delta-finding loop in primegen() with a much more elaborate system in its own source file, with unit tests and everything. Immediate benefits: - fixes a theoretical possibility of overflowing the target number of bits, if the random number was so close to the top of the range that the addition of delta * factor pushed it over. However, this only happened with negligible probability. - fixes a directional bias in delta-finding. The previous code incremented the number repeatedly until it found a value coprime to all the right things, which meant that a prime preceded by a particularly long sequence of numbers with tiny factors was more likely to be chosen. Now we select candidate delta values at random, that bias should be eliminated. - changes the semantics of the outermost primegen() function to make them easier to use, because now the caller specifies the 'bits' and 'firstbits' values for the actual returned prime, rather than having to account for the factor you're multiplying it by in DSA. DSA client code is correspondingly adjusted. Future benefits: - having the candidate generation in a separate function makes it easy to reuse in alternative prime generation strategies - the available constraints support applications such as Maurer's algorithm for generating provable primes, or strong primes for RSA in which both p-1 and p+1 have a large factor. So those become things we could experiment with in future.
2020-02-23 14:30:03 +00:00
X(pcs, PrimeCandidateSource *, pcs_free(v)) \
Introduce a vtable system for prime generation. The functions primegen() and primegen_add_progress_phase() are gone. In their place is a small vtable system with two methods corresponding to them, plus the usual admin of allocating and freeing contexts. This API change is the starting point for being able to drop in different prime generation algorithms at run time in response to user configuration.
2020-02-29 09:10:47 +00:00
X(pgc, PrimeGenerationContext *, primegen_free_context(v)) \
New 'Pockle' object, for verifying primality. This implements an extended form of primality verification using certificates based on Pocklington's theorem. You make a Pockle object, and then try to convince it that one number after another is prime, by means of providing it with a list of prime factors of p-1 and a primitive root. (Or just by saying 'this prime is small enough for you to check yourself'.) Pocklington's theorem requires you to have factors of p-1 whose product is at least the square root of p. I've extended that to support factorisations only as big as the cube root, via an extension of the theorem given in Maurer's paper on generating provable primes. The Pockle object is more or less write-only: it has no methods for reading out its contents. Its only output channel is the return value when you try to insert a prime into it: if it isn't sufficiently convinced that your prime is prime, it will return an error code. So anything for which it returns POCKLE_OK you can be confident of. I'm going to use this for provable prime generation. But exposing this part of the system as an object in its own right means I can write a set of unit tests for this specifically. My negative tests exercise all the different ways a certification can be erroneous or inadequate; the positive tests include proofs of primality of various primes used in elliptic-curve crypto. The Poly1305 proof in particular is taken from a proof in DJB's paper, which has exactly the form of a Pocklington certificate only written in English.
2020-02-23 15:16:30 +00:00
X(pockle, Pockle *, pockle_free(v)) \
Add some tests of Miller-Rabin to cryptsuite. I'm about to rewrite the Miller-Rabin testing code, so let's start by introducing a test suite that the old version passes, and then I can make sure the new one does too.
2021-08-27 16:43:40 +00:00
X(millerrabin, MillerRabin *, miller_rabin_free(v)) \
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
/* end of list */
typedef struct Value Value;
enum ValueType {
#define VALTYPE_ENUM(n,t,f) VT_##n,
VALUE_TYPES(VALTYPE_ENUM)
#undef VALTYPE_ENUM
Whitespace rationalisation of entire code base. The number of people has been steadily increasing who read our source code with an editor that thinks tab stops are 4 spaces apart, as opposed to the traditional tty-derived 8 that the PuTTY code expects. So I've been wondering for ages about just fixing it, and switching to a spaces-only policy throughout the code. And I recently found out about 'git blame -w', which should make this change not too disruptive for the purposes of source-control archaeology; so perhaps now is the time. While I'm at it, I've also taken the opportunity to remove all the trailing spaces from source lines (on the basis that git dislikes them, and is the only thing that seems to have a strong opinion one way or the other). Apologies to anyone downstream of this code who has complicated patch sets to rebase past this change. I don't intend it to be needed again.
2019-09-08 19:29:00 +00:00
};
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
testcrypt: fix a technically illegal forward-typedef. Every compiler that's so far seen testcrypt.c has tolerated me writing 'typedef enum ValueType ValueType' before actually saying what 'enum ValueType' is, but one just pointed out that it's not actually legal standard C to do that. Moved the typedef to after the enum.
2019-01-11 19:13:27 +00:00
typedef enum ValueType ValueType;
Add lots of missing 'static' keywords. A trawl through the code with -Wmissing-prototypes and -Wmissing-variable-declarations turned up a lot of things that should have been internal to a particular source file, but were accidentally global. Keep the namespace clean by making them all static. (Also, while I'm here, a couple of them were missing a 'const': the ONE and ZERO arrays in sshcrcda.c, and EMPTY_WINDOW_TITLE in terminal.c.)
2020-01-29 06:22:01 +00:00
static const char *const type_names[] = {
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
#define VALTYPE_NAME(n,t,f) #n,
VALUE_TYPES(VALTYPE_NAME)
#undef VALTYPE_NAME
};
Introduce a vtable system for prime generation. The functions primegen() and primegen_add_progress_phase() are gone. In their place is a small vtable system with two methods corresponding to them, plus the usual admin of allocating and freeing contexts. This API change is the starting point for being able to drop in different prime generation algorithms at run time in response to user configuration.
2020-02-29 09:10:47 +00:00
#define VALTYPE_TYPEDEF(n,t,f) \
typedef t TD_val_##n; \
typedef t *TD_out_val_##n;
VALUE_TYPES(VALTYPE_TYPEDEF)
#undef VALTYPE_TYPEDEF
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
struct Value {
/*
* Protocol identifier assigned to this value when it was created.
* Lives in the same malloced block as this Value object itself.
*/
ptrlen id;
/*
* Type of the value.
*/
ValueType type;
/*
* Union of all the things it could hold.
*/
union {
#define VALTYPE_UNION(n,t,f) t vu_##n;
VALUE_TYPES(VALTYPE_UNION)
#undef VALTYPE_UNION
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
char *bare_string;
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
};
};
static int valuecmp(void *av, void *bv)
{
Value *a = (Value *)av, *b = (Value *)bv;
return ptrlen_strcmp(a->id, b->id);
}
static int valuefind(void *av, void *bv)
{
ptrlen *a = (ptrlen *)av;
Value *b = (Value *)bv;
return ptrlen_strcmp(*a, b->id);
}
static tree234 *values;
static Value *value_new(ValueType vt)
{
static uint64_t next_index = 0;
char *name = dupprintf("%s%"PRIu64, type_names[vt], next_index++);
size_t namelen = strlen(name);
Value *val = snew_plus(Value, namelen+1);
memcpy(snew_plus_get_aux(val), name, namelen+1);
val->id.ptr = snew_plus_get_aux(val);
val->id.len = namelen;
val->type = vt;
Value *added = add234(values, val);
assert(added == val);
sfree(name);
return val;
}
#define VALTYPE_RETURNFN(n,t,f) \
void return_val_##n(strbuf *out, t v) { \
Value *val = value_new(VT_##n); \
val->vu_##n = v; \
put_datapl(out, val->id); \
put_byte(out, '\n'); \
}
VALUE_TYPES(VALTYPE_RETURNFN)
#undef VALTYPE_RETURNFN
static ptrlen get_word(BinarySource *in)
{
ptrlen toret;
toret.ptr = get_ptr(in);
toret.len = 0;
while (get_avail(in) && get_byte(in) != ' ')
toret.len++;
return toret;
}
testcrypt: move TD_foo declarations up the file. A completely trivial change which I only put in its own commit so that the next one will have a less confusing-looking diff.
2021-11-22 18:35:08 +00:00
typedef uintmax_t TD_uint;
typedef bool TD_boolean;
typedef ptrlen TD_val_string_ptrlen;
typedef char *TD_val_string_asciz;
typedef BinarySource *TD_val_string_binarysource;
typedef unsigned *TD_out_uint;
typedef BinarySink *TD_out_val_string_binarysink;
typedef const char *TD_opt_val_string_asciz;
typedef char **TD_out_val_string_asciz;
typedef char **TD_out_opt_val_string_asciz;
typedef const char **TD_out_opt_val_string_asciz_const;
typedef ssh_hash *TD_consumed_val_hash;
typedef const ssh_hashalg *TD_hashalg;
typedef const ssh2_macalg *TD_macalg;
typedef const ssh_keyalg *TD_keyalg;
typedef const ssh_cipheralg *TD_cipheralg;
typedef const ssh_kex *TD_dh_group;
typedef const ssh_kex *TD_ecdh_alg;
typedef RsaSsh1Order TD_rsaorder;
typedef key_components *TD_keycomponents;
typedef const PrimeGenerationPolicy *TD_primegenpolicy;
typedef struct mpint_list TD_mpint_list;
typedef PockleStatus TD_pocklestatus;
typedef struct mr_result TD_mr_result;
typedef Argon2Flavour TD_argon2flavour;
typedef FingerprintType TD_fptype;
typedef HttpDigestHash TD_httpdigesthash;
testcrypt: move enumerated types out into another header. Now all the lookup functions like get_hashalg() and get_primegenpolicy() are autogenerated by macro expansion from a header a little like the existing testcrypt.h. But the macros are much simpler this time. This doesn't really change anything, _except_ that now I'll be able to reinclude the same header to do other things with the list of enumerated types.
2021-11-22 18:38:36 +00:00
#define BEGIN_ENUM_TYPE(name) \
static bool enum_translate_##name(ptrlen valname, TD_##name *out) { \
static const struct { \
const char *key; \
TD_##name value; \
} mapping[] = {
#define ENUM_VALUE(name, value) {name, value},
#define END_ENUM_TYPE(name) \
}; \
for (size_t i = 0; i < lenof(mapping); i++) \
if (ptrlen_eq_string(valname, mapping[i].key)) { \
if (out) \
*out = mapping[i].value; \
return true; \
} \
return false; \
} \
\
static TD_##name get_##name(BinarySource *in) { \
ptrlen valname = get_word(in); \
TD_##name out; \
if (enum_translate_##name(valname, &out)) \
return out; \
else \
fatal_error("%s '%.*s': not found", \
#name, PTRLEN_PRINTF(valname)); \
}
#include "testcrypt-enum.h"
#undef BEGIN_ENUM_TYPE
#undef ENUM_VALUE
#undef END_ENUM_TYPE
HTTP proxy: implement Digest authentication. In http.c, this drops in reasonably neatly alongside the existing support for Basic, now that we're waiting for an initial 407 response from the proxy to tell us which auth mechanism it would prefer to use. The rest of this patch is mostly contriving to add testcrypt support for the function in cproxy.c that generates the complicated output header to go in the HTTP request: you need about a dozen assorted parameters, the actual response hash has two more hashes in its preimage, and there's even an option to hash the username as well if necessary. Much more complicated than CHAP (which is just plain HMAC-MD5), so it needs testing! Happily, RFC 7616 comes with some reasonably useful test cases, and I've managed to transcribe them directly into cryptsuite.py and demonstrate that my response-generator agrees with them. End-to-end testing of the whole system was done against Squid 4.13 (specifically, the squid package in Debian bullseye, version 4.13-10).
2021-11-20 14:56:32 +00:00
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
static uintmax_t get_uint(BinarySource *in)
{
ptrlen word = get_word(in);
char *string = mkstr(word);
uintmax_t toret = strtoumax(string, NULL, 0);
sfree(string);
return toret;
}
RSA generation: option to generate strong primes. A 'strong' prime, as defined by the Handbook of Applied Cryptography, is a prime p such that each of p-1 and p+1 has a large prime factor, and that the large factor q of p-1 is such that q-1 in turn _also_ has a large prime factor. HoAC says that making your RSA key using primes of this form defeats some factoring algorithms - but there are other faster algorithms to which it makes no difference. So this is probably not a useful precaution in practice. However, it has been recommended in the past by some official standards, and it's easy to implement given the new general facility in PrimeCandidateSource that lets you ask for your prime to satisfy an arbitrary modular congruence. (And HoAC also says there's no particular reason _not_ to use strong primes.) So I provide it as an option, just in case anyone wants to select it. The change to the key generation algorithm is entirely in sshrsag.c, and is neatly independent of the prime-generation system in use. If you're using Maurer provable prime generation, then the known factor q of p-1 can be used to help certify p, and the one for q-1 to help with q in turn; if you switch to probabilistic prime generation then you still get an RSA key with the right structure, except that every time the definition says 'prime factor' you just append '(probably)'. (The probabilistic version of this procedure is described as 'Gordon's algorithm' in HoAC section 4.4.2.)
2020-03-02 06:52:09 +00:00
static bool get_boolean(BinarySource *in)
{
return ptrlen_eq_string(get_word(in), "true");
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
static Value *lookup_value(ptrlen word)
{
Value *val = find234(values, &word, valuefind);
if (!val)
fatal_error("id '%.*s': not found", PTRLEN_PRINTF(word));
return val;
}
static Value *get_value(BinarySource *in)
{
return lookup_value(get_word(in));
}
typedef void (*finaliser_fn_t)(strbuf *out, void *ctx);
struct finaliser {
finaliser_fn_t fn;
void *ctx;
};
static struct finaliser *finalisers;
Add lots of missing 'static' keywords. A trawl through the code with -Wmissing-prototypes and -Wmissing-variable-declarations turned up a lot of things that should have been internal to a particular source file, but were accidentally global. Keep the namespace clean by making them all static. (Also, while I'm here, a couple of them were missing a 'const': the ONE and ZERO arrays in sshcrcda.c, and EMPTY_WINDOW_TITLE in terminal.c.)
2020-01-29 06:22:01 +00:00
static size_t nfinalisers, finalisersize;
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
static void add_finaliser(finaliser_fn_t fn, void *ctx)
{
New array-growing macros: sgrowarray and sgrowarrayn. The idea of these is that they centralise the common idiom along the lines of if (logical_array_len >= physical_array_size) { physical_array_size = logical_array_len * 5 / 4 + 256; array = sresize(array, physical_array_size, ElementType); } which happens at a zillion call sites throughout this code base, with different random choices of the geometric factor and additive constant, sometimes forgetting them completely, and generally doing a lot of repeated work. The new macro sgrowarray(array,size,n) has the semantics: here are the array pointer and its physical size for you to modify, now please ensure that the nth element exists, so I can write into it. And sgrowarrayn(array,size,n,m) is the same except that it ensures that the array has size at least n+m (so sgrowarray is just the special case where m=1). Now that this is a single centralised implementation that will be used everywhere, I've also gone to more effort in the implementation, with careful overflow checks that would have been painful to put at all the previous call sites. This commit also switches over every use of sresize(), apart from a few where I really didn't think it would gain anything. A consequence of that is that a lot of array-size variables have to have their types changed to size_t, because the macros require that (they address-take the size to pass to the underlying function).
2019-02-28 20:07:30 +00:00
sgrowarray(finalisers, finalisersize, nfinalisers);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
finalisers[nfinalisers].fn = fn;
finalisers[nfinalisers].ctx = ctx;
nfinalisers++;
}
static void run_finalisers(strbuf *out)
{
for (size_t i = 0; i < nfinalisers; i++)
finalisers[i].fn(out, finalisers[i].ctx);
nfinalisers = 0;
}
static void finaliser_return_value(strbuf *out, void *ctx)
{
Value *val = (Value *)ctx;
put_datapl(out, val->id);
put_byte(out, '\n');
}
New 'Pockle' object, for verifying primality. This implements an extended form of primality verification using certificates based on Pocklington's theorem. You make a Pockle object, and then try to convince it that one number after another is prime, by means of providing it with a list of prime factors of p-1 and a primitive root. (Or just by saying 'this prime is small enough for you to check yourself'.) Pocklington's theorem requires you to have factors of p-1 whose product is at least the square root of p. I've extended that to support factorisations only as big as the cube root, via an extension of the theorem given in Maurer's paper on generating provable primes. The Pockle object is more or less write-only: it has no methods for reading out its contents. Its only output channel is the return value when you try to insert a prime into it: if it isn't sufficiently convinced that your prime is prime, it will return an error code. So anything for which it returns POCKLE_OK you can be confident of. I'm going to use this for provable prime generation. But exposing this part of the system as an object in its own right means I can write a set of unit tests for this specifically. My negative tests exercise all the different ways a certification can be erroneous or inadequate; the positive tests include proofs of primality of various primes used in elliptic-curve crypto. The Poly1305 proof in particular is taken from a proof in DJB's paper, which has exactly the form of a Pocklington certificate only written in English.
2020-02-23 15:16:30 +00:00
static void finaliser_sfree(strbuf *out, void *ctx)
{
sfree(ctx);
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
#define VALTYPE_GETFN(n,t,f) \
static Value *unwrap_value_##n(Value *val) { \
ValueType expected = VT_##n; \
if (expected != val->type) \
fatal_error("id '%.*s': expected %s, got %s", \
PTRLEN_PRINTF(val->id), \
type_names[expected], type_names[val->type]); \
return val; \
} \
static Value *get_value_##n(BinarySource *in) { \
return unwrap_value_##n(get_value(in)); \
} \
static t get_val_##n(BinarySource *in) { \
return get_value_##n(in)->vu_##n; \
}
VALUE_TYPES(VALTYPE_GETFN)
#undef VALTYPE_GETFN
static ptrlen get_val_string_ptrlen(BinarySource *in)
{
return ptrlen_from_strbuf(get_val_string(in));
}
static char *get_val_string_asciz(BinarySource *in)
{
return get_val_string(in)->s;
}
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
static strbuf *get_opt_val_string(BinarySource *in);
static char *get_opt_val_string_asciz(BinarySource *in)
{
strbuf *sb = get_opt_val_string(in);
return sb ? sb->s : NULL;
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
static mp_int **get_out_val_mpint(BinarySource *in)
{
Value *val = value_new(VT_mpint);
add_finaliser(finaliser_return_value, val);
return &val->vu_mpint;
}
New 'Pockle' object, for verifying primality. This implements an extended form of primality verification using certificates based on Pocklington's theorem. You make a Pockle object, and then try to convince it that one number after another is prime, by means of providing it with a list of prime factors of p-1 and a primitive root. (Or just by saying 'this prime is small enough for you to check yourself'.) Pocklington's theorem requires you to have factors of p-1 whose product is at least the square root of p. I've extended that to support factorisations only as big as the cube root, via an extension of the theorem given in Maurer's paper on generating provable primes. The Pockle object is more or less write-only: it has no methods for reading out its contents. Its only output channel is the return value when you try to insert a prime into it: if it isn't sufficiently convinced that your prime is prime, it will return an error code. So anything for which it returns POCKLE_OK you can be confident of. I'm going to use this for provable prime generation. But exposing this part of the system as an object in its own right means I can write a set of unit tests for this specifically. My negative tests exercise all the different ways a certification can be erroneous or inadequate; the positive tests include proofs of primality of various primes used in elliptic-curve crypto. The Poly1305 proof in particular is taken from a proof in DJB's paper, which has exactly the form of a Pocklington certificate only written in English.
2020-02-23 15:16:30 +00:00
struct mpint_list {
size_t n;
mp_int **integers;
};
static struct mpint_list get_mpint_list(BinarySource *in)
{
size_t n = get_uint(in);
struct mpint_list mpl;
mpl.n = n;
mpl.integers = snewn(n, mp_int *);
for (size_t i = 0; i < n; i++)
mpl.integers[i] = get_val_mpint(in);
add_finaliser(finaliser_sfree, mpl.integers);
return mpl;
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
static void finaliser_return_uint(strbuf *out, void *ctx)
{
unsigned *uval = (unsigned *)ctx;
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "%u\n", *uval);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
sfree(uval);
}
static unsigned *get_out_uint(BinarySource *in)
{
unsigned *uval = snew(unsigned);
add_finaliser(finaliser_return_uint, uval);
return uval;
}
static BinarySink *get_out_val_string_binarysink(BinarySource *in)
{
Value *val = value_new(VT_string);
val->vu_string = strbuf_new();
add_finaliser(finaliser_return_value, val);
return BinarySink_UPCAST(val->vu_string);
}
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
static void return_val_string_asciz_const(strbuf *out, const char *s);
static void return_val_string_asciz(strbuf *out, char *s);
static void finaliser_return_opt_string_asciz(strbuf *out, void *ctx)
{
char **valp = (char **)ctx;
char *val = *valp;
sfree(valp);
if (!val)
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "NULL\n");
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
else
return_val_string_asciz(out, val);
}
static char **get_out_opt_val_string_asciz(BinarySource *in)
{
char **valp = snew(char *);
*valp = NULL;
add_finaliser(finaliser_return_opt_string_asciz, valp);
return valp;
}
static void finaliser_return_opt_string_asciz_const(strbuf *out, void *ctx)
{
const char **valp = (const char **)ctx;
const char *val = *valp;
sfree(valp);
if (!val)
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "NULL\n");
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
else
return_val_string_asciz_const(out, val);
}
static const char **get_out_opt_val_string_asciz_const(BinarySource *in)
{
const char **valp = snew(const char *);
*valp = NULL;
add_finaliser(finaliser_return_opt_string_asciz_const, valp);
return valp;
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
static BinarySource *get_val_string_binarysource(BinarySource *in)
{
strbuf *sb = get_val_string(in);
BinarySource *src = snew(BinarySource);
BinarySource_BARE_INIT(src, sb->u, sb->len);
add_finaliser(finaliser_sfree, src);
return src;
}
Introduce a vtable system for prime generation. The functions primegen() and primegen_add_progress_phase() are gone. In their place is a small vtable system with two methods corresponding to them, plus the usual admin of allocating and freeing contexts. This API change is the starting point for being able to drop in different prime generation algorithms at run time in response to user configuration.
2020-02-29 09:10:47 +00:00
#define GET_CONSUMED_FN(type) \
typedef TD_val_##type TD_consumed_val_##type; \
static TD_val_##type get_consumed_val_##type(BinarySource *in) \
{ \
Value *val = get_value_##type(in); \
TD_val_##type toret = val->vu_##type; \
del234(values, val); \
sfree(val); \
return toret; \
}
GET_CONSUMED_FN(hash)
GET_CONSUMED_FN(pcs)
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
static void return_int(strbuf *out, intmax_t u)
{
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "%"PRIdMAX"\n", u);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
}
static void return_uint(strbuf *out, uintmax_t u)
{
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "0x%"PRIXMAX"\n", u);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
}
static void return_boolean(strbuf *out, bool b)
{
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "%s\n", b ? "true" : "false");
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
}
New 'Pockle' object, for verifying primality. This implements an extended form of primality verification using certificates based on Pocklington's theorem. You make a Pockle object, and then try to convince it that one number after another is prime, by means of providing it with a list of prime factors of p-1 and a primitive root. (Or just by saying 'this prime is small enough for you to check yourself'.) Pocklington's theorem requires you to have factors of p-1 whose product is at least the square root of p. I've extended that to support factorisations only as big as the cube root, via an extension of the theorem given in Maurer's paper on generating provable primes. The Pockle object is more or less write-only: it has no methods for reading out its contents. Its only output channel is the return value when you try to insert a prime into it: if it isn't sufficiently convinced that your prime is prime, it will return an error code. So anything for which it returns POCKLE_OK you can be confident of. I'm going to use this for provable prime generation. But exposing this part of the system as an object in its own right means I can write a set of unit tests for this specifically. My negative tests exercise all the different ways a certification can be erroneous or inadequate; the positive tests include proofs of primality of various primes used in elliptic-curve crypto. The Poly1305 proof in particular is taken from a proof in DJB's paper, which has exactly the form of a Pocklington certificate only written in English.
2020-02-23 15:16:30 +00:00
static void return_pocklestatus(strbuf *out, PockleStatus status)
{
switch (status) {
default:
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "POCKLE_BAD_STATUS_VALUE\n");
New 'Pockle' object, for verifying primality. This implements an extended form of primality verification using certificates based on Pocklington's theorem. You make a Pockle object, and then try to convince it that one number after another is prime, by means of providing it with a list of prime factors of p-1 and a primitive root. (Or just by saying 'this prime is small enough for you to check yourself'.) Pocklington's theorem requires you to have factors of p-1 whose product is at least the square root of p. I've extended that to support factorisations only as big as the cube root, via an extension of the theorem given in Maurer's paper on generating provable primes. The Pockle object is more or less write-only: it has no methods for reading out its contents. Its only output channel is the return value when you try to insert a prime into it: if it isn't sufficiently convinced that your prime is prime, it will return an error code. So anything for which it returns POCKLE_OK you can be confident of. I'm going to use this for provable prime generation. But exposing this part of the system as an object in its own right means I can write a set of unit tests for this specifically. My negative tests exercise all the different ways a certification can be erroneous or inadequate; the positive tests include proofs of primality of various primes used in elliptic-curve crypto. The Poly1305 proof in particular is taken from a proof in DJB's paper, which has exactly the form of a Pocklington certificate only written in English.
2020-02-23 15:16:30 +00:00
break;
#define STATUS_CASE(id) \
case id: \
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "%s\n", #id); \
New 'Pockle' object, for verifying primality. This implements an extended form of primality verification using certificates based on Pocklington's theorem. You make a Pockle object, and then try to convince it that one number after another is prime, by means of providing it with a list of prime factors of p-1 and a primitive root. (Or just by saying 'this prime is small enough for you to check yourself'.) Pocklington's theorem requires you to have factors of p-1 whose product is at least the square root of p. I've extended that to support factorisations only as big as the cube root, via an extension of the theorem given in Maurer's paper on generating provable primes. The Pockle object is more or less write-only: it has no methods for reading out its contents. Its only output channel is the return value when you try to insert a prime into it: if it isn't sufficiently convinced that your prime is prime, it will return an error code. So anything for which it returns POCKLE_OK you can be confident of. I'm going to use this for provable prime generation. But exposing this part of the system as an object in its own right means I can write a set of unit tests for this specifically. My negative tests exercise all the different ways a certification can be erroneous or inadequate; the positive tests include proofs of primality of various primes used in elliptic-curve crypto. The Poly1305 proof in particular is taken from a proof in DJB's paper, which has exactly the form of a Pocklington certificate only written in English.
2020-02-23 15:16:30 +00:00
break;
POCKLE_STATUSES(STATUS_CASE);
#undef STATUS_CASE
}
}
Add some tests of Miller-Rabin to cryptsuite. I'm about to rewrite the Miller-Rabin testing code, so let's start by introducing a test suite that the old version passes, and then I can make sure the new one does too.
2021-08-27 16:43:40 +00:00
static void return_mr_result(strbuf *out, struct mr_result result)
{
if (!result.passed)
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "failed\n");
Add some tests of Miller-Rabin to cryptsuite. I'm about to rewrite the Miller-Rabin testing code, so let's start by introducing a test suite that the old version passes, and then I can make sure the new one does too.
2021-08-27 16:43:40 +00:00
else if (!result.potential_primitive_root)
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "passed\n");
Add some tests of Miller-Rabin to cryptsuite. I'm about to rewrite the Miller-Rabin testing code, so let's start by introducing a test suite that the old version passes, and then I can make sure the new one does too.
2021-08-27 16:43:40 +00:00
else
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "passed+ppr\n");
Add some tests of Miller-Rabin to cryptsuite. I'm about to rewrite the Miller-Rabin testing code, so let's start by introducing a test suite that the old version passes, and then I can make sure the new one does too.
2021-08-27 16:43:40 +00:00
}
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
static void return_val_string_asciz_const(strbuf *out, const char *s)
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
{
strbuf *sb = strbuf_new();
put_data(sb, s, strlen(s));
return_val_string(out, sb);
}
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
static void return_val_string_asciz(strbuf *out, char *s)
{
return_val_string_asciz_const(out, s);
sfree(s);
}
testcrypt: refactor return_opt_foo functions. There are already three tediously similar functions that wrap a NULL check around some existing function to return one or another kind of pointer, and I'm about to want to add another one. Make a macro so that it's easy to make more functions identical to these three.
2019-04-28 08:59:28 +00:00
#define NULLABLE_RETURN_WRAPPER(type_name, c_type) \
static void return_opt_##type_name(strbuf *out, c_type ptr) \
{ \
if (!ptr) \
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "NULL\n"); \
testcrypt: refactor return_opt_foo functions. There are already three tediously similar functions that wrap a NULL check around some existing function to return one or another kind of pointer, and I'm about to want to add another one. Make a macro so that it's easy to make more functions identical to these three.
2019-04-28 08:59:28 +00:00
else \
return_##type_name(out, ptr); \
}
testcrypt: allow ssh2_cipher_new to return NULL. No cipher construction function _currently_ returns NULL, but one's about to start, so the testcrypt system will have to be able to cope. This is the first time a function in the testcrypt API has had an 'opt' type as its return value rather than an argument. But it works just the same in reverse: the wire protocol emits the special identifer "NULL" when the optional return value is absent, and the Python module catches that and rewrites it as Python 'None'.
2019-01-11 06:47:39 +00:00
testcrypt: fix the rsa_ssh1_encrypt wrapper. It wasn't expanding the output strbuf to the full size of the key modulus, so the output delivered to Python was only a part of the mpint it should have been. (Also, that was logically speaking a buffer overrun - we were writing to the strbuf buffer beyond its length - although in practice I think the _physical_ size of the buffer was large enough not to show it up even under ASan. In any case, a buffer overrun only in the test suite, and in a function I hadn't even got round to testing, is about the best place to have one.) While I'm here, I've also changed the way that the testcrypt wrapper on rsa_ssh1_encrypt indicates failure: now we have the 'opt_' mechanism, it can do that by returning None rather than "".
2020-01-09 19:16:58 +00:00
NULLABLE_RETURN_WRAPPER(val_string, strbuf *)
testcrypt: refactor return_opt_foo functions. There are already three tediously similar functions that wrap a NULL check around some existing function to return one or another kind of pointer, and I'm about to want to add another one. Make a macro so that it's easy to make more functions identical to these three.
2019-04-28 08:59:28 +00:00
NULLABLE_RETURN_WRAPPER(val_string_asciz, char *)
cmdgen: add a --dump option. Also spelled '-O text', this takes a public or private key as input, and produces on standard output a dump of all the actual numbers involved in the key: the exponent and modulus for RSA, the p,q,g,y parameters for DSA, the affine x and y coordinates of the public elliptic curve point for ECC keys, and all the extra bits and pieces in the private keys too. Partly I expect this to be useful to me for debugging: I've had to paste key files a few too many times through base64 decoders and hex dump tools, then manually decode SSH marshalling and paste the result into the Python REPL to get an integer object. Now I should be able to get _straight_ to text I can paste into Python. But also, it's a way that other applications can use the key generator: if you need to generate, say, an RSA key in some format I don't support (I've recently heard of an XML-based one, for example), then you can run 'puttygen -t rsa --dump' and have it print the elements of a freshly generated keypair on standard output, and then all you have to do is understand the output format.
2020-02-17 19:53:19 +00:00
NULLABLE_RETURN_WRAPPER(val_string_asciz_const, const char *)
testcrypt: refactor return_opt_foo functions. There are already three tediously similar functions that wrap a NULL check around some existing function to return one or another kind of pointer, and I'm about to want to add another one. Make a macro so that it's easy to make more functions identical to these three.
2019-04-28 08:59:28 +00:00
NULLABLE_RETURN_WRAPPER(val_cipher, ssh_cipher *)
NULLABLE_RETURN_WRAPPER(val_hash, ssh_hash *)
testcrypt: allow ssh_key constructors to fail. If key decoding or verification fails, those functions can return NULL. Recognise that in the testcrypt API, so that I can test them.
2019-04-28 08:59:28 +00:00
NULLABLE_RETURN_WRAPPER(val_key, ssh_key *)
Add a test for RSA key exchange. It demonstrates a successful round trip from a source integer to ciphertext and back, and also I've hardcoded the ciphertext I got from the first attempt so that future changes to the code won't be able to change it without me noticing.
2019-12-15 20:13:13 +00:00
NULLABLE_RETURN_WRAPPER(val_mpint, mp_int *)
cryptsuite: test hardware and software SHA, if possible. Like the AES code before it, I've now exposed the explicit _sw and _hw vtables for SHA-256 and SHA-1 through the testcrypt system, and now cryptsuite will run the standard test vectors for those hashes over both implementations, on a platform where more than one is available.
2019-01-23 18:54:12 +00:00
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
static void handle_hello(BinarySource *in, strbuf *out)
{
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "hello, world\n");
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
}
static void rsa_free(RSAKey *rsa)
{
freersakey(rsa);
sfree(rsa);
}
static void free_value(Value *val)
{
switch (val->type) {
#define VALTYPE_FREE(n,t,f) case VT_##n: { t v = val->vu_##n; (f); break; }
VALUE_TYPES(VALTYPE_FREE)
#undef VALTYPE_FREE
}
sfree(val);
}
static void handle_free(BinarySource *in, strbuf *out)
{
Value *val = get_value(in);
del234(values, val);
free_value(val);
}
static void handle_newstring(BinarySource *in, strbuf *out)
{
strbuf *sb = strbuf_new();
while (get_avail(in)) {
char c = get_byte(in);
if (c == '%') {
char hex[3];
hex[0] = get_byte(in);
if (hex[0] != '%') {
hex[1] = get_byte(in);
hex[2] = '\0';
c = strtoul(hex, NULL, 16);
}
}
put_byte(sb, c);
}
return_val_string(out, sb);
}
static void handle_getstring(BinarySource *in, strbuf *out)
{
strbuf *sb = get_val_string(in);
for (size_t i = 0; i < sb->len; i++) {
char c = sb->s[i];
if (c > ' ' && c < 0x7F && c != '%') {
put_byte(out, c);
} else {
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "%%%02X", 0xFFU & (unsigned)c);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
}
}
put_byte(out, '\n');
}
static void handle_mp_literal(BinarySource *in, strbuf *out)
{
ptrlen pl = get_word(in);
char *str = mkstr(pl);
mp_int *mp = mp__from_string_literal(str);
sfree(str);
return_val_mpint(out, mp);
}
static void handle_mp_dump(BinarySource *in, strbuf *out)
{
mp_int *mp = get_val_mpint(in);
for (size_t i = mp_max_bytes(mp); i-- > 0 ;)
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, "%02X", mp_get_byte(mp, i));
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
put_byte(out, '\n');
}
testcrypt: introduce and use 'checkenum' protocol query. This allows the Python side of testcrypt to check in advance if a given string is a valid element of an enumeration, and if not, cleanly throw a Python-level exception without terminating the testcrypt subprocess. Should be useful in both manual use (when I'm trying something out by hand and make a typo or misremember a spelling), and automated use (if I make the same kind of error in cryptsuite.py then the exception dump will make more sense). In order to do this, the new handle_checkenum() function has to recognise all the enumerated types by name and match them up to their lookup functions - which is just the kind of thing that can now be done easily be reincluding testcrypt-enum.h with different #defines.
2021-11-22 18:39:45 +00:00
static void handle_checkenum(BinarySource *in, strbuf *out)
{
ptrlen type = get_word(in);
ptrlen value = get_word(in);
bool ok = false;
#define BEGIN_ENUM_TYPE(name) \
if (ptrlen_eq_string(type, #name)) \
ok = enum_translate_##name(value, NULL);
#define ENUM_VALUE(name, value)
#define END_ENUM_TYPE(name)
#include "testcrypt-enum.h"
#undef BEGIN_ENUM_TYPE
#undef ENUM_VALUE
#undef END_ENUM_TYPE
put_dataz(out, ok ? "ok\n" : "bad\n");
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
static void random_queue(ptrlen pl)
{
bufchain_add(&random_data_queue, pl.ptr, pl.len);
}
static size_t random_queue_len(void)
{
return bufchain_size(&random_data_queue);
}
static void random_clear(void)
{
testcrypt: allow random_read() to use a full PRNG. This still isn't the true random generator used in the live tools: it's deterministic, for repeatable testing. The Python side of testcrypt can now call random_make_prng(), which will instantiate a PRNG with the given seed. random_clear() still gets rid of it. So I can still have some tests control the precise random numbers received by the function under test, but for others (especially key generation, with its uncertainty about how much randomness it will actually use) I can just say 'here, have a seed, generate as much stuff from that seed as you need'.
2020-02-19 19:12:59 +00:00
if (test_prng) {
prng_free(test_prng);
test_prng = NULL;
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
bufchain_clear(&random_data_queue);
}
testcrypt: allow random_read() to use a full PRNG. This still isn't the true random generator used in the live tools: it's deterministic, for repeatable testing. The Python side of testcrypt can now call random_make_prng(), which will instantiate a PRNG with the given seed. random_clear() still gets rid of it. So I can still have some tests control the precise random numbers received by the function under test, but for others (especially key generation, with its uncertainty about how much randomness it will actually use) I can just say 'here, have a seed, generate as much stuff from that seed as you need'.
2020-02-19 19:12:59 +00:00
static void random_make_prng(const ssh_hashalg *hashalg, ptrlen seed)
{
random_clear();
test_prng = prng_new(hashalg);
prng_seed_begin(test_prng);
put_datapl(test_prng, seed);
prng_seed_finish(test_prng);
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
mp_int *monty_identity_wrapper(MontyContext *mc)
{
return mp_copy(monty_identity(mc));
}
mp_int *monty_modulus_wrapper(MontyContext *mc)
{
return mp_copy(monty_modulus(mc));
}
Adopt the new hash API functions where they're useful. This commit switches as many ssh_hash_free / ssh_hash_new pairs as possible to reuse the previous hash object via ssh_hash_reset. Also a few other cleanups: use the wrapper function hash_simple() where possible, and I've also introduced ssh_hash_digest_nondestructive() and switched to that where possible as well.
2019-12-15 09:57:30 +00:00
strbuf *ssh_hash_digest_wrapper(ssh_hash *h)
{
strbuf *sb = strbuf_new();
void *p = strbuf_append(sb, ssh_hash_alg(h)->hlen);
ssh_hash_digest(h, p);
return sb;
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
strbuf *ssh_hash_final_wrapper(ssh_hash *h)
{
strbuf *sb = strbuf_new();
void *p = strbuf_append(sb, ssh_hash_alg(h)->hlen);
ssh_hash_final(h, p);
return sb;
}
testcrypt: fix param name in ssh_cipher_setiv_wrapper. Spotted in passing that a cut-and-paste error made the parameter name 'key' rather than 'iv'. Harmless, but wrong.
2021-11-21 11:46:00 +00:00
void ssh_cipher_setiv_wrapper(ssh_cipher *c, ptrlen iv)
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
{
testcrypt: fix param name in ssh_cipher_setiv_wrapper. Spotted in passing that a cut-and-paste error made the parameter name 'key' rather than 'iv'. Harmless, but wrong.
2021-11-21 11:46:00 +00:00
if (iv.len != ssh_cipher_alg(c)->blksize)
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
fatal_error("ssh_cipher_setiv: needs exactly %d bytes",
ssh_cipher_alg(c)->blksize);
testcrypt: fix param name in ssh_cipher_setiv_wrapper. Spotted in passing that a cut-and-paste error made the parameter name 'key' rather than 'iv'. Harmless, but wrong.
2021-11-21 11:46:00 +00:00
ssh_cipher_setiv(c, iv.ptr);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
}
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
void ssh_cipher_setkey_wrapper(ssh_cipher *c, ptrlen key)
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
{
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
if (key.len != ssh_cipher_alg(c)->padded_keybytes)
fatal_error("ssh_cipher_setkey: needs exactly %d bytes",
ssh_cipher_alg(c)->padded_keybytes);
ssh_cipher_setkey(c, key.ptr);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
}
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
strbuf *ssh_cipher_encrypt_wrapper(ssh_cipher *c, ptrlen input)
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
{
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
if (input.len % ssh_cipher_alg(c)->blksize)
fatal_error("ssh_cipher_encrypt: needs a multiple of %d bytes",
ssh_cipher_alg(c)->blksize);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
strbuf *sb = strbuf_new();
put_datapl(sb, input);
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
ssh_cipher_encrypt(c, sb->u, sb->len);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
return sb;
}
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
strbuf *ssh_cipher_decrypt_wrapper(ssh_cipher *c, ptrlen input)
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
{
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
if (input.len % ssh_cipher_alg(c)->blksize)
fatal_error("ssh_cipher_decrypt: needs a multiple of %d bytes",
ssh_cipher_alg(c)->blksize);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
strbuf *sb = strbuf_new();
put_datapl(sb, input);
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
ssh_cipher_decrypt(c, sb->u, sb->len);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
return sb;
}
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
strbuf *ssh_cipher_encrypt_length_wrapper(ssh_cipher *c, ptrlen input,
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
unsigned long seq)
{
if (input.len != 4)
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
fatal_error("ssh_cipher_encrypt_length: needs exactly 4 bytes");
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
strbuf *sb = strbuf_new();
put_datapl(sb, input);
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
ssh_cipher_encrypt_length(c, sb->u, sb->len, seq);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
return sb;
}
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
strbuf *ssh_cipher_decrypt_length_wrapper(ssh_cipher *c, ptrlen input,
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
unsigned long seq)
{
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
if (input.len % ssh_cipher_alg(c)->blksize)
fatal_error("ssh_cipher_decrypt_length: needs exactly 4 bytes");
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
strbuf *sb = strbuf_new();
put_datapl(sb, input);
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
ssh_cipher_decrypt_length(c, sb->u, sb->len, seq);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
return sb;
}
strbuf *ssh2_mac_genresult_wrapper(ssh2_mac *m)
{
strbuf *sb = strbuf_new();
void *u = strbuf_append(sb, ssh2_mac_alg(m)->len);
ssh2_mac_genresult(m, u);
return sb;
}
bool dh_validate_f_wrapper(dh_ctx *dh, mp_int *f)
{
return dh_validate_f(dh, f) == NULL;
}
void ssh_hash_update(ssh_hash *h, ptrlen pl)
{
put_datapl(h, pl);
}
void ssh2_mac_update(ssh2_mac *m, ptrlen pl)
{
put_datapl(m, pl);
}
static RSAKey *rsa_new(void)
{
RSAKey *rsa = snew(RSAKey);
memset(rsa, 0, sizeof(RSAKey));
return rsa;
}
strbuf *rsa_ssh1_encrypt_wrapper(ptrlen input, RSAKey *key)
{
/* Fold the boolean return value in C into the string return value
testcrypt: fix the rsa_ssh1_encrypt wrapper. It wasn't expanding the output strbuf to the full size of the key modulus, so the output delivered to Python was only a part of the mpint it should have been. (Also, that was logically speaking a buffer overrun - we were writing to the strbuf buffer beyond its length - although in practice I think the _physical_ size of the buffer was large enough not to show it up even under ASan. In any case, a buffer overrun only in the test suite, and in a function I hadn't even got round to testing, is about the best place to have one.) While I'm here, I've also changed the way that the testcrypt wrapper on rsa_ssh1_encrypt indicates failure: now we have the 'opt_' mechanism, it can do that by returning None rather than "".
2020-01-09 19:16:58 +00:00
* for this purpose, by returning NULL on failure */
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
strbuf *sb = strbuf_new();
put_datapl(sb, input);
testcrypt: fix the rsa_ssh1_encrypt wrapper. It wasn't expanding the output strbuf to the full size of the key modulus, so the output delivered to Python was only a part of the mpint it should have been. (Also, that was logically speaking a buffer overrun - we were writing to the strbuf buffer beyond its length - although in practice I think the _physical_ size of the buffer was large enough not to show it up even under ASan. In any case, a buffer overrun only in the test suite, and in a function I hadn't even got round to testing, is about the best place to have one.) While I'm here, I've also changed the way that the testcrypt wrapper on rsa_ssh1_encrypt indicates failure: now we have the 'opt_' mechanism, it can do that by returning None rather than "".
2020-01-09 19:16:58 +00:00
put_padding(sb, key->bytes - input.len, 0);
if (!rsa_ssh1_encrypt(sb->u, input.len, key)) {
strbuf_free(sb);
return NULL;
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
return sb;
}
strbuf *rsa_ssh1_decrypt_pkcs1_wrapper(mp_int *input, RSAKey *key)
{
/* Again, return "" on failure */
strbuf *sb = strbuf_new();
if (!rsa_ssh1_decrypt_pkcs1(input, key, sb))
New functions to shrink a strbuf. These are better than my previous approach of just assigning to sb->len, because firstly they check by assertion that the new length is within range, and secondly they preserve the invariant that the byte stored in the buffer just after the length runs out is \0. Switched to using the new functions everywhere a grep could turn up opportunities.
2020-01-21 20:16:28 +00:00
strbuf_clear(sb);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
return sb;
}
Expose des_{en,de}crypt_xdmauth in testcrypt. This allows me to remove another diagnostic main() that I just found lurking at the bottom of sshdes.c, which was there to allow manual untangling of XDM-AUTHORIZATION-1 strings when debugging X forwarding. Now you can ask the same kind of question at the interactive Python prompt, without having to manually compile anything. For example, the query you might previously have asked by building the sshdes test program and running $ ./sshdes 090a0b0c0d0e0f10 0123456789abcd decrypt(090a0b0c0d0e0f10,0123456789abcd) = ab53fd65ae7f4ec3 encrypt(090a0b0c0d0e0f10,0123456789abcd) = 7065d20441f5abe3 you can now run using the standard testcrypt (bearing in mind that the actual library function takes the key argument first): $ python -i test/testcrypt.py >>> from binascii import hexlify as H, unhexlify as U >>> H(des_decrypt_xdmauth(U('0123456789abcd'),U('090a0b0c0d0e0f10'))) 'ab53fd65ae7f4ec3' >>> H(des_encrypt_xdmauth(U('0123456789abcd'),U('090a0b0c0d0e0f10'))) '7065d20441f5abe3'
2019-01-04 08:23:17 +00:00
strbuf *des_encrypt_xdmauth_wrapper(ptrlen key, ptrlen data)
{
if (key.len != 7)
fatal_error("des_encrypt_xdmauth: key must be 7 bytes long");
if (data.len % 8 != 0)
fatal_error("des_encrypt_xdmauth: data must be a multiple of 8 bytes");
strbuf *sb = strbuf_new();
put_datapl(sb, data);
des_encrypt_xdmauth(key.ptr, sb->u, sb->len);
return sb;
}
strbuf *des_decrypt_xdmauth_wrapper(ptrlen key, ptrlen data)
{
if (key.len != 7)
fatal_error("des_decrypt_xdmauth: key must be 7 bytes long");
if (data.len % 8 != 0)
fatal_error("des_decrypt_xdmauth: data must be a multiple of 8 bytes");
strbuf *sb = strbuf_new();
put_datapl(sb, data);
des_decrypt_xdmauth(key.ptr, sb->u, sb->len);
return sb;
}
Add tests of auxiliary encryption functions. All the things like des_encrypt_xdmauth and aes256_encrypt_pubkey are at risk of changing their behaviour if I rewrite the underlying code, so even if I don't have any externally verified test cases, I should still have _something_ to keep me confident that they work the same way today that they worked yesterday.
2019-01-18 06:39:35 +00:00
strbuf *des3_encrypt_pubkey_wrapper(ptrlen key, ptrlen data)
{
if (key.len != 16)
fatal_error("des3_encrypt_pubkey: key must be 16 bytes long");
if (data.len % 8 != 0)
fatal_error("des3_encrypt_pubkey: data must be a multiple of 8 bytes");
strbuf *sb = strbuf_new();
put_datapl(sb, data);
des3_encrypt_pubkey(key.ptr, sb->u, sb->len);
return sb;
}
strbuf *des3_decrypt_pubkey_wrapper(ptrlen key, ptrlen data)
{
if (key.len != 16)
fatal_error("des3_decrypt_pubkey: key must be 16 bytes long");
if (data.len % 8 != 0)
fatal_error("des3_decrypt_pubkey: data must be a multiple of 8 bytes");
strbuf *sb = strbuf_new();
put_datapl(sb, data);
des3_decrypt_pubkey(key.ptr, sb->u, sb->len);
return sb;
}
strbuf *des3_encrypt_pubkey_ossh_wrapper(ptrlen key, ptrlen iv, ptrlen data)
{
if (key.len != 24)
fatal_error("des3_encrypt_pubkey_ossh: key must be 24 bytes long");
if (iv.len != 8)
fatal_error("des3_encrypt_pubkey_ossh: iv must be 8 bytes long");
if (data.len % 8 != 0)
fatal_error("des3_encrypt_pubkey_ossh: data must be a multiple of 8 bytes");
strbuf *sb = strbuf_new();
put_datapl(sb, data);
des3_encrypt_pubkey_ossh(key.ptr, iv.ptr, sb->u, sb->len);
return sb;
}
strbuf *des3_decrypt_pubkey_ossh_wrapper(ptrlen key, ptrlen iv, ptrlen data)
{
if (key.len != 24)
fatal_error("des3_decrypt_pubkey_ossh: key must be 24 bytes long");
if (iv.len != 8)
fatal_error("des3_encrypt_pubkey_ossh: iv must be 8 bytes long");
if (data.len % 8 != 0)
fatal_error("des3_decrypt_pubkey_ossh: data must be a multiple of 8 bytes");
strbuf *sb = strbuf_new();
put_datapl(sb, data);
des3_decrypt_pubkey_ossh(key.ptr, iv.ptr, sb->u, sb->len);
return sb;
}
Add an IV argument to aes_{en,de}crypt_pubkey. No functional change: currently, the IV passed in is always zero (except in the test suite). But this prepares to change that in a future revision of the key file format.
2021-02-18 17:48:06 +00:00
strbuf *aes256_encrypt_pubkey_wrapper(ptrlen key, ptrlen iv, ptrlen data)
Add tests of auxiliary encryption functions. All the things like des_encrypt_xdmauth and aes256_encrypt_pubkey are at risk of changing their behaviour if I rewrite the underlying code, so even if I don't have any externally verified test cases, I should still have _something_ to keep me confident that they work the same way today that they worked yesterday.
2019-01-18 06:39:35 +00:00
{
if (key.len != 32)
fatal_error("aes256_encrypt_pubkey: key must be 32 bytes long");
Add an IV argument to aes_{en,de}crypt_pubkey. No functional change: currently, the IV passed in is always zero (except in the test suite). But this prepares to change that in a future revision of the key file format.
2021-02-18 17:48:06 +00:00
if (iv.len != 16)
fatal_error("aes256_encrypt_pubkey: iv must be 16 bytes long");
Add tests of auxiliary encryption functions. All the things like des_encrypt_xdmauth and aes256_encrypt_pubkey are at risk of changing their behaviour if I rewrite the underlying code, so even if I don't have any externally verified test cases, I should still have _something_ to keep me confident that they work the same way today that they worked yesterday.
2019-01-18 06:39:35 +00:00
if (data.len % 16 != 0)
fatal_error("aes256_encrypt_pubkey: data must be a multiple of 16 bytes");
strbuf *sb = strbuf_new();
put_datapl(sb, data);
Add an IV argument to aes_{en,de}crypt_pubkey. No functional change: currently, the IV passed in is always zero (except in the test suite). But this prepares to change that in a future revision of the key file format.
2021-02-18 17:48:06 +00:00
aes256_encrypt_pubkey(key.ptr, iv.ptr, sb->u, sb->len);
Add tests of auxiliary encryption functions. All the things like des_encrypt_xdmauth and aes256_encrypt_pubkey are at risk of changing their behaviour if I rewrite the underlying code, so even if I don't have any externally verified test cases, I should still have _something_ to keep me confident that they work the same way today that they worked yesterday.
2019-01-18 06:39:35 +00:00
return sb;
}
Add an IV argument to aes_{en,de}crypt_pubkey. No functional change: currently, the IV passed in is always zero (except in the test suite). But this prepares to change that in a future revision of the key file format.
2021-02-18 17:48:06 +00:00
strbuf *aes256_decrypt_pubkey_wrapper(ptrlen key, ptrlen iv, ptrlen data)
Add tests of auxiliary encryption functions. All the things like des_encrypt_xdmauth and aes256_encrypt_pubkey are at risk of changing their behaviour if I rewrite the underlying code, so even if I don't have any externally verified test cases, I should still have _something_ to keep me confident that they work the same way today that they worked yesterday.
2019-01-18 06:39:35 +00:00
{
if (key.len != 32)
fatal_error("aes256_decrypt_pubkey: key must be 32 bytes long");
Add an IV argument to aes_{en,de}crypt_pubkey. No functional change: currently, the IV passed in is always zero (except in the test suite). But this prepares to change that in a future revision of the key file format.
2021-02-18 17:48:06 +00:00
if (iv.len != 16)
fatal_error("aes256_encrypt_pubkey: iv must be 16 bytes long");
Add tests of auxiliary encryption functions. All the things like des_encrypt_xdmauth and aes256_encrypt_pubkey are at risk of changing their behaviour if I rewrite the underlying code, so even if I don't have any externally verified test cases, I should still have _something_ to keep me confident that they work the same way today that they worked yesterday.
2019-01-18 06:39:35 +00:00
if (data.len % 16 != 0)
fatal_error("aes256_decrypt_pubkey: data must be a multiple of 16 bytes");
strbuf *sb = strbuf_new();
put_datapl(sb, data);
Add an IV argument to aes_{en,de}crypt_pubkey. No functional change: currently, the IV passed in is always zero (except in the test suite). But this prepares to change that in a future revision of the key file format.
2021-02-18 17:48:06 +00:00
aes256_decrypt_pubkey(key.ptr, iv.ptr, sb->u, sb->len);
Add tests of auxiliary encryption functions. All the things like des_encrypt_xdmauth and aes256_encrypt_pubkey are at risk of changing their behaviour if I rewrite the underlying code, so even if I don't have any externally verified test cases, I should still have _something_ to keep me confident that they work the same way today that they worked yesterday.
2019-01-18 06:39:35 +00:00
return sb;
}
Replace PuTTY's PRNG with a Fortuna-like system. This tears out the entire previous random-pool system in sshrand.c. In its place is a system pretty close to Ferguson and Schneier's 'Fortuna' generator, with the main difference being that I use SHA-256 instead of AES for the generation side of the system (rationale given in comment). The PRNG implementation lives in sshprng.c, and defines a self- contained data type with no state stored outside the object, so you can instantiate however many of them you like. The old sshrand.c still exists, but in place of the previous random pool system, it's just become a client of sshprng.c, whose job is to hold a single global instance of the PRNG type, and manage its reference count, save file, noise-collection timers and similar administrative business. Advantages of this change include: - Fortuna is designed with a more varied threat model in mind than my old home-grown random pool. For example, after any request for random numbers, it automatically re-seeds itself, so that if the state of the PRNG should be leaked, it won't give enough information to find out what past outputs _were_. - The PRNG type can be instantiated with any hash function; the instance used by the main tools is based on SHA-256, an improvement on the old pool's use of SHA-1. - The new PRNG only uses the completely standard interface to the hash function API, instead of having to have privileged access to the internal SHA-1 block transform function. This will make it easier to revamp the hash code in general, and also it means that hardware-accelerated versions of SHA-256 will automatically be used for the PRNG as well as for everything else. - The new PRNG can be _tested_! Because it has an actual (if not quite explicit) specification for exactly what the output numbers _ought_ to be derived from the hashes of, I can (and have) put tests in cryptsuite that ensure the output really is being derived in the way I think it is. The old pool could have been returning any old nonsense and it would have been very hard to tell for sure.
2019-01-22 22:42:41 +00:00
strbuf *prng_read_wrapper(prng *pr, size_t size)
{
strbuf *sb = strbuf_new();
prng_read(pr, strbuf_append(sb, size), size);
return sb;
}
void prng_seed_update(prng *pr, ptrlen data)
{
put_datapl(pr, data);
}
Add tests of the CRC compensation detector. I remembered the existence of that module while I was changing the API of the CRC functions. It's still quite possibly the only code in PuTTY not written specifically _for_ PuTTY, so it definitely deserves a bit of a test suite. In order to expose it through the ptrlen-centric testcrypt system, I've added some missing 'const' in the detector module itself, but otherwise I've left the detector code as it was.
2019-01-14 21:19:38 +00:00
bool crcda_detect(ptrlen packet, ptrlen iv)
{
if (iv.len != 0 && iv.len != 8)
fatal_error("crcda_detect: iv must be empty or 8 bytes long");
if (packet.len % 8 != 0)
fatal_error("crcda_detect: packet must be a multiple of 8 bytes");
struct crcda_ctx *ctx = crcda_make_context();
bool toret = detect_attack(ctx, packet.ptr, packet.len,
iv.len ? iv.ptr : NULL);
crcda_free_context(ctx);
return toret;
}
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
ssh_key *ppk_load_s_wrapper(BinarySource *src, char **comment,
const char *passphrase, const char **errorstr)
{
ssh2_userkey *uk = ppk_load_s(src, passphrase, errorstr);
if (uk == SSH2_WRONG_PASSPHRASE) {
/* Fudge this special return value */
*errorstr = "SSH2_WRONG_PASSPHRASE";
return NULL;
}
if (uk == NULL)
return NULL;
ssh_key *toret = uk->key;
*comment = uk->comment;
sfree(uk);
return toret;
}
int rsa1_load_s_wrapper(BinarySource *src, RSAKey *rsa, char **comment,
const char *passphrase, const char **errorstr)
{
int toret = rsa1_load_s(src, rsa, passphrase, errorstr);
*comment = rsa->comment;
rsa->comment = NULL;
return toret;
}
Restore the ability to write out PPK v2. This commit adds the capability in principle to ppk_save_sb, by adding a fmt_version field in the save parameters structure. As yet it's not connected up to any user interface in PuTTYgen, but I think I'll need to, because currently there's no way at all to convert PPK v3 back to v2, and surely people will need to interoperate with older installations of PuTTY, or with other PPK-consuming software.
2021-02-21 22:29:10 +00:00
strbuf *ppk_save_sb_wrapper(
ssh_key *key, const char *comment, const char *passphrase,
unsigned fmt_version, Argon2Flavour flavour,
uint32_t mem, uint32_t passes, uint32_t parallel)
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
{
Introduce PPK file format version 3. This removes both uses of SHA-1 in the file format: it was used as the MAC protecting the key file against tamperproofing, and also used in the key derivation step that converted the user's passphrase to cipher and MAC keys. The MAC is simply upgraded from HMAC-SHA-1 to HMAC-SHA-256; it is otherwise unchanged in how it's applied (in particular, to what data). The key derivation is totally reworked, to be based on Argon2, which I've just added to the code base. This should make stolen encrypted key files more resistant to brute-force attack. Argon2 has assorted configurable parameters for memory and CPU usage; the new key format includes all those parameters. So there's no reason we can't have them under user control, if a user wants to be particularly vigorous or particularly lightweight with their own key files. They could even switch to one of the other flavours of Argon2, if they thought side channels were an especially large or small risk in their particular environment. In this commit I haven't added any UI for controlling that kind of thing, but the PPK loading function is all set up to cope, so that can all be added in a future commit without having to change the file format. While I'm at it, I've also switched the CBC encryption to using a random IV (or rather, one derived from the passphrase along with the cipher and MAC keys). That's more like normal SSH-2 practice.
2021-02-20 10:17:45 +00:00
/*
* For repeatable testing purposes, we never want a timing-dependent
* choice of password hashing parameters, so this is easy.
*/
ppk_save_parameters save_params;
Fix determinism failures in cgtest. Thanks to Pavel and his CI for pointing out what I'd forgotten: the automated test of cmdgen.c expects that round-tripping a PPK file to some other format and back will regenerate the identical file. Of course, with a randomised salt in the new-look password hash, that isn't true any more in normal usage. Fixed by adding an option in the existing parameters structure to provide a salt override. That shouldn't be used anywhere except cgtest, but in cgtest, it restores the determinism we need. Another potential (but not guaranteed) source of difference is the automatic time-scaling of the Argon2 parameter choice. So I've turned that off too, while I'm at it.
2021-02-21 14:37:59 +00:00
memset(&save_params, 0, sizeof(save_params));
Restore the ability to write out PPK v2. This commit adds the capability in principle to ppk_save_sb, by adding a fmt_version field in the save parameters structure. As yet it's not connected up to any user interface in PuTTYgen, but I think I'll need to, because currently there's no way at all to convert PPK v3 back to v2, and surely people will need to interoperate with older installations of PuTTY, or with other PPK-consuming software.
2021-02-21 22:29:10 +00:00
save_params.fmt_version = fmt_version;
Introduce PPK file format version 3. This removes both uses of SHA-1 in the file format: it was used as the MAC protecting the key file against tamperproofing, and also used in the key derivation step that converted the user's passphrase to cipher and MAC keys. The MAC is simply upgraded from HMAC-SHA-1 to HMAC-SHA-256; it is otherwise unchanged in how it's applied (in particular, to what data). The key derivation is totally reworked, to be based on Argon2, which I've just added to the code base. This should make stolen encrypted key files more resistant to brute-force attack. Argon2 has assorted configurable parameters for memory and CPU usage; the new key format includes all those parameters. So there's no reason we can't have them under user control, if a user wants to be particularly vigorous or particularly lightweight with their own key files. They could even switch to one of the other flavours of Argon2, if they thought side channels were an especially large or small risk in their particular environment. In this commit I haven't added any UI for controlling that kind of thing, but the PPK loading function is all set up to cope, so that can all be added in a future commit without having to change the file format. While I'm at it, I've also switched the CBC encryption to using a random IV (or rather, one derived from the passphrase along with the cipher and MAC keys). That's more like normal SSH-2 practice.
2021-02-20 10:17:45 +00:00
save_params.argon2_flavour = flavour;
save_params.argon2_mem = mem;
save_params.argon2_passes_auto = false;
save_params.argon2_passes = passes;
save_params.argon2_parallelism = parallel;
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
ssh2_userkey uk;
uk.key = key;
uk.comment = dupstr(comment);
Introduce PPK file format version 3. This removes both uses of SHA-1 in the file format: it was used as the MAC protecting the key file against tamperproofing, and also used in the key derivation step that converted the user's passphrase to cipher and MAC keys. The MAC is simply upgraded from HMAC-SHA-1 to HMAC-SHA-256; it is otherwise unchanged in how it's applied (in particular, to what data). The key derivation is totally reworked, to be based on Argon2, which I've just added to the code base. This should make stolen encrypted key files more resistant to brute-force attack. Argon2 has assorted configurable parameters for memory and CPU usage; the new key format includes all those parameters. So there's no reason we can't have them under user control, if a user wants to be particularly vigorous or particularly lightweight with their own key files. They could even switch to one of the other flavours of Argon2, if they thought side channels were an especially large or small risk in their particular environment. In this commit I haven't added any UI for controlling that kind of thing, but the PPK loading function is all set up to cope, so that can all be added in a future commit without having to change the file format. While I'm at it, I've also switched the CBC encryption to using a random IV (or rather, one derived from the passphrase along with the cipher and MAC keys). That's more like normal SSH-2 practice.
2021-02-20 10:17:45 +00:00
strbuf *toret = ppk_save_sb(&uk, passphrase, &save_params);
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
sfree(uk.comment);
return toret;
}
strbuf *rsa1_save_sb_wrapper(RSAKey *key, const char *comment,
const char *passphrase)
{
key->comment = dupstr(comment);
strbuf *toret = rsa1_save_sb(key, passphrase);
sfree(key->comment);
key->comment = NULL;
return toret;
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
#define return_void(out, expression) (expression)
New vtable API for keygen progress reporting. The old API was one of those horrible things I used to do when I was young and foolish, in which you have just one function, and indicate which of lots of things it's doing by passing in flags. It was crying out to be replaced with a vtable. While I'm at it, I've reworked the code on the Windows side that decides what to do with the progress bar, so that it's based on actually justifiable estimates of probability rather than magic integer constants. Since computers are generally faster now than they were at the start of this project, I've also decided there's no longer any point in making the fixed final part of RSA key generation bother to report progress at all. So the progress bars are now only for the variable part, i.e. the actual prime generations. (This is a reapplication of commit a7bdefb39, without the Miller-Rabin refactoring accidentally folded into it. Also this time I've added -lm to the link options, which for some reason _didn't_ cause me a link failure last time round. No idea why not.)
2020-02-23 15:29:40 +00:00
static ProgressReceiver null_progress = { .vt = &null_progress_vt };
Add primegen() to the testcrypt API. I just found I wanted to generate a prime with particular properties, and I knew PuTTY's prime generator could manage it, so it was easier to add this function to testcrypt for occasional manual use than to look for another prime-generator with the same feature set! I've wrapped the function so as to remove the three progress- reporting parameters.
2019-02-09 15:35:02 +00:00
Introduce a vtable system for prime generation. The functions primegen() and primegen_add_progress_phase() are gone. In their place is a small vtable system with two methods corresponding to them, plus the usual admin of allocating and freeing contexts. This API change is the starting point for being able to drop in different prime generation algorithms at run time in response to user configuration.
2020-02-29 09:10:47 +00:00
mp_int *primegen_generate_wrapper(
PrimeGenerationContext *ctx, PrimeCandidateSource *pcs)
Add primegen() to the testcrypt API. I just found I wanted to generate a prime with particular properties, and I knew PuTTY's prime generator could manage it, so it was easier to add this function to testcrypt for occasional manual use than to look for another prime-generator with the same feature set! I've wrapped the function so as to remove the three progress- reporting parameters.
2019-02-09 15:35:02 +00:00
{
Introduce a vtable system for prime generation. The functions primegen() and primegen_add_progress_phase() are gone. In their place is a small vtable system with two methods corresponding to them, plus the usual admin of allocating and freeing contexts. This API change is the starting point for being able to drop in different prime generation algorithms at run time in response to user configuration.
2020-02-29 09:10:47 +00:00
return primegen_generate(ctx, pcs, &null_progress);
Add primegen() to the testcrypt API. I just found I wanted to generate a prime with particular properties, and I knew PuTTY's prime generator could manage it, so it was easier to add this function to testcrypt for occasional manual use than to look for another prime-generator with the same feature set! I've wrapped the function so as to remove the three progress- reporting parameters.
2019-02-09 15:35:02 +00:00
}
RSA generation: option to generate strong primes. A 'strong' prime, as defined by the Handbook of Applied Cryptography, is a prime p such that each of p-1 and p+1 has a large prime factor, and that the large factor q of p-1 is such that q-1 in turn _also_ has a large prime factor. HoAC says that making your RSA key using primes of this form defeats some factoring algorithms - but there are other faster algorithms to which it makes no difference. So this is probably not a useful precaution in practice. However, it has been recommended in the past by some official standards, and it's easy to implement given the new general facility in PrimeCandidateSource that lets you ask for your prime to satisfy an arbitrary modular congruence. (And HoAC also says there's no particular reason _not_ to use strong primes.) So I provide it as an option, just in case anyone wants to select it. The change to the key generation algorithm is entirely in sshrsag.c, and is neatly independent of the prime-generation system in use. If you're using Maurer provable prime generation, then the known factor q of p-1 can be used to help certify p, and the one for q-1 to help with q in turn; if you switch to probabilistic prime generation then you still get an RSA key with the right structure, except that every time the definition says 'prime factor' you just append '(probably)'. (The probabilistic version of this procedure is described as 'Gordon's algorithm' in HoAC section 4.4.2.)
2020-03-02 06:52:09 +00:00
RSAKey *rsa1_generate(int bits, bool strong, PrimeGenerationContext *pgc)
Expose key generation functions in testcrypt. They're not much use for 'real' key generation, since like all the other randomness-using testcrypt functions, they need you to have explicitly queued up some random data. But for generating keys for test purposes, they have the great virtue that they deliver the key in the internal format, where we can generate all the various public and private blobs from it as well as the on-disk formats. A minor change to one of the keygen functions itself: rsa_generate now fills in the 'bits' and 'bytes' fields of the returned RSAKey, without which it didn't actually work to try to generate a public blob from it. (We'd never noticed before, because no previous client of rsa_generate even tried that.)
2020-01-09 07:21:30 +00:00
{
RSAKey *rsakey = snew(RSAKey);
RSA generation: option to generate strong primes. A 'strong' prime, as defined by the Handbook of Applied Cryptography, is a prime p such that each of p-1 and p+1 has a large prime factor, and that the large factor q of p-1 is such that q-1 in turn _also_ has a large prime factor. HoAC says that making your RSA key using primes of this form defeats some factoring algorithms - but there are other faster algorithms to which it makes no difference. So this is probably not a useful precaution in practice. However, it has been recommended in the past by some official standards, and it's easy to implement given the new general facility in PrimeCandidateSource that lets you ask for your prime to satisfy an arbitrary modular congruence. (And HoAC also says there's no particular reason _not_ to use strong primes.) So I provide it as an option, just in case anyone wants to select it. The change to the key generation algorithm is entirely in sshrsag.c, and is neatly independent of the prime-generation system in use. If you're using Maurer provable prime generation, then the known factor q of p-1 can be used to help certify p, and the one for q-1 to help with q in turn; if you switch to probabilistic prime generation then you still get an RSA key with the right structure, except that every time the definition says 'prime factor' you just append '(probably)'. (The probabilistic version of this procedure is described as 'Gordon's algorithm' in HoAC section 4.4.2.)
2020-03-02 06:52:09 +00:00
rsa_generate(rsakey, bits, strong, pgc, &null_progress);
Expose key generation functions in testcrypt. They're not much use for 'real' key generation, since like all the other randomness-using testcrypt functions, they need you to have explicitly queued up some random data. But for generating keys for test purposes, they have the great virtue that they deliver the key in the internal format, where we can generate all the various public and private blobs from it as well as the on-disk formats. A minor change to one of the keygen functions itself: rsa_generate now fills in the 'bits' and 'bytes' fields of the returned RSAKey, without which it didn't actually work to try to generate a public blob from it. (We'd never noticed before, because no previous client of rsa_generate even tried that.)
2020-01-09 07:21:30 +00:00
rsakey->comment = NULL;
return rsakey;
}
RSA generation: option to generate strong primes. A 'strong' prime, as defined by the Handbook of Applied Cryptography, is a prime p such that each of p-1 and p+1 has a large prime factor, and that the large factor q of p-1 is such that q-1 in turn _also_ has a large prime factor. HoAC says that making your RSA key using primes of this form defeats some factoring algorithms - but there are other faster algorithms to which it makes no difference. So this is probably not a useful precaution in practice. However, it has been recommended in the past by some official standards, and it's easy to implement given the new general facility in PrimeCandidateSource that lets you ask for your prime to satisfy an arbitrary modular congruence. (And HoAC also says there's no particular reason _not_ to use strong primes.) So I provide it as an option, just in case anyone wants to select it. The change to the key generation algorithm is entirely in sshrsag.c, and is neatly independent of the prime-generation system in use. If you're using Maurer provable prime generation, then the known factor q of p-1 can be used to help certify p, and the one for q-1 to help with q in turn; if you switch to probabilistic prime generation then you still get an RSA key with the right structure, except that every time the definition says 'prime factor' you just append '(probably)'. (The probabilistic version of this procedure is described as 'Gordon's algorithm' in HoAC section 4.4.2.)
2020-03-02 06:52:09 +00:00
ssh_key *rsa_generate_wrapper(int bits, bool strong,
PrimeGenerationContext *pgc)
Expose key generation functions in testcrypt. They're not much use for 'real' key generation, since like all the other randomness-using testcrypt functions, they need you to have explicitly queued up some random data. But for generating keys for test purposes, they have the great virtue that they deliver the key in the internal format, where we can generate all the various public and private blobs from it as well as the on-disk formats. A minor change to one of the keygen functions itself: rsa_generate now fills in the 'bits' and 'bytes' fields of the returned RSAKey, without which it didn't actually work to try to generate a public blob from it. (We'd never noticed before, because no previous client of rsa_generate even tried that.)
2020-01-09 07:21:30 +00:00
{
RSA generation: option to generate strong primes. A 'strong' prime, as defined by the Handbook of Applied Cryptography, is a prime p such that each of p-1 and p+1 has a large prime factor, and that the large factor q of p-1 is such that q-1 in turn _also_ has a large prime factor. HoAC says that making your RSA key using primes of this form defeats some factoring algorithms - but there are other faster algorithms to which it makes no difference. So this is probably not a useful precaution in practice. However, it has been recommended in the past by some official standards, and it's easy to implement given the new general facility in PrimeCandidateSource that lets you ask for your prime to satisfy an arbitrary modular congruence. (And HoAC also says there's no particular reason _not_ to use strong primes.) So I provide it as an option, just in case anyone wants to select it. The change to the key generation algorithm is entirely in sshrsag.c, and is neatly independent of the prime-generation system in use. If you're using Maurer provable prime generation, then the known factor q of p-1 can be used to help certify p, and the one for q-1 to help with q in turn; if you switch to probabilistic prime generation then you still get an RSA key with the right structure, except that every time the definition says 'prime factor' you just append '(probably)'. (The probabilistic version of this procedure is described as 'Gordon's algorithm' in HoAC section 4.4.2.)
2020-03-02 06:52:09 +00:00
return &rsa1_generate(bits, strong, pgc)->sshk;
Expose key generation functions in testcrypt. They're not much use for 'real' key generation, since like all the other randomness-using testcrypt functions, they need you to have explicitly queued up some random data. But for generating keys for test purposes, they have the great virtue that they deliver the key in the internal format, where we can generate all the various public and private blobs from it as well as the on-disk formats. A minor change to one of the keygen functions itself: rsa_generate now fills in the 'bits' and 'bytes' fields of the returned RSAKey, without which it didn't actually work to try to generate a public blob from it. (We'd never noticed before, because no previous client of rsa_generate even tried that.)
2020-01-09 07:21:30 +00:00
}
Introduce a vtable system for prime generation. The functions primegen() and primegen_add_progress_phase() are gone. In their place is a small vtable system with two methods corresponding to them, plus the usual admin of allocating and freeing contexts. This API change is the starting point for being able to drop in different prime generation algorithms at run time in response to user configuration.
2020-02-29 09:10:47 +00:00
ssh_key *dsa_generate_wrapper(int bits, PrimeGenerationContext *pgc)
Expose key generation functions in testcrypt. They're not much use for 'real' key generation, since like all the other randomness-using testcrypt functions, they need you to have explicitly queued up some random data. But for generating keys for test purposes, they have the great virtue that they deliver the key in the internal format, where we can generate all the various public and private blobs from it as well as the on-disk formats. A minor change to one of the keygen functions itself: rsa_generate now fills in the 'bits' and 'bytes' fields of the returned RSAKey, without which it didn't actually work to try to generate a public blob from it. (We'd never noticed before, because no previous client of rsa_generate even tried that.)
2020-01-09 07:21:30 +00:00
{
Spelling: standardise on "DSA", not "DSS". This code base has always been a bit confused about which spelling it likes to use to refer to that signature algorithm. The SSH protocol id is "ssh-dss". But everyone I know refers to it as the Digital Signature _Algorithm_, not the Digital Signature _Standard_. When I moved everything down into the crypto subdir, I took the opportunity to rename sshdss.c to dsa.c. Now I'm doing the rest of the job: all internal identifiers and code comments refer to DSA, and the spelling "dss" only survives in externally visible identifiers that have to remain constant. (Such identifiers include the SSH protocol id, and also the string id used to identify the key type in PuTTY's own host key cache. We can't change the latter without causing everyone a backwards-compatibility headache, and if we _did_ ever decide to do that, we'd surely want to do a much more thorough job of making the cache format more sensible!)
2021-04-22 17:28:35 +00:00
struct dsa_key *dsakey = snew(struct dsa_key);
dsa_generate(dsakey, bits, pgc, &null_progress);
return &dsakey->sshk;
Expose key generation functions in testcrypt. They're not much use for 'real' key generation, since like all the other randomness-using testcrypt functions, they need you to have explicitly queued up some random data. But for generating keys for test purposes, they have the great virtue that they deliver the key in the internal format, where we can generate all the various public and private blobs from it as well as the on-disk formats. A minor change to one of the keygen functions itself: rsa_generate now fills in the 'bits' and 'bytes' fields of the returned RSAKey, without which it didn't actually work to try to generate a public blob from it. (We'd never noticed before, because no previous client of rsa_generate even tried that.)
2020-01-09 07:21:30 +00:00
}
ssh_key *ecdsa_generate_wrapper(int bits)
{
struct ecdsa_key *ek = snew(struct ecdsa_key);
New vtable API for keygen progress reporting. The old API was one of those horrible things I used to do when I was young and foolish, in which you have just one function, and indicate which of lots of things it's doing by passing in flags. It was crying out to be replaced with a vtable. While I'm at it, I've reworked the code on the Windows side that decides what to do with the progress bar, so that it's based on actually justifiable estimates of probability rather than magic integer constants. Since computers are generally faster now than they were at the start of this project, I've also decided there's no longer any point in making the fixed final part of RSA key generation bother to report progress at all. So the progress bars are now only for the variable part, i.e. the actual prime generations. (This is a reapplication of commit a7bdefb39, without the Miller-Rabin refactoring accidentally folded into it. Also this time I've added -lm to the link options, which for some reason _didn't_ cause me a link failure last time round. No idea why not.)
2020-02-23 15:29:40 +00:00
if (!ecdsa_generate(ek, bits)) {
Expose key generation functions in testcrypt. They're not much use for 'real' key generation, since like all the other randomness-using testcrypt functions, they need you to have explicitly queued up some random data. But for generating keys for test purposes, they have the great virtue that they deliver the key in the internal format, where we can generate all the various public and private blobs from it as well as the on-disk formats. A minor change to one of the keygen functions itself: rsa_generate now fills in the 'bits' and 'bytes' fields of the returned RSAKey, without which it didn't actually work to try to generate a public blob from it. (We'd never noticed before, because no previous client of rsa_generate even tried that.)
2020-01-09 07:21:30 +00:00
sfree(ek);
return NULL;
}
return &ek->sshk;
}
ssh_key *eddsa_generate_wrapper(int bits)
{
struct eddsa_key *ek = snew(struct eddsa_key);
New vtable API for keygen progress reporting. The old API was one of those horrible things I used to do when I was young and foolish, in which you have just one function, and indicate which of lots of things it's doing by passing in flags. It was crying out to be replaced with a vtable. While I'm at it, I've reworked the code on the Windows side that decides what to do with the progress bar, so that it's based on actually justifiable estimates of probability rather than magic integer constants. Since computers are generally faster now than they were at the start of this project, I've also decided there's no longer any point in making the fixed final part of RSA key generation bother to report progress at all. So the progress bars are now only for the variable part, i.e. the actual prime generations. (This is a reapplication of commit a7bdefb39, without the Miller-Rabin refactoring accidentally folded into it. Also this time I've added -lm to the link options, which for some reason _didn't_ cause me a link failure last time round. No idea why not.)
2020-02-23 15:29:40 +00:00
if (!eddsa_generate(ek, bits)) {
Expose key generation functions in testcrypt. They're not much use for 'real' key generation, since like all the other randomness-using testcrypt functions, they need you to have explicitly queued up some random data. But for generating keys for test purposes, they have the great virtue that they deliver the key in the internal format, where we can generate all the various public and private blobs from it as well as the on-disk formats. A minor change to one of the keygen functions itself: rsa_generate now fills in the 'bits' and 'bytes' fields of the returned RSAKey, without which it didn't actually work to try to generate a public blob from it. (We'd never noticed before, because no previous client of rsa_generate even tried that.)
2020-01-09 07:21:30 +00:00
sfree(ek);
return NULL;
}
return &ek->sshk;
}
cmdgen: add a --dump option. Also spelled '-O text', this takes a public or private key as input, and produces on standard output a dump of all the actual numbers involved in the key: the exponent and modulus for RSA, the p,q,g,y parameters for DSA, the affine x and y coordinates of the public elliptic curve point for ECC keys, and all the extra bits and pieces in the private keys too. Partly I expect this to be useful to me for debugging: I've had to paste key files a few too many times through base64 decoders and hex dump tools, then manually decode SSH marshalling and paste the result into the Python REPL to get an integer object. Now I should be able to get _straight_ to text I can paste into Python. But also, it's a way that other applications can use the key generator: if you need to generate, say, an RSA key in some format I don't support (I've recently heard of an XML-based one, for example), then you can run 'puttygen -t rsa --dump' and have it print the elements of a freshly generated keypair on standard output, and then all you have to do is understand the output format.
2020-02-17 19:53:19 +00:00
size_t key_components_count(key_components *kc) { return kc->ncomponents; }
const char *key_components_nth_name(key_components *kc, size_t n)
{
return (n >= kc->ncomponents ? NULL :
kc->components[n].name);
}
const char *key_components_nth_str(key_components *kc, size_t n)
{
return (n >= kc->ncomponents ? NULL :
kc->components[n].is_mp_int ? NULL :
kc->components[n].text);
}
mp_int *key_components_nth_mp(key_components *kc, size_t n)
{
return (n >= kc->ncomponents ? NULL :
!kc->components[n].is_mp_int ? NULL :
mp_copy(kc->components[n].mp));
}
New 'Pockle' object, for verifying primality. This implements an extended form of primality verification using certificates based on Pocklington's theorem. You make a Pockle object, and then try to convince it that one number after another is prime, by means of providing it with a list of prime factors of p-1 and a primitive root. (Or just by saying 'this prime is small enough for you to check yourself'.) Pocklington's theorem requires you to have factors of p-1 whose product is at least the square root of p. I've extended that to support factorisations only as big as the cube root, via an extension of the theorem given in Maurer's paper on generating provable primes. The Pockle object is more or less write-only: it has no methods for reading out its contents. Its only output channel is the return value when you try to insert a prime into it: if it isn't sufficiently convinced that your prime is prime, it will return an error code. So anything for which it returns POCKLE_OK you can be confident of. I'm going to use this for provable prime generation. But exposing this part of the system as an object in its own right means I can write a set of unit tests for this specifically. My negative tests exercise all the different ways a certification can be erroneous or inadequate; the positive tests include proofs of primality of various primes used in elliptic-curve crypto. The Poly1305 proof in particular is taken from a proof in DJB's paper, which has exactly the form of a Pocklington certificate only written in English.
2020-02-23 15:16:30 +00:00
PockleStatus pockle_add_prime_wrapper(Pockle *pockle, mp_int *p,
struct mpint_list mpl, mp_int *witness)
{
return pockle_add_prime(pockle, p, mpl.integers, mpl.n, witness);
}
Add implementation of the Argon2 password hash. This is going to be used in the new version of the PPK file format. It was the winner of the Password Hashing Context, which I think makes it a reasonable choice. Argon2 comes in three flavours: one with no data dependency in its memory addressing, one with _deliberate_ data dependency (intended to serialise computation, to hinder parallel brute-forcing), and a hybrid form that starts off data-independent and then switches over to the dependent version once the sensitive input data has been adequately mixed around. I test all three in the test suite; the side-channel tester can only expect Argon2i to pass; and, following the spec's recommendation, I'll be using Argon2id for the actual key file encryption.
2021-02-13 17:30:12 +00:00
strbuf *argon2_wrapper(Argon2Flavour flavour, uint32_t mem, uint32_t passes,
uint32_t parallel, uint32_t taglen,
ptrlen P, ptrlen S, ptrlen K, ptrlen X)
{
strbuf *out = strbuf_new();
argon2(flavour, mem, passes, parallel, taglen, P, S, K, X, out);
return out;
}
Expose openssh_bcrypt() to testcrypt, and test it. I happened to notice in passing that this function doesn't have any tests (although it will have been at least somewhat tested by the cmdgen interop test system). This involved writing a wrapper that passes the passphrase and salt as ptrlens, and I decided it made more sense to make the same change to the original function too and adjust the call sites appropriately. I derived a test case by getting OpenSSH itself to make an encrypted key file, and then using the inputs and output from the password hash operation that decrypted it again.
2021-12-24 09:56:30 +00:00
strbuf *openssh_bcrypt_wrapper(ptrlen passphrase, ptrlen salt,
unsigned rounds, unsigned outbytes)
{
strbuf *out = strbuf_new();
openssh_bcrypt(passphrase, salt, rounds,
strbuf_append(out, outbytes), outbytes);
return out;
}
Break up crypto modules containing HW acceleration. This applies to all of AES, SHA-1, SHA-256 and SHA-512. All those source files previously contained multiple implementations of the algorithm, enabled or disabled by ifdefs detecting whether they would work on a given compiler. And in order to get advanced machine instructions like AES-NI or NEON crypto into the output file when the compile flags hadn't enabled them, we had to do nasty stuff with compiler-specific pragmas or attributes. Now we can do the detection at cmake time, and enable advanced instructions in the more sensible way, by compile-time flags. So I've broken up each of these modules into lots of sub-pieces: a file called (e.g.) 'foo-common.c' containing common definitions across all implementations (such as round constants), one called 'foo-select.c' containing the top-level vtable(s), and a separate file for each implementation exporting just the vtable(s) for that implementation. One advantage of this is that it depends a lot less on compiler- specific bodgery. My particular least favourite part of the previous setup was the part where I had to _manually_ define some Arm ACLE feature macros before including <arm_neon.h>, so that it would define the intrinsics I wanted. Now I'm enabling interesting architecture features in the normal way, on the compiler command line, there's no need for that kind of trick: the right feature macros are already defined and <arm_neon.h> does the right thing. Another change in this reorganisation is that I've stopped assuming there's just one hardware implementation per platform. Previously, the accelerated vtables were called things like sha256_hw, and varied between FOO-NI and NEON depending on platform; and the selection code would simply ask 'is hw available? if so, use hw, else sw'. Now, each HW acceleration strategy names its vtable its own way, and the selection vtable has a whole list of possibilities to iterate over looking for a supported one. So if someone feels like writing a second accelerated implementation of something for a given platform - for example, I've heard you can use plain NEON to speed up AES somewhat even without the crypto extension - then it will now have somewhere to drop in alongside the existing ones.
2021-04-19 05:42:12 +00:00
strbuf *get_implementations_commasep(ptrlen alg)
{
strbuf *out = strbuf_new();
put_datapl(out, alg);
if (ptrlen_startswith(alg, PTRLEN_LITERAL("aes"), NULL)) {
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, ",%.*s_sw", PTRLEN_PRINTF(alg));
Break up crypto modules containing HW acceleration. This applies to all of AES, SHA-1, SHA-256 and SHA-512. All those source files previously contained multiple implementations of the algorithm, enabled or disabled by ifdefs detecting whether they would work on a given compiler. And in order to get advanced machine instructions like AES-NI or NEON crypto into the output file when the compile flags hadn't enabled them, we had to do nasty stuff with compiler-specific pragmas or attributes. Now we can do the detection at cmake time, and enable advanced instructions in the more sensible way, by compile-time flags. So I've broken up each of these modules into lots of sub-pieces: a file called (e.g.) 'foo-common.c' containing common definitions across all implementations (such as round constants), one called 'foo-select.c' containing the top-level vtable(s), and a separate file for each implementation exporting just the vtable(s) for that implementation. One advantage of this is that it depends a lot less on compiler- specific bodgery. My particular least favourite part of the previous setup was the part where I had to _manually_ define some Arm ACLE feature macros before including <arm_neon.h>, so that it would define the intrinsics I wanted. Now I'm enabling interesting architecture features in the normal way, on the compiler command line, there's no need for that kind of trick: the right feature macros are already defined and <arm_neon.h> does the right thing. Another change in this reorganisation is that I've stopped assuming there's just one hardware implementation per platform. Previously, the accelerated vtables were called things like sha256_hw, and varied between FOO-NI and NEON depending on platform; and the selection code would simply ask 'is hw available? if so, use hw, else sw'. Now, each HW acceleration strategy names its vtable its own way, and the selection vtable has a whole list of possibilities to iterate over looking for a supported one. So if someone feels like writing a second accelerated implementation of something for a given platform - for example, I've heard you can use plain NEON to speed up AES somewhat even without the crypto extension - then it will now have somewhere to drop in alongside the existing ones.
2021-04-19 05:42:12 +00:00
#if HAVE_AES_NI
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, ",%.*s_ni", PTRLEN_PRINTF(alg));
Break up crypto modules containing HW acceleration. This applies to all of AES, SHA-1, SHA-256 and SHA-512. All those source files previously contained multiple implementations of the algorithm, enabled or disabled by ifdefs detecting whether they would work on a given compiler. And in order to get advanced machine instructions like AES-NI or NEON crypto into the output file when the compile flags hadn't enabled them, we had to do nasty stuff with compiler-specific pragmas or attributes. Now we can do the detection at cmake time, and enable advanced instructions in the more sensible way, by compile-time flags. So I've broken up each of these modules into lots of sub-pieces: a file called (e.g.) 'foo-common.c' containing common definitions across all implementations (such as round constants), one called 'foo-select.c' containing the top-level vtable(s), and a separate file for each implementation exporting just the vtable(s) for that implementation. One advantage of this is that it depends a lot less on compiler- specific bodgery. My particular least favourite part of the previous setup was the part where I had to _manually_ define some Arm ACLE feature macros before including <arm_neon.h>, so that it would define the intrinsics I wanted. Now I'm enabling interesting architecture features in the normal way, on the compiler command line, there's no need for that kind of trick: the right feature macros are already defined and <arm_neon.h> does the right thing. Another change in this reorganisation is that I've stopped assuming there's just one hardware implementation per platform. Previously, the accelerated vtables were called things like sha256_hw, and varied between FOO-NI and NEON depending on platform; and the selection code would simply ask 'is hw available? if so, use hw, else sw'. Now, each HW acceleration strategy names its vtable its own way, and the selection vtable has a whole list of possibilities to iterate over looking for a supported one. So if someone feels like writing a second accelerated implementation of something for a given platform - for example, I've heard you can use plain NEON to speed up AES somewhat even without the crypto extension - then it will now have somewhere to drop in alongside the existing ones.
2021-04-19 05:42:12 +00:00
#endif
#if HAVE_NEON_CRYPTO
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, ",%.*s_neon", PTRLEN_PRINTF(alg));
Break up crypto modules containing HW acceleration. This applies to all of AES, SHA-1, SHA-256 and SHA-512. All those source files previously contained multiple implementations of the algorithm, enabled or disabled by ifdefs detecting whether they would work on a given compiler. And in order to get advanced machine instructions like AES-NI or NEON crypto into the output file when the compile flags hadn't enabled them, we had to do nasty stuff with compiler-specific pragmas or attributes. Now we can do the detection at cmake time, and enable advanced instructions in the more sensible way, by compile-time flags. So I've broken up each of these modules into lots of sub-pieces: a file called (e.g.) 'foo-common.c' containing common definitions across all implementations (such as round constants), one called 'foo-select.c' containing the top-level vtable(s), and a separate file for each implementation exporting just the vtable(s) for that implementation. One advantage of this is that it depends a lot less on compiler- specific bodgery. My particular least favourite part of the previous setup was the part where I had to _manually_ define some Arm ACLE feature macros before including <arm_neon.h>, so that it would define the intrinsics I wanted. Now I'm enabling interesting architecture features in the normal way, on the compiler command line, there's no need for that kind of trick: the right feature macros are already defined and <arm_neon.h> does the right thing. Another change in this reorganisation is that I've stopped assuming there's just one hardware implementation per platform. Previously, the accelerated vtables were called things like sha256_hw, and varied between FOO-NI and NEON depending on platform; and the selection code would simply ask 'is hw available? if so, use hw, else sw'. Now, each HW acceleration strategy names its vtable its own way, and the selection vtable has a whole list of possibilities to iterate over looking for a supported one. So if someone feels like writing a second accelerated implementation of something for a given platform - for example, I've heard you can use plain NEON to speed up AES somewhat even without the crypto extension - then it will now have somewhere to drop in alongside the existing ones.
2021-04-19 05:42:12 +00:00
#endif
} else if (ptrlen_startswith(alg, PTRLEN_LITERAL("sha256"), NULL) ||
ptrlen_startswith(alg, PTRLEN_LITERAL("sha1"), NULL)) {
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, ",%.*s_sw", PTRLEN_PRINTF(alg));
Break up crypto modules containing HW acceleration. This applies to all of AES, SHA-1, SHA-256 and SHA-512. All those source files previously contained multiple implementations of the algorithm, enabled or disabled by ifdefs detecting whether they would work on a given compiler. And in order to get advanced machine instructions like AES-NI or NEON crypto into the output file when the compile flags hadn't enabled them, we had to do nasty stuff with compiler-specific pragmas or attributes. Now we can do the detection at cmake time, and enable advanced instructions in the more sensible way, by compile-time flags. So I've broken up each of these modules into lots of sub-pieces: a file called (e.g.) 'foo-common.c' containing common definitions across all implementations (such as round constants), one called 'foo-select.c' containing the top-level vtable(s), and a separate file for each implementation exporting just the vtable(s) for that implementation. One advantage of this is that it depends a lot less on compiler- specific bodgery. My particular least favourite part of the previous setup was the part where I had to _manually_ define some Arm ACLE feature macros before including <arm_neon.h>, so that it would define the intrinsics I wanted. Now I'm enabling interesting architecture features in the normal way, on the compiler command line, there's no need for that kind of trick: the right feature macros are already defined and <arm_neon.h> does the right thing. Another change in this reorganisation is that I've stopped assuming there's just one hardware implementation per platform. Previously, the accelerated vtables were called things like sha256_hw, and varied between FOO-NI and NEON depending on platform; and the selection code would simply ask 'is hw available? if so, use hw, else sw'. Now, each HW acceleration strategy names its vtable its own way, and the selection vtable has a whole list of possibilities to iterate over looking for a supported one. So if someone feels like writing a second accelerated implementation of something for a given platform - for example, I've heard you can use plain NEON to speed up AES somewhat even without the crypto extension - then it will now have somewhere to drop in alongside the existing ones.
2021-04-19 05:42:12 +00:00
#if HAVE_SHA_NI
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, ",%.*s_ni", PTRLEN_PRINTF(alg));
Break up crypto modules containing HW acceleration. This applies to all of AES, SHA-1, SHA-256 and SHA-512. All those source files previously contained multiple implementations of the algorithm, enabled or disabled by ifdefs detecting whether they would work on a given compiler. And in order to get advanced machine instructions like AES-NI or NEON crypto into the output file when the compile flags hadn't enabled them, we had to do nasty stuff with compiler-specific pragmas or attributes. Now we can do the detection at cmake time, and enable advanced instructions in the more sensible way, by compile-time flags. So I've broken up each of these modules into lots of sub-pieces: a file called (e.g.) 'foo-common.c' containing common definitions across all implementations (such as round constants), one called 'foo-select.c' containing the top-level vtable(s), and a separate file for each implementation exporting just the vtable(s) for that implementation. One advantage of this is that it depends a lot less on compiler- specific bodgery. My particular least favourite part of the previous setup was the part where I had to _manually_ define some Arm ACLE feature macros before including <arm_neon.h>, so that it would define the intrinsics I wanted. Now I'm enabling interesting architecture features in the normal way, on the compiler command line, there's no need for that kind of trick: the right feature macros are already defined and <arm_neon.h> does the right thing. Another change in this reorganisation is that I've stopped assuming there's just one hardware implementation per platform. Previously, the accelerated vtables were called things like sha256_hw, and varied between FOO-NI and NEON depending on platform; and the selection code would simply ask 'is hw available? if so, use hw, else sw'. Now, each HW acceleration strategy names its vtable its own way, and the selection vtable has a whole list of possibilities to iterate over looking for a supported one. So if someone feels like writing a second accelerated implementation of something for a given platform - for example, I've heard you can use plain NEON to speed up AES somewhat even without the crypto extension - then it will now have somewhere to drop in alongside the existing ones.
2021-04-19 05:42:12 +00:00
#endif
#if HAVE_NEON_CRYPTO
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, ",%.*s_neon", PTRLEN_PRINTF(alg));
Break up crypto modules containing HW acceleration. This applies to all of AES, SHA-1, SHA-256 and SHA-512. All those source files previously contained multiple implementations of the algorithm, enabled or disabled by ifdefs detecting whether they would work on a given compiler. And in order to get advanced machine instructions like AES-NI or NEON crypto into the output file when the compile flags hadn't enabled them, we had to do nasty stuff with compiler-specific pragmas or attributes. Now we can do the detection at cmake time, and enable advanced instructions in the more sensible way, by compile-time flags. So I've broken up each of these modules into lots of sub-pieces: a file called (e.g.) 'foo-common.c' containing common definitions across all implementations (such as round constants), one called 'foo-select.c' containing the top-level vtable(s), and a separate file for each implementation exporting just the vtable(s) for that implementation. One advantage of this is that it depends a lot less on compiler- specific bodgery. My particular least favourite part of the previous setup was the part where I had to _manually_ define some Arm ACLE feature macros before including <arm_neon.h>, so that it would define the intrinsics I wanted. Now I'm enabling interesting architecture features in the normal way, on the compiler command line, there's no need for that kind of trick: the right feature macros are already defined and <arm_neon.h> does the right thing. Another change in this reorganisation is that I've stopped assuming there's just one hardware implementation per platform. Previously, the accelerated vtables were called things like sha256_hw, and varied between FOO-NI and NEON depending on platform; and the selection code would simply ask 'is hw available? if so, use hw, else sw'. Now, each HW acceleration strategy names its vtable its own way, and the selection vtable has a whole list of possibilities to iterate over looking for a supported one. So if someone feels like writing a second accelerated implementation of something for a given platform - for example, I've heard you can use plain NEON to speed up AES somewhat even without the crypto extension - then it will now have somewhere to drop in alongside the existing ones.
2021-04-19 05:42:12 +00:00
#endif
} else if (ptrlen_startswith(alg, PTRLEN_LITERAL("sha512"), NULL)) {
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, ",%.*s_sw", PTRLEN_PRINTF(alg));
Break up crypto modules containing HW acceleration. This applies to all of AES, SHA-1, SHA-256 and SHA-512. All those source files previously contained multiple implementations of the algorithm, enabled or disabled by ifdefs detecting whether they would work on a given compiler. And in order to get advanced machine instructions like AES-NI or NEON crypto into the output file when the compile flags hadn't enabled them, we had to do nasty stuff with compiler-specific pragmas or attributes. Now we can do the detection at cmake time, and enable advanced instructions in the more sensible way, by compile-time flags. So I've broken up each of these modules into lots of sub-pieces: a file called (e.g.) 'foo-common.c' containing common definitions across all implementations (such as round constants), one called 'foo-select.c' containing the top-level vtable(s), and a separate file for each implementation exporting just the vtable(s) for that implementation. One advantage of this is that it depends a lot less on compiler- specific bodgery. My particular least favourite part of the previous setup was the part where I had to _manually_ define some Arm ACLE feature macros before including <arm_neon.h>, so that it would define the intrinsics I wanted. Now I'm enabling interesting architecture features in the normal way, on the compiler command line, there's no need for that kind of trick: the right feature macros are already defined and <arm_neon.h> does the right thing. Another change in this reorganisation is that I've stopped assuming there's just one hardware implementation per platform. Previously, the accelerated vtables were called things like sha256_hw, and varied between FOO-NI and NEON depending on platform; and the selection code would simply ask 'is hw available? if so, use hw, else sw'. Now, each HW acceleration strategy names its vtable its own way, and the selection vtable has a whole list of possibilities to iterate over looking for a supported one. So if someone feels like writing a second accelerated implementation of something for a given platform - for example, I've heard you can use plain NEON to speed up AES somewhat even without the crypto extension - then it will now have somewhere to drop in alongside the existing ones.
2021-04-19 05:42:12 +00:00
#if HAVE_NEON_SHA512
Generalise strbuf_catf() into put_fmt(). marshal.h now provides a macro put_fmt() which allows you to write arbitrary printf-formatted data to an arbitrary BinarySink. We already had this facility for strbufs in particular, in the form of strbuf_catf(). That was able to take advantage of knowing the inner structure of a strbuf to minimise memory allocation (it would snprintf directly into the strbuf's existing buffer if possible). For a general black-box BinarySink we can't do that, so instead we dupvprintf into a temporary buffer. For consistency, I've removed strbuf_catf, and converted all uses of it into the new put_fmt - and I've also added an extra vtable method in the BinarySink API, so that put_fmt can still use strbuf_catf's more efficient memory management when talking to a strbuf, and fall back to the simpler strategy when that's not available.
2021-11-19 10:23:32 +00:00
put_fmt(out, ",%.*s_neon", PTRLEN_PRINTF(alg));
Break up crypto modules containing HW acceleration. This applies to all of AES, SHA-1, SHA-256 and SHA-512. All those source files previously contained multiple implementations of the algorithm, enabled or disabled by ifdefs detecting whether they would work on a given compiler. And in order to get advanced machine instructions like AES-NI or NEON crypto into the output file when the compile flags hadn't enabled them, we had to do nasty stuff with compiler-specific pragmas or attributes. Now we can do the detection at cmake time, and enable advanced instructions in the more sensible way, by compile-time flags. So I've broken up each of these modules into lots of sub-pieces: a file called (e.g.) 'foo-common.c' containing common definitions across all implementations (such as round constants), one called 'foo-select.c' containing the top-level vtable(s), and a separate file for each implementation exporting just the vtable(s) for that implementation. One advantage of this is that it depends a lot less on compiler- specific bodgery. My particular least favourite part of the previous setup was the part where I had to _manually_ define some Arm ACLE feature macros before including <arm_neon.h>, so that it would define the intrinsics I wanted. Now I'm enabling interesting architecture features in the normal way, on the compiler command line, there's no need for that kind of trick: the right feature macros are already defined and <arm_neon.h> does the right thing. Another change in this reorganisation is that I've stopped assuming there's just one hardware implementation per platform. Previously, the accelerated vtables were called things like sha256_hw, and varied between FOO-NI and NEON depending on platform; and the selection code would simply ask 'is hw available? if so, use hw, else sw'. Now, each HW acceleration strategy names its vtable its own way, and the selection vtable has a whole list of possibilities to iterate over looking for a supported one. So if someone feels like writing a second accelerated implementation of something for a given platform - for example, I've heard you can use plain NEON to speed up AES somewhat even without the crypto extension - then it will now have somewhere to drop in alongside the existing ones.
2021-04-19 05:42:12 +00:00
#endif
}
return out;
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
#define OPTIONAL_PTR_FUNC(type) \
typedef TD_val_##type TD_opt_val_##type; \
static TD_opt_val_##type get_opt_val_##type(BinarySource *in) { \
ptrlen word = get_word(in); \
if (ptrlen_eq_string(word, "NULL")) \
return NULL; \
return unwrap_value_##type(lookup_value(word))->vu_##type; \
}
Merge the ssh1_cipher type into ssh2_cipher. The aim of this reorganisation is to make it easier to test all the ciphers in PuTTY in a uniform way. It was inconvenient that there were two separate vtable systems for the ciphers used in SSH-1 and SSH-2 with different functionality. Now there's only one type, called ssh_cipher. But really it's the old ssh2_cipher, just renamed: I haven't made any changes to the API on the SSH-2 side. Instead, I've removed ssh1_cipher completely, and adapted the SSH-1 BPP to use the SSH-2 style API. (The relevant differences are that ssh1_cipher encapsulated both the sending and receiving directions in one object - so now ssh1bpp has to make a separate cipher instance per direction - and that ssh1_cipher automatically initialised the IV to all zeroes, which ssh1bpp now has to do by hand.) The previous ssh1_cipher vtable for single-DES has been removed completely, because when converted into the new API it became identical to the SSH-2 single-DES vtable; so now there's just one vtable for DES-CBC which works in both protocols. The other two SSH-1 ciphers each had to stay separate, because 3DES is completely different between SSH-1 and SSH-2 (three layers of CBC structure versus one), and Blowfish varies in endianness and key length between the two. (Actually, while I'm here, I've only just noticed that the SSH-1 Blowfish cipher mis-describes itself in log messages as Blowfish-128. In fact it passes the whole of the input key buffer, which has length SSH1_SESSION_KEY_LENGTH == 32 bytes == 256 bits. So it's actually Blowfish-256, and has been all along!)
2019-01-17 18:06:08 +00:00
OPTIONAL_PTR_FUNC(cipher)
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
OPTIONAL_PTR_FUNC(mpint)
Expose the key-file marshalling functions in testcrypt. This will allow me to write tests for them.
2020-01-06 19:58:25 +00:00
OPTIONAL_PTR_FUNC(string)
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
/*
* HERE BE DRAGONS: the horrible C preprocessor business that reads
Move some tests into the test subdirectory. Now testcrypt has _two_ header files, that's more files than I want at the top level, so I decided to move it. It has a good claim to live in either 'test' or 'crypto', but in the end I decided it wasn't quite specific enough to crypto (it already also tests things in keygen and proxy), and also, the Python half of the mechanism already lives in 'test', so it can live alongside that. Having done that, it seemed silly to leave testsc and testzlib at the top level: those have 'test' in the names as well, so they can go in the test subdir as well. While I'm renaming, also renamed testcrypt.h to testcrypt-func.h to distinguish it from the new testcrypt-enum.h.
2021-11-22 18:50:12 +00:00
* testcrypt-func.h and generates a marshalling wrapper for each
* exported function.
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
*
* In an ideal world, we would start from a specification like this in
Move some tests into the test subdirectory. Now testcrypt has _two_ header files, that's more files than I want at the top level, so I decided to move it. It has a good claim to live in either 'test' or 'crypto', but in the end I decided it wasn't quite specific enough to crypto (it already also tests things in keygen and proxy), and also, the Python half of the mechanism already lives in 'test', so it can live alongside that. Having done that, it seemed silly to leave testsc and testzlib at the top level: those have 'test' in the names as well, so they can go in the test subdir as well. While I'm renaming, also renamed testcrypt.h to testcrypt-func.h to distinguish it from the new testcrypt-enum.h.
2021-11-22 18:50:12 +00:00
* testcrypt-func.h
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
*
testcrypt-funcs.h: remove extra parens round argument lists. They were there to work around that annoying feature of VS's preprocessor when it expands __VA_ARGS__ into the argument list of another macro. But I've just thought of a workaround that I can apply in testcrypt.c itself, so that those parens don't have to appear in every function definition in the header file. The trick is, instead of writing destination_macro(__VA_ARGS__) you instead write JUXTAPOSE(destination_macro, (__VA_ARGS__)) where JUXTAPOSE is defined to be a macro that simply expands its two arguments next to each other: #define JUXTAPOSE(first, second) first second This works because the arguments to JUXTAPOSE get macro-expanded _before_ passing them to JUXTAPOSE itself - the same reason that the standard tricks with STR_INNER and CAT_INNER work (as seen in defs.h here). So this defuses the magic behaviour of commas expanded from __VA_ARGS__, and causes the destination macro to get all its arguments in the expected places again.
2021-11-28 09:39:49 +00:00
* FUNC(val_foo, example, ARG(val_bar, bar), ARG(uint, n))
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
*
* and generate a wrapper function looking like this:
*
* static void handle_example(BinarySource *in, strbuf *out) {
* TD_val_bar bar = get_val_bar(in);
* TD_uint n = get_uint(in);
* return_val_foo(out, example(bar, n));
* }
*
* which would read the marshalled form of each function argument in
* turn from the input BinarySource via the get_<type>() function
* family defined in this file; assign each argument to a local
* variable; call the underlying C function with all those arguments;
* and then call a function of the return_<type>() family to marshal
* the output value into the output strbuf to be sent to standard
* output.
*
* With a more general macro processor such as m4, or custom code in
* Perl or Python, or a helper program like llvm-tblgen, we could just
Move some tests into the test subdirectory. Now testcrypt has _two_ header files, that's more files than I want at the top level, so I decided to move it. It has a good claim to live in either 'test' or 'crypto', but in the end I decided it wasn't quite specific enough to crypto (it already also tests things in keygen and proxy), and also, the Python half of the mechanism already lives in 'test', so it can live alongside that. Having done that, it seemed silly to leave testsc and testzlib at the top level: those have 'test' in the names as well, so they can go in the test subdir as well. While I'm renaming, also renamed testcrypt.h to testcrypt-func.h to distinguish it from the new testcrypt-enum.h.
2021-11-22 18:50:12 +00:00
* do that directly, reading function specifications from
* testcrypt-func.h and writing out exactly the above. But we don't
* have a fully general macro processor (since everything in that
* category introduces an extra build dependency that's awkward on
* plain Windows, or requires compiling and running a helper program
* which is awkward in a cross-compile). We only have cpp. And in cpp,
* a macro can't expand one of its arguments differently in two parts
* of its own expansion. So we have to be more clever.
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
*
* In place of the above code, I instead generate three successive
* declarations for each function. In simplified form they would look
* like this:
*
* typedef struct ARGS_example {
* TD_val_bar bar;
* TD_uint n;
* } ARGS_example;
*
* static inline ARGS_example get_args_example(BinarySource *in) {
* ARGS_example args;
* args.bar = get_val_bar(in);
* args.n = get_uint(in);
* return args;
* }
*
* static void handle_example(BinarySource *in, strbuf *out) {
* ARGS_example args = get_args_example(in);
* return_val_foo(out, example(args.bar, args.n));
* }
*
* Each of these mentions the arguments and their types just _once_,
* so each one can be generated by a single expansion of the FUNC(...)
Move some tests into the test subdirectory. Now testcrypt has _two_ header files, that's more files than I want at the top level, so I decided to move it. It has a good claim to live in either 'test' or 'crypto', but in the end I decided it wasn't quite specific enough to crypto (it already also tests things in keygen and proxy), and also, the Python half of the mechanism already lives in 'test', so it can live alongside that. Having done that, it seemed silly to leave testsc and testzlib at the top level: those have 'test' in the names as well, so they can go in the test subdir as well. While I'm renaming, also renamed testcrypt.h to testcrypt-func.h to distinguish it from the new testcrypt-enum.h.
2021-11-22 18:50:12 +00:00
* specification in testcrypt-func.h, with FUNC and ARG and VOID
* defined to appropriate values.
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
*
* Or ... *nearly*. In fact, I left out several details there, but
* it's a good starting point to understand the full version.
*
* To begin with, several of the variable names shown above are
* actually named with an ugly leading underscore, to minimise the
* chance of them colliding with real parameter names. (You could
* easily imagine 'out' being the name of a parameter to one of the
* wrapped functions.) Also, we memset the whole structure to zero at
* the start of get_args_example() to avoid compiler warnings about
* uninitialised stuff, and insert a precautionary '(void)args;' in
* handle_example to avoid a similar warning about _unused_ stuff.
*
* The big problem is the commas that have to appear between arguments
* in the final call to the actual C function. Those can't be
* generated by expanding the ARG macro itself, or you'd get one too
* many - either a leading comma or a trailing comma. Trailing commas
* are legal in a Python function call, but unfortunately C is not yet
* so enlightened. (C permits a trailing comma in a struct or array
* initialiser, and is coming round to it in enums, but hasn't yet
* seen the light about function calls or function prototypes.)
*
* So the commas must appear _between_ ARG(...) specifiers. And that
* means they unavoidably appear in _every_ expansion of FUNC() (or
testcrypt-funcs.h: remove extra parens round argument lists. They were there to work around that annoying feature of VS's preprocessor when it expands __VA_ARGS__ into the argument list of another macro. But I've just thought of a workaround that I can apply in testcrypt.c itself, so that those parens don't have to appear in every function definition in the header file. The trick is, instead of writing destination_macro(__VA_ARGS__) you instead write JUXTAPOSE(destination_macro, (__VA_ARGS__)) where JUXTAPOSE is defined to be a macro that simply expands its two arguments next to each other: #define JUXTAPOSE(first, second) first second This works because the arguments to JUXTAPOSE get macro-expanded _before_ passing them to JUXTAPOSE itself - the same reason that the standard tricks with STR_INNER and CAT_INNER work (as seen in defs.h here). So this defuses the magic behaviour of commas expanded from __VA_ARGS__, and causes the destination macro to get all its arguments in the expected places again.
2021-11-28 09:39:49 +00:00
* rather, every expansion that uses the variadic argument list at
* all). Therefore, we need to ensure they're harmless in the other
* two functions as well.
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
*
* In the get_args_example() function above, there's no real problem.
* The list of assignments can perfectly well be separated by commas
* instead of semicolons, so that it becomes a single expression-
* statement instead of a sequence of them; the comma operator still
* defines a sequence point, so it's fine.
*
* But what about the structure definition of ARGS_example?
*
* To get round that, we fill the structure with pointless extra
* cruft, in the form of an extra 'int' field before and after each
* actually useful argument field. So the real structure definition
* ends up looking more like this:
*
* typedef struct ARGS_example {
* int _predummy_bar;
* TD_val_bar bar;
* int _postdummy_bar, _predummy_n;
* TD_uint n;
* int _postdummy_n;
* } ARGS_example;
*
* Those extra 'int' fields are ignored completely at run time. They
* might cause a runtime space cost if the struct doesn't get
* completely optimised away when get_args_example is inlined into
* handle_example, but even if so, that's OK, this is a test program
* whose memory usage isn't critical. The real point is that, in
* between each pair of real arguments, there's a declaration
* containing *two* int variables, and in between them is the vital
* comma that we need!
*
Move some tests into the test subdirectory. Now testcrypt has _two_ header files, that's more files than I want at the top level, so I decided to move it. It has a good claim to live in either 'test' or 'crypto', but in the end I decided it wasn't quite specific enough to crypto (it already also tests things in keygen and proxy), and also, the Python half of the mechanism already lives in 'test', so it can live alongside that. Having done that, it seemed silly to leave testsc and testzlib at the top level: those have 'test' in the names as well, so they can go in the test subdir as well. While I'm renaming, also renamed testcrypt.h to testcrypt-func.h to distinguish it from the new testcrypt-enum.h.
2021-11-22 18:50:12 +00:00
* So in that pass through testcrypt-func.h, the ARG(type, name) macro
* has to expand to the weird piece of text
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
*
* _predummy_name; // terminating the previous int declaration
* TD_type name; // declaring the thing we actually wanted
* int _postdummy_name // new declaration ready to see a comma
*
* so that a comma-separated list of pieces of expansion like that
* will fall into just the right form to be the core of the above
* expanded structure definition. Then we just need to put in the
* 'int' after the open brace, and the ';' before the closing brace,
* and we've got everything we need to make it all syntactically legal.
*
* Finally, what if a wrapped function has _no_ arguments? Two out of
* three uses of the argument list here need some kind of special case
* for that. That's why you have to write 'VOID' explicitly in an
Move some tests into the test subdirectory. Now testcrypt has _two_ header files, that's more files than I want at the top level, so I decided to move it. It has a good claim to live in either 'test' or 'crypto', but in the end I decided it wasn't quite specific enough to crypto (it already also tests things in keygen and proxy), and also, the Python half of the mechanism already lives in 'test', so it can live alongside that. Having done that, it seemed silly to leave testsc and testzlib at the top level: those have 'test' in the names as well, so they can go in the test subdir as well. While I'm renaming, also renamed testcrypt.h to testcrypt-func.h to distinguish it from the new testcrypt-enum.h.
2021-11-22 18:50:12 +00:00
* empty argument list in testcrypt-func.h: we make VOID expand to
* whatever is needed to avoid a syntax error in that special case.
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
*/
testcrypt: ability to handle more function parameters. I'm about to add a function of particularly large arity.
2021-02-20 16:47:37 +00:00
testcrypt-funcs.h: remove extra parens round argument lists. They were there to work around that annoying feature of VS's preprocessor when it expands __VA_ARGS__ into the argument list of another macro. But I've just thought of a workaround that I can apply in testcrypt.c itself, so that those parens don't have to appear in every function definition in the header file. The trick is, instead of writing destination_macro(__VA_ARGS__) you instead write JUXTAPOSE(destination_macro, (__VA_ARGS__)) where JUXTAPOSE is defined to be a macro that simply expands its two arguments next to each other: #define JUXTAPOSE(first, second) first second This works because the arguments to JUXTAPOSE get macro-expanded _before_ passing them to JUXTAPOSE itself - the same reason that the standard tricks with STR_INNER and CAT_INNER work (as seen in defs.h here). So this defuses the magic behaviour of commas expanded from __VA_ARGS__, and causes the destination macro to get all its arguments in the expected places again.
2021-11-28 09:39:49 +00:00
/*
* Workarounds for an awkwardness in Visual Studio's preprocessor,
* which disagrees with everyone else about what happens if you expand
* __VA_ARGS__ into the argument list of another macro. gcc and clang
* will treat the commas expanding from __VA_ARGS__ as argument
* separators, whereas VS will make them all part of a single argument
* to the secondary macro. We want the former behaviour, so we use
* the following workaround to enforce it.
*
* Each of these JUXTAPOSE macros simply places its arguments side by
* side. But the arguments are macro-expanded before JUXTAPOSE is
* called at all, so we can do this:
*
* JUXTAPOSE(macroname, (__VA_ARGS__))
* -> JUXTAPOSE(macroname, (foo, bar, baz))
* -> macroname (foo, bar, baz)
*
* and this preliminary expansion causes the commas to be treated
* normally by the time VS gets round to expanding the inner macro.
*
* We need two differently named JUXTAPOSE macros, because we have to
* do this trick twice: once to turn FUNC and FUNC_WRAPPED in
* testcrypt-funcs.h into the underlying common FUNC_INNER, and again
* to expand the final function call. And you can't expand a macro
* inside text expanded from the _same_ macro, so we have to do the
* outer and inner instances of this trick using macros of different
* names.
*/
#define JUXTAPOSE1(first, second) first second
#define JUXTAPOSE2(first, second) first second
Extend testcrypt macros up to 12-ary functions. I'm about to add a monster new function that takes 12 arguments.
2021-11-20 14:28:36 +00:00
testcrypt-funcs.h: remove extra parens round argument lists. They were there to work around that annoying feature of VS's preprocessor when it expands __VA_ARGS__ into the argument list of another macro. But I've just thought of a workaround that I can apply in testcrypt.c itself, so that those parens don't have to appear in every function definition in the header file. The trick is, instead of writing destination_macro(__VA_ARGS__) you instead write JUXTAPOSE(destination_macro, (__VA_ARGS__)) where JUXTAPOSE is defined to be a macro that simply expands its two arguments next to each other: #define JUXTAPOSE(first, second) first second This works because the arguments to JUXTAPOSE get macro-expanded _before_ passing them to JUXTAPOSE itself - the same reason that the standard tricks with STR_INNER and CAT_INNER work (as seen in defs.h here). So this defuses the magic behaviour of commas expanded from __VA_ARGS__, and causes the destination macro to get all its arguments in the expected places again.
2021-11-28 09:39:49 +00:00
#define FUNC(outtype, fname, ...) \
JUXTAPOSE1(FUNC_INNER, (outtype, fname, fname, __VA_ARGS__))
#define FUNC_WRAPPED(outtype, fname, ...) \
JUXTAPOSE1(FUNC_INNER, (outtype, fname, fname##_wrapper, __VA_ARGS__))
testcrypt.h: invent FUNC_WRAPPED. FUNC_WRAPPED is an alternative keyword to FUNC which you can use to introduce a function specification in testcrypt.h, indicating that the function is _not_ the one of the same name used in the main PuTTY code, but instead a wrapper on it in testcrypt.c whose API was reworked to be more friendly to translation into Python. There are a lot of those wrappers already, and previously they passed without comment in testcrypt.h, and were put into service by #defining over the top of each name before expanding the marshalling functions. Now, all those #defines are gone, because the use of FUNC_WRAPPED in testcrypt.h is enough to clue in the marshalling wrapper to be generated with a call to foo_wrapper() instead of foo(). Mostly the purpose of this is to make testcrypt.h a bit more self-documenting: if you see FUNC_WRAPPED, you know not to be confused by the Python and C function definitions totally failing to match.
2021-11-21 13:03:34 +00:00
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
#define ARG(type, arg) _predummy_##arg; TD_##type arg; int _postdummy_##arg
#define VOID _voiddummy
testcrypt-funcs.h: remove extra parens round argument lists. They were there to work around that annoying feature of VS's preprocessor when it expands __VA_ARGS__ into the argument list of another macro. But I've just thought of a workaround that I can apply in testcrypt.c itself, so that those parens don't have to appear in every function definition in the header file. The trick is, instead of writing destination_macro(__VA_ARGS__) you instead write JUXTAPOSE(destination_macro, (__VA_ARGS__)) where JUXTAPOSE is defined to be a macro that simply expands its two arguments next to each other: #define JUXTAPOSE(first, second) first second This works because the arguments to JUXTAPOSE get macro-expanded _before_ passing them to JUXTAPOSE itself - the same reason that the standard tricks with STR_INNER and CAT_INNER work (as seen in defs.h here). So this defuses the magic behaviour of commas expanded from __VA_ARGS__, and causes the destination macro to get all its arguments in the expected places again.
2021-11-28 09:39:49 +00:00
#define FUNC_INNER(outtype, fname, realname, ...) \
testcrypt.h: invent FUNC_WRAPPED. FUNC_WRAPPED is an alternative keyword to FUNC which you can use to introduce a function specification in testcrypt.h, indicating that the function is _not_ the one of the same name used in the main PuTTY code, but instead a wrapper on it in testcrypt.c whose API was reworked to be more friendly to translation into Python. There are a lot of those wrappers already, and previously they passed without comment in testcrypt.h, and were put into service by #defining over the top of each name before expanding the marshalling functions. Now, all those #defines are gone, because the use of FUNC_WRAPPED in testcrypt.h is enough to clue in the marshalling wrapper to be generated with a call to foo_wrapper() instead of foo(). Mostly the purpose of this is to make testcrypt.h a bit more self-documenting: if you see FUNC_WRAPPED, you know not to be confused by the Python and C function definitions totally failing to match.
2021-11-21 13:03:34 +00:00
typedef struct ARGS_##fname { \
testcrypt-funcs.h: remove extra parens round argument lists. They were there to work around that annoying feature of VS's preprocessor when it expands __VA_ARGS__ into the argument list of another macro. But I've just thought of a workaround that I can apply in testcrypt.c itself, so that those parens don't have to appear in every function definition in the header file. The trick is, instead of writing destination_macro(__VA_ARGS__) you instead write JUXTAPOSE(destination_macro, (__VA_ARGS__)) where JUXTAPOSE is defined to be a macro that simply expands its two arguments next to each other: #define JUXTAPOSE(first, second) first second This works because the arguments to JUXTAPOSE get macro-expanded _before_ passing them to JUXTAPOSE itself - the same reason that the standard tricks with STR_INNER and CAT_INNER work (as seen in defs.h here). So this defuses the magic behaviour of commas expanded from __VA_ARGS__, and causes the destination macro to get all its arguments in the expected places again.
2021-11-28 09:39:49 +00:00
int __VA_ARGS__; \
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
} ARGS_##fname;
Move some tests into the test subdirectory. Now testcrypt has _two_ header files, that's more files than I want at the top level, so I decided to move it. It has a good claim to live in either 'test' or 'crypto', but in the end I decided it wasn't quite specific enough to crypto (it already also tests things in keygen and proxy), and also, the Python half of the mechanism already lives in 'test', so it can live alongside that. Having done that, it seemed silly to leave testsc and testzlib at the top level: those have 'test' in the names as well, so they can go in the test subdir as well. While I'm renaming, also renamed testcrypt.h to testcrypt-func.h to distinguish it from the new testcrypt-enum.h.
2021-11-22 18:50:12 +00:00
#include "testcrypt-func.h"
testcrypt.h: invent FUNC_WRAPPED. FUNC_WRAPPED is an alternative keyword to FUNC which you can use to introduce a function specification in testcrypt.h, indicating that the function is _not_ the one of the same name used in the main PuTTY code, but instead a wrapper on it in testcrypt.c whose API was reworked to be more friendly to translation into Python. There are a lot of those wrappers already, and previously they passed without comment in testcrypt.h, and were put into service by #defining over the top of each name before expanding the marshalling functions. Now, all those #defines are gone, because the use of FUNC_WRAPPED in testcrypt.h is enough to clue in the marshalling wrapper to be generated with a call to foo_wrapper() instead of foo(). Mostly the purpose of this is to make testcrypt.h a bit more self-documenting: if you see FUNC_WRAPPED, you know not to be confused by the Python and C function definitions totally failing to match.
2021-11-21 13:03:34 +00:00
#undef FUNC_INNER
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
#undef ARG
#undef VOID
#define ARG(type, arg) _args.arg = get_##type(_in)
#define VOID ((void)0)
testcrypt-funcs.h: remove extra parens round argument lists. They were there to work around that annoying feature of VS's preprocessor when it expands __VA_ARGS__ into the argument list of another macro. But I've just thought of a workaround that I can apply in testcrypt.c itself, so that those parens don't have to appear in every function definition in the header file. The trick is, instead of writing destination_macro(__VA_ARGS__) you instead write JUXTAPOSE(destination_macro, (__VA_ARGS__)) where JUXTAPOSE is defined to be a macro that simply expands its two arguments next to each other: #define JUXTAPOSE(first, second) first second This works because the arguments to JUXTAPOSE get macro-expanded _before_ passing them to JUXTAPOSE itself - the same reason that the standard tricks with STR_INNER and CAT_INNER work (as seen in defs.h here). So this defuses the magic behaviour of commas expanded from __VA_ARGS__, and causes the destination macro to get all its arguments in the expected places again.
2021-11-28 09:39:49 +00:00
#define FUNC_INNER(outtype, fname, realname, ...) \
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
static inline ARGS_##fname get_args_##fname(BinarySource *_in) { \
ARGS_##fname _args; \
memset(&_args, 0, sizeof(_args)); \
testcrypt-funcs.h: remove extra parens round argument lists. They were there to work around that annoying feature of VS's preprocessor when it expands __VA_ARGS__ into the argument list of another macro. But I've just thought of a workaround that I can apply in testcrypt.c itself, so that those parens don't have to appear in every function definition in the header file. The trick is, instead of writing destination_macro(__VA_ARGS__) you instead write JUXTAPOSE(destination_macro, (__VA_ARGS__)) where JUXTAPOSE is defined to be a macro that simply expands its two arguments next to each other: #define JUXTAPOSE(first, second) first second This works because the arguments to JUXTAPOSE get macro-expanded _before_ passing them to JUXTAPOSE itself - the same reason that the standard tricks with STR_INNER and CAT_INNER work (as seen in defs.h here). So this defuses the magic behaviour of commas expanded from __VA_ARGS__, and causes the destination macro to get all its arguments in the expected places again.
2021-11-28 09:39:49 +00:00
__VA_ARGS__; \
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
return _args; \
Extend testcrypt macros up to 12-ary functions. I'm about to add a monster new function that takes 12 arguments.
2021-11-20 14:28:36 +00:00
}
Move some tests into the test subdirectory. Now testcrypt has _two_ header files, that's more files than I want at the top level, so I decided to move it. It has a good claim to live in either 'test' or 'crypto', but in the end I decided it wasn't quite specific enough to crypto (it already also tests things in keygen and proxy), and also, the Python half of the mechanism already lives in 'test', so it can live alongside that. Having done that, it seemed silly to leave testsc and testzlib at the top level: those have 'test' in the names as well, so they can go in the test subdir as well. While I'm renaming, also renamed testcrypt.h to testcrypt-func.h to distinguish it from the new testcrypt-enum.h.
2021-11-22 18:50:12 +00:00
#include "testcrypt-func.h"
testcrypt.h: invent FUNC_WRAPPED. FUNC_WRAPPED is an alternative keyword to FUNC which you can use to introduce a function specification in testcrypt.h, indicating that the function is _not_ the one of the same name used in the main PuTTY code, but instead a wrapper on it in testcrypt.c whose API was reworked to be more friendly to translation into Python. There are a lot of those wrappers already, and previously they passed without comment in testcrypt.h, and were put into service by #defining over the top of each name before expanding the marshalling functions. Now, all those #defines are gone, because the use of FUNC_WRAPPED in testcrypt.h is enough to clue in the marshalling wrapper to be generated with a call to foo_wrapper() instead of foo(). Mostly the purpose of this is to make testcrypt.h a bit more self-documenting: if you see FUNC_WRAPPED, you know not to be confused by the Python and C function definitions totally failing to match.
2021-11-21 13:03:34 +00:00
#undef FUNC_INNER
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
#undef ARG
#undef VOID
#define ARG(type, arg) _args.arg
#define VOID
testcrypt-funcs.h: remove extra parens round argument lists. They were there to work around that annoying feature of VS's preprocessor when it expands __VA_ARGS__ into the argument list of another macro. But I've just thought of a workaround that I can apply in testcrypt.c itself, so that those parens don't have to appear in every function definition in the header file. The trick is, instead of writing destination_macro(__VA_ARGS__) you instead write JUXTAPOSE(destination_macro, (__VA_ARGS__)) where JUXTAPOSE is defined to be a macro that simply expands its two arguments next to each other: #define JUXTAPOSE(first, second) first second This works because the arguments to JUXTAPOSE get macro-expanded _before_ passing them to JUXTAPOSE itself - the same reason that the standard tricks with STR_INNER and CAT_INNER work (as seen in defs.h here). So this defuses the magic behaviour of commas expanded from __VA_ARGS__, and causes the destination macro to get all its arguments in the expected places again.
2021-11-28 09:39:49 +00:00
#define FUNC_INNER(outtype, fname, realname, ...) \
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
static void handle_##fname(BinarySource *_in, strbuf *_out) { \
ARGS_##fname _args = get_args_##fname(_in); \
(void)_args; /* suppress warning if no actual arguments */ \
testcrypt-funcs.h: remove extra parens round argument lists. They were there to work around that annoying feature of VS's preprocessor when it expands __VA_ARGS__ into the argument list of another macro. But I've just thought of a workaround that I can apply in testcrypt.c itself, so that those parens don't have to appear in every function definition in the header file. The trick is, instead of writing destination_macro(__VA_ARGS__) you instead write JUXTAPOSE(destination_macro, (__VA_ARGS__)) where JUXTAPOSE is defined to be a macro that simply expands its two arguments next to each other: #define JUXTAPOSE(first, second) first second This works because the arguments to JUXTAPOSE get macro-expanded _before_ passing them to JUXTAPOSE itself - the same reason that the standard tricks with STR_INNER and CAT_INNER work (as seen in defs.h here). So this defuses the magic behaviour of commas expanded from __VA_ARGS__, and causes the destination macro to get all its arguments in the expected places again.
2021-11-28 09:39:49 +00:00
return_##outtype(_out, JUXTAPOSE2(realname, (__VA_ARGS__))); \
Extend testcrypt macros up to 12-ary functions. I'm about to add a monster new function that takes 12 arguments.
2021-11-20 14:28:36 +00:00
}
Move some tests into the test subdirectory. Now testcrypt has _two_ header files, that's more files than I want at the top level, so I decided to move it. It has a good claim to live in either 'test' or 'crypto', but in the end I decided it wasn't quite specific enough to crypto (it already also tests things in keygen and proxy), and also, the Python half of the mechanism already lives in 'test', so it can live alongside that. Having done that, it seemed silly to leave testsc and testzlib at the top level: those have 'test' in the names as well, so they can go in the test subdir as well. While I'm renaming, also renamed testcrypt.h to testcrypt-func.h to distinguish it from the new testcrypt-enum.h.
2021-11-22 18:50:12 +00:00
#include "testcrypt-func.h"
testcrypt.h: invent FUNC_WRAPPED. FUNC_WRAPPED is an alternative keyword to FUNC which you can use to introduce a function specification in testcrypt.h, indicating that the function is _not_ the one of the same name used in the main PuTTY code, but instead a wrapper on it in testcrypt.c whose API was reworked to be more friendly to translation into Python. There are a lot of those wrappers already, and previously they passed without comment in testcrypt.h, and were put into service by #defining over the top of each name before expanding the marshalling functions. Now, all those #defines are gone, because the use of FUNC_WRAPPED in testcrypt.h is enough to clue in the marshalling wrapper to be generated with a call to foo_wrapper() instead of foo(). Mostly the purpose of this is to make testcrypt.h a bit more self-documenting: if you see FUNC_WRAPPED, you know not to be confused by the Python and C function definitions totally failing to match.
2021-11-21 13:03:34 +00:00
#undef FUNC_INNER
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
#undef ARG
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
static void process_line(BinarySource *in, strbuf *out)
{
ptrlen id = get_word(in);
Fix technical-UB uses of the preprocessor. A recent test-compile at high warning level points out that if you define a macro with a ... at the end of the parameter list, then every call should at least include the comma before the variadic part. That is, if you #define MACRO(x,y,...) then you shouldn't call MACRO(1,2) with no comma after the 2. But that's what I had done in one of my definitions of FUNC0 in the fiddly testcrypt system. In a similar vein, it's a mistake to use the preprocessor 'defined' operator when it's expanded from another macro. Adjusted the setup of BB_OK in mpint_i.h to avoid doing that. (Neither of these has yet caused a problem in any real compile, but best to fix them before they do.)
2020-01-29 06:13:41 +00:00
#define DISPATCH_INTERNAL(cmdname, handler) do { \
if (ptrlen_eq_string(id, cmdname)) { \
handler(in, out); \
return; \
} \
} while (0)
#define DISPATCH_COMMAND(cmd) DISPATCH_INTERNAL(#cmd, handle_##cmd)
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
DISPATCH_COMMAND(hello);
DISPATCH_COMMAND(free);
DISPATCH_COMMAND(newstring);
DISPATCH_COMMAND(getstring);
DISPATCH_COMMAND(mp_literal);
DISPATCH_COMMAND(mp_dump);
testcrypt: introduce and use 'checkenum' protocol query. This allows the Python side of testcrypt to check in advance if a given string is a valid element of an enumeration, and if not, cleanly throw a Python-level exception without terminating the testcrypt subprocess. Should be useful in both manual use (when I'm trying something out by hand and make a typo or misremember a spelling), and automated use (if I make the same kind of error in cryptsuite.py then the exception dump will make more sense). In order to do this, the new handle_checkenum() function has to recognise all the enumerated types by name and match them up to their lookup functions - which is just the kind of thing that can now be done easily be reincluding testcrypt-enum.h with different #defines.
2021-11-22 18:39:45 +00:00
DISPATCH_COMMAND(checkenum);
Fix technical-UB uses of the preprocessor. A recent test-compile at high warning level points out that if you define a macro with a ... at the end of the parameter list, then every call should at least include the comma before the variadic part. That is, if you #define MACRO(x,y,...) then you shouldn't call MACRO(1,2) with no comma after the 2. But that's what I had done in one of my definitions of FUNC0 in the fiddly testcrypt system. In a similar vein, it's a mistake to use the preprocessor 'defined' operator when it's expanded from another macro. Adjusted the setup of BB_OK in mpint_i.h to avoid doing that. (Neither of these has yet caused a problem in any real compile, but best to fix them before they do.)
2020-01-29 06:13:41 +00:00
#undef DISPATCH_COMMAND
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
testcrypt-funcs.h: remove extra parens round argument lists. They were there to work around that annoying feature of VS's preprocessor when it expands __VA_ARGS__ into the argument list of another macro. But I've just thought of a workaround that I can apply in testcrypt.c itself, so that those parens don't have to appear in every function definition in the header file. The trick is, instead of writing destination_macro(__VA_ARGS__) you instead write JUXTAPOSE(destination_macro, (__VA_ARGS__)) where JUXTAPOSE is defined to be a macro that simply expands its two arguments next to each other: #define JUXTAPOSE(first, second) first second This works because the arguments to JUXTAPOSE get macro-expanded _before_ passing them to JUXTAPOSE itself - the same reason that the standard tricks with STR_INNER and CAT_INNER work (as seen in defs.h here). So this defuses the magic behaviour of commas expanded from __VA_ARGS__, and causes the destination macro to get all its arguments in the expected places again.
2021-11-28 09:39:49 +00:00
#define FUNC_INNER(outtype, fname, realname, ...) \
testcrypt.h: invent FUNC_WRAPPED. FUNC_WRAPPED is an alternative keyword to FUNC which you can use to introduce a function specification in testcrypt.h, indicating that the function is _not_ the one of the same name used in the main PuTTY code, but instead a wrapper on it in testcrypt.c whose API was reworked to be more friendly to translation into Python. There are a lot of those wrappers already, and previously they passed without comment in testcrypt.h, and were put into service by #defining over the top of each name before expanding the marshalling functions. Now, all those #defines are gone, because the use of FUNC_WRAPPED in testcrypt.h is enough to clue in the marshalling wrapper to be generated with a call to foo_wrapper() instead of foo(). Mostly the purpose of this is to make testcrypt.h a bit more self-documenting: if you see FUNC_WRAPPED, you know not to be confused by the Python and C function definitions totally failing to match.
2021-11-21 13:03:34 +00:00
DISPATCH_INTERNAL(#fname,handle_##fname);
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
#define ARG1(type, arg)
#define ARGN(type, arg)
#define VOID
Move some tests into the test subdirectory. Now testcrypt has _two_ header files, that's more files than I want at the top level, so I decided to move it. It has a good claim to live in either 'test' or 'crypto', but in the end I decided it wasn't quite specific enough to crypto (it already also tests things in keygen and proxy), and also, the Python half of the mechanism already lives in 'test', so it can live alongside that. Having done that, it seemed silly to leave testsc and testzlib at the top level: those have 'test' in the names as well, so they can go in the test subdir as well. While I'm renaming, also renamed testcrypt.h to testcrypt-func.h to distinguish it from the new testcrypt-enum.h.
2021-11-22 18:50:12 +00:00
#include "testcrypt-func.h"
testcrypt.h: invent FUNC_WRAPPED. FUNC_WRAPPED is an alternative keyword to FUNC which you can use to introduce a function specification in testcrypt.h, indicating that the function is _not_ the one of the same name used in the main PuTTY code, but instead a wrapper on it in testcrypt.c whose API was reworked to be more friendly to translation into Python. There are a lot of those wrappers already, and previously they passed without comment in testcrypt.h, and were put into service by #defining over the top of each name before expanding the marshalling functions. Now, all those #defines are gone, because the use of FUNC_WRAPPED in testcrypt.h is enough to clue in the marshalling wrapper to be generated with a call to foo_wrapper() instead of foo(). Mostly the purpose of this is to make testcrypt.h a bit more self-documenting: if you see FUNC_WRAPPED, you know not to be confused by the Python and C function definitions totally failing to match.
2021-11-21 13:03:34 +00:00
#undef FUNC_INNER
Rewrite the testcrypt.c macro system. Yesterday's commit 52ee636b092c199 which further extended the huge pile of arity-specific annoying wrapper macros pushed me over the edge and inspired me to give some harder thought to finding a way to handle all arities at once. And this time I found one! The new technique changes the syntax of the function specifications in testcrypt.h. In particular, they now have to specify a _name_ for each parameter as well as a type, because the macros generating the C marshalling wrappers will need a structure field for each parameter and cpp isn't flexible enough to generate names for those fields automatically. Rather than tediously name them arg1, arg2 etc, I've reused the names of the parameters from the prototypes or definitions of the underlying real functions (via a one-off auto-extraction process starting from the output of 'clang -Xclang -dump-ast' plus some manual polishing), which means testcrypt.h is now a bit more self-documenting. The testcrypt.py end of the mechanism is rewritten to eat the new format. Since it's got more complicated syntax and nested parens and things, I've written something a bit like a separated lexer/parser system in place of the previous crude regex matcher, which should enforce that the whole header file really does conform to the restricted syntax it has to fit into. The new system uses a lot less code in testcrypt.c, but I've made up for that by also writing a long comment explaining how it works, which was another thing the previous system lacked! Similarly, the new testcrypt.h has some long-overdue instructions at the top.
2021-11-21 10:27:30 +00:00
#undef ARG
#undef VOID
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
Fix technical-UB uses of the preprocessor. A recent test-compile at high warning level points out that if you define a macro with a ... at the end of the parameter list, then every call should at least include the comma before the variadic part. That is, if you #define MACRO(x,y,...) then you shouldn't call MACRO(1,2) with no comma after the 2. But that's what I had done in one of my definitions of FUNC0 in the fiddly testcrypt system. In a similar vein, it's a mistake to use the preprocessor 'defined' operator when it's expanded from another macro. Adjusted the setup of BB_OK in mpint_i.h to avoid doing that. (Neither of these has yet caused a problem in any real compile, but best to fix them before they do.)
2020-01-29 06:13:41 +00:00
#undef DISPATCH_INTERNAL
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
fatal_error("command '%.*s': unrecognised", PTRLEN_PRINTF(id));
}
static void free_all_values(void)
{
for (Value *val; (val = delpos234(values, 0)) != NULL ;)
free_value(val);
freetree234(values);
}
testcrypt: include a dputs() function. This allows me to compile testcrypt with -DDEBUG, even though it's not linked against the usual collection of platform-specific modules that normally provide dputs. I think the simplest possible dputs ('just output to stderr') is actually better for testcrypt, because that keeps it easy to compile for strange experimental platforms.
2019-01-20 21:44:51 +00:00
void dputs(const char *buf)
{
fputs(buf, stderr);
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
int main(int argc, char **argv)
{
testcrypt: add an -o option. This enables me to control where testcrypt both reads its input and writes its output. That in turn makes it convenient to run testcrypt itself in a separate Unix terminal window from its client Python, by making two named pipe files (say, 'i' and 'o'), running the client with PUTTY_TESTCRYPT="cat o & cat > i" in its environment, and in another window, running 'testcrypt -o o i'. And that in turn makes it easy to attach gdb to testcrypt, so I can easily debug its handling of whatever request the client sent.
2019-01-06 21:44:57 +00:00
const char *infile = NULL, *outfile = NULL;
bool doing_opts = true;
while (--argc > 0) {
char *p = *++argv;
if (p[0] == '-' && doing_opts) {
if (!strcmp(p, "-o")) {
if (--argc <= 0) {
fprintf(stderr, "'-o' expects a filename\n");
return 1;
}
outfile = *++argv;
} else if (!strcmp(p, "--")) {
doing_opts = false;
} else if (!strcmp(p, "--help")) {
printf("usage: testcrypt [INFILE] [-o OUTFILE]\n");
printf(" also: testcrypt --help display this text\n");
return 0;
} else {
fprintf(stderr, "unknown command line option '%s'\n", p);
return 1;
}
} else if (!infile) {
infile = p;
} else {
fprintf(stderr, "can only handle one input file name\n");
return 1;
}
}
FILE *infp = stdin;
if (infile) {
infp = fopen(infile, "r");
if (!infp) {
fprintf(stderr, "%s: open: %s\n", infile, strerror(errno));
return 1;
}
}
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
testcrypt: add an -o option. This enables me to control where testcrypt both reads its input and writes its output. That in turn makes it convenient to run testcrypt itself in a separate Unix terminal window from its client Python, by making two named pipe files (say, 'i' and 'o'), running the client with PUTTY_TESTCRYPT="cat o & cat > i" in its environment, and in another window, running 'testcrypt -o o i'. And that in turn makes it easy to attach gdb to testcrypt, so I can easily debug its handling of whatever request the client sent.
2019-01-06 21:44:57 +00:00
FILE *outfp = stdout;
if (outfile) {
outfp = fopen(outfile, "w");
if (!outfp) {
fprintf(stderr, "%s: open: %s\n", outfile, strerror(errno));
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
return 1;
}
}
values = newtree234(valuecmp);
atexit(free_all_values);
testcrypt: add an -o option. This enables me to control where testcrypt both reads its input and writes its output. That in turn makes it convenient to run testcrypt itself in a separate Unix terminal window from its client Python, by making two named pipe files (say, 'i' and 'o'), running the client with PUTTY_TESTCRYPT="cat o & cat > i" in its environment, and in another window, running 'testcrypt -o o i'. And that in turn makes it easy to attach gdb to testcrypt, so I can easily debug its handling of whatever request the client sent.
2019-01-06 21:44:57 +00:00
for (char *line; (line = chomp(fgetline(infp))) != NULL ;) {
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
BinarySource src[1];
BinarySource_BARE_INIT(src, line, strlen(line));
strbuf *sb = strbuf_new();
process_line(src, sb);
run_finalisers(sb);
size_t lines = 0;
for (size_t i = 0; i < sb->len; i++)
if (sb->s[i] == '\n')
lines++;
New wrapper macro for printf("%zu"), for old VS compat. A user reports that Visual Studio 2013 and earlier have printf implementations in their C library that don't support the 'z' modifier to indicate that an integer argument is size_t. The 'I' modifier apparently works in place of it. To avoid littering ifdefs everywhere, I've invented my own inttypes.h style macros to wrap size_t formatting directives, which are defined to %zu and %zx normally, or %Iu and %Ix in old-VS mode. Those are in defs.h, and they're used everywhere that a %z might otherwise get into the Windows build.
2020-01-26 10:59:07 +00:00
fprintf(outfp, "%"SIZEu"\n%s", lines, sb->s);
testcrypt: add an -o option. This enables me to control where testcrypt both reads its input and writes its output. That in turn makes it convenient to run testcrypt itself in a separate Unix terminal window from its client Python, by making two named pipe files (say, 'i' and 'o'), running the client with PUTTY_TESTCRYPT="cat o & cat > i" in its environment, and in another window, running 'testcrypt -o o i'. And that in turn makes it easy to attach gdb to testcrypt, so I can easily debug its handling of whatever request the client sent.
2019-01-06 21:44:57 +00:00
fflush(outfp);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
strbuf_free(sb);
sfree(line);
}
testcrypt: add an -o option. This enables me to control where testcrypt both reads its input and writes its output. That in turn makes it convenient to run testcrypt itself in a separate Unix terminal window from its client Python, by making two named pipe files (say, 'i' and 'o'), running the client with PUTTY_TESTCRYPT="cat o & cat > i" in its environment, and in another window, running 'testcrypt -o o i'. And that in turn makes it easy to attach gdb to testcrypt, so I can easily debug its handling of whatever request the client sent.
2019-01-06 21:44:57 +00:00
if (infp != stdin)
fclose(infp);
if (outfp != stdin)
fclose(outfp);
New test system for mp_int and cryptography. I've written a new standalone test program which incorporates all of PuTTY's crypto code, including the mp_int and low-level elliptic curve layers but also going all the way up to the implementations of the MAC, hash, cipher, public key and kex abstractions. The test program itself, 'testcrypt', speaks a simple line-oriented protocol on standard I/O in which you write the name of a function call followed by some inputs, and it gives you back a list of outputs preceded by a line telling you how many there are. Dynamically allocated objects are assigned string ids in the protocol, and there's a 'free' function that tells testcrypt when it can dispose of one. It's possible to speak that protocol by hand, but cumbersome. I've also provided a Python module that wraps it, by running testcrypt as a persistent subprocess and gatewaying all the function calls into things that look reasonably natural to call from Python. The Python module and testcrypt.c both read a carefully formatted header file testcrypt.h which contains the name and signature of every exported function, so it costs minimal effort to expose a given function through this test API. In a few cases it's necessary to write a wrapper in testcrypt.c that makes the function look more friendly, but mostly you don't even need that. (Though that is one of the motivations between a lot of API cleanups I've done recently!) I considered doing Python integration in the more obvious way, by linking parts of the PuTTY code directly into a native-code .so Python module. I decided against it because this way is more flexible: I can run the testcrypt program on its own, or compile it in a way that Python wouldn't play nicely with (I bet compiling just that .so with Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or attach a debugger to it. I can even recompile testcrypt for a different CPU architecture (32- vs 64-bit, or even running it on a different machine over ssh or under emulation) and still layer the nice API on top of that via the local Python interpreter. All I need is a bidirectional data channel.
2019-01-01 19:08:37 +00:00
return 0;
}
Reference in New Issue Copy Permalink