diff --git a/tests/openpgp/README b/tests/openpgp/README index d85ff3f07..b7ec6f784 100644 --- a/tests/openpgp/README +++ b/tests/openpgp/README @@ -1,257 +1,257 @@ # Emacs, this is an -*- org -*- file. * How to run the test suite ** tldr: How to run all tests fast. obj $ make check-all TESTFLAGS=--parallel You can use --parallel=N to request N parallel jobs. Hint: Tuck TESTFLAGS=--parallel in your environment. ** Running individual test suites or tests From your build directory, run obj $ make -C tests/openpgp check to run all tests or obj $ make -C tests/openpgp check TESTS=your-test.scm to run a specific test (or any number of tests separated by spaces). If you want to debug a test, add verbose=1 to see messages printed by spawned programs to their standard error stream, verbose=2 to see what programs are executed, or verbose=3 to see even more program output and exit codes. If you want to run gpg under valgrind add with_valgrind=1. ** Inspecting the test environment To inspect the environment in which tests are running, or to quickly create keys for debugging or testing, you can start a shell. There is one test that does just that: obj $ make -C tests/openpgp check TESTS=shell.scm PASS: tests/openpgp/setup.scm Load legacy test environment? [Y/n] y Drop 'batch' from gpg.conf? [Y/n] y Enjoy your test environment. Type 'exit' to exit it, it will be cleaned up after you. ... $ gpg -k Alfa gpg: NOTE: THIS IS A DEVELOPMENT VERSION! gpg: It is only intended for test purposes and should NOT be gpg: used in a production environment or with production keys! gpg: /tmp/gpgscm-20170809T144032-run-tests-PFfybw/trustdb.gpg: trustdb created pub dsa1024 1999-03-08 [SCA] A0FF4590BB6122EDEF6E3C542D727CC768697734 uid [ unknown] Alfa Test (demo key) uid [ unknown] Alpha Test (demo key) uid [ unknown] Alice (demo key) sub elg1024 1999-03-08 [E] PATH is adjusted so that you will use the tools from the build tree. Note that the directory is removed when you exit the shell. ** Passing options to the test driver You can set TESTFLAGS to pass flags to 'run-tests.scm'. For example, to speed up the test suite when bisecting, do obj $ make -C tests/openpgp check TESTFLAGS=--parallel See below for the arguments supported by the driver. ** Calling the test driver directly This is a bit tricky because one needs to manually set some environment variables. We should make that easier. See discussion below. From your build directory, do: obj $ srcdir=/tests/openpgp \ GPGSCM_PATH=/tests/gpgscm:/tests/openpgp \ $(pwd)/tests/gpgscm/gpgscm [gpgscm args] \ run-tests.scm [test suite runner args] *** Arguments supported by the test suite runner The test suite runner supports two modes of operation, '--sequential' and '--parallel'. By default the tests are run in sequential order, each one in a clean environment. You can specify the tests to run as positional arguments relative to srcdir (e.g. just 'version.scm'). Note that you do not have to specify setup.scm and finish.scm, they are executed implicitly. The test suite runner can be executed in any location that the current user can write to. It will create temporary files and directories, but will in general clean up all of them. *** Discussion of the various environment variables **** srcdir Must be set to the source of the openpgp test suite. Used to locate data files. **** GPGSCM_PATH Used to locate the Scheme library as well as code used by the test suite. -**** BIN_PREFIX -The test suite does not hardcode any paths to tools. If set it is -used to locate the tools to test, otherwise the test suite assumes to -be run from the build directory. -**** GPG_PRESET_PASSPHRASE -This tool is not installed by 'make install', hence we need to -explicitly override its position. In fact, the location of any tool -used by the test suite can be overridden this way. See defs.scm. +**** GNUPG_BUILD_ROOT +To locate the actual binaries under test the test suite requires that +those binaries and associate files are installed to a test location. +This envvar gives the root directory of the install tree. See +tests/gpgconf.ctl for the way we tell the GnuPG components this +location. Note that we can't use that envvar directlyr because this +would allow user scripts and other software to accidently mess up the +used components. **** argv[0] run-tests.scm depends on being able to re-exec gpgscm. It uses argv[0] for that. Therefore you must use an absolute path to invoke gpgscm. * How to write tests gpgscm provides a number of functions to aid you in writing tests, as well as bindings to process management abstractions provided by GnuPG. For the Scheme environment provided by TinySCHEME, see the TinySCHEME manual that is included in tests/gpgscm/Manual.txt. For a quick start, please have a look at various tests that are already implemented, e.g. 'encrypt.scm'. ** The test framework The functions info, error, and skip display their first argument and flush the output buffers. error and skip will also terminate the process, signaling that the test failed or should be skipped. (for-each-p msg proc list) will display msg, and call proc with each element of list while displaying the progress appropriately. for-each-p' is similar, but accepts another callback before the 'list' argument to format each item. for-each-p can be safely nested, and the inner progress indicator will be abbreviated using '.'. ** Debugging tests Say you are working on a new test called 'your-test.scm', you can run it on its own using obj $ make -C tests/openpgp check TESTS=your-test.scm but something isn't working as expected. There are several little gadgets that might help. The first one is 'trace', a function that prints the value given to it and evaluates to it. E.g. (trace (+ 2 3)) prints '5' and evaluates to 5. Also, there is an 'assert' macro that aborts the execution if its argument does not evaluate to a trueish value. Feel free to express invariants with it. You can also get an interactive repl by dropping (interactive-repl (current-environment)) anywhere you like. Or, if you want to examine the environment from an operating system shell, use (interactive-shell) ** Interfacing with gpg defs.scm defines several convenience functions. Say you want to parse the colon output from gpg, there is gpg-with-colons that splits the result at newlines and colons, so you can use the result like this: (define (fpr some-key) (list-ref (assoc "fpr" (gpg-with-colons `(--with-fingerprint --list-secret-keys ,some-key))) 9)) Or if you want to count all non-revoked uids for a given key, do (define (count-uids-of-secret-key some-key) (length (filter (lambda (x) (and (string=? "uid" (car x)) (string=? "u" (cadr x)))) (gpg-with-colons `(--with-fingerprint --list-secret-keys ,some-key))))) ** Temporary files (lettmp ) will create and delete temporary files that you can use in . (with-temporary-working-directory ) will create a temporary director, change to that, and clean it up after executing ). make-temporary-file will create a temporary file. You can optionally provide an argument to that function that will serve as tag so you can distinguish the files for debugging. remove-temporary-file will delete a file created using make-temporary-file. ** Monadic transformer and pipe support Tests often perform sequential transformations on files, or connect processes using pipes. To aid you in this, the test framework provides two monadic data structures. (Currently, the implementation mashes the 'bind' operation together with the application of the monad. Also, there is no 'return' operation. I guess all of that could be implemented on top of call/cc, but it isn't at the moment.) *** pipe The pipe monad constructs pipe lines. It consists of a function pipe:do that binds the functions together and manages the execution of the child processes, a family of functions that act as sources, a function to spawn processes, and a family of functions acting as sinks. Sources are pipe:open, pipe:defer, pipe:echo. To spawn a process use pipe:spawn, or the convenience function pipe:gpg. To sink the data use pipe:splice, or pipe:write-to. Example: (pipe:do (pipe:echo "3\n1\n2\n") (pipe:spawn '("/usr/bin/sort")) (pipe:write-to "sorted" (logior O_WRONLY O_CREAT) #o600)) Caveats: Due to the single-threaded nature of gpgscm you cannot use both a source and sink that is implemented in Scheme. pipe:defer and pipe:echo are executing in gpgscm, and so does pipe:splice. *** tr The transformer monad describes sequential file transformations. There is one source function, tr:open. To describe a transformation using some process, use tr:spawn, tr:gpg, or tr:pipe-do. There are several sinks, although sink is not quite the right term, because the data is not consumed, and hence one can use them at any position. The "sinks" are tr:write-to, tr:call-with-content, tr:assert-identity, and tr:assert-weak-identity. A somewhat contrived example demonstrating many functions is: (tr:do (tr:pipe-do (pipe:echo "3\n1\n2\n") (pipe:spawn '("/usr/bin/sort"))) (tr:write-to "reference") (tr:call-with-content (lambda (c) (echo "currently, c contains" (string-length c) "bytes"))) (tr:spawn "" '("/usr/bin/gcc" -x c "-E" -o **out** **in**)) (tr:pipe-do (pipe:spawn '("/bin/grep" -v "#"))) (tr:assert-identity "reference")) Caveats: As a convenience, gpgscm allows one to specify command line arguments as Scheme symbols. Scheme symbols, however, are case-insensitive, and get converted to lower case. Therefore, the -E argument must be given as a string in the example above. Similarly, you need to quote numerical values. ** Process management If you just need to execute a single command, there is (call-with-fds cmdline infd outfd errfd) which executes cmdline with the given file descriptors bound to it, and waits for its completion returning the status code. There is (call cmdline) which is similar, but calls the command with a closed stdin, connecting stdout and stderr to stderr if gpgscm is executed with --verbose. (call-check cmdline) raises an exception if the command does not return 0. (call-popen cmdline input) calls a command, writes input to its stdin, and returns any output from stdout, or raises an exception containing stderr on failure. * Sample messages diff --git a/tests/openpgp/defs.scm b/tests/openpgp/defs.scm index 86d312f82..61e3fdbda 100644 --- a/tests/openpgp/defs.scm +++ b/tests/openpgp/defs.scm @@ -1,524 +1,520 @@ ;; Common definitions for the OpenPGP test scripts. ;; ;; Copyright (C) 2016, 2017 g10 Code GmbH ;; ;; This file is part of GnuPG. ;; ;; GnuPG is free software; you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation; either version 3 of the License, or ;; (at your option) any later version. ;; ;; GnuPG is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public License ;; along with this program; if not, see . ;; ;; Constants. ;; (define usrname1 "one@example.com") (define usrpass1 "def") (define usrname2 "two@example.com") (define usrpass2 "") (define usrname3 "three@example.com") (define usrpass3 "") (define dsa-usrname1 "pgp5") ;; we use the sub key because we do not yet have the logic to derive ;; the first encryption key from a keyblock (I guess) (Well of course ;; we have this by now and the notation below will lookup the primary ;; first and then search for the encryption subkey.) (define dsa-usrname2 "0xCB879DE9") (define keys (package (define (new fpr grip uids subkeys) (package)) (define (subkey fpr grip) (package)) (define alfa (new "A0FF4590BB6122EDEF6E3C542D727CC768697734" "76F7E2B35832976B50A27A282D9B87E44577EB66" '("alfa@example.net" "alpha@example.net") (list (subkey "3B3FBC948FE59301ED629EFB6AE6D7EE46A871F8" "A0747D5F9425E6664F4FFBEED20FBCA79FDED2BD")))) (define one (new "289B0EF1D105E124B6F626020EF77096D74C5F22" "50B2D4FA4122C212611048BC5FC31BD44393626E" '("one@example.com") (list (subkey "EB467DCA4AD7676A6A62B2ABABAB28A247BE2775" "7E201E28B6FEB2927B321F443205F4724EBE637E")))) (define two (new "C1DEBB34EA8B71009EAFA474973D50E1C40FDECF" "343D8AF79796EE107D645A2787A9D9252F924E6F" '("two@example.com") (list (subkey "CD3D0F5701CBFCACB2A4907305A37887B27907AA" "8B5ABF3EF9EB8D96B91A0B8C2C4401C91C834C34")))))) (define key-file1 "samplekeys/rsa-rsa-sample-1.asc") (define key-file2 "samplekeys/ed25519-cv25519-sample-1.asc") (define plain-files '("plain-1" "plain-2" "plain-3" "plain-large")) (define data-files '("data-500" "data-9000" "data-32000" "data-80000")) (define exp-files '()) (define all-files (append plain-files data-files)) (let ((verbose (string->number (getenv "verbose")))) (if (number? verbose) (*set-verbose!* verbose))) (define (qualify executable) (string-append executable (getenv "EXEEXT"))) (define (getenv' key default) (let ((value (getenv key))) (if (string=? "" value) default value))) (define (percent-decode s) (define (decode c) (if (and (> (length c) 2) (char=? #\% (car c))) (integer->char (string->number (string #\# #\x (cadr c) (caddr c)))) #f)) (let loop ((i 0) (c (string->list s)) (r (make-string (string-length s)))) (if (null? c) (substring r 0 i) (let ((decoded (decode c))) (string-set! r i (if decoded decoded (car c))) (loop (+ 1 i) (if decoded (cdddr c) (cdr c)) r))))) (assert (equal? (percent-decode "") "")) (assert (equal? (percent-decode "%61") "a")) (assert (equal? (percent-decode "foob%61r") "foobar")) (define (percent-encode s) (define (encode c) `(#\% ,@(string->list (number->string (char->integer c) 16)))) (let loop ((acc '()) (cs (reverse (string->list s)))) (if (null? cs) (list->string acc) (case (car cs) ((#\: #\%) (loop (append (encode (car cs)) acc) (cdr cs))) (else (loop (cons (car cs) acc) (cdr cs))))))) (assert (equal? (percent-encode "") "")) (assert (equal? (percent-encode "%61") "%2561")) (assert (equal? (percent-encode "foob%61r") "foob%2561r")) +;; Note that the entry for pinentry relies on the fact that +;; GNUPG_BUILD_ROOT has the bin,libexec,share directories (where we +;; have installed versions of the tools under test) as well as the +;; openpgp directory. The second element in each list is an envvar which +;; can be used to specifiy a different tool than the installed one. (define tools - '((gpgv "GPGV" "g10/gpgv") - (gpg-connect-agent "GPG_CONNECT_AGENT" "tools/gpg-connect-agent") - (gpgconf "GPGCONF" "tools/gpgconf") + '((gpgv "GPGV" "bin/gpgv") + (gpg-connect-agent "GPG_CONNECT_AGENT" "bin/gpg-connect-agent") + (gpgconf "GPGCONF" "bin/gpgconf") (gpg-preset-passphrase "GPG_PRESET_PASSPHRASE" - "agent/gpg-preset-passphrase") - (gpgtar "GPGTAR" "tools/gpgtar") - (gpg-zip "GPGZIP" "tools/gpg-zip") - (pinentry "PINENTRY" "tests/openpgp/fake-pinentry"))) - -(define bin-prefix (getenv "BIN_PREFIX")) -(define installed? (not (string=? "" bin-prefix))) -(define with-valgrind? (not (string=? (getenv "with_valgrind") ""))) + "libexec/gpg-preset-passphrase") + (gpgtar "GPGTAR" "bin/gpgtar") + (pinentry "PINENTRY" "openpgp/fake-pinentry"))) (define (tool-hardcoded which) (let ((t (assoc which tools))) (getenv' (cadr t) - (qualify (if installed? - (string-append bin-prefix "/" (basename (caddr t))) - (string-append (getenv "objdir") "/" (caddr t))))))) + (qualify (string-append (getenv "GNUPG_BUILD_ROOT") + "/" (caddr t)))))) + +(define with-valgrind? (not (string=? (getenv "with_valgrind") ""))) ;; You can splice VALGRIND into your argument vector to run programs ;; under valgrind. For example, to run valgrind on gpg, you may want ;; to redefine gpg: ;; ;; (set! gpg `(,@valgrind ,@gpg)) ;; (define valgrind '("/usr/bin/valgrind" -q --leak-check=no --track-origins=yes --error-exitcode=154 --exit-on-first-error=yes)) -(unless installed? - (setenv "GNUPG_BUILDDIR" (getenv "objdir") #t)) - (define (gpg-conf . args) (gpg-conf' "" args)) (define (gpg-conf' input args) (let ((s (call-popen `(,(tool-hardcoded 'gpgconf) - ,@(if installed? '() - (list '--build-prefix (getenv "objdir"))) ,@args) input))) (map (lambda (line) (map percent-decode (string-split line #\:))) (string-split-newlines s)))) (define :gc:c:name car) (define :gc:c:description cadr) (define :gc:c:pgmname caddr) (define (:gc:o:name x) (list-ref x 0)) (define (:gc:o:flags x) (string->number (list-ref x 1))) (define (:gc:o:level x) (string->number (list-ref x 2))) (define (:gc:o:description x) (list-ref x 3)) (define (:gc:o:type x) (string->number (list-ref x 4))) (define (:gc:o:alternate-type x) (string->number (list-ref x 5))) (define (:gc:o:argument-name x) (list-ref x 6)) (define (:gc:o:default-value x) (list-ref x 7)) (define (:gc:o:default-argument x) (list-ref x 8)) (define (:gc:o:value x) (if (< (length x) 10) "" (list-ref x 9))) (define (gpg-config component key) (package (define (value) (let* ((conf (assoc key (gpg-conf '--list-options component))) (type (:gc:o:type conf)) (value (:gc:o:value conf))) (case type ((0 2 3) (string->number value)) ((1 32) (substring value 1 (string-length value)))))) (define (update value) (let ((value' (cond ((string? value) (string-append "\"" value)) ((number? value) (number->string value)) (else (throw "Unsupported value" value))))) (gpg-conf' (string-append key ":0:" (percent-encode value')) `(--change-options ,component)))) (define (clear) (gpg-conf' (string-append key ":16:") `(--change-options ,component))))) (define gpg-components (apply gpg-conf '(--list-components))) (define (tool which) (case which ((gpg gpg-agent scdaemon gpgsm keyboxd dirmngr) (:gc:c:pgmname (assoc (symbol->string which) gpg-components))) (else (tool-hardcoded which)))) (define (gpg-has-option? option) (string-contains? (call-popen `(,(tool 'gpg) --dump-options) "") option)) (define have-opt-always-trust (catch #f (with-ephemeral-home-directory (lambda ()) (lambda ()) (call-check `(,(tool 'gpg) --gpgconf-test --always-trust))) #t)) (define GPG `(,(tool 'gpg) --no-permission-warning ,@(if have-opt-always-trust '(--always-trust) '()))) (define GPGV `(,(tool 'gpgv))) (define PINENTRY (tool 'pinentry)) (define (tr:gpg input args) (tr:spawn input `(,@GPG --output **out** ,@args **in**))) (define (pipe:gpg args) (pipe:spawn `(,@GPG --output - ,@args))) (define (gpg-with-colons args) (let ((s (call-popen `(,@GPG --with-colons ,@args) ""))) (map (lambda (line) (string-split line #\:)) (string-split-newlines s)))) ;; Convenient accessors for the colon output. (define (:type x) (string->symbol (list-ref x 0))) (define (:length x) (string->number (list-ref x 2))) (define (:alg x) (string->number (list-ref x 3))) (define (:expire x) (list-ref x 6)) (define (:fpr x) (list-ref x 9)) (define (:cap x) (list-ref x 11)) (define (have-public-key? key) (catch #f (pair? (filter (lambda (l) (and (equal? 'fpr (:type l)) (equal? key::fpr (:fpr l)))) (gpg-with-colons `(--list-keys ,key::fpr)))))) (define (have-secret-key? key) (catch #f (pair? (filter (lambda (l) (and (equal? 'fpr (:type l)) (equal? key::fpr (:fpr l)))) (gpg-with-colons `(--list-secret-keys ,key::fpr)))))) (define (have-secret-key-file? key) (file-exists? (path-join (getenv "GNUPGHOME") "private-keys-v1.d" (string-append key::grip ".key")))) (define (get-config what) (string-split (caddar (gpg-with-colons `(--list-config ,what))) #\;)) (define all-pubkey-algos (delay (get-config "pubkeyname"))) (define all-hash-algos (delay (get-config "digestname"))) (define all-cipher-algos (delay (get-config "ciphername"))) (define all-compression-algos (delay (get-config "compressname"))) (define (have-pubkey-algo? x) (not (not (member x (force all-pubkey-algos))))) (define (have-hash-algo? x) (not (not (member x (force all-hash-algos))))) (define (have-cipher-algo? x) (not (not (member x (force all-cipher-algos))))) (define (have-compression-algo? x) (not (not (member x (force all-compression-algos))))) (define (gpg-pipe args0 args1 errfd) (lambda (source sink) (let* ((p (pipe)) (task0 (spawn-process-fd `(,@GPG ,@args0) source (:write-end p) errfd)) (_ (close (:write-end p))) (task1 (spawn-process-fd `(,@GPG ,@args1) (:read-end p) sink errfd))) (close (:read-end p)) (wait-processes (list GPG GPG) (list task0 task1) #t)))) (setenv "GPG_AGENT_INFO" "" #t) (setenv "GNUPGHOME" (getcwd) #t) (define GNUPGHOME (getcwd)) ;; ;; GnuPG helper. ;; ;; Call GPG to obtain the hash sums. Either specify an input file in ;; ARGS, or an string in INPUT. Returns a list of ( ;; "") lists. (define (gpg-hash-string args input) (map (lambda (line) (let ((p (string-split line #\:))) (list (string->number (cadr p)) (caddr p)))) (string-split-newlines (call-popen `(,@GPG --with-colons ,@args) input)))) ;; Dearmor a file. (define (dearmor source-name sink-name) (pipe:do (pipe:open source-name (logior O_RDONLY O_BINARY)) (pipe:spawn `(,@GPG --dearmor)) (pipe:write-to sink-name (logior O_WRONLY O_CREAT O_BINARY) #o600))) (define (gpg-dump-packets source-name sink-name) (pipe:do (pipe:open source-name (logior O_RDONLY O_BINARY)) (pipe:spawn `(,@GPG --list-packets)) (pipe:write-to sink-name (logior O_WRONLY O_CREAT O_BINARY) #o600))) ;; ;; Support for test environment creation and teardown. ;; (define (make-test-data filename size) (call-with-binary-output-file filename (lambda (port) (display (make-random-string size) port)))) (define (create-file name . lines) (catch #f (unlink name)) (letfd ((fd (open name (logior O_WRONLY O_CREAT O_BINARY) #o600))) (let ((port (fdopen fd "wb"))) (for-each (lambda (line) (display line port) (newline port)) lines)))) (define (create-gpghome) (log "Creating test environment...") (srandom (getpid)) (make-test-data "random_seed" 600) (log "Creating configuration files") (if (flag "--use-keyring" *args*) (create-file "pubring.gpg")) (create-file "common.conf" (if (flag "--use-keyboxd" *args*) "use-keyboxd" "#use-keyboxd") (string-append "keyboxd-program " (tool 'keyboxd)) ) (create-file "gpg.conf" ;;"log-file socket:///tmp/S.wklog" ;;"verbose" "no-greeting" "no-secmem-warning" "no-permission-warning" "batch" "no-auto-key-retrieve" "no-auto-key-locate" "allow-weak-digest-algos" "allow-old-cipher-algos" "ignore-mdc-error" (if have-opt-always-trust "no-auto-check-trustdb" "#no-auto-check-trustdb") (string-append "agent-program " (tool 'gpg-agent) "|--debug-quick-random\n") ) (create-file "keyboxd.conf" ;;"log-file socket:///tmp/S.wklog" ;;"verbose" ;;"debug ipc" ) (create-file "gpg-agent.conf" "allow-preset-passphrase" "no-grab" "enable-ssh-support" "s2k-count 65536" (string-append "pinentry-program " (tool 'pinentry)) "disable-scdaemon")) ;; Initialize the test environment, install appropriate configuration ;; and start the agent, without any keys. (define (setup-environment) (create-gpghome) (start-agent)) (define (setup-environment-no-atexit) (create-gpghome) (start-agent #t)) (define (create-sample-files) (log "Creating sample data files") (for-each (lambda (size) (make-test-data (string-append "data-" (number->string size)) size)) '(500 9000 32000 80000)) (log "Unpacking samples") (for-each (lambda (name) (dearmor (in-srcdir "tests" "openpgp" (string-append name "o.asc")) name)) plain-files)) (define (create-legacy-gpghome) (create-sample-files) (log "Storing private keys") (for-each (lambda (name) (dearmor (in-srcdir "tests" "openpgp" "privkeys" (string-append name ".asc")) (string-append "private-keys-v1.d/" name ".key"))) '("50B2D4FA4122C212611048BC5FC31BD44393626E" "7E201E28B6FEB2927B321F443205F4724EBE637E" "13FDB8809B17C5547779F9D205C45F47CE0217CE" "343D8AF79796EE107D645A2787A9D9252F924E6F" "8B5ABF3EF9EB8D96B91A0B8C2C4401C91C834C34" "0D6F6AD4C4C803B25470F9104E9F4E6A4CA64255" "FD692BD59D6640A84C8422573D469F84F3B98E53" "76F7E2B35832976B50A27A282D9B87E44577EB66" "A0747D5F9425E6664F4FFBEED20FBCA79FDED2BD" "00FE67F28A52A8AA08FFAED20AF832DA916D1985" "1DF48228FEFF3EC2481B106E0ACA8C465C662CC5" "A2832820DC9F40751BDCD375BB0945BA33EC6B4C" "ADE710D74409777B7729A7653373D820F67892E0" "CEFC51AF91F68A2904FBFF62C4F075A4785B803F" "1E28F20E41B54C2D1234D896096495FF57E08D18" "EB33B687EB8581AB64D04852A54453E85F3DF62D" "C6A6390E9388CDBAD71EAEA698233FE5E04F001E" "D69102E0F5AC6B6DB8E4D16DA8E18CF46D88CAE3")) (log "Importing public demo and test keys") (for-each (lambda (file) (call-check `(,@GPG --yes --import ,(in-srcdir "tests" "openpgp" file)))) (list "pubdemo.asc" "pubring.asc" key-file1)) (pipe:do (pipe:open (in-srcdir "tests" "openpgp" "pubring.pkr.asc") (logior O_RDONLY O_BINARY)) (pipe:spawn `(,@GPG --dearmor)) (pipe:spawn `(,@GPG --yes --import)))) (define (preset-passphrases) (log "Presetting passphrases") ;; one@example.com (call-check `(,(tool 'gpg-preset-passphrase) --preset --passphrase def "50B2D4FA4122C212611048BC5FC31BD44393626E")) (call-check `(,(tool 'gpg-preset-passphrase) --preset --passphrase def "7E201E28B6FEB2927B321F443205F4724EBE637E")) ;; alpha@example.net (call-check `(,(tool 'gpg-preset-passphrase) --preset --passphrase abc "76F7E2B35832976B50A27A282D9B87E44577EB66")) (call-check `(,(tool 'gpg-preset-passphrase) --preset --passphrase abc "A0747D5F9425E6664F4FFBEED20FBCA79FDED2BD"))) ;; Initialize the test environment, install appropriate configuration ;; and start the agent, with the keys from the legacy test suite. (define (setup-legacy-environment) (create-gpghome) (if (member "--unpack-tarball" *args*) (begin (call-check `(,(tool 'gpgtar) --extract --directory=. ,(cadr *args*))) (start-agent)) (begin (start-agent) (create-legacy-gpghome))) (preset-passphrases)) ;; Create the socket dir and start the agent. (define (start-agent . args) (log "Starting gpg-agent...") (let ((gnupghome (getenv "GNUPGHOME"))) (if (null? args) (atexit (lambda () (with-home-directory gnupghome (stop-agent)))))) (catch (log "Warning: Creating socket directory failed:" (car *error*)) (gpg-conf '--create-socketdir)) (call-check `(,(tool 'gpg-connect-agent) --verbose ,(string-append "--agent-program=" (tool 'gpg-agent) "|--debug-quick-random") /bye))) ;; Stop the agent and other daemons and remove the socket dir. (define (stop-agent) (log "Stopping gpg-agent...") (gpg-conf '--kill 'all) (catch (log "Warning: Removing socket directory failed.") (gpg-conf '--remove-socketdir))) ;; Get the trust level for KEYID. Any remaining arguments are simply ;; passed to GPG. ;; ;; This function only supports keys with a single user id. (define (gettrust keyid . args) (let ((trust (list-ref (assoc "pub" (gpg-with-colons `(,@args --list-keys ,keyid))) 1))) (unless (and (= 1 (string-length trust)) (member (string-ref trust 0) (string->list "oidreqnmfuws-"))) (fail "Bad trust value:" trust)) trust)) ;; Check that KEYID's trust level matches EXPECTED-TRUST. Any ;; remaining arguments are simply passed to GPG. ;; ;; This function only supports keys with a single user id. (define (checktrust keyid expected-trust . args) (let ((trust (apply gettrust `(,keyid ,@args)))) (unless (string=? trust expected-trust) (fail keyid ": Expected trust to be" expected-trust "but got" trust)))) ;; ;; Enable checking with valgrind if the envvar "with_valgrind" is set ;; (when with-valgrind? (set! gpg `(,@valgrind ,@gpg))) ;;(set! *args* (append *args* (list "--use-keyboxd"))) ;; end diff --git a/tests/tpm2dtests/defs.scm b/tests/tpm2dtests/defs.scm index 2a0910945..eb840206f 100644 --- a/tests/tpm2dtests/defs.scm +++ b/tests/tpm2dtests/defs.scm @@ -1,473 +1,464 @@ ;; Common definitions for the OpenPGP test scripts. ;; ;; Copyright (C) 2016, 2017 g10 Code GmbH ;; ;; This file is part of GnuPG. ;; ;; GnuPG is free software; you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation; either version 3 of the License, or ;; (at your option) any later version. ;; ;; GnuPG is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public License ;; along with this program; if not, see . (let ((verbose (string->number (getenv "verbose")))) (if (number? verbose) (*set-verbose!* verbose))) (define (qualify executable) (string-append executable (getenv "EXEEXT"))) (define (getenv' key default) (let ((value (getenv key))) (if (string=? "" value) default value))) (define (percent-decode s) (define (decode c) (if (and (> (length c) 2) (char=? #\% (car c))) (integer->char (string->number (string #\# #\x (cadr c) (caddr c)))) #f)) (let loop ((i 0) (c (string->list s)) (r (make-string (string-length s)))) (if (null? c) (substring r 0 i) (let ((decoded (decode c))) (string-set! r i (if decoded decoded (car c))) (loop (+ 1 i) (if decoded (cdddr c) (cdr c)) r))))) (assert (equal? (percent-decode "") "")) (assert (equal? (percent-decode "%61") "a")) (assert (equal? (percent-decode "foob%61r") "foobar")) (define (percent-encode s) (define (encode c) `(#\% ,@(string->list (number->string (char->integer c) 16)))) (let loop ((acc '()) (cs (reverse (string->list s)))) (if (null? cs) (list->string acc) (case (car cs) ((#\: #\%) (loop (append (encode (car cs)) acc) (cdr cs))) (else (loop (cons (car cs) acc) (cdr cs))))))) (assert (equal? (percent-encode "") "")) (assert (equal? (percent-encode "%61") "%2561")) (assert (equal? (percent-encode "foob%61r") "foob%2561r")) (define tools - '((gpgv "GPGV" "g10/gpgv") - (gpg-connect-agent "GPG_CONNECT_AGENT" "tools/gpg-connect-agent") - (gpgconf "GPGCONF" "tools/gpgconf") + '((gpgv "GPGV" "bin/gpgv") + (gpg-connect-agent "GPG_CONNECT_AGENT" "bin/gpg-connect-agent") + (gpgconf "GPGCONF" "bin/gpgconf") (gpg-preset-passphrase "GPG_PRESET_PASSPHRASE" - "agent/gpg-preset-passphrase") - (gpgtar "GPGTAR" "tools/gpgtar") - (gpg-zip "GPGZIP" "tools/gpg-zip") - (pinentry "PINENTRY" "tests/openpgp/fake-pinentry") - (tpm2daemon "TPM2DAEMON" "tpm2d/tpm2daemon"))) - -(define bin-prefix (getenv "BIN_PREFIX")) -(define installed? (not (string=? "" bin-prefix))) + "libexec/gpg-preset-passphrase") + (gpgtar "GPGTAR" "bin/gpgtar") + (tpm2daemon "TPM2DAEMON" "libexec/tpm2daemon") + (pinentry "PINENTRY" "openpgp/fake-pinentry"))) + (define with-valgrind? (not (string=? (getenv "with_valgrind") ""))) (define (tool-hardcoded which) (let ((t (assoc which tools))) (getenv' (cadr t) - (qualify (if installed? - (string-append bin-prefix "/" (basename (caddr t))) - (string-append (getenv "objdir") "/" (caddr t))))))) + (qualify (string-append (getenv "GNUPG_BUILD_ROOT") + "/" (caddr t)))))) ;; You can splice VALGRIND into your argument vector to run programs ;; under valgrind. For example, to run valgrind on gpg, you may want ;; to redefine gpg: ;; ;; (set! gpg `(,@valgrind ,@gpg)) ;; (define valgrind '("/usr/bin/valgrind" -q --leak-check=no --track-origins=yes --error-exitcode=154 --exit-on-first-error=yes)) -(unless installed? - (setenv "GNUPG_BUILDDIR" (getenv "objdir") #t)) - (define (gpg-conf . args) (gpg-conf' "" args)) (define (gpg-conf' input args) (let ((s (call-popen `(,(tool-hardcoded 'gpgconf) - ,@(if installed? '() - (list '--build-prefix (getenv "objdir"))) ,@args) input))) (map (lambda (line) (map percent-decode (string-split line #\:))) (string-split-newlines s)))) (define :gc:c:name car) (define :gc:c:description cadr) (define :gc:c:pgmname caddr) (define (:gc:o:name x) (list-ref x 0)) (define (:gc:o:flags x) (string->number (list-ref x 1))) (define (:gc:o:level x) (string->number (list-ref x 2))) (define (:gc:o:description x) (list-ref x 3)) (define (:gc:o:type x) (string->number (list-ref x 4))) (define (:gc:o:alternate-type x) (string->number (list-ref x 5))) (define (:gc:o:argument-name x) (list-ref x 6)) (define (:gc:o:default-value x) (list-ref x 7)) (define (:gc:o:default-argument x) (list-ref x 8)) (define (:gc:o:value x) (if (< (length x) 10) "" (list-ref x 9))) (define (gpg-config component key) (package (define (value) (let* ((conf (assoc key (gpg-conf '--list-options component))) (type (:gc:o:type conf)) (value (:gc:o:value conf))) (case type ((0 2 3) (string->number value)) ((1 32) (substring value 1 (string-length value)))))) (define (update value) (let ((value' (cond ((string? value) (string-append "\"" value)) ((number? value) (number->string value)) (else (throw "Unsupported value" value))))) (gpg-conf' (string-append key ":0:" (percent-encode value')) `(--change-options ,component)))) (define (clear) (gpg-conf' (string-append key ":16:") `(--change-options ,component))))) (define gpg-components (apply gpg-conf '(--list-components))) (define (tool which) (case which ((gpg gpg-agent scdaemon gpgsm dirmngr) (:gc:c:pgmname (assoc (symbol->string which) gpg-components))) (else (tool-hardcoded which)))) (define (gpg-has-option? option) (string-contains? (call-popen `(,(tool 'gpg) --dump-options) "") option)) (define have-opt-always-trust (catch #f (with-ephemeral-home-directory (lambda ()) (lambda ()) (call-check `(,(tool 'gpg) --gpgconf-test --always-trust))) #t)) (define GPG `(,(tool 'gpg) --no-permission-warning ,@(if have-opt-always-trust '(--always-trust) '()))) (define GPGV `(,(tool 'gpgv))) (define PINENTRY (tool 'pinentry)) (define TPM2DAEMON (tool 'tpm2daemon)) (define (tr:gpg input args) (tr:spawn input `(,@GPG --output **out** ,@args **in**))) (define (pipe:gpg args) (pipe:spawn `(,@GPG --output - ,@args))) (define (gpg-with-colons args) (let ((s (call-popen `(,@GPG --with-colons ,@args) ""))) (map (lambda (line) (string-split line #\:)) (string-split-newlines s)))) (define (secinfo name) (assoc "sec" (gpg-with-colons `(--list-secret-key ,name)))) (define (ssbinfo name) (assoc "ssb" (gpg-with-colons `(--list-secret-key ,name)))) (define (fingerprint name) (:fpr (assoc "fpr" (gpg-with-colons `(--list-secret-key ,name))))) ;; convenient accessors for sec (define (:cardinfo x) (list-ref x 14)) ;; Convenient accessors for the colon output of pub. (define (:type x) (string->symbol (list-ref x 0))) (define (:length x) (string->number (list-ref x 2))) (define (:alg x) (string->number (list-ref x 3))) (define (:expire x) (list-ref x 6)) (define (:fpr x) (list-ref x 9)) (define (:cap x) (list-ref x 11)) (define (have-public-key? key) (catch #f (pair? (filter (lambda (l) (and (equal? 'fpr (:type l)) (equal? key::fpr (:fpr l)))) (gpg-with-colons `(--list-keys ,key::fpr)))))) (define (have-secret-key? key) (catch #f (pair? (filter (lambda (l) (and (equal? 'fpr (:type l)) (equal? key::fpr (:fpr l)))) (gpg-with-colons `(--list-secret-keys ,key::fpr)))))) (define (have-secret-key-file? key) (file-exists? (path-join (getenv "GNUPGHOME") "private-keys-v1.d" (string-append key::grip ".key")))) (define (get-config what) (string-split (caddar (gpg-with-colons `(--list-config ,what))) #\;)) (define all-pubkey-algos (delay (get-config "pubkeyname"))) (define all-hash-algos (delay (get-config "digestname"))) (define all-cipher-algos (delay (get-config "ciphername"))) (define all-compression-algos (delay (get-config "compressname"))) (define (have-pubkey-algo? x) (not (not (member x (force all-pubkey-algos))))) (define (have-hash-algo? x) (not (not (member x (force all-hash-algos))))) (define (have-cipher-algo? x) (not (not (member x (force all-cipher-algos))))) (define (have-compression-algo? x) (not (not (member x (force all-compression-algos))))) (define (gpg-pipe args0 args1 errfd) (lambda (source sink) (let* ((p (pipe)) (task0 (spawn-process-fd `(,@GPG ,@args0) source (:write-end p) errfd)) (_ (close (:write-end p))) (task1 (spawn-process-fd `(,@GPG ,@args1) (:read-end p) sink errfd))) (close (:read-end p)) (wait-processes (list GPG GPG) (list task0 task1) #t)))) ;; ;; Do we have a software tpm ;; (define have-swtpm? (not (and (string=? "" (getenv "TPMSERVER")) (string=? "" (getenv "SWTPM"))))) (setenv "GPG_AGENT_INFO" "" #t) (setenv "GNUPGHOME" (getcwd) #t) (if have-swtpm? (setenv "TPM_INTERFACE_TYPE" "socsim" #t)) (define GNUPGHOME (getcwd)) ;; ;; GnuPG helper. ;; ;; Call GPG to obtain the hash sums. Either specify an input file in ;; ARGS, or an string in INPUT. Returns a list of ( ;; "") lists. (define (gpg-hash-string args input) (map (lambda (line) (let ((p (string-split line #\:))) (list (string->number (cadr p)) (caddr p)))) (string-split-newlines (call-popen `(,@GPG --with-colons ,@args) input)))) ;; Dearmor a file. (define (dearmor source-name sink-name) (pipe:do (pipe:open source-name (logior O_RDONLY O_BINARY)) (pipe:spawn `(,@GPG --dearmor)) (pipe:write-to sink-name (logior O_WRONLY O_CREAT O_BINARY) #o600))) (define (gpg-dump-packets source-name sink-name) (pipe:do (pipe:open source-name (logior O_RDONLY O_BINARY)) (pipe:spawn `(,@GPG --list-packets)) (pipe:write-to sink-name (logior O_WRONLY O_CREAT O_BINARY) #o600))) ;; ;; Support for test environment creation and teardown. ;; (define (make-test-data filename size) (call-with-binary-output-file filename (lambda (port) (display (make-random-string size) port)))) (define (create-file name . lines) (catch #f (unlink name)) (letfd ((fd (open name (logior O_WRONLY O_CREAT O_BINARY) #o600))) (let ((port (fdopen fd "wb"))) (for-each (lambda (line) (display line port) (newline port)) lines)))) (define (create-gpghome) (log "Creating test environment...") (srandom (getpid)) (make-test-data "random_seed" 600) (log "Creating configuration files") (if (flag "--use-keyring" *args*) (create-file "pubring.gpg")) (create-file "gpg.conf" ;;"log-file socket:///tmp/S.wklog" ;;"verbose" "no-greeting" "no-secmem-warning" "no-permission-warning" "batch" "no-auto-key-retrieve" "no-auto-key-locate" "allow-weak-digest-algos" "ignore-mdc-error" (if have-opt-always-trust "no-auto-check-trustdb" "#no-auto-check-trustdb") (string-append "agent-program " (tool 'gpg-agent) "|--debug-quick-random\n") (if (flag "--use-keyboxd" *args*) "use-keyboxd" "#use-keyboxd") ) (create-file "gpg-agent.conf" "allow-preset-passphrase" "debug-all" "log-file gpg-agent.log" "no-grab" "enable-ssh-support" "s2k-count 65536" (string-append "pinentry-program " (tool 'pinentry)) (string-append "tpm2daemon-program " (tool 'tpm2daemon)) "disable-scdaemon") (create-file "msg.txt" "This is a test of TPM signing and encryption" "With two lines of text")) ;; Initialize the test environment, install appropriate configuration ;; and start the agent, without any keys. (define (setup-environment) (create-gpghome) (start-agent) (start-tpm)) (define (setup-environment-no-atexit) (create-gpghome) (start-agent #t)) ;; Initialize the test environment, install appropriate configuration ;; and start the agent, with the keys from the legacy test suite. (define (setup-legacy-environment) (create-gpghome) (if (member "--unpack-tarball" *args*) (begin (call-check `(,(tool 'gpgtar) --extract --directory=. ,(cadr *args*))) (start-agent)) (begin (start-agent) (create-legacy-gpghome))) (preset-passphrases)) ;; start the tpm server (define (start-tpm) (if have-swtpm? (begin (define pid (call-check `(,(in-srcdir "tests" "tpm2dtests" "start_sw_tpm.sh")))) (if (not (null? pid)) (atexit (lambda () (call-check `("/bin/kill" ,pid)))))))) ;; Create the socket dir and start the agent. (define (start-agent . args) (log "Starting gpg-agent...") (let ((gnupghome (getenv "GNUPGHOME"))) (if (null? args) (atexit (lambda () (with-home-directory gnupghome (stop-agent)))))) (catch (log "Warning: Creating socket directory failed:" (car *error*)) (gpg-conf '--create-socketdir)) (call-check `(,(tool 'gpg-connect-agent) --verbose ,(string-append "--agent-program=" (tool 'gpg-agent) "|--debug-quick-random") /bye))) ;; Stop the agent and other daemons and remove the socket dir. (define (stop-agent) (log "Stopping gpg-agent...") (gpg-conf '--kill 'all) (catch (log "Warning: Removing socket directory failed.") (gpg-conf '--remove-socketdir))) ;; Get the trust level for KEYID. Any remaining arguments are simply ;; passed to GPG. ;; ;; This function only supports keys with a single user id. (define (gettrust keyid . args) (let ((trust (list-ref (assoc "pub" (gpg-with-colons `(,@args --list-keys ,keyid))) 1))) (unless (and (= 1 (string-length trust)) (member (string-ref trust 0) (string->list "oidreqnmfuws-"))) (fail "Bad trust value:" trust)) trust)) ;; Check that KEYID's trust level matches EXPECTED-TRUST. Any ;; remaining arguments are simply passed to GPG. ;; ;; This function only supports keys with a single user id. (define (checktrust keyid expected-trust . args) (let ((trust (apply gettrust `(,keyid ,@args)))) (unless (string=? trust expected-trust) (fail keyid ": Expected trust to be" expected-trust "but got" trust)))) (define (keytotpm name select) (let ((result (call-with-io `(,@GPG --command-fd=0 --edit-key ,name ,select keytotpm) "y\n"))) (if (= 0 (:retcode result)) (:stdout result) (throw "keytotpm failed" (:stderr result))))) (define (quick-gen name algo) (info "creating TPM " algo " key") (call-check `(,@GPG --quick-generate-key ,name ,algo)) (keytotpm name "key 0") (unless (string=? (:cardinfo (secinfo name)) "TPM-Protected") (throw "key is not in the TPM"))) (define (quick-add name algo) (info "adding TPM encryption " algo " key") (call-check `(,@GPG --quick-add-key ,(fingerprint name) ,algo "encr")) (keytotpm name "key 1") (unless (string=? (:cardinfo (ssbinfo name)) "TPM-Protected") (throw "Added key is not in the TPM"))) (define (check-sig name) (info "checking TPM signing") (call-check `(,@GPG --default-key ,name --sign msg.txt)) (call-check `(,@GPG --verify msg.txt.gpg)) (unlink "msg.txt.gpg")) (define (check-encrypt name) (info "Checking TPM decryption") (call-check `(,@GPG --recipient ,name --encrypt msg.txt)) (call-check `(,@GPG --output msg.out.txt --decrypt msg.txt.gpg)) (unless (file=? "msg.txt" "msg.out.txt") (throw "File did not decrypt to the same message")) (unlink "msg.out.txt") (unlink "msg.txt.gpg")) ;; ;; Tests are very simple: create primary key in TPM add encryption key ;; in TPM (verifies TPM primary can certify secondary), sign a message ;; with primary key and check signature encrypt a message with ;; encryption key and check signature ;; (define (test-tpm name algo) (quick-gen name algo) (quick-add name algo) (check-sig name) (check-encrypt name)) ;; ;; Enable checking with valgrind if the envvar "with_valgrind" is set ;; (when with-valgrind? (set! gpg `(,@valgrind ,@gpg))) ;;(set! *args* (append *args* (list "--use-keyboxd"))) ;; end