Commit d50b5d54 authored by Ricardo Constantino's avatar Ricardo Constantino Committed by Michael Niedermayer

rtmpdh: Initialize gcrypt before using it

Either disabling or init'ing secure memory is required after the use
of gcry_check_version. From a look at the functions rtmpdh uses, I
noticed none require the use of secure memory, so we disable it [1][2].

This resolves some errors returned by rtmpdh code with uninitialized
gcrypt, especifically:
Fatal: failed to create the RNG lock: Invalid argument
FATAL: failed to acquire the FSM lock in libgrypt: Invalid argument

Version "1.5.4" was arbitrarily chosen. An older version probably works
as well, but I couldn't compile older versions to test on my machine.

[1]
https://gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html
[2]
https://www.gnupg.org/documentation/manuals/gcrypt/Controlling-the-library.htmlSigned-off-by: 's avatarRicardo Constantino <wiiaboo@gmail.com>
Signed-off-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
parent d637a587
...@@ -97,7 +97,16 @@ ...@@ -97,7 +97,16 @@
mpz_fdiv_r_2exp(bn, bn, num_bits); \ mpz_fdiv_r_2exp(bn, bn, num_bits); \
} while (0) } while (0)
#elif CONFIG_GCRYPT #elif CONFIG_GCRYPT
#define bn_new(bn) bn = gcry_mpi_new(1) #define bn_new(bn) \
do { \
if (!gcry_control(GCRYCTL_INITIALIZATION_FINISHED_P)) { \
if (!gcry_check_version("1.5.4")) \
return AVERROR(EINVAL); \
gcry_control(GCRYCTL_DISABLE_SECMEM, 0); \
gcry_control(GCRYCTL_INITIALIZATION_FINISHED, 0); \
} \
bn = gcry_mpi_new(1); \
} while (0)
#define bn_free(bn) gcry_mpi_release(bn) #define bn_free(bn) gcry_mpi_release(bn)
#define bn_set_word(bn, w) gcry_mpi_set_ui(bn, w) #define bn_set_word(bn, w) gcry_mpi_set_ui(bn, w)
#define bn_cmp(a, b) gcry_mpi_cmp(a, b) #define bn_cmp(a, b) gcry_mpi_cmp(a, b)
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment