Hi Adrien,

That is indeed puzzling. It looks like the server checks the key share sent by the client and the server and it doesn’t seem to be on the curve. This can mean one of two things:
  1. The key share has been altered by for example a memory corruption error and the resulting point is not on the curve.
  2. Both Mbed TLS and OpenSSL see the same point and one of them thinks that it is on the curve and the other one doesn’t - someone has a bug

If you can confirm that both OpenSSL and Mbed TLS see the same point in these cases, and send us such a point, we can investigate the bug. If it turns out that they see different points, there might be a memory issue or perhaps a bug in parsing somewhere.

Cheers,
Janos



From: Adrien LEGER via mbed-tls <mbed-tls@lists.trustedfirmware.org>
Date: Monday, 23 February 2026 at 10:12
To: mbed-tls@lists.trustedfirmware.org <mbed-tls@lists.trustedfirmware.org>
Subject: [mbed-tls] [TLS 1.2][ECDHE][RX65N] point is not on curve (secp256r1)

Hello,
I am using mbedTLS 3.6.5 on a Renesas RX65N with compiler ccrx.
I am implementing a TLS 1.2 server using:
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
- ECDSA P-256 server certificate
- ECDHE secp256r1

During the handshake (I use  openssl s_client), I get:
    >>> TLS 1.2, Alert [length 0002], fatal illegal_parameter

       02 2f

   140605661713728:error:1012606B:elliptic curve routines:EC_POINT_set_affine_coordinates:point is not on curve:../crypto/ec/ec_lib.c:812:

   140605661713728:error:141A4132:SSL routines:tls_process_ske_ecdhe:bad ecpoint:../ssl/statem/statem_clnt.c:2229:
   mbedtls_ecdh_make_params() returned -0x4C80 (MBEDTLS_ERR_ECP_INVALID_KEY)

This happens right after:
      ssl_tls12_server.c:4304: server state: 4

      ssl_tls12_server.c:3234: => write server key exchange

      ssl_tls12_server.c:2971: ECDHE curve: secp256r1

      ssl_tls12_server.c:3075: value of 'ECDH: Q(X)' (256 bits) is:

      ssl_tls12_server.c:3075:  f0 7e c6 f3 cc 41 71 bb a8 01 0b cc 3a 8a 5e 72

      ssl_tls12_server.c:3075:  9d db bc d9 a1 5a 04 91 47 44 e0 ff 6f 42 de b3

      ssl_tls12_server.c:3075: value of 'ECDH: Q(Y)' (255 bits) is:

      ssl_tls12_server.c:3075:  5e ba af af 86 55 1a 6e 04 a8 97 b4 13 12 c2 3c

      ssl_tls12_server.c:3075:  a3 2e 00 a4 2d 44 e8 63 bf 98 08 74 81 94 5f 5e

      ssl_tls12_server.c:3130: pick hash algorithm 9 for signing

      ssl_tls.c:9231: Perform mbedtls-based computation of digest of ServerKeyExchange

      ssl_tls12_server.c:3148: dumping 'parameters hash' (32 bytes)

      ssl_tls12_server.c:3148: 0000:  2d b3 aa 62 c4 5a 87 18 39 a6 b6 91 0e 6d fb 81  -..b.Z..9....m..

      ssl_tls12_server.c:3148: 0010:  f7 55 38 54 33 1d 30 cc 85 83 10 2e 39 5c 5d 67  .U8T3.0.....9\]g

      ssl_tls12_server.c:3296: dumping 'my signature' (72 bytes)

      ssl_tls12_server.c:3296: 0000:  30 46 02 21 00 ee 81 dd 1f 32 62 66 57 5c 90 31  0F.!.....2bfW\.1

      ssl_tls12_server.c:3296: 0010:  a9 84 2a c4 e8 ee 6a c5 f0 db 39 01 58 d5 9c e3  ..*...j...9.X...

      ssl_tls12_server.c:3296: 0020:  6e e6 bd 04 25 02 21 00 f5 c6 89 97 d8 dd 2f 93  n...%.!......./.

      ssl_tls12_server.c:3296: 0030:  d0 11 19 f7 0a e7 c4 6b ae 27 b8 d5 db b4 a9 2c  .......k.'.....,

      ssl_tls12_server.c:3296: 0040:  2f ec 2e b4 53 1a 72 01


I suspect an entropy / RNG issue. My RNG initialization is:
- custom entropy source based on XXX
- added via mbedtls_entropy_add_source(...)
- CTR_DRBG seeded with personalization string "debug-seed"
Do you see any problem in this setup ? Do not hesitate if you need any other information.

I'm new to cryptography and currently learning TLS with mbedTLS.
Thanks in advance,

Adrien.