Don't trap if an encoding error occurs in wcrtomb()

The POSIX definition of wcrtomb
(http://pubs.opengroup.org/onlinepubs/9699919799/functions/wcrtomb.html)
states:

"When wc is not a valid wide character, an encoding error shall occur.
In this case, the function shall store the value of the macro [EILSEQ]
in errno and shall return (size_t)-1; the conversion state shall be
undefined."

The fortify-headers implementation of wcrtomb interprets the result -1
as 18446744073709551615 bytes. Since this is the highest 64-bit number
possible, it is pretty safe to say this will always be larger than any
buffer provided to wcrtomb. Therefore, it traps.

Fixes bug https://bugs.alpinelinux.org/issues/7681.

Patch by A. Wilcox <AWilcox@Wilcox-Tech.com>

1 files changed, 3 insertions, 1 deletions

diff --git a/include/wchar.h b/include/wchar.h
index 08a817b..7394598 100644
--- a/include/wchar.h
+++ b/include/wchar.h
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2015-2016 Dimitris Papastamos <sin@2f30.org>
+ * Copyright (C) 2015-2017 Dimitris Papastamos <sin@2f30.org>
  *
  * Permission to use, copy, modify, and/or distribute this software for any
  * purpose with or without fee is hereby granted.
@@ -109,6 +109,8 @@ _FORTIFY_FN(wcrtomb) size_t wcrtomb(char *__s, wchar_t __w, mbstate_t *__st)
 
         if (__s) {
                 __r = __orig_wcrtomb(__buf, __w, __st);
+                if (__r == (size_t)-1)
+                        return __r;
                 if (__r > __b)
                         __builtin_trap();
                 memcpy(__s, __buf, __r);