Re: openldap client GSSAPI authentication segfaults in fbsd8stablei386



On Thu, Jul 15, 2010 at 12:14:58AM +0300, Reko Turja wrote:
I had similar issue with 8-RELEASE and cyrus-sasl2 with cyrus-saslauthd linked against system kerberos.

(uname -a xxx.xxx.xxx 8.0-RELEASE-p3 FreeBSD 8.0-RELEASE-p3 #1:
Sat Jun 12 00:39:22 EEST 2010
root@xxxxxxxxxxx:/usr/obj/usr/src/sys/WWW i386)

The problem manifested itself with pretty much the same backtrace
when using cyradm tool for administering cyrus mailboxes and due
time constraints I solved my issue by removing all the gssapi
plugin libs from /usr/local/lib/sasl2, so my solution isn't really
applicable in your case.

After building perl, cyrus-sasl2 and userland/kernel with debug symbols
I was able to get the following backtrace.

#0 free (ptr=0x280871c0) at /usr/src/lib/libc/stdlib/malloc.c:3889
3889 arena_dalloc(chunk->arena, chunk, ptr);
[New Thread 286ae140 (LWP 100273)]
(gdb) bt
#0 free (ptr=0x280871c0) at /usr/src/lib/libc/stdlib/malloc.c:3889
#1 0x2899ed32 in gss_release_buffer (minor_status=0xbfbfe4b8,
buffer=0x280871cc) at /usr/src/lib/libgssapi/gss_release_buffer.c:41
#2 0x2899e6e2 in _gss_mg_error (m=0x28a86480, maj=851968, min=2) at /usr/src/lib/libgssapi/gss_display_status.c:240
#3 0x2899afd9 in gss_init_sec_context (minor_status=0xbfbfe5a4, initiator_cred_handle=0x0, context_handle=0x28630164,
target_name=0x2861b380, input_mech_type=0x0, req_flags=58, time_req=0, input_chan_bindings=0x0, input_token=0x0,
actual_mech_type=0x0, output_token=0xbfbfe5a8, ret_flags=0xbfbfe594, time_rec=0x0)
at /usr/src/lib/libgssapi/gss_init_sec_context.c:156
#4 0x289936c9 in gssapi_client_mech_step (conn_context=0x28630160, params=0x28a97080, serverin=0x0, serverinlen=0,
prompt_need=0xbfbfe8b0, clientout=0xbfbfe8ac, clientoutlen=0xbfbfe8a8, oparams=0x28a95860) at gssapi.c:1418
#5 0x285810f6 in sasl_client_step (conn=0x28a95000, serverin=0x0,
serverinlen=0, prompt_need=0xbfbfe8b0, clientout=0xbfbfe8ac,
clientoutlen=0xbfbfe8a8) at client.c:655
#6 0x28580ef7 in sasl_client_start (conn=0x28a95000,
mechlist=0x2861b360 "GSSAPI DIGEST-MD5 CRAM-MD5 ",
prompt_need=0xbfbfe8b0,
clientout=0xbfbfe8ac, clientoutlen=0xbfbfe8a8, mech=0xbfbfe8b8) at client.c:603
#7 0x2856a94c in imclient_authenticate () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Cyrus/IMAP/IMAP.so
#8 0x28566f5e in XS_Cyrus__IMAP__authenticate () from
/usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Cyrus/IMAP/IMAP.so
#9 0x281d8f30 in Perl_pp_entersub () at pp_hot.c:2888
#10 0x281878bc in Perl_runops_debug () at dump.c:1968
#11 0x280d80a9 in S_run_body (oldscope=1) at perl.c:2431
#12 0x280d7535 in perl_run (my_perl=0x28601100) at perl.c:2349
#13 0x08048930 in main (argc=6, argv=0xbfbfec44, env=0xbfbfec60) at perlmain.c:117

I'm complete GDB-idjit though so any help in getting usable information
from the following trace would be appreciated - I have the dump etc.
stored away for further digging of course.

The dump is only useful if it's used on that exact system it came from,
so don't rebuild any of the software/etc. until we get this figured out.

The code in question:

src/lib/libgssapi/gss_display_status.c

186 struct mg_thread_ctx {
187 gss_OID mech;
188 OM_uint32 maj_stat;
189 OM_uint32 min_stat;
190 gss_buffer_desc maj_error;
191 gss_buffer_desc min_error;
192 };
...
231 void
232 _gss_mg_error(struct _gss_mech_switch *m, OM_uint32 maj, OM_uint32 min)
233 {
234 OM_uint32 major_status, minor_status;
235 OM_uint32 message_content;
236 struct mg_thread_ctx *mg;
237
238 mg = &last_error_context;
239
240 gss_release_buffer(&minor_status, &mg->maj_error);
241 gss_release_buffer(&minor_status, &mg->min_error);

src/lib/libgssapi/gss_release_buffer.c

34 OM_uint32
35 gss_release_buffer(OM_uint32 *minor_status,
36 gss_buffer_t buffer)
37 {
38
39 *minor_status = 0;
40 if (buffer->value)
41 free(buffer->value);

src/include/gssapi/gssapi.h

106 typedef struct gss_buffer_desc_struct {
107 size_t length;
108 void *value;
109 } gss_buffer_desc, *gss_buffer_t;

So I'm going to take a shot in the dark at this one, with some
supporting evidence.

It looks like gss_release_buffer.c:40 is improper/incomplete -- it's
only checking to see if the pointer is non-NULL, and then blindly calls
free(). It doesn't bother to check to see if buffer->length is
non-zero.

It seems gss_release_buffer() is basically mindless about what it frees
anyway, which is bad design given the complexity of this thing. Well,
some MIT Kerberos dudes complained about it too, and solved it by adding
"magic numbers" to the structure so that you can tell what "type" of
buffer it is:

http://krbdev.mit.edu/rt/Ticket/Display.html?id=2145

The patch/code there is in no way shape or form compatible with what we
have in our repo... but I did notice something interesting in their
gss_release_buffer() routine/diff which correlates with my idea:

- if ((buffer->length) &&
- (buffer->value)) {

They're checking both buffer->length *and* buffer->value. We're not.

That said, can you please execute the following in gdb and provide
the output?

(gdb) p/x gss_release_buffer.c::buffer->value
(gdb) p/x gss_release_buffer.c::buffer->length

Furthermore, relevant bug (PR 144754) indicates there's an easier way to
induce this problem, so I'm going to see if I can reproduce it here
locally. It's almost certainly the same problem but induced via a
slightly different context.

http://lists.freebsd.org/pipermail/freebsd-bugs/2010-March/038956.html

I'll report back once I poke around with that.

--
| Jeremy Chadwick jdc@xxxxxxxxxxxx |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |

_______________________________________________
freebsd-stable@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@xxxxxxxxxxx"