Re: A couple of CoSign design and deployment questions

[kevin mcgowan <clunis@xxxxxxxxx>]

On Mar 23, 2004, at 8:24 PM, Brett Lomas wrote:

> Hi Kevin (and others :) ),
>
> I have a few more questions about Cosign:
>
> 1. Why was it decided to store the cookies on disk? There are a couple 
> of
> reasons I ask this question. Linux/Unix default to only allowing around
> 65500 objects under a single directory. This in itself wouldn't 
> normally be
> a problem, but because the cosign and cosign service cookies (for the
> daemon) are stored in one directory; this will fill up very quickly. 
> If it
> is suggested we go ahead with CoSign I will be suggesting to 
> management that
> we spend some time changing the storage to either a shared memory table
> (because the cosignd and children have a shared parent) OR change it 
> to use
> a DB backend for storage (and it may also possible handle the 
> replication to
> other cosignd server databases). What do you think of these?

These are completely justifiable concerns, I had them myself when we 
started.  We chose the filesystem because:  it was simple to write, 
easy to debug/test with standard unix tools, its contents survive a 
reboot, and it was adequate for cosign's needs.  At peak load we're 
seeing approximately 120,000 objects in /var/cosign/daemon on our 
cosign servers.  I'm certain there's a hard limit (for files in a 
directory), but in our previous Solaris deploy we found that we ran 
into performance problems long before we hit a real limit. In our 
current Linux deploy we've yet to see any noticeable slowdown.  If 
someone does encounter such a slowdown, there are, as you point out, 
several options for addressing the situation.  This is the forum in 
which those alternatives would be discussed/hashed out.

> Also having the
> files named the same as the cookie is big security risk (in my 
> opinion)...
> it could possibly lead to exploits, although I am still doing my 
> security
> review and have not found a concrete example yet.

I welcome the review and am eager to learn what you discover.  I feel 
confident that we've explored the possibilities here (note the checks 
in daemon/command.c for '/'), but having someone else explore them too 
is fantastic.  What sort of security vulnerabilities are you concerned 
about?

> 2. What size is your deployment at UM? Can you give me some stats, 
> like the
> hardware you are using and the number of authentications etc you 
> service a
> day. Only if this is not too much of an effort on your part, because I 
> will
> be stressing the application myself, I am just curious.

So far this month we are seeing approximately 180,000 LOGINs per day on 
a monday - thursday, as few as half that on a weekend day.  These are 
LOGIN events, of course, not unique individuals.  These 180,000 weekday 
LOGINs are associated with approximately 32,000 LOGOUT events.

We REGISTER approximately 255,000 service cookies on an average 
weekday.  120,000 of these are for mail.umich.edu.  The rest are 
divided among the roughly 50 other cosign-protected services in active 
use on campus.

We're running our weblogin service on three 1u linux boxes.  Each 
machine has dual 2.8 Ghz Xeons and 4 gigs of ram.  We're using hardware 
this (relatively) "beefy" only because it was pretty much the cheapest 
thing we were willing to deploy a service on.  We have three of them so 
we can have the service physically located in multiple server rooms.  
Load average on these machines is typically around .2 (point two). :)

> 3. Have you successfully deployed CoSign to an n-tiered application? I
> specifically ask because will needing the chosen implementation to be 
> able
> to SSO to our Cyrus IMAP server via Horde/IMP

We have several N-tier applications deployed:

    o afs.umich.edu -- a web-based AFS file manager, Horde's gollem 
running with the user's AFS token.
    o kpasswd.cgi -- kpasswd, but a cgi
    o mail.umich.edu -- IMP, see below
    o directory.umich.edu -- gui client to our ldap directory service
    o flume -- our web log analysis software (runs reports of user's 
web statistics and writes the report to the appropriate directory in 
AFS).

mail.umich.edu is our most popular web app right now, seeing upwards of 
15,000 simultaneous users at any given moment on weekday afternoons 
with, as I said above, roughly 120,000 users per day and 60,000 - 
70,000 unique users accessing the service per month.

We made a few changes to mod_cosign a couple of years ago to correctly 
set up the GSSAPI environment that IMP/c-client needed.  Otherwise IMP 
should just work out of the box with Cosign.  I can put you in touch 
with Liam Hoekenga, the person primarily responsible for our IMP 
installation, if you have specific questions there.

Kevin

                                      ... "In, as you say, the mud." ...