Re: replication behind load balancer

[Brett Lomas <b.lomas@xxxxxxxxxxxxxx>]

Hi all,

we have a current working cosign behind a foundry load balancer. It has
been tested and is working quite well. I had to change the cosign code a
little bit to handle this, which are yet to sent back to the umich guys
(Jo, i will do this ASAP). But the main changes are:

1. The cosign daemon can be told to listen on more than one IP address,
and be told which certs to associate to which IP. So one (6663) is the
external facing cert, and the other on 6664 is the internal facing cert.

2. The CGI can be given a number of cosign hosts to talk to and a port
to talk on.

3. A foundry puts of a lot more pressure on the replication! So to
combat this I have allowed for what i call a proxy check. If a daemon
gets a check which it doesn't know about it will go to all its
replication host and issuse a PROXYCHECK, if the other host has it, it
will first return the normal CHECK response which can be forward to the
client and then also any replication events right there and then for the
ticket to be stored locally.

I am sure there was a number of other tweaks too.

If anyone does want this code noe I can get it for you, but of course it
would not be support by the umich guys. I am in the process of sending
these changes back to them and should be done in a few days (we have a 5
day weekend here in NZ at the moment :) )


Thanks

Brett

On Fri, 2005-03-25 at 03:46, johanna bromberg craig wrote:
> On Mar 23, 2005, at 4:58 PM, David Alexander wrote:
> >
> >
> > I added the opposite blade's IP address to the /etc/hosts file on each 
> > blade.  Now, communication between cosignd processes is not crossing 
> > the load balancer, so I think I am just dealing with cosign 
> > configuration issues at this point.
> >
> > The documentation is a little thin on replication.  What should the 
> > consign.conf file contain on each host?  Now that I can use the host 
> > names of the individual blades, I created client certs for 'cosign11' 
> > and 'cosign12'.  I have tried various permutations in the cosign.conf 
> > file, but I still get the error "f_starttls: No access for 
> > cosign1[12]" in the log file.  I get a "CHILD xxxxx talking to itself" 
> > error in the other host's log file.  I was also getting a "cosign[12] 
> > is not a daemon" error at some point.
> >
> > Can someone tell me more about these errors?
> >
> 
> I think from your setup, cosign11 and cosign12 ( or whatever the cert 
> names are that the daemon is using ) need to both be permitted as cgis. 
> The "talking to itself" bit is basically so we can use a multiple A 
> record like weblogin.umich.edu ( which contains a.umich.edu, 
> b.umich.edu, and c.umich.edu ), and when we get the individual host 
> records a only replicates to b and c, not itself. That's what the 
> daemon flag/command is for ( f_daemon ).
> 
> In pusherhosts() ( in pusher.c ) we call gethostbyname() which I 
> believe is aware of /etc/hosts.
> 
> The "%s is not a daemon" message comes from the CN of the host trying 
> to do the replication ( aka the replicator ) not being listed as a cgi 
> in the cosign.conf of the replicatee. The No access message is probably 
> the same thing.
> 
> Since you have 2 hosts with 2 separate IPs, at least for testing 
> purposes I'd just start with this do this:
> 
> host A:
> 
> cosign.conf:
> 
> cgi	a.weblogin.ohio.edu ( just needs itself, and this is the CN for the 
> cert A is running with, not domain name )
> 
> %/usr/local/sbin/cosignd -h b.weblogin.ohio.edu
> 
> ( you also want to run monster the same way - %/usr/local/sbin/monster 
> -h b.weblogin.ohio.edu )
> 
> 
> and on host B you should also have "a.weblogin.ohio.edu" permitted in 
> cosign.conf, and just the regular
> % /usr/local/sbin/cosignd ( and /usr/local/sbin/monster, too ).
> 
> So A should replicate to B, one way. And only A would also house the 
> CGI.
> 
> Once that's up and happy, you can do the exact same thing on B, so now 
> permit A and B both as CGIs on each cosign.conf, and have A do % 
> cosignd -h b.weblogin.ohio.edu and B do % cosignd -h 
> a.weblogin.ohio.edu
> 
> I can see why this might need a bit more documentation. :) We're 
> working on a few small tweaks for 1.8.1 release, so I'll try and add 
> something to that or just put it up on the website.
> 
> Also, as a side note, I'm not sure fail over in the clients will work 
> right if both cosign servers are behind a load balancer. I haven't 
> drawn it all out yet, but the basic idea behind client fail over is 
> that the filters ask every weblogin server they are aware of ( A, B, 
> and C ) based on the lookup they did at startup. If they are talking to 
> a load balancer that only had 1 IP/name, they might only get A when in 
> fact B happen to have the most  up-to-date information. Like I said, I 
> haven't really thought about this though, so I might be wrong, too.
> 
> In any case, let us know if the above suggestions help with replication 
> and/or what else we can do to help.
> 
> -Johanna
> 
> 
> 
> 
> 
> 
>