= SINDES Support and Configuration in Quattor = [[TracNav]] == Obtaining SINDES == == Installing and configuring SINDES == * Setup instructions based on those at at BeGrid [http://quattor.begrid.be/trac/centralised-begrid-v4/wiki/SINDES] == Sindes Installation == * What is SINDES? You might want to read [https://twiki.cern.ch/twiki/pub/FIOgroup/SinDes/presentation-poulhies-27-sept-2005.pdf presentation on SINDES] to know what SINDES is. * we only use it for the certificate distribution/management part === Configuration step through guide === * Use the cb-v5 yum repository (if not yet done, go [https://quattor.begrid.be/trac/centralised-begrid-v5/wiki/BEgridClient#AII here]) * This should get everything else that's needed {{{ yum install SINDES-ca }}} * edit /etc/sindes/ca.config, make sure to set O, OU and CN. * Is it possible that the CN of the CA can't be the machines FQDN because of conflict with the certificate for the webserver (which is not the CA certificate, but a certificate signed by the CA with the CN of the host of the webserver, being the same as the CA? ;) * eg (the CA after CN is not a typo) {{{ [ req_distinguished_name ] O = IIHE OU = GRID CN = Local BEgrid client CA }}} * edit /etc/sindes/sindesrc: in [MISC], set correctly domain (and use it correctly in sindessh !) * Default validity of the certificates: * the current version of sindes-bootstrap-ca will generate a CA certificate with a 10 year lifetime. (if this is not ok for you, run '''grep 3650 /usr/sbin/sindes-bootstrap-ca''' and mofiy it there!) * The generated certificates also have a lifetime of 10 years. If this is not Ok for you, you can change the default lifetime in /etc/sindes/ca.config: {{{ default_days = 3650 }}} * run "sindes-bootstrap-ca -a" to generate all certificates etc, check output carefully. This also generates a rpm with the CA certifiacte called eg SINDES-ca-certificate--0.1-1.noarch.rpm * you need to provide a passphrase for the apache.key. If you do this, everytime you want to restart httpd, it will prompt for a passwd (see the [wiki:SINDES#Troubleshooting Troubleshooting] section to disable password) * Email the SINDES-ca-certificate--0.1-1.noarch.rpm to begrid@belnet.be, so an administrator can upload it to the quattor-repository. * [http://quattor.begrid.be/trac/centralised-begrid-v5/wiki/BEgridClient#Optional:SetupaccesstoBEgridSWREPrepository instructions on how to do it yourself] * you can check the validity of the genereated certificates in /etc/sindes/cert by running {{{ openssl x509 -in /etc/sindes/certs/ca.crt -noout -text openssl x509 -in /etc/sindes/certs/apache.crt -noout -text }}} * edit /etc/httpd/conf.d/sindes-ssl.conf: * make sure that the /var/www/https/profiles directory is not reachable by any other virtualhost (ie, if the {{{DocumentRoot}}} is unique, it shouldn't). * Replace ''{{{DocumentRoot /var/www/html/https}}}'' by ''{{{DocumentRoot /var/www/https}}}''. * because we will use it only for certificate-management, some of the configuration it is already in ssl.conf, BUT we should use a new virtualhost (port 444) for sindes and profiles. * also, the rewrite rules should be made for the profile directory. * add this line in the beginning of the virtual host setting: (so after ) {{{ HostnameLookups On }}} * add the following lines at the end of the correct virtual host (the one running on port 444, the default sindes-ssl.config file also has a virtualhost on 445): {{{ #### BEGIN AII STUFF #### RewriteMap ACLmap txt:/var/www/acl/ACLmap.txt RewriteCond ${ACLmap:%{REMOTE_HOST}|NO} NO RewriteRule ^/profiles/.*$ /profiles/profile_%{REMOTE_HOST}.xml Options +Indexes SSLOptions +StdEnvVars SSLRequireSSL SSLVerifyClient require SSLOptions +StrictRequire SSLVerifyDepth 1 SSLOptions +OptRenegotiate SSLRequire %{SSL_CLIENT_S_DN_CN} eq %{REMOTE_HOST} #### END AII STUFF #### }}} * make ACLmap in /var/www/acl/ACLmap.txt. It's needed for aii-shellfe to avoid remapping. Replace by the actual fully qualified domain name of your machine. {{{ mkdir /var/www/acl/ echo "$(hostname -f) YES" > /var/www/acl/ACLmap.txt }}} * give correct premission to the /var/log/sindes repository: {{{ chown -R apache.apache /var/log/sindes }}} * create the https directory: {{{ mkdir /var/www/https/ }}} * disable SElinux (if not already done) * check if the file '''/selinux/enforce''' exists * if not, then is SElinux disabled * if it exists, do {{{ echo "echo 0 > /selinux/enforce" >> /etc/rc.local /etc/rc.local }}} * restart httpd (will prompt for passwd. To get rid of this, see [http://quattor.begrid.be/trac/centralised-begrid-v5/wiki/SINDES#Troubleshooting remove passwd from apache.key in /etc/sindes/keys]) {{{ service httpd restart }}} === Useful commands === '''About the domain set in sindesrc: if you have machine called node20-1.wn.iihe.ac.be and you set the domain to iihe.ac.be, the target should be node20-1.wn''' * to use SINDES, switch to {{{sindes}}} user. It uses it's own special sindes shell {{{ su - sindes }}} * set timelimit of 1000s to target and grant the permission to retrieve it {{{ acl -set -length 1000 -grant -target node20-1.wn }}} * give overview of status {{{ acl -print }}} * give info about certificate {{{ cert -check -target node20-1.wn }}} * revoke certificate {{{ cert -revoke -target node20-1.wn }}} ==== Manual certificate revocation ==== (in case the cert -revoke doesn't work) The hard way is not very userfriendly, though it can be easily wrapped I think. You have to grep for a line starting with 'V' and containing your CN in your ca.db.index. You'll get something like: {{{ V 101215134729Z 02 unknown /O=CERN/OU=FIO-DS/CN=lxservb02.cern.ch ^^ }}} Then, you know your certificate has serial '02' (hex), so you can run: {{{ /usr/bin/openssl ca -config /etc/sindes/ca.config -batch -revoke /var/sindes/CA/ca/ca.db.certs/02.pem -crl_reason cessationOfOperation }}} === Manual client installation === * install SINDES-client and SINDES-ca-certificate- rpms: * configure /etc/sindes/get-cert.conf ( O and OU must match the ones in ca.config (use quotes!!); domain can be empty) {{{ # Https server HTTP_SEL="https://" HTTPS_SERVER="q3.iihe.ac.be" #domain name, to be removed from the hostname if it's a FQDN DOMAIN_NAME="" # Organisation and Unit: CRT_O="IIHE" CRT_OU="GRID" }}} * run sindes-get-certificate === SINDES+AII === * modify /etc/aii-shellfe.conf to make sure it is setup correctly {{{ cdburl = https://f.q.d.n:444/profiles cert_file = /etc/sindes/certs/apache.crt key_file = /etc/sindes/keys/apache.key ca_file = /etc/sindes/certs/ca.crt profile_prefix = profile_ use_fqdn = 1 }}} {{{ #!comment * TIP: use ca_dir instead of ca_file (in AII, ccm.conf and aii-shellfe.conf) }}} * to install a new node, one now has to: * aii-shellfe --configure * aii-shellfe --install * you can already start the installation * set the acl with acl -set -length 1000 -grant -target ==== Renewal of CA cert ==== '''On the quattor client side:''' * remove all existing certificates issued (run this command only if you know what your are doing): {{{ sindes-bootstrap-ca -c }}} * increase the {{{RELEASE}}} parameter in {{{sindes-bootstrap-ca}}} so it will create the new CA rpm with a different rpm version. * then generate new ones by looking at section [wiki:SINDES#Configurationstepthroughguide Configuration step through guide] * grant permission retrieve a new certificate {{{ su - sindes acl -set -grant -length 5000 -target nodeXXX.wn }}} '''On the every node side:''' * install the new CA certificate of the quattor client: {{{ rpm -Uvh http://quattor.begrid.be//begrid/swrep/noarch_sindes/SINDES-ca-certificate-q3-0.1-5.noarch.rpm }}} * to avoid using globus version of openssl (gives -batch not found error): {{{ export PATH=/usr/bin/:$PATH }}} * finally, renew the node certificate with: {{{ sindes-get-certificate -f }}} ==== Troubleshooting ==== * if you have troubles with {{{sindes-get-certificate}}}, login on the machine and first do {{{ export DEBUG=1 }}} This will turn on the debug and will allow you to rerun the commands used. * most curl commands used by sindes run with {{{curl -f -s}}} for very silent running and failing. To debug, it's best to rerun them with {{{curl -v}}} * logfiles can be found in /var/log/httpd and /var/log/sindes * remove passwd from apache.key in /etc/sindes/keys {{{ cd /etc/sindes/keys;mv apache.key apache.key-2;openssl rsa -in apache.key-2 -out apache.key;chown apache.apache apache.key;chmod 400 apache.key; cd - }}} * full cleanup: because high dependencies, best is to * backup config-files (ca.config, sindesrc and sindes-ssl.conf) * and run: {{{ rpm -e perl-SINDES-Shell-0.5-30 perl-SINDES-common-0.5-24 SINDES-Shell-bin-0.5-34 perl-SINDES-GetCertificate-0.9.99-1 SINDES-ca-0.9.99-2 rm -Rf /etc/sindes /var/sindes yum install SINDES-ca }}} * Make sure nslookup of the client fqdn works on the SINDES-server. * on the client: when a problem arises with ccm-fetch, use curl to see if profiles are fetchable: {{{ curl --cert /etc/sindes/certs/client_cert_key.pem --cacert /etc/sindes/certs/ca.crt }}} * a direct route to sindes server (ie the DN in the apache.crt) is needed. Passing through a NAT-box first will make it fail. Therefore you need: * extra line in aii with explicit route BEFORE sindes-setup (don't forget to sleep!!) * hard route config in the profiles (or ccm-fetch will not work) * aii-installack.cgi failure: * add a "--use_fqdn " to aii-installack.cgi * aii-installack.cgi also needs a direct route to the cgi-server. * if more than one crt resides in the CA crt dir, run in that directory (get the {{{Makefile.crt}}} from {{{/etc/httpd/ssl.crt}}}) {{{ make -f Makefile.crt }}} * When I wanted to renew the certificate of a workernode behind NAT, it failed when I run "sindes-get-certificate -f": {{{ ... Something went wrong while contacting CA (curl returned 22) }}} The error is because the WN connects to the quattor client via NAT instead of direct. I solved this by adding this in /etc/hosts: {{{ 192.168.10.4 gridy4.begrid.be }}} This is the private ip of our quattor client. === Links === * Wiki pages [https://twiki.cern.ch/twiki/bin/view/FIOgroup/SinDes] * CVS access: {{{ export CVSROOT=:pserver:anonymous@isscvs.cern.ch:/local/reps/fio cvs co fabric/SINDES }}}