Changes between Version 2 and Version 3 of Doc/BasicConfig/SINDES

Timestamp:

Feb 15, 2011, 2:13:08 PM (13 years ago)

Author:

/C=UK/O=eScience/OU=CLRC/L=RAL/CN=ian collier

Comment:

--

Doc/BasicConfig/SINDES

@@ -7 +7 @@
 
 == Installing and configuring SINDES ==
-* Setup instructions at BeGrid [http://quattor.begrid.be/trac/centralised-begrid-v4/wiki/SINDES]
+* Setup instructions based on those at at BeGrid [http://quattor.begrid.be/trac/centralised-begrid-v4/wiki/SINDES]
+
+
+== Sindes Installation ==
+ * What is SINDES? You might want to read [https://twiki.cern.ch/twiki/pub/FIOgroup/SinDes/presentation-poulhies-27-sept-2005.pdf presentation on SINDES] to know what SINDES is.
+  * we only use it for the certificate distribution/management part
+=== Configuration step through guide ===
+ * Use the cb-v5 yum repository (if not yet done, go [https://quattor.begrid.be/trac/centralised-begrid-v5/wiki/BEgridClient#AII here])
+ * This should get everything else that's needed
+
+{{{
+  yum install SINDES-ca
+}}}
+
+ * edit /etc/sindes/ca.config, make sure to set O, OU and CN. 
+  * Is it possible that the CN of the CA can't be the machines FQDN because of conflict with the certificate for the webserver (which is not the CA certificate, but a certificate signed by the CA with the CN of the host of the webserver, being the same as the CA? ;)
+  * eg (the CA after CN is not a typo)
+
+{{{
+ [ req_distinguished_name ]
+ O                      = IIHE
+ OU                     = GRID
+ CN                     = Local BEgrid client CA
+}}}
+
+ * edit /etc/sindes/sindesrc: in [MISC], set correctly domain (and use it correctly in sindessh !)
+ * Default validity of the certificates:
+  * the current version of sindes-bootstrap-ca will generate a CA certificate with a 10 year lifetime. (if this is not ok for you, run '''grep 3650 /usr/sbin/sindes-bootstrap-ca''' and mofiy it there!)
+  * The generated certificates also have a lifetime of 10 years. If this is not Ok for you, you can change the default lifetime in /etc/sindes/ca.config: 
+{{{
+  default_days            = 3650
+}}}
+
+ * run "sindes-bootstrap-ca -a" to generate all certificates etc, check output carefully. This also generates a rpm with the CA certifiacte called eg SINDES-ca-certificate-<machine>-0.1-1.noarch.rpm
+  * you need to provide a passphrase for the apache.key. If you do this, everytime you want to restart httpd, it will prompt for a passwd (see the [wiki:SINDES#Troubleshooting Troubleshooting] section to disable password)
+  * Email the SINDES-ca-certificate-<machine>-0.1-1.noarch.rpm to begrid@belnet.be, so an administrator can upload it to the quattor-repository.
+   * [http://quattor.begrid.be/trac/centralised-begrid-v5/wiki/BEgridClient#Optional:SetupaccesstoBEgridSWREPrepository instructions on how to do it yourself]
+  * you can check the validity of the genereated certificates in /etc/sindes/cert by running
+{{{
+  openssl x509 -in /etc/sindes/certs/ca.crt -noout -text
+  openssl x509 -in /etc/sindes/certs/apache.crt -noout -text
+}}}
+
+ * edit /etc/httpd/conf.d/sindes-ssl.conf:
+  * make sure that the /var/www/https/profiles directory is not reachable by any other virtualhost (ie, if the {{{DocumentRoot}}} is unique, it shouldn't). 
+    * Replace ''{{{DocumentRoot /var/www/html/https}}}'' by ''{{{DocumentRoot /var/www/https}}}''.
+  * because we will use it only for certificate-management, some of the configuration it is already in ssl.conf, BUT we should use a new virtualhost (port 444) for sindes and profiles.
+  * also, the rewrite rules should be made for the profile directory.
+  * add this line in the beginning of the virtual host setting: (so after <virtualhost _default_:444>)
+{{{
+  HostnameLookups On
+}}}
+
+  * add the following lines at the end of the correct virtual host (the one running on port 444, the default sindes-ssl.config file also has a virtualhost on 445):
+
+{{{
+        #### BEGIN AII STUFF ####
+        RewriteMap ACLmap txt:/var/www/acl/ACLmap.txt
+        RewriteCond ${ACLmap:%{REMOTE_HOST}|NO} NO
+        RewriteRule ^/profiles/.*$ /profiles/profile_%{REMOTE_HOST}.xml
+ 
+        <Directory "/var/www/https/profiles">
+        Options +Indexes
+        SSLOptions +StdEnvVars
+        SSLRequireSSL
+        SSLVerifyClient require
+        SSLOptions +StrictRequire
+        SSLVerifyDepth 1
+        SSLOptions +OptRenegotiate
+        SSLRequire %{SSL_CLIENT_S_DN_CN} eq %{REMOTE_HOST}
+        </Directory>
+        #### END AII STUFF ####
+}}}
+
+ * make ACLmap in /var/www/acl/ACLmap.txt. It's needed for aii-shellfe to avoid remapping. Replace <fqdn> by the actual fully qualified domain name of your machine. 
+
+{{{
+  mkdir /var/www/acl/
+  echo "$(hostname -f) YES" > /var/www/acl/ACLmap.txt
+}}}
+
+ * give correct premission to the /var/log/sindes repository:
+
+{{{
+  chown -R apache.apache /var/log/sindes
+}}}
+
+ * create the https directory:
+
+{{{
+  mkdir /var/www/https/
+}}}
+
+ * disable SElinux (if not already done)
+  * check if the file '''/selinux/enforce''' exists
+   * if not, then is SElinux disabled
+   * if it exists, do 
+{{{
+echo "echo 0 >  /selinux/enforce" >> /etc/rc.local
+/etc/rc.local  
+}}}
+
+ * restart httpd (will prompt for passwd. To get rid of this, see [http://quattor.begrid.be/trac/centralised-begrid-v5/wiki/SINDES#Troubleshooting remove passwd from apache.key in /etc/sindes/keys])
+
+{{{
+  service httpd restart
+}}}
+
+
+=== Useful commands ===
+'''About the domain set in sindesrc: if you have machine called node20-1.wn.iihe.ac.be and you set the domain to iihe.ac.be, the target should be node20-1.wn'''
+ * to use SINDES, switch to {{{sindes}}} user. It uses it's own special sindes shell
+{{{
+su - sindes
+}}}
+ * set timelimit of 1000s to target and grant the permission to retrieve it
+
+{{{
+  acl -set -length 1000 -grant -target node20-1.wn
+}}}
+
+ * give overview of status
+
+{{{
+  acl -print
+}}}
+
+ * give info about certificate
+
+{{{
+  cert -check -target node20-1.wn
+}}}
+
+ * revoke certificate
+
+{{{
+  cert -revoke -target node20-1.wn
+}}}
+
+==== Manual certificate revocation ====
+(in case the cert -revoke doesn't work)
+The hard way is not very userfriendly, though it can be easily wrapped I think. 
+You have to grep for a line starting with 'V' and containing your CN in your ca.db.index. You'll get something like:
+
+{{{
+   V 101215134729Z 02 unknown /O=CERN/OU=FIO-DS/CN=lxservb02.cern.ch
+   ^^
+}}}
+
+Then, you know your certificate has serial '02' (hex), so you can run:
+
+{{{
+  /usr/bin/openssl ca -config /etc/sindes/ca.config -batch -revoke 
+  /var/sindes/CA/ca/ca.db.certs/02.pem -crl_reason cessationOfOperation
+}}}
+
+=== Manual client installation ===
+ * install SINDES-client and SINDES-ca-certificate-<machine> rpms:
+ * configure /etc/sindes/get-cert.conf ( O and OU must match the ones in ca.config (use quotes!!); domain can be empty)
+
+{{{
+  # Https server
+  HTTP_SEL="https://"
+  HTTPS_SERVER="q3.iihe.ac.be"
+  #domain name, to be removed from the hostname if it's a FQDN
+  DOMAIN_NAME=""
+  # Organisation and Unit:
+  CRT_O="IIHE"
+  CRT_OU="GRID"
+}}}
+
+ * run sindes-get-certificate
+=== SINDES+AII ===
+ * modify /etc/aii-shellfe.conf to make sure it is setup correctly
+{{{
+  cdburl = https://f.q.d.n:444/profiles
+  cert_file = /etc/sindes/certs/apache.crt
+  key_file = /etc/sindes/keys/apache.key
+  ca_file = /etc/sindes/certs/ca.crt
+  profile_prefix = profile_
+  use_fqdn = 1
+}}}
+
+{{{
+#!comment
+  * TIP: use ca_dir instead of ca_file (in AII, ccm.conf and aii-shellfe.conf)
+}}}
+ * to install a new node, one now has to:
+  * aii-shellfe --configure
+  * aii-shellfe --install
+  * you can already start the installation
+  * set the acl with acl -set -length 1000 -grant -target
+
+==== Renewal of CA cert ====
+'''On the quattor client side:'''
+ * remove all existing certificates issued (run this command only if you know what your are doing):
+
+{{{
+  sindes-bootstrap-ca -c
+}}}
+ * increase the {{{RELEASE}}} parameter in {{{sindes-bootstrap-ca}}} so it will create the new CA rpm with a different rpm version.
+ * then generate new ones by looking at section [wiki:SINDES#Configurationstepthroughguide Configuration step through guide]
+ * grant permission retrieve a new certificate
+ 
+{{{
+ su - sindes
+ acl -set -grant -length 5000 -target nodeXXX.wn
+}}}
+
+'''On the every node side:'''
+ * install the new CA certificate of the quattor client:
+
+{{{
+  rpm -Uvh http://quattor.begrid.be//begrid/swrep/noarch_sindes/SINDES-ca-certificate-q3-0.1-5.noarch.rpm
+}}}
+
+ * to avoid using globus version of openssl (gives -batch not found error):
+
+{{{
+  export PATH=/usr/bin/:$PATH
+}}}
+
+ * finally, renew the node certificate with:
+
+{{{
+  sindes-get-certificate -f
+}}}
+
+==== Troubleshooting ====
+ * if you have troubles with {{{sindes-get-certificate}}}, login on the machine and first do
+{{{
+export DEBUG=1
+}}}
+  This will turn on the debug and will allow you to rerun the commands used.
+  * most curl commands used by sindes run with {{{curl -f -s}}} for very silent running and failing. To debug, it's best to rerun them with {{{curl -v}}}
+ * logfiles can be found in /var/log/httpd and /var/log/sindes
+ * remove passwd from apache.key in /etc/sindes/keys
+
+{{{
+  cd /etc/sindes/keys;mv apache.key apache.key-2;openssl rsa -in apache.key-2 -out apache.key;chown apache.apache apache.key;chmod 400 apache.key; cd -
+}}}
+
+ * full cleanup: because high dependencies, best is to
+  * backup config-files (ca.config, sindesrc and sindes-ssl.conf)
+  * and run:
+
+{{{
+  rpm -e perl-SINDES-Shell-0.5-30 perl-SINDES-common-0.5-24 SINDES-Shell-bin-0.5-34 perl-SINDES-GetCertificate-0.9.99-1 SINDES-ca-0.9.99-2
+  rm -Rf /etc/sindes /var/sindes
+  yum install SINDES-ca
+}}}
+
+ * Make sure nslookup of the client fqdn works on the SINDES-server.
+ * on the client: when a problem arises with ccm-fetch, use curl to see if profiles are fetchable:
+
+{{{
+  curl --cert /etc/sindes/certs/client_cert_key.pem --cacert /etc/sindes/certs/ca.crt <link to profile>
+}}}
+
+ * a direct route to sindes server (ie the DN in the apache.crt) is needed. Passing through a NAT-box first will make it fail. Therefore you need:
+  * extra line in aii with explicit route BEFORE sindes-setup (don't forget to sleep!!)
+  * hard route config in the profiles (or ccm-fetch will not work)
+ * aii-installack.cgi failure:
+  * add a "--use_fqdn " to aii-installack.cgi
+  * aii-installack.cgi also needs a direct route to the cgi-server.
+ * if more than one crt resides in the CA crt dir, run in that directory (get the {{{Makefile.crt}}} from {{{/etc/httpd/ssl.crt}}})
+{{{
+  make -f Makefile.crt
+}}}
+
+ * When I wanted to renew the certificate of a workernode behind NAT, it failed when I run "sindes-get-certificate -f":
+{{{
+  ...
+  Something went wrong while contacting CA (curl returned 22)
+}}}
+ The error is because the WN connects to the quattor client via NAT instead of direct. I solved this by adding this in /etc/hosts:
+{{{
+192.168.10.4    gridy4.begrid.be
+}}}
+This is the private ip of our quattor client.
+
+
+=== Links ===
+ * Wiki pages [https://twiki.cern.ch/twiki/bin/view/FIOgroup/SinDes]
+ * CVS access:
+
+{{{
+  export CVSROOT=:pserver:anonymous@isscvs.cern.ch:/local/reps/fio
+  cvs co fabric/SINDES
+}}}