How-To: Shibboleth „LocalDynamic“

06.01.2021 | Administrator


We also provide technical information an experience to all interested developers in e-learning. Following article is about movin Shibboleth configuration from MetadataProvider type="XML" -> „LocalDynamic“.

Overview

Shibboleth SP configuration with standard XML Metadata Provider and full eduGain metadata takes a long time on startup, depending on machine/vm power several minutes. In this document we explain moving to „LocalDynamic“ Metadata Provider.

In standard CentOS config additionally httpd did not start until shibd was started. Even though this can be reconfigured, still startup is long, we looked for a way to make this faster. In Shibboleth docs we found a method „LocalDynamic“, but no additional info how to set this up.

In this document we describe a way to use „LocalDynamic“ instead of XML Metadata Provider.

XML Metadata Provider

In most config how-to we find metadata provider sections like the following (can be more than 1 element:  

<MetadataProvider type="XML" url="<uri_to_metadata>"

   backingFilePath="<local_cache_file>" reloadInterval="3600"

   backgroundInitialize="true" ignoreTransport="true" >

   <MetadataFilter type="RequireValidUntil" maxValidityInterval="2419200"/>

   <MetadataFilter type="Signature" certificate=”<path_to_certificate>"/>

</MetadataProvider>  

Followed by  AttributeExtractor, AttributeResolver and AttributeFilter

The MetadataProvider type="XML" fetches the data from given URL , checks signature and stores it internally, renewing with given interval.

Unfortunately, this must be done also on startup and with large eduGain this requires time.

We change this to simple line

<MetadataProvider type="LocalDynamic" sourceDirectory="/var/cache/shibboleth/ed"/>

Which reads data from a local directory "/var/cache/shibboleth/ed”, startup is now very fast, (service shibd restart takes less than a second now on our VM), but: directory must be filled and regularly updated. In the following parts we describe how to do this.

Transport

First: we did not work on transport, downloading the metadata on our own for several reasons:

  • It’s not recommended, for security reasons
  • Would require more work!
  • Fill the LocalDynamic sourceDirectory and optionally
  • Create a discofeed json

So, we reserve one machine in our network with traditional MetadataProvider type="XML", and fetch copies from there, then machine with traditional MetadataProvider type="XML", can be any machine! The local copies can be found at attribute “backingFilePath”

The only 2 tasks are now:

MetadataProvider XML -> LocalDynamic

According to docs content of the sourceDirectory are multiple files each containing one single IdP EntityDescriptor with filename is the sha hash of IdP entityId.

hash generation (bash):

echo -n "<idp_entityId>" | openssl sha1

hash generation (java):

MessageDigest md = MessageDigest.getInstance("SHA-1");

String hash = new String(md.digest("<idp_entityId>"));

So, we wrote a simple app parsing the huge MetadataProvider xml files, extracting the IdP EntityDescriptor and generating files into LocalDynamic sourceDirectory.

Important: filenames are <hash>.xml (suffix .xml is important)

This can be additionally optimized by writing only changed IdP EntityDescriptors, each file should look like this (example file for TUM Munich):

Filename: b71c9974637b9c39e1bb57c784dfe100b5036408.xml:

<?xml version="1.0" encoding="UTF-8"?>

    <Extensions>

       […]

DiscoFeed

Only necessary when you use shibboleth-ds

In the same process creating the files of LocalDynamic sourceDirectory we also write a complete json discofeed.json file, depending on local setup. We place that in /etc/shibboleth-ds/ then do following changes:

In /etc/shibboleth-ds/idpselect_config.js

Changed:

to

this.dataSource = './discofeed.json';

Added Alias /shibboleth-ds/discofeed.json in /etc/shibboleth-ds/shibboleth-ds.conf

The structure of discofeed.json is simple: Array of

{

  "entityID" : "<entityId>",

  "DisplayNames" : [ {

    "value" : "<name1>",

    "lang" : "en"

}],

  "Descriptions" : [ {

    "value" : ">desciption1>",

    "lang" : "en"

  }],

  "Logos" : [ {

    "value" : "<logo url or also base64 data>",

    "width" : "<width>",

    "height" : "<height>"

 } ]

}

entityID + is necessary + at least ONE DisplayName!

More DisplayNames for more languages are optional, also descriptions and logos are optional!

Parsing this information out of a EntityDescriptor is very easy.

Putting to life

Added some error checking and monitoring of the process finally introduced this into production and test machines. Monitoring of shibboleth is quite crucial, you don’t want to end up in a stuck metadata loading process, so please make sure you monitor error entries on the machines with traditional MetadataProvider XML and the app generating LocalDynamic sourceDirectory files and DiscoFeed json.

Now we find following lines in shibd.log:

2021-01-06 14:05:59 INFO OpenSAML.MetadataProvider.Dynamic [1] [default]: resolving metadata for (<entityId>)

2021-01-06 14:05:59 INFO OpenSAML.MetadataProvider.Dynamic [1] [default]: caching resolved metadata for ((<entityId>))

2021-01-06 14:05:59 INFO OpenSAML.MetadataProvider.Dynamic [1] [default]: next refresh of metadata for ((<entityId>)) no sooner than 28800 seconds

For configuration testing machines LocalDynamic is very convenient that the startup process of shibd is lighting fast, things can be tried out very quickly even with production metadata in place.

Interested in the app? Let us know, it’s written in Java, we configured it in crontab hourly, can think of putting it into open source…