Tips & Tricks:  Reading FRED API data in RapidMiner

Recently I wanted to load in some XML data from the Federal Reserve Economic Data (FRED) website.  I didn’t want to use any extensions, just native RapidMiner operators. The FRED has a great API that lets you pull in all kinds of free economic and financial data (in JSON and XML format) – all you need to do is apply for a free FRED API key.

Accessing this data is really easy in RapidMiner Studio and all it takes is two operators: Open File and Read XML.
overall-process

 

 

 

 

 

 

 

In order to get any output, you have to form the access URL for that data. The way the URL is formed will tell the FRED data source to return what you want and in the format you want it. It requires you to enter in your API key and pass some parameters for the asset type, start dates, end dates, etc. You can read more about the FRED URL parameters here.

I was interested in accessing Euro/USD time series data and the only FRED API data available for this currency was the date and closing price (time series). This was sufficient for my needs and the symbol for the Euro/USD currency is DEXUSEU.

The resulting URL that I had to form looked like this:

https://api.stlouisfed.org/fred/series/observations?series_id=DEXUSEU&api_key=myAPIkey

Note: myAPIkey is my personal API key, you’ll have to substitute your own.

When I accessed the URL in my browser, I got the following raw XML results.

raw-xml

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

So far, so good.

Loading the data into RapidMiner

Now that we have a working URL, the trick is to access it and load it into RapidMiner.  The way to do that is to plug the URL into an Open File operator.  You set the “Resource Type” parameter to URL and then paste in your URL.

resource-type

 

 

 

 

 

 

 

Next you have to use the Read XML operator to be able to parse in the XML data correctly.  This operation can be a bit tricky but it’s a lot easier if you use the Read XML Import Configuration Wizard.  Through this Wizard you can select the nested XML data and parse it right into RapidMiner, like below.

xpath

 

 

 

 

 

 

 

 

 

 

 

 

 

xpath-attributes

 

 

 

 

 

 

 

 

 

attributes-metadata

 

 

 

 

 

 

 

 

 

The Result

Once you’ve gotten this all configured you can hit the Play button.  This is the result that is returned.

FRED API data process-results

 

 

 

 

 

 

 

 

 

 

You can now process the time series downstream by renaming the attributes and using our free Series extension.  If you are interested in trying this process yourself, just download the zip file for this process and import it into your RapidMiner Studio. All you need to do is supply your own FRED API key in the URL where it says “myAPIKey.”

Showing 2 comments
  • Brian T.

    Great tip! If you are looking for additional economic or financial data, there are a lot of available free datasets at Quandl (https://www.quandl.com/). You can load the data from there using the same URL/API key combination as you demonstrate here. They also have multiple formats (csv, xml, json).

    • Thomas Ott

      Hi Brian,
      You are correct. There are tons of great data sources out there that you can easily pull into RapidMiner! Perhaps we should start a thread in the community about where to find cool data sources.