Monday, December 23, 2013

XML the xfMap path

Most of the people I meet in the Geo-information sector are aware of FME as a translation tool  from one format to another.

That is originally what FME was built for (20 years ago) and is still one of the jobs where FME transcend all the rest of the available tools.

But FME is much more than a format translation tool, is as good as any GIS software available , and in my opinion it blows away most of them.

I constantly get amazed looks from costumers, when explaining them that most of their requirements can be done with FME, and in most cases in a single workspace.

The ability to dazzle with FME is not restricted to the spatial domain, FME's support of non spatial formats (for example the Excel writer, the shapefile of non spatial) makes it possible for non spatial people to benefit from FME when a serious user is around.

XML is another format where FME can be of great benefit, since almost any imaginable operation on XML can be done in FME, the combination of spatial and non spatial makes FME a truly versatile tool.

Since nowadays you can find XML almost everywhere, from OGC services styling to cadaster data sources all is served via XML.
The only main difference in the data is the schema (data model) and that can range from simple to extremely complex.

In FME the advised way of approaching XML data is by using feature paths, this makes XML handling very easy and does not require any in depth knowledge of the data schema.
However in some cases using feature path doesn't (and sometimes can't) handle the XML as good as xfMap.
xfMaps are the basis of XML handling in FME and if you know your xfMap then handling XML becomes super easy. The main drawback is that you do need in depth knowledge of the data schema.

The big advantage of using  xfMap over feature paths is the ability to read very large XML files, I suspect that when using feature paths the file is read into the computers memory, this can be an issue when dealing with huge volumes of data, even if your machine is supercharged and all of your FME settings are in place.

When using xfMap the file that crashed your machine trying to handle it with feature paths, will now easily load the same data.

I have tested this with the new BRK (new cadastral registration) data, available from the Dutch Kadaster.
Trying to parse the XML (260MB) with feature paths, just crashed my machine (and I do have a reasonable machine with the FME settings set properly).
With a very simple (and powerful) xfMap the same file loaded within 10 seconds and was ready to be transformed.

So to wrap it up, use xfMap if:
  1. The data volume is large and the feature paths configuration doesn't comply.
  2. The data schema is static
  3. You have too much time on your hands.

This is the last post for 2013, if you have already had some glimpses of what is to be expected for us with FME 2014, then I am sure you are as excited as I am!

A prosperous 2014 to us all.