LCM 11.1.2.3 now supports planning essbase data (after patching)

A quick update from me as I noticed an 11.1.2.3 patch was recently released for Shared Services:

Patch 17307819: PSU 11.1.2.3.050

The patch mainly addresses the long awaited LCM backward compatibility to 11.1.1.4 but requires patch 11.1.1.4.500 to be applied which was also released at the same time.

I was looking through the list of defect fixes and the following caught my attention:

17348625 - Support is needed for the migration of Essbase data for Planning application using Lifecycle Management.

Over time LCM has matured into a well-established tool which does simplify the whole migration piece and is streets ahead of the world we used to live in, yes it does have some drawbacks and bugs but with each release it has improved and now supports the majority of EPM products.

LCM has supported Essbase data migration since 11.1.2.0 but this was for pure Essbase applications and the functionality has been missing for planning applications until now.

The patch requires applying on all servers in an EPM environment and updates not only Shared Services but EPMA and Planning as well.

Once applied new Essbase data artifacts are available for planning application in Shared Services.

The data migration supports both BSO and ASO planning applications.

The BSO export is a full text data export in column format, the ASO export is of course level 0 (not in column format)

 The export is the same as using LCM with Essbase applications except with planning the txt file extension is included while with essbase there is no file extension though it is still text format.

If the LCM planning export includes the Essbaase data artifact then the output content will not be zipped and be in exploded format as a folder.

If the planning essbase databases are large in size then be aware the export could take a while to complete and the database will be in read-only mode while the export is active.

It is worth understanding how the export is processed as it could potentially cause problems.

The export looks to be executed from the planning server and at first is exported to the
<EPM_ORACLE_INSTANCE>\tmp directory

 The export file is then renamed and moved to the LCM import/export directory.

If the export is large then this could impact the available space on the epm instance drive and add additional network traffic as the file is moved around.

It is possible to change the location of the tmp directory by updating java temporary directory for the planning web application server.

On windows this can be achieved by updating the registry.

For Unix type deployments then setCustomParamsPlanning.sh can be updated.

I did also notice a slight issue if the account running the planning managed server does not have access to the LCM import/export directory (in my case the directory was on a share) then the LCM process will complete successfully but the LCM Essbase Data folder will not exist, the data export will still run to the temporay directory but that is as far as it goes even though it does not fail, this does not happen for any of the other planning artifacts and did throw me for a while but updating the permissions resolved the problem.

When importing the data using LCM the data is not cleared out and it is the equivalent of importing with overwrite values.

There is no option to clear the data which would be nice and I have always thought it would be great if there was an option to completely clear out the target planning application before running an LCM import, maybe one for the future?

If importing to a planning ASO database then any aggregations will need to be run as this is not handled by LCM.

I did test using the EPM clone utility which I blogged about here and can confirm once patched it does automatically export/import the planning data.

EPM 11.1.2.3 clone utility

Whenever I complete a build of an environment I test out basic functionality of the installed products with a set of sample applications, reports and integrations, if there are a number of environments to build this process can sometimes be a little tedious so I was looking for a way to speed up and simplify the process.

In 11.1.2.3 a new command line utility appeared which allows the cloning of environments and it basically uses LCM (Lifecycle Management) as the engine to achieve this.

I have known about the utility since it was released but never got round to testing it out and was wondering whether it could help with my situation.

In theory the utility could be used to clone any environment and possibly take scheduled snapshots.

There are some points before using the utility worth highlighting:

• It only operates against LCM enabled products. (if you are not sure which are enabled then have a read here)
  
  
• It does export Essbase and HFM data and as it operates in serial this can not only be an extremely slow process the size of the export could be huge. It doesn’t look like there is a parameter to exclude the data.
  
  
• LCM does not yet export Planning data so this needs to be addressed separately.
  (Shared Services Patch 11.1.2.3.050+ supports planning data)
  
  
• You can only import in an environment where there are no applications for Planning, Financial Management, and Profitability and Cost Management.
  
  
• It does not export deployment metadata which makes sense as you wouldn’t want to transfer that across environments unless you wanted to corrupt it.

Before I put the utility to the test I created two like for like 11.1.2.3 environments and in the first environment I created the following:

• Shared Services provisioned users.
• Essbase applications (BSO/ASO) with data, rules, scripts, report, variables.
• Default sample planning application which was initialized and refreshed.
• Calculation Manager rules, rulesets and variables.
• Full statutory HFM sample application with data.
• Financial Reporting reports for Essbase, Planning and HFM.
• FDMEE integrations for Essbase, Planning and HFM

The Planning and HFM application were both classic and not EPMA which I will explain the reason for later.

These sample applications should give a good representation of a standard EPM deployment and provide a simple test for the cloning utility.

The utility is broken into export – epm_cloneexport.bat/sh and import - epm_cloneimport.bat/sh and these can be found in

<MIDDLEWARE_HOME>\user_projects\<instancename>\bin

To run the utility there is only additional piece that is required and that is a properties file which contains the admin username and password.
 

To run the export from command line the following syntax is required.

epm_cloneexport.bat properties_filename

Once executed the utility will cycle through all the available artifacts in the environment and export them.

The properties file is updated and the password is encrypted after the first use so there are no security implications.

The full export log is available at:

<MIDDLEWARE_HOME>\user_projects\<instancename>\diagnostics\logs\migration

The utility outputs one folder named EPM_CloneExport which is in the environments LCM import/export location.

Beneath the EPM_CloneExport folder the output is broken into each product area just like with any LCM export.

 In Shared Services you can access the full export under the File System.

The options available are exactly the same as standard with LCM and it is possible to use them at full export level or by individual product.

In Shared Services viewing the Migration Status Report will also display the output from running the utility.

Right so that is the export taken care of the next step is to copy the EPM_CloneExport folder over to LCM import/export directory on the target environment, as usual I hit the windows path limit on some of the folders so used robocopy to move them over to the other environment.

Before running the import using the clone utility I created the planning applications data source with the same name as the source as the utility will not be able to create the planning application without this in place.

I then edited the Users.csv file which contains the native users and removed the admin user as otherwise it will update the password for the admin account with the same password as the source environment

 To run the import from command line the properties file needs recreating and then the following syntax is required:

epm_cloneimport.bat properties_filename

Once executed the utility will cycle through all the tasks defined in the import.xml file (automatically generated from the export) and due to the different artifact dependencies it should hopefully import them in the correct order. 

 
 It is possible to edit the xml file and remove tasks if they are not required for the import.

The import acts in the same way as the export and produces a log in the same location and the results are viewable in the Migration Status Report within Shared Services.

The report is indicating that the import failed and by selecting the failed status it provides further information.

The import failed on just two Essbase custom defined functions which are used by Calculation Manager, the comment field can only be a maximum of 256 characters and both these functions seem to have comments which are longer.

I am not sure how Calculation Manager manages to register them in the first place in the source environment if they exceed the limit.

Even though the CDF LCM files don’t have a file extension they are all in XML format and can be opened with any text editor.

As a workaround the comment can be updated in the file to shorten the length to less than 256 characters and then imported through the standard LCM mechanism through Shared Services.

If you get hit with the following error then in most cases it can be ignored.

The reason the error is generated is if the source environment does not have any associated artifacts available then it will not create a resource directory, when importing the utility will try to locate and directory and as it does not exist generate an error.

Once the import has successfully completed then there may be a few manual updates to carry out for example if the Financial Reporting database connections are different between source and target such as HFM cluster name.

For FDMEE you would need to copy application data files and update the root folder values

Now I have a cloned environment without too much hassle but remember this was with classic applications and not EPMA ones, the reason I left the applications as classic is that usually with LCM and EPMA the applications have be created through EPMA by deploying before importing the remaining applications artifacts, maybe the utility had somehow got around this but I would be shocked if it had.

I ran another clone export from the source environment but this time I had converted the planning application to EPMA before running the utility.

I cleared down the target environment by reverting to a previous snapshot, copied the EPM_CloneExport folder across, fixed the Essbase CDF issue and then ran the clone import utility again.

As I expected the import failed with the following errors:

Basically because the planning application is now EPMA enabled the application definition file will be at the EPMA level and not the planning application level, as the application does not yet exist LCM tries to create the application as classic but fails because the definition file is not there.

The FDMEE artifacts then fail to import because the planning application does not exist.

How about if I try deploying the application from EPMA after the failure?

This creates the planning application so I can try running the clone import again.

The import will not run because one of the utility rules:

• You can only import in an environment where there are no applications for Planning, Financial Management, and Profitability and Cost Management.

It is interesting that the error states “Please run the epm environment reset script”, I am not sure which script this is referring to but if anybody does know then please let me know.

This is where the clone utility exposes a major flaw as how it is possible to clone an environment which has EPMA enabled applications if it can’t create the applications and the applications can’t already exist.

Once again when working with EPMA you feel the pain.

I suppose the workaround would be after the import fails to import the application and any other artifacts that failed through the standard LCM route.

Using this method is not idea but at least it should be successful.

Alternatively for my testing strategy I could create the applications as classic in the source environment then run the clone export and finally convert them to EPMA.

Well there we have it if there are only classic applications in the environment the utility works quite well but if EPMA is in the mix then it adds in additional complications.

If in future releases the utility can get around this issue and provide the option to export with or without data it could prove to be quite useful.