EPM 11.1.2.1 compact deployment update

In a previous post I went through the steps on deploying EPM 11.1.2.1 WebLogic based web applications in compact mode.

There were a couple of questions that were left unanswered questions in the post and I have been asked a few times what the answer is to them so I thought I would write up my findings.

Now if you are not intending of deploying in compact mode or have not read my previous post on the steps then this will probably be of no interest to you so you can go and do something far more interesting instead.

On to the first question, when deploying in compact mode there are a number of dependent products that need to be installed before running the deployment, in my case the deployment failed the first time due to not having HPS (Hyperion Performance Scorecard) installed and I didn’t really have any intention on using the product but at the time I just installed it to be able to proceed with the deployment.

I said that it may be possible to edit a few XML files to get around the need to have the dependent products installed, as it was niggling at me whether it could be done I went back and had a look.

The intention was to be able to deploy without having to install HPS and here are the steps that I carried out before running the compact deployment.




Details of the required librarie, dependent products and deployment details are stored within the following file
<MIDDLEWARE_HOME>/EPMSystem11R1/common/templates/compact/epm_system_11.1.2.1.jar



The file can be opened with a compression utility like 7zip so that all the XML files within the jar can be accessed and then edited, if you are planning on following these steps I would recommend taking a backup of epm_system_11.1.2.1.jar first.



The first file that was edited was template-info.xml, this file contains all the dependent libraries and deployment information the line containing “Oracle EPM HPS libraries” was deleted and then the file was saved.



The next file config-groups.xml contains all the products that will be deployed if they are already installed, two lines relating to HPS were removed.



Another line containing “hps_datasource” was removed from the same file.



The section highlighted above was also removed from the file and then the file was saved.



Within the config directory the file config.xml was edited and the above section relating to HPS was removed.



The datasource section relating to HPS shown above was also removed and the file was saved.



Finally the file hps_datasource-jdbc.xml was deleted from the jdbc directory within the jar file.

Now the compact deployment can be executed and it should not fail if HPS has not been installed, for full details on how to deploy in compact mode go back to my previous blog.

It is possible to remove other dependent products from the deployment by using this method to try and save some extra memory.

If you have products installed that are not on the dependent product list but are in the deployment list e.g. ERPi, Disclosure Management and you don’t want them deploying using the compact method then you will either to need to uninstall them and then install them after deploying in compact mode or use the method of editing the XML files as explained in this post.

I was going to go through my second question next but unfortunately I have not got any spare time left today so I will write it up in the next couple of days and then post it.

ODI Series – Essbase related bug fixes

There have been a couple of patches released lately for ODI which address issues with the essbase adaptor and I thought it was worth going through them as I have blogged about these specific problems in the past.

• Patch 10.1.3.6.7_01 contains the following fix - 9046176 - Wrong export order of members when using LKM Hyperion Essbase Metadata to SQL.A new column called sortid is added to the all the dimensions to order the records. This column is automatically populated using a counter.
  
  
• Patch 10.1.3.6.8 contains the following fix - 8826035 - Extract member's descriptions when we have more than one alias table in Hyperion.

Both patches contain updates to the knowledge module “LKM Hyperion Essbase (Metadata) to SQL” and updates to underlying Java code.

These bug fixes have not made it to ODI 11g yet but it might be possible to use the KM and Java files from the above patches (I have not tested this theory).

I will go through the wrong export order of members issue first, if you are not familiar with extracting essbase metadata using ODI then you can read an article I wrote on how to do it here.



Let’s use the measures dimension in Sample.Basic for the example, if a simple interface is created using the “LKM Hyperion Essbase (Metadata) to SQL” to extract the measures dimension then you would expect the output order of members to reflect the order in the outline.



As you can see the order is not correct, the dimension member “Measures” is the last in the order instead of being the first.

Now what I have done in the past is use a RDBMS target and add an ID column, in Oracle the ID column can be populated using a Sequence and for SQL Server an identity column property can be used.



This produces the above output.



If you reverse the output using SQL then “Measures” is outputted first though the order still is not correct.



If you were to build an outline based on that order you would get an outline like the one displayed on the left when what you want is the order of the outline on the right, basically all the levels are in the wrong order and not just the top level.

I was hoping that the patch release was going resolve the issue fully but let me go through it and see the output.

Once the patch has been applied make sure the “LKM Hyperion Essbase (Metadata) to SQL” is replaced using the import_replace option.



If the essbase database model is reversed again an extra numeric column called sortid is available within each dimension datastore.



A new interface was created to extract the measures dimension and sortid is mapped from the source to ID in the target.

To change the order of the output then a slight change to the IKM being used is required.



In my example I am using “IKM SQL to File Append” so the KM was edited and step “Insert new rows” was updated, the following code was added to the “Command on Source” -
ORDER BY <%=snpRef.getColList("","[EXPRESSION] ","","","UD1")%> DESC

This means the output will be ordered based on UD1 in the mapping of the interface.



UD1 was selected on the sortid column so the output will be ordered by this column and the interface was executed.



In my opinion the fix does not resolve the order issue fully, it has added a numeric counter to the output but the levels are still in the wrong order.

On to the second fix available in patch 10.1.3.6.8, this addresses the issue with being able to only extract the default alias table information.

I originally blogged about this issue here and went through a workaround by customizing the LKM and updating the Java API.



The patch release adds a new option in the LKM to enter which alias table to extract from; the idea is the same as what I used in my workaround so I had a quick look at the bug on “My Oracle Support”



I had a quick peak at the Java code in the new patch release and it is the same as what I wrote, so it looks my workaround has made it into official Oracle software, not that anybody in Oracle contacted me about it :)

There are a number of other essbase/planning/HFM related bug fixes in the latest patches so if you are experiencing any issues it is always worth have a look through the Readme.

EPM 11.1.2.1 compact deployment

The release of EPM 11.1.2 brought many changes across the product set and with the supporting architecture, gone was the ability to deploy web applications using Tomcat and only one option of WebLogic remained. This brought with it a much bigger footprint in terms of memory and pushed many down alternative paths to be able to run the system.

There are many reasons why high memory requirements can be restrictive, for instance if you wanted to run the full system for development, training, demos etc and intended on hosting on a laptop, yes it is possible if you pay for very high end spec machine or look to the cloud for inspiration but surely there were other possibilities.

With the release of 11.1.2.1 an additional option called compact deployment has become available which allows WebLogic based web applications to be deployed to a single managed server instead of a separate server for each product, so in theory reducing the overall memory requirement.

Today I am going to go through the process of deploying using the compact server method, now there are a few things to be aware of first.

• It is aimed at 64bit operating systems.


• Is available for new/upgraded 11.1.2.1 installations.


• Financial Management and Financial Close management are not included; these will still need to be deployed using the standard deployment method, though saying that it is still possible to use a large percentage of FM functionality without the WebLogic piece running.


• If the managed server crashes for any reason then all the web application running within it will be taken down.


• This deployment method should not be considered for anything more than a development environment, it wouldn’t be supported either.

There are two methods of deploying in compact mode, one when products are configured using a single database and the other when configured using multiple databases, today I am going to cover the single database method as it makes perfect sense to only use one database in this sort of deployment.

Now I am not going to go through how to install EPM 11.1.2.1 as I have covered that in the past and it’s not the purpose of this blog, I am starting off with a fresh install of EPM 11.1.2.1 deployed on a windows 2008 R2 64bit machine, the process covered today should be pretty much the same for other operating systems and if it is unix type system then it will just be a matter of running a .sh instead of .bat



Before deploying the web applications the other areas for each product such as database configurations have to be carried out first, so the configurator was started using <MIDDLEWARE_HOME>/EPMSystem11R1common/config/11.1.2.0/config-manual.bat

Usually the configurator is started using configtool.bat but using configtool-manual.bat starts the configurator as if the web applications are to be deployed manually, this means there will be no options available for web application deployment.



As this a fresh install a new EPM Oracle instance has to be created, I stuck with the defaults which creates an instance with the name of epmsystem1, the instance name is required later in the process.



The first-time configuration option is selected and details of the database are entered, this will only have to be entered once as all products are going to be deployed using one database.



All the required options will already be selected notice that there are no options available to deploy to application server.

It is also worth noting that the essbase server configuration option is not available either, now you can leave configuring essbase until later or do what I did and configure all the above first and then run configtool.cmd, deselect all and only select Essbase > “Configure Essbase Server”.

Once all the products have been configured then it is time to move on to the web applications compact deployment.





To start up the compact deployment open a command prompt and go to <MIDDLEWARE_HOME>/user_projects/epmsystem1/bin/compact and execute “runCompactDeploy.bat”

Four pieces of information are required:-

Weblogic Domain name – this should be the same name as the domain name created in the EPM configurator, the default being EPMSystem (though I used epmsystem1 as the domain name in my example as it was a fresh install with no domain created)

Weblogic Domain user name/password – I stuck with the default username used when configuring EPM which is epm_admin and the password is your choice, you will need these account details if you intend on logging into the WebLogic admin console for the domain.

Database user password – this is the password of the account used for configuring the one database in the configurator.

Once entered the compact deployment should spring into life and all should be good…



Unfortunately I hit a bit of an issue and thought I would explain why it happened as I am sure I won’t be the only one that gets hit with it.

The compact deployment ran through and by watching the command window intently I noticed errors appear and in the end was awarded with a pop up message stating startEPMSystem.bat did not exist.

The problem started to manifest when a WebLogic script is executed to create the domain and add the EPM library templates.

The exception was “The following prerequisites were found to be missing: Oracle EPM HPS libraries – 11.1.2.1”

It seems that there are a number of mandatory products that need to be installed before the compact deployment can be carried out.

I found the details of the required libraries in the following file which is used in the compact deployment -
<MIDDLEWARE_HOME>/EPMSystem11R1/common/templates/compact/epm_system_11.1.2.1.jar



Within the jar is a file named template-info.xml which contains the required libraries, the libraries are located at <MIDDLEWARE_HOME>/EPMSystem11R1/common/templates/applications

It looks like the following products may need to installed before running the compact deployment –
Foundation
Disclosure Management
Reporting and Analysis Framework
EAS
ERPi
Planning
Financial Reporting
Web Analysis
HPS (Hyperion Performance Scorecard)

So depending on what products have been installed will determine the exception generated, in my case the only product I had not installed was Performance Scorecard and that is why the error around “Oracle EPM HPS libraries – 11.1.2.1” was being generated.

It may be possible to update the xml file to alter the dependencies but it is not something I have attempted. (Update: it is possible and I have wrote a post on how to do it, have a read here)

This left me the only option to install Performance scorecard and then configured the non web app elements.

Once complete “runCompactDeploy.bat” was executed again.



This time everything ran smoothly and no errors were generated in the compactDeploy window, once completed another command window opens and starts up the WebLogic admin server for epmsystem1, a final command window opened starting up all the EPM web applications in one managed WebLogic server by executing <MIDDLEWARE_HOME>/user_projects/domains/epmsystem1/bin/startEPMSystem.bat



If you log into the WebLogic admin console for the epmsystem1 domain you will notice a server has been created called “EPMServer0” running on port 9000, this means all the EPM web application will be running against that port.

To log in to the admin console go to http://localhost:7001/console and enter the username/password that was entered earlier when running the compact deployment batch script



To view all the web applications deployed in the managed server click on “EPMServer0” and go to deployments tab.

The admin server can be stopped to save memory as it is not required and the "EPMServer0" managed server can be run indepently.



It is possible to test logging into some of the products using port 9000 but for all the products to communicate correctly through workspace the final step was to configure the http web server element.



Once the web server was successfully configured using OHS the web applications were all accessible via port 19000

To start the managed server execute - <MIDDLEWARE_HOME>/user_projects/domains/epmsystem1/bin/startEPMSystem.bat

To stop the managed server execute - <MIDDLEWARE_HOME>/user_projects/domains/epmsystem1/bin/stopEPMSystem.bat

No windows service is created by default to start/stop the managed server, I am sure it is possible to create one and maybe it is something I will have a look at if I ever get the chance.
(Update: the steps to create a windows service can be found here)



A quick look at the Java process running the WebLogic managed server indicated the minimum memory being consumed was 1.3GB, so if you compared that to starting the entire set of EPM web applications in a standard deployment then there is a considerable difference, starting up just “Foundation” and “R&A Framework” web applications can consume around 1GB.



Depending on the available memory on the machine being deployed on the JVM maximum memory can be increased and with it being 64bit then there is less of a constraint on an upper limit like with 32bit.

To increase the JVM edit <MIDDLEWARE_HOME>/user_projects/domains/epmsystem1/bin/setManagedParamsEPMServer.bat and update the value for -Xmx then restart the managed server.

I have not had a chance to really put the system through its paces but the performance seems reasonable considering what it will be used for, I recommend you try it out and make up your own opinion but at least it opens the door for deployments on machines with a lower memory capacity.

Planning 11.1.2.1 – Upgrade Wizard

The recent release of planning 11.1.2.1 brings new application administration features which are worth a quick mention.



These can be accessed from workspace under Administer > Classic Application Administration > Planning Administration.
Even though it is being accessed through Classic Administration it also applies to EPMA enabled applications.



The Upgrade Wizard is a new addition.




Previously if any data source elements had changed you would have to edit each data source individually, now it is possible to mass update this information.

So for instance if say the essbase server was rehosted or account information had changed then all the data sources can be updated at once.



Select the data sources that require updates, enter the new information, apply then save.



In previous versions if you wanted to upgrade a planning application to the latest release you would have to log into the planning application and select migrate, it is now possible to upgrade multiple applications at once which saves time if you have just upgraded planning.



In case of upgrade failure the upgrade process for each planning application is logged in <MIDDLEWARE_HOME>/user_projects/epmsystem1/Planning/planning1

and in the format of

PlanningAppUpgradeLog_<APPLICATION_NAME>.txt

If you have never understood what happens when a planning application is upgraded it is worth having a look at the log as it will give you a good idea of the steps taken.

The last option is “Update Reporting Essbase Servers”



If you are using the Reporting Application functionality essbase server connection information changes then it is possible update this for single or multiple applications.

I did notice a security flaw with the new functionality as if you go directly to http://<planning_server>:8300/HyperionPlanning/UpgradeWizard.jsp then anybody can access the Upgrade Wizard pages without the need for authentication, I am sure this will get fixed in subsequent patches but something to be aware of at the moment.

Planning 11.1.2.1 – Automate pushing of Reporting Application data

One of the new features in planning 11.1.2 was the ability to map and push data to a reporting application, I covered the reporting functionality in an earlier blog which you can read here, one of the drawbacks was if you wanted to push the data to an essbase database it had to be done manually, depending on the size of the data the export/import could take a while and it would be something you would want to schedule out of hours.

With the release of 11.1.2.1 a new command line utility called PushData has been created, the utility provides the functionality to automate the pushing of data for any Reporting Applications that have been set up in a planning application.



The utility is available with the other planning utilities in the directory <MIDDLEWARE_HOME>/user_projects/epmsystem1/Planning/planning1

The parameters to use the utility are -

PushData [-f:passwordFile] /U:username /A:sourceApplication / M:applicationMapping [/C]

Most of the parameters are self-explanatory, if optional /C is used then the data on the target is cleared first.
If you want to understand how to create a password file have a read here

PushData.cmd /U:admin /A:PlanSamp /M:PLANASO /C

The utility also produces a log available at -
<MIDDLEWARE_HOME/user_projects/epmsystem1/diagnostics/diagnostics/logs/planning/PushData.log



The log will capture whether the pushing of data was successful or not and any members that are skipped due to not existing in the target.

It is also possible to check the status through the Job Console within the planning application.

I did notice some strange anomalies in the log as shown above.

After successfully running the command line utility it can be batched up and scheduled using your chosen scheduler to run at required intervals.

Planning 11.1.2.1 - Exporting metadata

I have lost count of the amount times the question has been asked “how can I extract a planning applications outline”, until now the options have been pretty limited, one of the possible methods is to use LCM but it has its limitations and is not in the format that suits everybody.

Since the arrival of version 11 the planning Outline Load utility has done wonders and has been improving with each release, it should have been around well before 11 but at least it did arrive.

Now the utility is mainly aimed at classic planning applications, yes there are still many clients holding out with classic and why not.

With the recent release of 11.1.2.1 the utility brings in the option to export dimension metadata which I didn't see mentioned in the planning new features document, so the utility has evolved as not just a loader but an extractor as well.

The utility is available on the machine where planning has been deployed in the folder <MIDDLEWARE_HOME>\user_projects\epmsystem1\Planning\planning1\

I am not going to go through all the parameters of the utility as I have already covered the utility when it was first released

To extract a dimensions metadata it is as simple as executing:-

Outlineload /A:application /U:username /D:DimensionToExtract /E:outputFilename

For example OutlineLoad /A:PlanSamp /U:admin /D:Account /E:F:/PlanExports/acc_export.csv

If you want to include logging then you can use the parameters /L:LogFilename /X:exceptionFileName, if you don’t want to enter a password you can you the –F:passwordFile, how to set up a password file is available read here



The generated output CSV file is exactly the same easy manageable format that is required to load a dimension using the utility.

It is also possible to extract attribute dimension metadata.



To extract an attribute dimension all that is required is to provide the attribute name using the /D: parameter.



Output format is MemberName/Parent/Alias.



It is possible to export all the Smart List metadata within an application.



The parameter required for exporting Smart Lists is /DS:HSP_SMARTLISTS





Once again the output format matches the one for importing.

To export exchange rates tables the parameter used would be /DX:HSP_RATES



All the exchange rates tables available within the application are exported to the output CSV file.



The utility supports the exporting of planning unit hierarchies (which was available in 11.1.2) and the defining parameter is /D:PlanningUnitHierName

Enjoy!