Sunday, 12 July 2009

ODI Series – Executing Hyperion Business Rules

At a conference I was attending a few weeks back a question was asked – “Is it possible to run business rules through ODI?”.
Now it is certainly true that none of the adaptors have the functionality to run business rules but I thought I would go through a couple of ways that it is possible to execute them.

This is going to be based around classic business rules and not calc manager ones. I always thought the available functionality outside EAS has been pretty poor for business rules, you have the choice of the not so great web launcher or the command line launcher, and there has never been an API that I feel would have been a lot more useful.

Anyway in your business processes you may have business rules that are run a scheduled basis and you want to integrate them with your current ODI processes.

The first option is to use the OS functionality in ODI procedures, I am going to use the command line launcher in version 11 which is pretty much the same as previous versions except you have to use an encrypted password file. I am also going to be using a windows O/S but the example could easily be modified to fit a different environment.

The documentation for the command line launcher is completely out of date but the usage should be :-

CmdLineLauncher [-p:] -Sservername -Uusername [-rBusiness Rule Name | -sSequence Name] [-fRun Time Prompts file]

The important parameter to watch out for is –fRun Time Prompts file as even if you are not using run time prompts you still need to provide information on which database you want the rule to be run against.
The run time prompts file can be generated from within EAS by right clicking on a rule and choosing “Automate Launch Variables” and this will generate a file.

The ExecDB parameter is always required, all parameters are paired together with ::
You can manually edit the file to add in more run time prompts if required.

If you are using version 11 then you will have to create an encrypted password file first, there is a utility (PasswordEncryption.bat) that is in the same directory as the command line utility (\Hyperion\products\Essbase\eas\console\bin)

So an example to run a business using the command line launcher would be

Ok, so we need to transfer this example to ODI. In the designer you will need to create a new procedure, add a new command and set it against “Operating System” technology.

To use the command shell it is good practice to start with cmd /C this means the command will be terminated after execution

If you execute the procedure as it is then it will fail.

Not the most useful error code but what you need to realise is that the command is not being executed in the directory of the utility, if you have a look at the command line launcher batch script you will see one of the reasons why it is failing

The script uses a relative path so none of the java files will be added to the classpath, I created an environment variable to point to the EAS directory and updated the batch script to use it.

It is up to you whether you want to use an environment variable or just set one up in the batch script.

You also need to be aware that when calling the command line launcher from ODI you need to specify the location of the password file and run time prompts file otherwise it will not locate the files.

Once again, this can be done by hard coding the path or using environment variables.

Another option would be use ODI variables and they could be set a run time if required. (If you want to know more about ODI variables have a read of this previous blog)

You could set a variable up for each parameter of the command line call.

This way the variables could be set when you run the integration or they could be constants or set by retrieving the information from tables or files, there are many different options to choose from.

This time when you execute the procedure it should through without any problems.

So that is one method which does what it is supposed to but is a little limited, the next option would to be use a little piece of Java to call the launcher, now unfortunately there is no HBR launcher API but that is not going to stop me and I will show you how it can be done.

Before you start you will need to use a number jar files in your classpath, most of them are displayed in the batch file but I found that it required an extra jar file to get it working (xalan.jar). I just copied them all to another directory.

You will also need the HBRCLient.propeties file as the java code looks for this file by default.

Basically you just need to pass in the server name, username, password, rule name, the RTP file location and the location of the file, this is set as a system property (HBR_HOME)

The main method is not required this is just to test the java code was functioning; it can be removed after testing.

In using the java example you don’t need to provide the location of the password file you can pass it straight into the method, unfortunately you still need to provide the location of the run time prompts file. I did have a look whether you could get round this and it is possible but you would have to change the core java code and I didn’t want to head in that direction.

If you are interested then in the LaunchManager class there is HashTable that stores the run time prompt properties.

If this is changed from private to public you could probably push the RTP values into it, as it is currently private you can’t access it from outside of the class.

Once you have the code working and executing the rules then it can be compiled and put into a jar file.

To be able to use this java in ODI you need to let ODI know where all the jar files are, there are few ways of doing this, you can just copy all the files in the oracledi\drivers directory, make sure you restart any ODI GUI and agents.

Another way would be to update the classpath in odiparams file in the \oracledi\bin directory.

This will dynamically add all the jar files to the classpath from the directory you specify, in my case E:\Java\HBRlauncer

If you are running an agent as service then you will have to update a configuration file that is called when the service is started, it is snpsagent.conf and is located in \oracledi\tools\wrapper\conf

I just added a new line with location to the jar files.
One thing to watch out for with the above methods is not to include xercesImpl.jar as it causes conflicts with the other classes that ODI uses.

You can just remove the file and the code will still function correctly as it will use a class already referenced by ODI.

With the files in place you can now create a new procedure in ODI to call the java method, the procedure could use “Java BeanShell” or “Jython” technology.

I am going to use Jython the same technology that the Hyperion KMs use.

In the example I just import the class in, set up the variables and pass them into the Java method.

Once again you can set use ODI variables instead of hard coding all the parameters.

If you any of the parameters are incorrect it should raise an error.

If the rule has run successfully and you are running an agent as a service then it will be logged in \OraHome_1\oracledi\bin\agentservice.log

The hbrlaunch.log will also be populated with information about the execution just like it would if it was run from EAS or planning.

The only issue at the moment with this solution is that you have to manually edit the RTP file each time you want to make changes, well as usual there are ways around this and you can let ODI do the work and create the RTP file for you.

You could have the information stored in a db table, a file or an ODI variable and create an interface to generate the file. I am going to show how this can be done using an ODI variable that could be populated at run time.

First you need to create a File Datastore point to the directory and name of the file.

Create one column with a length that will be long enough to hold RTP information.

Now create a variable of text type to hold the RTP information

Create a new interface, set the staging area (I am using SQL Server as the staging for this example) and drag the file Datastore that was just created to the target.

Next in the mapping I added the reference to ODI variable and used a replace function to replace spaces, this is because with a ODI text variable the lines are separated by a space, I replaced the space with a line break char(10).

This means it should write a new line for each line of the text variable.

I know your asking, but what if the value element of your variable has spaces in it, well if that is the case you could add something to distinguish each separate line for example.

Then use the replace as

replace('#HBR_Variable','@NEWLINE@ ',char(10))

In the flow section of the interface the IKM to use is “IKM SQL to File Append”, though it will require some changes as if you try and run the interface it will generate an error in its current state.

This is because the IKM is based on the interface having a source and we are not using a source so the SQL generated is incorrect.

As you can see there is no table name, to get around this, duplicate the “IKM SQL to File Append” and give it a new name

In the details tab, you can delete the Command “Insert column headers” as it is not required and then edit the Command “Insert new rows”

In the source tab you view the code that is generating the SQL when the IKM is used

As you can see it has from and then an ODI command to retrieve the table name, this can be updated, if you are using SQL server as the Staging Area then you can remove everything from just before from.

If you are going to be using Oracle as the staging area you would change it to “from dual”

So what this means it will ignore generating any source table name and will just run SQL to display what you have in the target mapping of the interface, in this example it will be the text from the ODI variable.

Once you have saved it you can update the IKM in the flow of the interface to use the newly updated IKM.

I have used TRUNCATE option so the file is recreated each time.

When you execute the interface and check the steps in the operator you can now see that it has generated the correct SQL.

The output file now has all the required information to be used with HBR launcher.

A package can be created to bring it all together.

If you generate a scenario from the package and integrate some of the web functionality I have shown how to create in previous blogs, you will have your own HBR web launcher with the benefit of being able to add other ODI elements into the package.