One of interesting new features for EPM configuration in 11.1.2.3 is the ability to vertically scale based Java web applications through the configurator, yes it was possible to vertically scale in previous versions though I am not sure how supported the process was unless you were in the exalytics world and it certainly was not as slick as it is now.
At present the following EPM components do not support vertical scaling:
The EPM instance the web application has been deployed against is named epmsystem1.
In 11.1.2.3 the instance name are included in the display and service name of the service which plays a part in vertically scaling as the service name will be unique which was not possible in previous versions.
If you take a look at the servers in the WebLogic admin console there is a single managed server “Planning0” which is part of cluster “Planning”
To begin the vertically scaling process first start up the EPM configurator.
A new EPM instance is required to be created.
Only “Deploy to Application Server” is required for the product component as the other options have already been configured with the first deployment.
The Port and the Managed Server Name have incremented by one so now will be deployed as Planning1 on port 8301.
The port increase is another new feature in 11.1.2.3
“Port Manager, a feature of EPM System Configurator, manages port uniqueness by using Oracle Hyperion Shared Services Registry to check if a port is in use. Port Manager auto-increments ports so that ports not already in use are always shown.”
Once the web application has been successfully deployed then if you take a look at the window services there will be a new service for the vertically scaled web application.
A new instance will be created in the file system which means there will be a duplication of many of the files already created in the original instance.
It is also means that you will need to manage the logs under diagnostics in multiple places but there is no need to worry anymore because you have the new log analysis tool :)
Within the file system for the Weblogic EPMSystem domain there will be an additional executable which runs as the java web app process.
If you take a look in the WebLogic admin console the newly created managed server will be available and added to the existing cluster.
The web application has now been vertically scaled but at the moment the http web server is not aware of the additional server in the WebLogic cluster.
In the OHS mod_wl_ohs (the module which handles requests to be proxied from Oracle HTTP Server to Oracle WebLogic Server) configuration file there is still only the original server on port 8300.
The “WeblogicCluster” (click to the link to find more information) parameter is required to proxy a list of back-end servers that are clustered, or to perform load balancing among non-clustered managed server instances.
To allow the second instance on port 8301 to be configured with the http server the simple to solution is to run the EPM configurator again and choose to configure the foundation web server.
After configuring the file should now contain the server with the additional port.
To confirm everything has been configured correctly a new EPM deployment report can be fired up.
Once the web applications have been started there will be the two managed server processes running and both should be accessible through the test URL.
So there we have it, vertically scaling is definitely an option worth considering.
At present the following EPM components do not support vertical scaling:
- All Financial Management components
- EPM Architect Dimension Server
- All Strategic Finance components
- All FDM components
- Essbase Integration Services components
So what is vertically scaling web applications? Well in the EPM context it basically means the process of adding additional instances of already deployed J2EE web applications on the same server they were deployed on.
We are now in an age of not being confined to the dark days of 32bit limits with much more horsepower and memory starting to become the norm on single server 64bit machines so it can make sense to vertically scale the web applications.
With vertical scaling, the servers processing power, CPU usage, and JVM heap sizes are the main factors in deciding how many server instances should be run on one machine.
So as an example you have a machine with a planning web application deployment which is notorious for being resource hungry but you are still finding that the machine is being underutilized, the JVM has been tuned and there is a large percentage of memory available and the CPU usage is nothing to write home about so maybe this is a good candidate for vertically scaling.
Another consideration is the possibility of some form of high availability, if you have multiple instances of the same web application not only can you load balance to share the load but you are still operational if one of the web application crashes, ok if the machine goes kaboom then you in trouble but there is nothing stopping you horizontally scaling as well.
There is a great piece of research done on “One JVM (64bit) vs Two JVMs(64bit) on a 64bit OS” from the Oracle product assurance engineering team which is available in a EPM Infrastructure performance tuning guide white paper, the test concludes that there was a 20% performance improvement when using multiple JVM instances. The improvement came from the reduction in the time spent in Garbage Collection and a significant reduction in lock contention and more efficient use of the CPU shared cache and memory.
Anyway let’s have a look at vertically scaling a java web application, in my example I will be scaling planning on a windows machine but the process is the same for different java web applications and *nix based systems.
Planning has already been deployed to the default port of 8300.
We are now in an age of not being confined to the dark days of 32bit limits with much more horsepower and memory starting to become the norm on single server 64bit machines so it can make sense to vertically scale the web applications.
With vertical scaling, the servers processing power, CPU usage, and JVM heap sizes are the main factors in deciding how many server instances should be run on one machine.
So as an example you have a machine with a planning web application deployment which is notorious for being resource hungry but you are still finding that the machine is being underutilized, the JVM has been tuned and there is a large percentage of memory available and the CPU usage is nothing to write home about so maybe this is a good candidate for vertically scaling.
Another consideration is the possibility of some form of high availability, if you have multiple instances of the same web application not only can you load balance to share the load but you are still operational if one of the web application crashes, ok if the machine goes kaboom then you in trouble but there is nothing stopping you horizontally scaling as well.
There is a great piece of research done on “One JVM (64bit) vs Two JVMs(64bit) on a 64bit OS” from the Oracle product assurance engineering team which is available in a EPM Infrastructure performance tuning guide white paper, the test concludes that there was a 20% performance improvement when using multiple JVM instances. The improvement came from the reduction in the time spent in Garbage Collection and a significant reduction in lock contention and more efficient use of the CPU shared cache and memory.
Anyway let’s have a look at vertically scaling a java web application, in my example I will be scaling planning on a windows machine but the process is the same for different java web applications and *nix based systems.
Planning has already been deployed to the default port of 8300.
The EPM instance the web application has been deployed against is named epmsystem1.
In 11.1.2.3 the instance name are included in the display and service name of the service which plays a part in vertically scaling as the service name will be unique which was not possible in previous versions.
If you take a look at the servers in the WebLogic admin console there is a single managed server “Planning0” which is part of cluster “Planning”
To begin the vertically scaling process first start up the EPM configurator.
A new EPM instance is required to be created.
Only “Deploy to Application Server” is required for the product component as the other options have already been configured with the first deployment.
The Port and the Managed Server Name have incremented by one so now will be deployed as Planning1 on port 8301.
The port increase is another new feature in 11.1.2.3
“Port Manager, a feature of EPM System Configurator, manages port uniqueness by using Oracle Hyperion Shared Services Registry to check if a port is in use. Port Manager auto-increments ports so that ports not already in use are always shown.”
Once the web application has been successfully deployed then if you take a look at the window services there will be a new service for the vertically scaled web application.
A new instance will be created in the file system which means there will be a duplication of many of the files already created in the original instance.
It is also means that you will need to manage the logs under diagnostics in multiple places but there is no need to worry anymore because you have the new log analysis tool :)
Within the file system for the Weblogic EPMSystem domain there will be an additional executable which runs as the java web app process.
If you take a look in the WebLogic admin console the newly created managed server will be available and added to the existing cluster.
The web application has now been vertically scaled but at the moment the http web server is not aware of the additional server in the WebLogic cluster.
In the OHS mod_wl_ohs (the module which handles requests to be proxied from Oracle HTTP Server to Oracle WebLogic Server) configuration file there is still only the original server on port 8300.
The “WeblogicCluster” (click to the link to find more information) parameter is required to proxy a list of back-end servers that are clustered, or to perform load balancing among non-clustered managed server instances.
To allow the second instance on port 8301 to be configured with the http server the simple to solution is to run the EPM configurator again and choose to configure the foundation web server.
After configuring the file should now contain the server with the additional port.
To confirm everything has been configured correctly a new EPM deployment report can be fired up.
Once the web applications have been started there will be the two managed server processes running and both should be accessible through the test URL.
So there we have it, vertically scaling is definitely an option worth considering.
I'd have to try, but I suspect that you won't need to re-run the web-server config task if you choose the shared-configuration option -- when you do that, mod_wl_ohs.conf is automatically updated whenever you deploy new web-apps, so it should receive this change as well.
ReplyDeleteYeah it does update it automatically if OHS is configured to a shared location, it is not so clever yet if the instance is removed.
ReplyDeleteOnce again, thank you for the wonderful blog.
ReplyDeleteSorry I am new to this terminology and this questions might be irrelevant but how will there be a performance improvement if we are to use workspace? Does that mean that we if one of the planning service goes down, it will use the other planning service?
Is there a document on how this can be achieved in 11.1.2.2?
Thanks,
hyperionEPM
Could you please give me a hint what you mean with "because you have the new log analysis tool" ?
ReplyDeletedoes it exists ?
John, What do you think about having two EPM instances from two different environments rather than scaling vertically to cluster two managed servers of the same. For example, how about you have
ReplyDeleteSERVER1:
- PLAN_PROD_MAIN
- QA_PROD_FAILOVER
SERVER2:
- PLAN_PROD_FAILOVER
- QA_PROD_MAIN
So server 2 is QA and server1 is PROD and they serve as each other's "fail over" server.
Have not tested but may play with it to see how it behaves. Just a thought.
Thanks!
Pablo
Yes the log analysis tool does exist - http://docs.oracle.com/cd/E40248_01/epm.1112/epm_troubleshooting/ch03s01.html
ReplyDeleteHi Pablo,
ReplyDeleteInteresting concept though I am not sure how it would work with a centralised reg.properties and only one epm instance home supported per server.
Drop me an email with more information as I interested to know your thoughts.
Cheers
John
FYI - I don't believe that you can do this on Linux. I was only able to vertically scale out Essbase. I gave it a try and the option to deploy Planning was not there.
ReplyDeleteFrom the Oracle documentation:
"EPM System components support vertical scaling as follows:
l Windows-based Java web applications except for Oracle Hyperion Financial Management
and Oracle Hyperion Strategic Finance Java web applications
l Oracle Essbase Server
l UNIX-based components"
Not sure what the "UNIX-based components" are, but I was only able to scale out Essbase.
Hi Chris,
ReplyDeleteIt is definitely possible to vertically scale the java web apps in Linux
Thanks John. I will take a look again.
ReplyDeleteHi John,
ReplyDeleteIs it possible to let us known the Pros & Cons of Hyperion Planning Vertical clustering?
In which situation, we have to apply the HP vertical
Thanks