Recently I have being working on a project where we were occasionally being hit with various web applications producing 404 errors for no apparent reason, once the issue had occurred the only way to resolve was to restart the web app.
There seemed to be no correlation with activity on the web application and the issue occurring, it was also affecting different products at different times. In the end the problem was traced back to the WebLogic web applications trying to access temporary files that no longer existed, the files were created when the web app was started up but for some reason were being deleted at random intervals.
This issue will only affect windows deployments and the reason for it happening is when the web applications are first deployed they are configured to use a temporary directory associated with the windows account that is carrying out the configuration.
Now the problem doesn’t occur just because of the temporary directory, the issue occurs when a remote desktop session is used to connect to the machine hosting the web apps and the same account is being used that deployed the EPM web applications, occasionally when the user logs out of the machine windows cleans up any temporary files that are associated with the account so this wipes out all the temporary files being used by WebLogic causing the web applications to fail.
There are a few ways to resolve this problem, the easiest being just don’t log into the machine using the account that configured the system, another is to change the configuration settings for terminal services and turn off the deletion of temporary files on exit or my personal preference is to go through the registry and update the temporary file location that is being used for each deployed web application.
This can be done by starting up the registry editor and locating
HKEY_LOCAL_MACHINE\SOFTWARE\Hyperion Solutions\<web application>0
On selecting one of the web applications there will be
“-Dweblogic.j2ee.application.tmpDir=…” usually associated with JVMOption3
Update the folder location to a permanent folder, an example permanent location that could be used is
<Middleware Home>\user_projects\domains\EPMSystem\servers\<WebAppName>\tmp
Repeat the process for each of the deployed web applications; once the registry changes have been made the web application will need to be restarted to take effect. I am sure this issue will be fixed in the imminent release of 11.1.2.1 as it is a known issue but I thought I would put a quick post out there just in case there are others that are hit with this problem and search for a resolution.
Worked like a charm. Thanks John!
ReplyDeleteWorked like a charm. Thanks John!
ReplyDeleteHi John, you can actually turn of terminal services temp variables, update the windows user temp variables (delete them) and rely on system variables and re-deploy. Works great except for Shared Services you should use the documented work around unless you have it patched as without Shared Services patch a re-deploy clobbers things.
ReplyDeleteThanks for the information, hopefully this shouldn't be an issue from now on with the issue being resolved in 11.1.2.1
ReplyDelete