Oracle WebLogic Server Optimal Flexible Architecture and Best Practices

Introduction

The OFA Standard is a set of installation guidelines that will give you faster, more reliable, Oracle Fusion Middleware WebLogic environments that require less work to maintain.

Nowadays due to the evolution of the storage systems we can easily state that the main purpose of the OFA standard is no more to improve the performances by spreading the Fusion Middleware components over the storage infrastructure, but it is really to provide a logical and efficient directory structure, easy to maintain.

Based on OFA, dbi services developed a WebLogic dbi Management Toolkit (WebLogic DMK) allowing to easily moving around the core directories of the oracle products. Dbi services also adapted the OFA standards to customers’ needs and introduced its WebLogic management experience in this document. Therefore some recommendations won't match exactly with the strict original OFA document. The purpose of this document is also to provide a clear presentation of the OFA architecture in order to be able to deploy it on any Oracle server infrastructure. In summary the core advantages of the OFA architecture are listed below:

• All servers have the same directory structure, decreasing the maintenance and troubleshooting efforts (and costs)

• Propose a structure to install Oracle Fusion middleware products

• Since all the servers use the same structure the deployment of scripts is easier from a central server

According to its Oracle Fusion Middleware experience, dbi services strongly advices the usage of the OFA standards, or at least to follow as much best practices as possible.

Fusion Middleware install directories

Fusion Middleware 12C ORACLE_HOME directories

The Fusion Middleware HOME should be created as below:

/u01/app/oracle/product/fmw

Directories created for a WebLogic Server install:

Directory	Purpose
/u01/app/oracle/product/fmw	Oracle software sub tree for release 12c products (12.2.x )
/u01/app/oracle/product/fmw/wlserver	Oracle home directories for Oracle WebLogic Server
/u01/app/oracle/product/fmw/logs	Installation and configuration log files.
/u01/app/oracle/product/coherence	Coherence cache directory
/u01/app/oracle/product/modules	Default libraries modules directory
/u01/app/oracle/product/utils	Utilities to install WebLogic patches
/u01/app/oracle/product/fmw/OPatch	Oracle Patch Installer utility directory
/u01/app/oracle/product/fmw/install	Install properties file directory
/u01/app/oracle/product/fmw/inventory	Installer inventory directory
/u01/app/oracle/product/fmw/oui	Oracle Universal Installer directory
/u01/app/oracle/product/fmw/oracle_common	The directory that contains the binary and library files that are common to all the Oracle Fusion Middleware products and features installed in the Oracle home
/u01/app/oracle/product/fmw/oracle_common	The directory that contains the binary and library files that are common to all the Oracle Fusion Middleware products and features installed in the Oracle home

Additional directories after installing a Fusion Middleware Component:

Directory	Purpose
/u01/app/oracle/product/fmw/[ProductDir]	The directory within the Oracle home, which contains the binary files associated with a Fusion Middleware Component.

Additional directory after a WebLogic Domain is created and default settings used for domain home

Directory	Purpose
/u01/app/oracle/product/fmw/user_projects	This directory contains all configuration files and data related to the domains.

Configuration data directories

Directory	Purpose
/u01/app/oracle/config	This directory contains the configuration and application data
/u01/app/oracle/config/domains	This directory contains the WebLogic Domains
/u01/app/oracle/config/applications	This directory contains the application data

OFA Naming guidelines

As OFA should be a "flexible" architecture we provide only guidelines and best practices concerning the naming convention of the several Fusion Middleware components.

The Fusion Middleware HOME directories naming convention changed during the last years. Because of the introduction of new Oracle products, the Fusion Middleware ORACLE_HOME directories have to be adapted in order to be easily identified.

Note that this structure does not allow identifying the patch level which has been installed. For this purpose dbi services advices to add a version file with prefix “weblogic_version_” with the fourth digit of the release concatenated to the prefix name, for instance:

• Weblogic_version_10.3.6 for a fusion Middleware 11G WebLogic 10.3.6 software installation.

• Weblogic_version_12.1.0.1 for a fusion Middleware 12C WebLogic 12.1.0.1 software installation.

Mount points

It is important to consider that each mount point represents a separate filesystem. In some cases, it should even be a filesystem stored of different disks (i.e. file systems for different WebLogic Domain). Below a list of components which should be separated over different physical disks:

Strongly advised:

• Oracle software and Configuration data (WebLogic Domains and/or Application data)

Recommended:

• Admin Server and Managed Servers log files

DBI OFA directory structure

The easiest way to present the OFA structure is to lay it out in a table. Below the table, several explanations are provided to explain this structure.

In this document we consider the fact that OFA is a "Flexible" architecture. Therefore we provide instructions but only advices and best practices each one is free to respect or not. Also the level of details until which OFA should be implemented can be defined by the customer according to its requirements.

Unix – Windows directory structure

In the following of this chapter, the directory separator “/” has to be replaced with “\” for windows environments. The column “Mount Point” describes the UNIX mount point and the Windows drives.

Mount Point	Directory	Owner	Description
Unix: /u01 Win: D:</p>		root	The /u01 mount po
	/app [ UNIX/Linux only ]	root	Application directory
	[/app]/weblogic	weblogic	basis for all Oracle Fusion Middleware components

WebLogic Middleware - Binary structure

Mount Point	Directory	Owner	Description
Unix: /u01 Win: D:</p>	[/app]/weblogic/product	weblogic	Directory containing the Oracle software installations
	[/app]/weblogic/product/Midddleware	weblogic	WebLogic Product directory
	[/app]/java	weblogic	JDK installation directory

WebLogic Middleware – Configuration structure for simple WebLogic domain

Mount Point	Directory	Owner	Description
Unix: /u02 Win: E:</p>	[/app]/weblogic/config	weblogic	Directory containing the domains and applications files
	[/app]/weblogic/config/domains/ <domain_name>	weblogic	WebLogic domains directories
	[/app]/weblogic/config/applications/ <domain_name>	weblogic	Applications directories

log files

Mount Point	Directory	Owner	Description
Unix: /u02 Win: E:</p>	[/app]/ weblogic/config/domains/ <domain_name>/servers/ /logs	weblogic	Default directory for the managed servers log files

WebLogic Middleware – Configuration structure for clustered WebLogic domain

Mount Point	Directory	Owner	Description
Unix: /u02 Win: E:</p>	/app/weblogic/config	weblogic	Base Directory for the WebLogic Domain
	/app/weblogic/config/domains/admin/ <domain_name>	weblogic	WebLogic domains directories for Admin server
	/app/weblogic/config/domains/ mservers/<domain_name>	weblogic	WebLogic domains directories for managed servers
	/app/weblogic/config/applications/ admin/<domain_name>	weblogic	Applications directories for the Admin server
	/app/weblogic/config/applications/ mservers/<domain_name>	weblogic	Applications directories for the managed servers

The domain separation in two sub directories admin and servers allows the admin server if configured to listen on a virtual IP address to be started on another machine in case of a hardware failure.

• If this installation directory is a shared drive, the virtual IP address needs to be declared on the other machine before starting the Admin Server.

• If not using a shared drive, the admin directory needs to be restored on another machine and the virtual IP address declared on this machine before starting the Admin Server.

Clustered WebLogic Domain with managed servers migration

If managed servers migration in case of machine failures is required:

• The “admin” directory and the “servers” directory in the domains have to be located on shared drives.

• Each WebLogic servers (admin or managed servers) have to be listening on virtual IP addresses.

• Persistence store and transaction logs must be backed by an highly available network file system or a database.

Two types of server migrations are possible:

• Manual WebLogic server’s migration is possible. The virtual IP address has to be declared on the other machine before starting the WebLogic server.

• Automatic WebLogic server’s migration is done using the node manager.

log files

Mount Point	Directory	Owner	Description
Unix: /u02 Win: E:</p>	[/app]/weblogic/config/domains/admin/ <domain_name>/servers/<server_name>/ logs	weblogic	Default directory for the Admin servers log files
	[/app]/weblogic/config/domains/mservers/<domain_name>/servers/<server_name>/logs	weblogic	Default directory for the managed servers log files

Administration structure

Middleware DMK

Mount Point	Directory	Owner	Description
Unix: /u01 Win: D:</p>	[/app]/weblogic/local/dmk	weblogic	WebLogic Management Kit

The DBI WebLogic Management kit sets environment variables, aliases and provide generic commands scripts to start, stop and monitor the WebLogic domains installed. Additional scripts to facilitate clustered WebLogic domain installation.

DBI Best Practices

OFA recommendations

DBI services recommends to follow the Optimal Flexible Architecture structure described in chapter 4.

Unix user

Each Product has resources requirements like “maximum number of user processes” or “maximum number of file descriptors”. To avoid troubles between several products, the recommendation is to have a separate user for the WebLogic server or Oracle Fusion Middleware and database software when installed on the same machine.

Node manager and start scripts

The usage of the node manager to start the Admin Server and the Managed Servers allows an automatic restart when a Managed Server goes down. The Node Manager will try to restart the failing Managed Server a few times depending on the “Max restarts between Intervals” and “restart Intervals” parameters set for each Managed Server. The node manager should be started at the startup of the hosting machine.

JVM start parameters

Heap Size settings

In production environments, set the minimum heap size and the maximum heap size to the same value to prevent wasting VM resources used to constantly grow and shrink the heap. This also applies to the New generation heap sizes (Sun) or Nursery size (Jrockit).

Out of Memory Exception

Most of the production environments first hurdle is “Out of Memory”. The parameters to create a heap dump when “Out of Memory” exception is raised should be put in place. This will allow to investigate on the reasons of this exception even.

-XX:+HeapDumpOnOutOfMemoryError-XX:HeapDumpPath=<path_to_heapDump_prof_file>

GC monitoring

In testing phase, monitoring the garbage collector might be interesting to adjust the heap or diagnose performance issues related to memory usage and garbage collector.

-verbose:gc –XX:+PrintGCTimeStamps –XX:+PrintGCDetails –loggc=<GC_log_file>

Where GC_log_file is: ${DOMAIN_HOME}/servers/${SERVER_NAME}/logs/verbosegc.log

Log files management

By default, the log files are generated in the servers’ logs directory. As explained earlier in this document, DBI recommends to configure the log files generation on a separate drive for customers who wants to keep a huge number of those. The DMK provides scripts to configure the logging on existing domains. DBI suggests the log files to be rotated date based at mid-night, and keep 180 files (near 6 months) when normal logging level is configured.