BEA-141196 The managed server could not update the configuration files

Error -

<Error> <Management> <BEA-141196> <The managed server could not update the configuration files during the registration with the deployment service. The update failed due to an exception:

weblogic.management.DeploymentException: [Deployer:149233]An unexpected error was encountered during the deployment process.

Solution –

1.       Config directory in the managed server could have got corrupted.

2.       Take a backup of the existing config directory of the server - <domain>/config

3.       Copy the config folder from Admin server to the Managed server.

4.       Restart the Managed server

BEA-001129: Received exception while creating connection for pool

While starting the Weblogic Admin / Managed servers, we get the Connection pool error 

Error Message - 

<Warning> <JDBC> <BEA-001129> <Received exception while creating connection for pool "conpoolname": ORA-01034: ORACLE not available

ORA-27101: shared memory realm does not exist

<Error> <JDBC> <BEA-001112> <Test "SELECT 1 FROM DUAL" set up for pool "Conpoolname" failed with exception: "oracle.jdbc.xa.OracleXAException".>

<Warning> <JDBC> <BEA-001129> <Received exception while creating connection for pool "Conpoolname": ORA-01034: ORACLE not available

ORA-27101: shared memory realm does not exist

Solution - 

The issue mostly is that the Datasource is not able to connect to the Oracle Database.

1. Login to the Admin server, navigate to Services -> Datasources

2. Go to Monitoring tab -> and do a Test Datasource.

3. If the issue still persists, go to the putty session, navigate to the <domain path>, and select 'config', 'jdbc' folders. check the jdbc.xml if the datasource values are correctly entered.

 4. After validation, restart the admin/managed server

Weblogic: Log files in Weblogic server

Log files in Weblogic server

Linux Tips: backup fstab

When copying or moving a fstab file, it is displays a permission denied error.

So use the following mechanism to backup or copy the fstab file by providing the destination file name.

Syntax - cp etc/fstab fstab

Linux Tips: backup crontab

How to save contents of a crontab into a text file as backup copy

Execute the command -

crontab -l > crontab.txt

OEM12c: Oracle Enterprise Manager - Creating a blackout

This post talks about creating a Blackout in Oracle Enterprise Manager.

We setup Enterprise Manager to monitor, notify any issues with the Database, Middleware and application stack. Enterprise manager sends alerts if it finds any issues with the systems or any thresholds being exceeded their limits.

If you need to perform a controlled activity like a restart or patch apply, then you may need to bring down the server or database. While you bring down, the EM will start bombarding with Alerts and this might cause unnecessary emails to be sent, and the important alerts might get missed.

Enterprise Manager provides an option to create a Blackout specifying the time and the targets during which no alert will be sent to the recipient

Steps to create a Blackout

1. Go to  Enterprise -> Monitoring -> Blackouts
    Click “Create” or if there is a previous one created that you want to use as template click “Create Like”
2. Provide 'Blackout Name'  (Eg. Restart OSB server). Comments and Reason are not required fields.
     Add targets (targets could be Database, Middleware, Cluster, Host servers, BI, Security suite, and other Oracle Products). Click Next button
3. Validate the Member Targets which are dependant on the 'Target' selected in the previous page.
4. Provide the schedule for the Blackout.
Blackout can be started immediately and provided with a duration.
Blackout can also be scheduled to start and end at certain times.
5. If the Blackout needs to be repeated weekly/daily, the Repeating drop down has to be selected with the appropriate option

BEA-000365 - Weblogic Admin / Managed server startup issues

Admin / Managed server startup Issues –

a. LDAP Files lok

<Error> <EmbeddedLDAP> <BEA-171519> <Could not obtain an exclusive lock to the embedded LDAP data files directory: /<domainhomepath>/<domainname>/servers/AdminServer/data/ldap/ldapfiles because another WebLogic Server is already using this directory. Ensure that the first WebLogic Server is completely shutdown and restart the server.>

<Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>

<Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>

<Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>

Suggestion  –

1.       Navigate to <domainhomepath>/<domainname>/servers/Adminserver/data/ldap/Idapfiles

2.       Rename the file “EmbeddedLDAP.lok” to “EmbeddedLDAP_bkp.bkp_lok”

3.       Restart the Admin server

b. Admin Server lok

weblogic.management.ManagementException: Unable to obtain lock on /<domainhomepath>/<domainname>/servers/AdminServer/tmp/AdminServer.lok. Server may already be running at weblogic.management.internal.ServerLocks.getServerLock(ServerLocks.java:206)

 <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>

 <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>

 <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>

Suggestion  –

1.       Navigate to <domainhomepath>/<domainname>/servers/Adminserver/tmp

2.       Rename the file “AdminServer.lok” to “AdminServer_bkp.bkp_lok”

3.       Restart the Admin server

c. WLS_ADMINSERVER DAT

<Emergency> <Store> <BEA-280060> <The persistent store "_WLS_AdminServer" encountered a fatal  error, and it must be shut down: weblogic.store.PersistentStoreFatalException: [Store:280105]The persistent file store "_WLS_AdminServer" cannot open file _WLS_ADMINSERVER000000.DAT.

The persistent file store "_WLS_AdminServer" cannot open file _WLS_ADMINSERVER000000.DAT.

weblogic.store.PersistentStoreFatalException: [Store:280105]The persistent file store "_WLS_AdminServer" cannot open file _WLS_ADMINSERVER000000.DAT.

Suggestion  –

1.       Navigate to <domainhomepath>/<domainname>/servers/Adminserver/data/store/default

2.       Rename the file “WLS_ADMINSERVER000000.DAT” to “WLS_ADMINSERVER000000_bkp.bDAT”

3.       Restart the Admin server

d. WLS_ADMINSERVER DAT

weblogic.store.PersistentStoreFatalException: [Store:280105]The persistent file store "WLS_DIAGNOSTICS" cannot open file WLS_DIAGNOSTICS000000.DAT.

<Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>

Suggestion  –

1.       Navigate to <domainhomepath>/<domainname>/servers/Adminserver/data/store/default

2.       Rename the file “WLS_DIAGNOSTICS.DAT” to “WLS_DIAGNOSTICS_bkp.bDAT”

3.       Restart the Admin server

Weblogic: Log Rotation in weblogic server

Log Rotation can be setup for Admin and Managed servers in a Oracle Weblogic server.

Below log files can be setup for log rotation

a. Server log
b. Access log
c. Domain log

Domain log rotation -

1. Login to the Admin console of the weblogic server
2. In the Domain structure, click on the Domain name.
3. Navigate to the 'Logging' tab and edit the Logging properties. Restart the Admin and Managed servers

Server and Access Log rotation -
1. Login to the Admin console of the weblogic server
2. Select 'Environments', click on 'Servers' and select the Admin or Managed server for which the logs needs to be rotated
3. Navigate to the 'Logging' tab, select General for Server log, and HTTP for access log and edit the Logging properties. Restart the Admin and Managed servers as appropriate.

File Rotation Steps -

1. Provide the 'Rotation Type' - By Time or By Date
2. If you want to setup 'RotationType' as 'By Time' then fill the following fields
             a. Begin Rotation Time - Eg. 23:45 (provide in 24 hrs)
             b. Rotation Interval - 24 (every 24 hrs or every 4 hrs)
             c. Limit number of retained files - 5 (if you need the old files to be removed from the system) - you can check this option, if you do not want to delete the old files
             d. Log file Rotation directory - Provide the absolute path of the rotation directory
3. If you want to setup 'Rotation Type' as 'By Date' then fill the following fields
              a. Rotation file size - 5000 (in kilobytes)
              b. Limit number of retained files - 5 (if you need the old files to be removed from the system) - you can check this option, if you do not want to delete the old files
             c. Log file Rotation directory - Provide the absolute path of the rotation directory

Linux Tips: Frequently used Unix commands

Some Unix commands and their usage which are being frequently used are listed in this post

df -h

Lists the free space data for the mount points in a human readable format.

Eg. 50G is 50 Gigabytes
      2T is 2 Terabytes

df -k

Lists the free space data in Kilo bytes format

To understand more about the df command, use df --help.