Increase Java Heap memory for OEM

 There are certain times when we observer OEM is performing slow or even unable to startup the Admin Server or OMS Server. We can look into few log files below and understand what is exactly causing the issue. Most of the times I have observed Java Heap issue or memory issue.

Sometime it happens due to an old process holding up the session and consuming the memory, the first step is to kill all the processes and remove some of the lok and temp files from the system.

Known error like:

java.lang.OutOfMemoryError: Java heap space.

Troubleshooting steps:

Kill all the old processes:

ps -ef | grep EMGC_ADMINSERVER

ps -ef | grep EMGC_OMS1

ps -ef | grep java

ps -ef | grep opmn

$kill -9 <PID>

The below temp files can be deleted:

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_ADMINSERVER/tmp/EMGC_ADMINSERVER.lok

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_OMS1/data/ldap/ldapfiles/EmbeddedLDAP.lok

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_OMS1/tmp/EMGC_OMS1.lok

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_ADMINSERVER/data/ldap/ldapfiles/EmbeddedLDAP.lok

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_OMS1/data/store/diagnostics/WLS_DIAGNOSTICS000000.DAT

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_OMS1/data/store/default/_WLS_EMGC_OMS1000000.DAT

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_ADMINSERVER/data/store/diagnostics/WLS_DIAGNOSTICS000000.DAT

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_ADMINSERVER/data/store/default/_WLS_EMGC_ADMINSERVER000000.DAT

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/config/config.lok

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_OMS1/tmp/EMGC_OMS1.lok

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_ADMINSERVER/tmp/EMGC_ADMINSERVER.lok

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_OMS1/data/nodemanager/*.lck, *.pid, *.state

$EM_INSTANCE_BASE/user_projects/domains/GCDomain/servers/EMGC_ADMINSERVER/data/nodemanager/*.lck, *.pid, *.state

Check current usage:

grep "JAVA Memory arguments" <OMS_INST>/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.out

--> This will give the output of the current OMS heap memory.

emctl get property -name OMS_HEAP_MAX

--> This needs Sysman password 

Fix:

Increase it to the desired value, example 4G.

  <OMS_HOME>/bin/emctl set property -name OMS_HEAP_MAX -value 4096M

Restart the OMS

emctl stop oms -all

emctl start oms

Hope this will fix the slowness and heap memory error.

Best wishes,

 Check progress on expdp and impdp:

In few cases we need to monitor the progress of an export or import job in oracle. below are the steps can be followed to get the details.

Datapump Client:

get the session details: 

SQL> select * from dba_datapump_jobs where state='EXECUTING';

attach the session:

from Command line:

    expdp system/********** attach=SYS_EXPORT_FULL_01

    Export > status

From Database:

Query 1: The % of work done for the running job can be found from this query:

  

set pages 200

set lines 300

col USERNAME for a15

col TARGET for a15

col OPNAME for a20

col "%DONE" for a7

SELECT b.username,
         a.sid,
         b.opname,
         b.target,
         ROUND (b.SOFAR * 100 / b.TOTALWORK, 0) || '%'       AS "%DONE",
         b.TIME_REMAINING,
         TO_CHAR (b.start_time, 'YYYY/MM/DD HH24:MI:SS')     start_time
    FROM v$session_longops b, v$session a
   WHERE a.sid = b.sid
ORDER BY 6;

Query 2: The work done so far and status of the job:

SELECT sl.sid,
       sl.serial#,
       sl.sofar,
       sl.totalwork,
       dp.owner_name,
       dp.state,
       dp.job_mode
  FROM v$session_longops sl, v$datapump_job dp
 WHERE sl.opname = dp.job_name AND sl.sofar != sl.totalwork;

Query 3: The amount of work done so far:

SELECT sl.sid,
       sl.serial#,
       sl.sofar,
       sl.MESSAGE,
       sl.totalwork,
       dp.owner_name,
       dp.state,
       dp.job_mode
  FROM v$session_longops sl, v$datapump_job dp
 WHERE sl.opname = dp.job_name;

Reference:

How To Monitor The Progress Of Datapump Jobs (Doc ID 1471766.1)

Best Wishes!!

Oracle Home remove/Detach from Inventory

Remove/Detach Oracle Home from Inventory

Prechecks:

Check /etc/oraInst.loc

go to the inventory location:

cd ContentsXML

cat inventory.xml and check which one needs to be detached.

inventory.xml example:

<?xml version = '1.0' encoding = 'UTF-8' standalone = 'yes'?>

<!-- Copyright (c) 1999, 2024, Oracle. All rights reserved. -->

<!-- Do not modify the contents of this file by hand. --><INVENTORY>

   <VERSION_INFO>

      <SAVED_WITH>13.9.4.0.0</SAVED_WITH>

      <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>

   </VERSION_INFO>

   <HOME_LIST>

      <HOME NAME="OraDB19Home1" LOC="/u01/app/oracle/19.3.0/" TYPE="O" IDX="4"/>

      <HOME NAME="agent13c1" LOC="/u01/app/em13c/agent_13.4.0.0.0" TYPE="O" IDX="3"/>

      <HOME NAME="agent13c2" LOC="/u01/app/em13c/agent_13.5.0.0.0" TYPE="O" IDX="6"/>

   </HOME_LIST>

   <COMPOSITEHOME_LIST/>

Oracle database archive space utilization

 Query to check Archive space Utilization

set pagesize 0 feedback off verify off heading off echo off

SELECT decode( nvl( space_used, 0),0, 0, ceil ( ( space_used / space_limit) * 100) ) pct_used FROM v$recovery_file_dest;