Tuesday, June 11, 2013

ORACLE_BASE not set in environment …!

Today, I will talk about the above issue and minor fundamental concept related to this issue. At first instance what we see/understand from the basic error, we try to check our environment variables. Glad every one does that, when it comes to oracle rac environment we need to check srvctl environmental variables

$ srvctl getenv database -d test
test:

If it’s does not set then we need to set as below

srvctl setenv database -d test -t "ORACLE_BASE=/u01/app/oracle"

Inspite of setting above parameters you might face the issue, then may proceed with removing and adding of srvctl entries – just incase your issue would have been resolved. If it got resolved, well good else then the trouble gets started. Even one of my colleague came across the same on one the development environments. I as verified the basic stuff, which I described and above and still the issue persists (I tried to verify the same using sqlplus and it’s works with out any issues).


Now, I tried to reviewed the logs of GRID ($GRID_HOME/log/server_name/agent/crsd/oraagent_oracle) which provides the valuable information during the instance_startup and communication across srvctl utility.

2013-06-10 07:20:12.357: [ora.test.db][2052462912] {1:1116:25189} [start] Utils::getCrsHome crsHome /u01/app/11.2.0.3/grid
2013-06-10 07:20:12.357: [ora.test.db][2052462912] {1:1116:25189} [start] clsnInstConnection::makeConnectStr LIBRARY_PATH1 ,LD_LIBRARY_PATH=
2013-06-10 07:20:12.357: [ora.test.db][2052462912] {1:1116:25189} [start] makeConnectStr = (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/u01/app/oracle/product/11.2.0.3_test/bin/oracle)(ARGV0=oracletest1)(ENVS='ORACLE_HOME=/u01/app/oracle/product/11.2.0.3_test,ORACLE_SID=test1,LD_LIBRARY_PATH=,ORACLE_BASE=/u01/app/oracle')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(CONNECT_DATA=(SID=test1)))
2013-06-10 07:20:12.358: [ora.test.db][2052462912] {1:1116:25189} [start] Container:start oracle home /u01/app/oracle/product/11.2.0.3_test
2013-06-10 07:20:12.359: [ora.test.db][2052462912] {1:1116:25189} [start] InstConnection::connectInt: server not attached

If we observe carefully, the srvctl is trying to communicate (if we observe the instance is got hanged prior to “alter database mount”) to oracle background process. Since, we didn’t able to receive confirmation from oracle process about “mounting” of instance.



SRVCTL tool is an tool which talks to database through oracle process (it needs to use communication through listener – in rac environment local_listener parameter helps that) where as sqlplus is tool which directly maps (get communicated with out listener help – we can get the same information when we debug with gdb debugger).


So I tried to verified the same once I was able to digest my self the error information, what lacks communication between srvctl and oracle background process (pmon which gets register to local_listener – listens on port).  The parameter was missing in spfile, so I have created “local_listener” entry in spfile and respective entry in tnsnames.ora and started the instance with SRVCTL.


This time it worked,the cluster instance came up with out any issues.


I hope it helps if some one came across this type of issue during their instance startup’s on RAC level.

2 comments:

saruamit4 said...

Hi,

Your observation seems not true to me. I believe there is no relation with local_listener parameter with srvctl utility at all. the message you are posting "[ora.test.db][2052462912] {1:1116:25189} [start] InstConnection::connectInt: server not attached". Usually this message will appear in the log even though you don't have any local_listener parameter at all configured.

have you tested after deleting all your listeners from environment and even without any local_listener parameter.

this parameter is required just only when you need to configure your listener on non-default port (any port other than 1521).

Pavan Kumar said...

Hi Saruamit,

Thanks for dropping an comment.
Yes, SRVCTL (connects through Oracle agent it needs to communicate to Oracle Instance), it's doesn't use IPC Socket communication it's only happens with mdsn process.

- Thanks
Pavan Kumar N

 
© 2013 Oracle Internals from basics to advanced....