Infinite loop in hasmap.put caused high CPU usage stuck threads in WebLogic Server
Infinite loop in hasmap.put caused high CPU usage stuck threads in WebLogic Server

Issue:

Weblogic threads going into a hang/ locked / stuck state with the following stack trace in thread dumps:

at java/util/HashMap.put(HashMap.java:1657)[optimized]
at java/util/HashSet.add(HashSet.java:194)[inlined]
at weblogic/rmi/internal/dgc/DGCServerImpl.addPhantomRef(DGCServerImpl.java:136)[inlined]
at weblogic/rmi/internal/BasicServerRef.getLocalRef(BasicServerRef.java:729)[inlined]
at weblogic/rmi/internal/BasicServerRef.getStubReference(BasicServerRef.java:709)[optimized]
at weblogic/rmi/cluster/ClusterableServerRef.getStubReference(ClusterableServerRef.java:193)[optimized]
at weblogic/rmi/internal/OIDManager.getReplacement(OIDManager.java:172)[optimized]
at weblogic/rmi/utils/io/RemoteObjectReplacer.replaceRemote(RemoteObjectReplacer.java:123)[inlined]
at weblogic/rmi/utils/io/RemoteObjectReplacer.replaceObject(RemoteObjectReplacer.java:104)[optimized]
at weblogic/rmi/utils/io/InteropObjectReplacer.replaceObject(InteropObjectReplacer.java:62)
at weblogic/utils/io/ChunkedObjectOutputStream.replaceObject(ChunkedObjectOutputStream.java:42)
at weblogic/utils/io/ChunkedObjectOutputStream$NestedObjectOutputStream.replaceObject(ChunkedObjectOutputStream.java:151)
at java/io/ObjectOutputStream.writeObject0(ObjectOutputStream.java:1045)[inlined]
at java/io/ObjectOutputStream.writeObject(ObjectOutputStream.java:302)[optimized]
at weblogic/jms/client/JMSConnectionFactory.writeExternal(JMSConnectionFactory.java:359)
at java/io/ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1310)
at java/io/ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1288)[inlined]
at java/io/ObjectOutputStream.writeObject0(ObjectOutputStream.java:1079)[inlined]
at java/io/ObjectOutputStream.writeObject(ObjectOutputStream.java:302)[optimized]
at weblogic/rjvm/MsgAbbrevOutputStream.writeObject(MsgAbbrevOutputStream.java:614)
at weblogic/utils/io/ChunkedObjectOutputStream.writeObject(ChunkedObjectOutputStream.java:73)
at weblogic/jndi/internal/RootNamingNode_WLSkel.invoke(ILweblogic/rmi/spi/InboundRequest;Lweblogic/rmi/spi/OutboundResponse;Ljava/lang/Object;)Lweblogic/rmi/spi/OutboundResponse;(Unknown Source)
at weblogic/rmi/internal/BasicServerRef.invoke(BasicServerRef.java:553)[optimized]
at weblogic/rmi/cluster/ClusterableServerRef.invoke(ClusterableServerRef.java:224)
at weblogic/rmi/internal/BasicServerRef$1.run(BasicServerRef.java:443)[inlined]
at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)[inlined]
at weblogic/security/service/SecurityManager.runAs(SecurityManager.java:147)[inlined]
at weblogic/rmi/internal/BasicServerRef.handleRequest(BasicServerRef.java:439)[inlined]
at weblogic/rmi/internal/BasicServerRef.access$300(BasicServerRef.java:61)[inlined]
at weblogic/rmi/internal/BasicServerRef$BasicExecuteRequest.run(BasicServerRef.java:983)[optimized]
at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]

Explanation:

  • This issue is a resultant of a Hashmap.put() going into a recursive loop causing the weblogic thread going into a stuck state and eventually causing the CPU utilization for the process to increase resulting in an unresponsive

Solution:

  • Oracle forums say that this to be BUG in the weblogic server, and has released a patch for the same
  • To get a fix for this issue: Open a support ticket with support along with the latest thread dumps so that they can diagnose the issue and provide relevant fixes

Cheers:)

Unni

comments here

December 7, 2009
175 words

Categories
Tags
example hugo blog

Connect. Socialize.

Sponsors