GlassFish Instance Synchronization Fails with an Unexpected End of File Error (Doc ID 1260336.1)

Last updated on NOVEMBER 05, 2016

Applies to:

Oracle GlassFish Server - Version 2.1 and later
Information in this document applies to any platform.

Symptoms

This problem can be encountered when an instance is synchronizing a large amount of data from the Domain Administration Server, which takes over 10 minutes to complete.

If the instance is being started from the command line, then the following error is seen:

# asadmin start-instance cl1-inst1
Unexpected end of file from server
CLI137 Command start-instance failed.


In the Node Agent's server.log file there will be the following error reported:

[#|2010-09-01T13:09:23.618+0100|SEVERE|sun-appserver2.1|javax.ee.enterprise.system.nodeagent|_ThreadID=11;_ThreadName=RMI TCP Connection(16)-192.168.1.121;cl1-inst1;FAILED!;686215;|NAGT0039:Synchronization for cl1-inst1 - FAILED! and took 686215 milliseconds|#]


From the log, the synchronization failed after 686215 milliseconds, or over 11 minutes. 

Please note that there will have been a number of other errors and exceptions logged prior to this log entry, for example:

[#|2010-09-01T13:09:23.548+0100|WARNING|sun-appserver2.1|javax.ee.enterprise.system.tools.synchronization|_ThreadID=10;_ThreadName=main;|SYNC029: There was an exception during synchronization. Please try again and a full synchronization will be initiated. If the problem persists for a server instance, you may backup and remove the cache repository (for example, directories under nodeagents/na1/server1/) and restart the node-agent (ex. asadmin stop-node-agent; start-node-agent --startinstances=false) and the server instance (ex. asadmin start-instance). All necessary configuration for the server instance (server1 in example) will be downloaded from Domain Administration Server by synchronization.
com.sun.enterprise.ee.synchronization.SynchronizationException: com.sun.enterprise.ee.synchronization.SynchronizationException: Error while retrieving file from central repository: ${com.sun.aas.instanceRoot}/config/
        at com.sun.enterprise.ee.synchronization.BaseSynchronizationDriver.synchronizeInternal(BaseSynchronizationDriver.java:293)
        at com.sun.enterprise.ee.synchronization.BaseSynchronizationDriver.synchronize(BaseSynchronizationDriver.java:140)
        at com.sun.enterprise.ee.synchronization.SynchronizationMain.main(SynchronizationMain.java:126)
Caused by: com.sun.enterprise.ee.synchronization.SynchronizationException: Error while retrieving file from central repository: ${com.sun.aas.instanceRoot}/config/
        at com.sun.enterprise.ee.synchronization.http.HttpGetCommand.execute(HttpGetCommand.java:139)
        at com.sun.enterprise.ee.synchronization.BaseRequestMediator.execute(BaseRequestMediator.java:90)
        at com.sun.enterprise.ee.synchronization.BaseRequestMediator.run(BaseRequestMediator.java:107)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.SocketException: Unexpected end of file from server
        at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:769)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:632)
        at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:766)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:632)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1049)
        at com.sun.enterprise.ee.synchronization.http.HttpGetCommand.execute(HttpGetCommand.java:125)
        ... 3 more
|#]


Please see the attached errors.txt file for the complete set of errors.

Changes

This problem can potentially be encountered when upgrading from a version of the GlassFish Server earlier than 2.1.1 patch 4, to a version between 2.1.1 patch 4 and patch 8.  It should only occur in domains which have very large applications and content that is synchronized between the Domain Administration Server and the instances in the domain.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms