Server error on "select from Class group by field" with 1,000,000+ records #3144
Comments
|
Is it possible it goes in timeout? After how much time it returns the error? Or this could be a RAM problem where order by could be RAM expensive. What's your cfg? |
|
It took around 60 secs before the server returned the error. My config file has the default settings. Here it is: <orient-server>
<handlers>
<handler class="com.orientechnologies.orient.graph.handler.OGraphServerHandler">
<parameters>
<parameter value="true" name="enabled"/>
<parameter value="50" name="graph.pool.max"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin">
<parameters>
<parameter value="${distributed}" name="enabled"/>
<parameter value="${ORIENTDB_HOME}/config/default-distributed-db-config.json" name="configuration.db.default"/>
<parameter value="${ORIENTDB_HOME}/config/hazelcast.xml" name="configuration.hazelcast"/>
<parameter value="com.orientechnologies.orient.server.distributed.conflict.ODefaultReplicationConflictResolver" name="conflict.resolver.impl"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.handler.OJMXPlugin">
<parameters>
<parameter value="false" name="enabled"/>
<parameter value="true" name="profilerManaged"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.handler.OAutomaticBackup">
<parameters>
<parameter value="false" name="enabled"/>
<parameter value="4h" name="delay"/>
<parameter value="backup" name="target.directory"/>
<parameter value="${DBNAME}-${DATE:yyyyMMddHHmmss}.zip" name="target.fileName"/>
<parameter value="9" name="compressionLevel"/>
<parameter value="1048576" name="bufferSize"/>
<parameter value="" name="db.include"/>
<parameter value="" name="db.exclude"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.handler.OServerSideScriptInterpreter">
<parameters>
<parameter value="true" name="enabled"/>
<parameter value="SQL" name="allowedLanguages"/>
</parameters>
</handler>
</handlers>
<network>
<sockets>
<socket implementation="com.orientechnologies.orient.server.network.OServerSSLSocketFactory" name="ssl">
<parameters>
<parameter value="false" name="network.ssl.clientAuth"/>
<parameter value="config/cert/orientdb.ks" name="network.ssl.keyStore"/>
<parameter value="password" name="network.ssl.keyStorePassword"/>
<parameter value="config/cert/orientdb.ks" name="network.ssl.trustStore"/>
<parameter value="password" name="network.ssl.trustStorePassword"/>
</parameters>
</socket>
<socket implementation="com.orientechnologies.orient.server.network.OServerSSLSocketFactory" name="https">
<parameters>
<parameter value="false" name="network.ssl.clientAuth"/>
<parameter value="config/cert/orientdb.ks" name="network.ssl.keyStore"/>
<parameter value="password" name="network.ssl.keyStorePassword"/>
<parameter value="config/cert/orientdb.ks" name="network.ssl.trustStore"/>
<parameter value="password" name="network.ssl.trustStorePassword"/>
</parameters>
</socket>
</sockets>
<protocols>
<protocol implementation="com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary" name="binary"/>
<protocol implementation="com.orientechnologies.orient.server.network.protocol.http.ONetworkProtocolHttpDb" name="http"/>
</protocols>
<listeners>
<listener protocol="binary" socket="default" port-range="2424-2430" ip-address="0.0.0.0"/>
<listener protocol="http" socket="default" port-range="2480-2490" ip-address="0.0.0.0">
<commands>
<command implementation="com.orientechnologies.orient.server.network.protocol.http.command.get.OServerCommandGetStaticContent" pattern="GET|www GET|studio/ GET| GET|*.htm GET|*.html GET|*.xml GET|*.jpeg GET|*.jpg GET|*.png GET|*.gif GET|*.js GET|*.css GET|*.swf GET|*.ico GET|*.txt GET|*.otf GET|*.pjs GET|*.svg GET|*.json GET|*.woff GET|*.ttf GET|*.svgz" stateful="false">
<parameters>
<entry value="Cache-Control: no-cache, no-store, max-age=0, must-revalidate\r\nPragma: no-cache" name="http.cache:*.htm *.html"/>
<entry value="Cache-Control: max-age=120" name="http.cache:default"/>
</parameters>
</command>
<command implementation="com.orientechnologies.orient.graph.server.command.OServerCommandGetGephi" pattern="GET|gephi/*" stateful="false"/>
</commands>
<parameters>
<parameter value="utf-8" name="network.http.charset"/>
</parameters>
</listener>
</listeners>
</network>
<storages/>
<users>
<user resources="*" password="******" name="root"/>
<user resources="connect,server.listDatabases,server.dblist" password="guest" name="guest"/>
</users>
<properties>
<entry value="1" name="db.pool.min"/>
<entry value="50" name="db.pool.max"/>
<entry value="true" name="profiler.enabled"/>
<entry value="info" name="log.console.level"/>
<entry value="fine" name="log.file.level"/>
</properties>
</orient-server>I tried to rerun the query while monitoring the RAM state. There were 22Gb of RAM available and again the query ended with the exception: [OServerPluginManager]Error on fetching record during browsing. The record has been skipped |
|
Please could you try the same query from console against the same database, but open as plocal? In this way you avoid the server and its timeouts. |
|
I tried what you suggested and got exactly the same error as I posted above. Error on fetching record during browsing. The record has been skipped |
|
Please could you try executing this query? |
|
It returned the number of distinct hist_ids: 315305 |
|
Please could you retry with 2.0-rc1 or last 2.0-SNAPSHOT? |
|
I tried the query with version 2.0-rc1 and still no success. My first attempt failed with the following exception: Again, I was monitoring the RAM state and there were about 19GB available. I noticed one strange thing. When the server starts it says that the heap size is 481MB: But when I stop it with the shutdown script it reports the heap size to be 7,138MB: I wonder if I need to configure the heap size manually or the server will increase it automatically? After this error the server stopped responding and I had to run the shutdown script about 6 times before the server acutally stopped. Then I restarted the server and retried the query several more times and every time I got the following exception: Now it takes a bit longer before the error occurs, about 2 minutes. |
|
Seems you have only half GB in server.sh |
|
Oh, I increased the heap size and now it works. Thank you very much! |
Hi!
I have a class containing 1,000,000+ records with the following properties start_of_life, end_of_life, hist_id. hist_id is shared among different versions of a record. The following query "select min(start_of_life), max(end_of_life), hist_id from SomeVertex group by hist_id" causes the server to throw an exception (SEVERE Internal server error: java.net.SocketException: Socket closed [ONetworkProtocolHttpDb])
Orientdb versions: 2.0-M2, 2.0-M3
Any ideas or advice?
The text was updated successfully, but these errors were encountered: