Skip to content
This repository has been archived by the owner on Jan 8, 2020. It is now read-only.

Oci8 Driver generating "Fetch out of sequence warning" #3958

Closed
wants to merge 1 commit into from
Closed

Oci8 Driver generating "Fetch out of sequence warning" #3958

wants to merge 1 commit into from

Conversation

ralphschindler
Copy link
Member

Zend\Db\Adapter\Driver\Oci8\Result line 151 is generating a oci_fetch_assoc(): ORA-01002: fetch out of sequence warning for any larger queries. Limiting the results to less than 50 stops the warning appearing but anything larger it appears.

Suppressing warnings with @oci_fetch_assoc get's rid of the warning and returns results as expected but obviously this isn't fixing the issue.

The warning is usually generated when a fetch is performed after a commit.

@hamilok
Copy link
Contributor

hamilok commented Mar 5, 2013

@oci_fetch_assoc classic workaround

@bartmcleod
Copy link
Contributor

I experienced the same problem while trying to fetch 600k+ records. Didn't try with less. Thanks for the workaround.

-Bart

Verstuurd vanaf mijn iPad

Op 5 mrt. 2013 om 08:30 heeft Phil Bennett [email protected] het volgende geschreven:

Zend\Db\Adapter\Driver\Oci8\Result line 151 is generating a oci_fetch_assoc(): ORA-01002: fetch out of sequence warning for any larger queries. Limiting the results to less than 50 stops the warning appearing but anything larger it appears.

Suppressing warnings with @oci_fetch_assoc get's rid of the warning and returns results as expected but obviously this isn't fixing the issue.

The warning is usually generated when a fetch is performed before a commit.


Reply to this email directly or view it on GitHub.

@philipobenito
Copy link
Author

It's just suppressing warnings but beware that this will also suppress any
other warnings so be careful.

On Tuesday, March 5, 2013, Bart McLeod wrote:

I experienced the same problem while trying to fetch 600k+ records. Didn't
try with less. Thanks for the workaround.

-Bart

Verstuurd vanaf mijn iPad

Op 5 mrt. 2013 om 08:30 heeft Phil Bennett <[email protected]<javascript:_e({}, 'cvml', '[email protected]');>>
het volgende geschreven:

Zend\Db\Adapter\Driver\Oci8\Result line 151 is generating a
oci_fetch_assoc(): ORA-01002: fetch out of sequence warning for any larger
queries. Limiting the results to less than 50 stops the warning appearing
but anything larger it appears.

Suppressing warnings with @oci_fetch_assoc get's rid of the warning and
returns results as expected but obviously this isn't fixing the issue.

The warning is usually generated when a fetch is performed before a
commit.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHubhttps://github.com//issues/3958#issuecomment-14451449
.

@bartmcleod
Copy link
Contributor

A collegue did a similar query with traditional php oci, he did not get the warning, do you know what might be the difference?

-Bart

Verstuurd vanaf mijn iPad

Op 5 mrt. 2013 om 12:08 heeft Phil Bennett [email protected] het volgende geschreven:

It's just suppressing warnings but beware that this will also suppress any
other warnings so beware.

On Tuesday, March 5, 2013, Bart McLeod wrote:

I experienced the same problem while trying to fetch 600k+ records. Didn't
try with less. Thanks for the workaround.

-Bart

Verstuurd vanaf mijn iPad

Op 5 mrt. 2013 om 08:30 heeft Phil Bennett <[email protected]<javascript:_e({}, 'cvml', '[email protected]');>>
het volgende geschreven:

Zend\Db\Adapter\Driver\Oci8\Result line 151 is generating a
oci_fetch_assoc(): ORA-01002: fetch out of sequence warning for any larger
queries. Limiting the results to less than 50 stops the warning appearing
but anything larger it appears.

Suppressing warnings with @oci_fetch_assoc get's rid of the warning and
returns results as expected but obviously this isn't fixing the issue.

The warning is usually generated when a fetch is performed before a
commit.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHubhttps://github.com//issues/3958#issuecomment-14451449
.


Reply to this email directly or view it on GitHub.

@philipobenito
Copy link
Author

It seems to be the order in which the query is built and committed, I'm
looking in to it so ill keep this thread informed

On Tuesday, March 5, 2013, Bart McLeod wrote:

A collegue did a similar query with traditional php oci, he did not get
the warning, do you know what might be the difference?

-Bart

Verstuurd vanaf mijn iPad

Op 5 mrt. 2013 om 12:08 heeft Phil Bennett <[email protected]<javascript:_e({}, 'cvml', '[email protected]');>>
het volgende geschreven:

It's just suppressing warnings but beware that this will also suppress
any
other warnings so beware.

On Tuesday, March 5, 2013, Bart McLeod wrote:

I experienced the same problem while trying to fetch 600k+ records.
Didn't
try with less. Thanks for the workaround.

-Bart

Verstuurd vanaf mijn iPad

Op 5 mrt. 2013 om 08:30 heeft Phil Bennett <[email protected]<javascript:_e({}, 'cvml', '[email protected]');><javascript:_e({},
'cvml', '[email protected] <javascript:_e({}, 'cvml',
'[email protected]');>');>>
het volgende geschreven:

Zend\Db\Adapter\Driver\Oci8\Result line 151 is generating a
oci_fetch_assoc(): ORA-01002: fetch out of sequence warning for any
larger
queries. Limiting the results to less than 50 stops the warning
appearing
but anything larger it appears.

Suppressing warnings with @oci_fetch_assoc get's rid of the warning
and
returns results as expected but obviously this isn't fixing the issue.

The warning is usually generated when a fetch is performed before a
commit.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub<
https://github.com/zendframework/zf2/issues/3958#issuecomment-14451449>
.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHubhttps://github.com//issues/3958#issuecomment-14452126
.

@hamilok
Copy link
Contributor

hamilok commented Mar 6, 2013

A collegue did a similar query with traditional php oci, he did not get the warning, do you know what might be the difference?

it exceeded the limit prefetch value?

@philipobenito
Copy link
Author

I don't think that's the issue as running the query in procedural PHP would have the same prefetch limit and that doesn't throw any warnings.

@hamilok
Copy link
Contributor

hamilok commented Mar 6, 2013

works for me

@philipobenito
Copy link
Author

What does? Changing the prefetch limit?

On Wednesday, March 6, 2013, Alexander Bonar wrote:

works for me


Reply to this email directly or view it on GitHubhttps://github.com//issues/3958#issuecomment-14512699
.

@ralphschindler
Copy link
Member

Is there a way someone could help me reproduce this? Perhaps with adding some large amount of data to the schema in https://github.com/ralphschindler/Zend_Db-Examples, then a script to reproduce the error that I can run?

@philipobenito
Copy link
Author

I can but it would have to be tomorrow if that's ok?

On Wednesday, March 13, 2013, Ralph Schindler wrote:

Is there a way someone could help me reproduce this? Perhaps with adding
some large amount of data to the schema in
https://github.com/ralphschindler/Zend_Db-Examples, then a script to
reproduce the error that I can run?


Reply to this email directly or view it on GitHubhttps://github.com//issues/3958#issuecomment-14859818
.

@ralphschindler
Copy link
Member

That's fine - whenever you can. Thanks, @happyaccidents

@cjbj
Copy link

cjbj commented Mar 13, 2013

Turning on oci_internal_debug(1) in the .php code may help to identify what OCI8 calls are being used. If you can recompile PHP OCI8, the patch below will print out SQL statements and whether the execute mode is autocommit or not:

diff --git a/ext/oci8/oci8_statement.c b/ext/oci8/oci8_statement.c
index 89facb0..ef5dc67 100644
--- a/ext/oci8/oci8_statement.c
+++ b/ext/oci8/oci8_statement.c
@@ -52,6 +52,11 @@ php_oci_statement *php_oci_statement_create (php_oci_connection *connection, cha
    if (!query_len) {
        /* do not allocate stmt handle for refcursors, we'll get it from OCIStmtPrepare2() */
        PHP_OCI_CALL(OCIHandleAlloc, (connection->env, (dvoid **)&(statement->stmt), OCI_HTYPE_STMT, 0, NULL));
+   } else {
+       if (OCI_G(debug_mode)) {
+           php_printf ("OCI8 DEBUG SQL: \"%s\" at (%s:%d) \n", query, __FILE__, __LINE__);
+       }
+
    }
            
    PHP_OCI_CALL(OCIHandleAlloc, (connection->env, (dvoid **)&(statement->err), OCI_HTYPE_ERROR, 0, NULL));
@@ -421,6 +426,9 @@ int php_oci_statement_execute(php_oci_statement *statement, ub4 mode TSRMLS_DC)
        case OCI_DESCRIBE_ONLY:
        case OCI_DEFAULT:
            /* only these are allowed */
+           if (OCI_G(debug_mode)) {
+               php_printf ("OCI8 DEBUG: Execute mode 0x%x at (%s:%d) \n", mode, __FILE__, __LINE__);
+           }
            break;
        default:
            php_error_docref(NULL TSRMLS_CC, E_WARNING, "Invalid execute mode given: %d", mode);

@bartmcleod
Copy link
Contributor

@ralphschindler Were you able to reproduce this?

@ralphschindler
Copy link
Member

I've not been given a reproduction case yet. ping @happyaccidents

@hamilok
Copy link
Contributor

hamilok commented May 7, 2013

This error may occur if you perform a FETCH on an active cursor after all records have been fetched. The option to resolve this error are:

Zend/Db/Adapter/Driver/Oci8/Result.php ( Updated)

    protected $currentData = null;

    public function rewind()
    {
        if ($this->position > 0) {
            throw new Exception\RuntimeException(
                'Oci8 results cannot be rewound for multiple iterations');
        }
    }

    public function next()
    {
        ++$this->position;
    }

    public function valid()
    {
        if ($this->currentData === false) {
            return false;
        }

        $this->currentData = oci_fetch_assoc($this->resource);

        return ($this->currentData !== false);
    }

    public function current()
    {
        return $this->currentData;
    }

    public function key()
    {
        return $this->position;
    }

@philipobenito
Copy link
Author

Been working on this today, came to a similar solution.

Will send PR and schema shortly (testing needed first)

On Tuesday, May 7, 2013, Alexander Bonar wrote:

This error may occur if you perform a FETCH on an active cursor after all
records have been fetched.

The option to resolve this error are ( with support ResultSet Buffer ) :

public function rewind()
{
    if ($this->position > 0) {
        throw new Exception\RuntimeException('Oci8 results cannot be rewound for multiple iterations');
    }
}

public function next()
{
    ++$this->position;
}

public function valid()
{
    // for ResultSet Buffer
    if ($this->numberOfRows > 0 && $this->numberOfRows < $this->position) {
        return false;
    }

    $this->currentData = oci_fetch_assoc($this->resource);
    $this->currentComplete = ($this->currentData !== false);

    // for ResultSet Buffer
    if ($this->currentComplete === false) {
        $this->numberOfRows = $this->position;
    }

    return $this->currentComplete;
}

public function current()
{
    return $this->currentData;
}

public function key()
{
    return $this->position;
}


Reply to this email directly or view it on GitHubhttps://github.com//issues/3958#issuecomment-17573808
.

@philipobenito
Copy link
Author

This issue is being caused by 1 too many calls being sent to Zend\Db\Adapter\Driver\Oci8\Result::loadData but this only happens when the data set is above a certain size. A temporary fix is here 5684b17918b167317cb54fcd317e3ae92d3a11ca but I am trying to trace where it is decided whether to make a call or stop. @ralphschindler could you shed any light on this? Still working on a large schema for you but it's taking longer than I planned.

@hamilok
Copy link
Contributor

hamilok commented May 8, 2013

You can reproduce this situation with a small set of data ( such as 5 records ) and the set option oci8.default_prefetch=3

The current implementation is too many calls to loadData function. This makes it difficult to read and understand the code.

Sorry for my english

@ralphschindler
Copy link
Member

@hamilok or @cjbj do either of you have documentation on the default_prefetch setting? I don't get much from the php.net/manual.

@cjbj changing the prefetch, does that affect how one might code? It seems like it should be a driver level optimization only?

@philipobenito
Copy link
Author

@ralphschindler The prefetch limit shouldn't have to be set by the driver though, it's still getting two false returns from oci_fetch_assoc(), it's just that if we're within the prefetch limit then it doesn't care, if we've exceeded prefetch it complains.

@ralphschindler
Copy link
Member

The only problem I see with @hamilok's patch is that it fails a common use case, which is getting a row from the result without iteration (consider this use case):

$result = $table->select(['id' => 5]);
if ($result) {
    $row = $result->current();
}

. In the current Result, the current looks like this:

    public function current()
    {
        if ($this->currentComplete == false) {
            if ($this->loadData() === false) {
                return false;
            }
        }

        return $this->currentData;
    }

The two primary use cases being: a) just call current to get the current data, or b) iterate using the iterate workflow to then call current to get a row.

@philipobenito
Copy link
Author

I've temporarily changed the check in loadData() to check if the previous call was false before calling oci_fetch_assoc(), this way, a single row should be fine as $currentData starts as null. My thoughts were that this check should be performed elsewhere though.

5684b17

@hamilok
Copy link
Contributor

hamilok commented May 9, 2013

@ralphschindler Excuse me, please. In my amendment is not considered a direct function call "current" on the resulting data set.

    public function current()
    {
+        if ($this->currentData === null) {
+            $this->valid();
+        }
        return $this->currentData;
    }

either the solution proposed @happyaccidents

either use an explicit calls

$result = $table->select(['id' => 5]);
if ($result->valid()) {
    $row = $result->current();
}

Oci Result fixed logic error resulting in sequence warning
@ralphschindler
Copy link
Member

Can you guys please test the patch in this PR? I believe it was a logic error. This works for me.

@cjbj
Copy link

cjbj commented May 10, 2013

To expand on @happyaccidents comment about being within the prefetch
limit it doesn't care, I can explain the underlying effect of
oci8.default_prefetch in reproducing the bug.

Prefetching reduces network traffic and DB interrupts etc by fetching
multiple rows of data and buffering them in the Oracle libraries used
by PHP OCI8. See the Underground PHP and Oracle Manual for a picture
(p. 174) of how prefetching works.

The design premise is that when oci_fetch_assoc() returns false, then
there is no more data. The application should never call
oci_fetch_assoc() again after this so it will never see an ORA error.
Due to the bug in ZF2 it is calling oci_fetch_assoc() too many times
and it is seeing the ORA-01002. (This error comes from the DB side of
the network)

The reason the error doesn't always occur is an optimization in the
Oracle libraries. When all rows can be fetched in one round trip to
the database (i.e. when the oci8.default_prefetch value is greater
than the number of rows in the queried table) then the Oracle client
libraries know that there is no more data. So no more networks trips
are initiated. Since the DB server is not invoked then the DB will
never generate the ORA-01002 error. So repeated calls to
oci_fetch_assoc() will simply return false to indicate there is no
more data.

For cases where oci8.default_prefetch is less than the number of rows
in the query result set, the ORA-01002 is generated by the DB when
oci_fetch_assoc() initiates some subsequent network trip to the DB.

@philipobenito
Copy link
Author

@cjbj 👍 well explained

@philipobenito
Copy link
Author

@ralphschindler yes, the fix works fine

weierophinney added a commit that referenced this pull request May 10, 2013
…ch-warnings

Oci8 Driver generating "Fetch out of sequence warning"
weierophinney added a commit that referenced this pull request May 10, 2013
@bartmcleod
Copy link
Contributor

Great this is fixed, works for me too.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants