Limit/offset doesn't work properly when using parameters and SQL Server drivers

[ralphschindler]

When using the native SQL Server driver, it seems that using limit and offset works fine with a query that has no parameters, but doesn't work with a query that has parameters. For example, the following will return the expected results:

$select = new \Zend\Db\Sql\Select(); 
$select->from('mytable')
        ->order('myfield')
        ->limit(5)
        ->offset(0);

$sql = new \Zend\Db\Sql\Sql($db);
$statement = $sql->prepareStatementForSqlObject($select);
$results = $statement->execute();

Whereas the following won't:

$select = new \Zend\Db\Sql\Select(); 
$select->from('mytable')
        ->where('aField = ?')
        ->order('myfield')
        ->limit(5)
        ->offset(0);

$sql = new \Zend\Db\Sql\Sql($db);
$statement = $sql->prepareStatementForSqlObject($select);
$results = $statement->execute(array(1));

I believe the reason has something to do with the order of the parameters getting messed up. To be more specific, it seems like when ParameterContainer calls getPositionalArray(), the offset/limit parameters come before the where parameters when it should actually be the opposite.

For the PDO SQL server, the problem is totally different. When trying to run the first case above, I get the following error:

Incorrect syntax near 'LIMIT'

That leads me to believe the ROW_NUMBER() function has not been implemented, not sure it's a bug or if it'll be unsupported for PDO, though I've searched and don't see anything saying that the drivers provided by Microsoft don't support it.

If I run the second case above, then I get the following error:

Invalid parameter number: mixed named and positional parameters

It does not throw this error if I change my where clause to use named parameters, but seems like I should be allowed to use positional or named parameters in my where clause as long as I stick to only one. Regardless, if I change to named parameters, I get the first error for syntax error near LIMIT.

[samsonasik]

have you tried ? :

use Zend\Db\Sql\Sql;
use Zend\Db\Sql\Where;

$sql = new Sql($dbAdapter);
$select = $sql->select();
$select->from('mytable');
$select->limit(5);

$statement = $sql->prepareStatementForSqlObject($select);
$result = $statement->execute();

return $result;

[tm8747a]

That doesn't work either, not for SQL server driver or PDO driver. First reason is the absence of an order clause, which seems to be mandatory, otherwise the query does something like this:

 ROW_NUMBER() OVER (SELECT 1)

That causes the following error to be thrown:

Incorrect syntax near the keyword 'SELECT'

If I add an order(), then the PDO driver goes back to throwing the LIMIT error (since LIMIT doesn't exist in SQL server) and the SQL Server driver doesn't return the proper results. If I add an offset(), then is works fine with the SQL Server driver, but only because there are no WHERE clause parameters in your example. If I add a where clause with a parameter in it, I don't get the proper results, as explained in my first post.

[tm8747a]

Looking further into this, this seems related to the bindParametersFromContainer() function in Zend\Db\Adapter\Driver\Sqlsrv\Statement. At the bottom, there's some code commented out that says "@todo bind errata". Looking at the ParameterContainer class, if I'm understanding the concept of errata properly, seems like this is meant to keep track of the order parameters should be in so that it can be corrected at the time you bind them to the query. Since this is not being done here, things break. This code is all rather confusing to me at this point, but I'm gonna dig in and see if I can come up with a solution based on what other drivers do.

[tm8747a]

Looks like I may have been off on the errata based on what I see in other drivers, but I did manage to change bindParametersFromContainer() to get this to work for the SQL Server driver (though I'd love for things to work with PDO since I'd like to be able to use named parameters in queries). Here's what I did, can't say it's a robust fix since it's not extensively tested, but I guess it could help somebody else:

protected function bindParametersFromContainer()
{
    $values = $this->parameterContainer->getNamedArray();
    // Check if this query uses limit and/or offset
    if (array_key_exists('limit', $values) || array_key_exists('offset', $values)) {
        // If query uses limit/offset, we need to re-order the parameters so that 
        // limit and offset related params are at the end 
        $vals = array();
        $limitParamVals = array();
        $limitParams = array('offset','limit','offsetForSum');
        // Loop through all parameters
        foreach ($values as $key=>$value) {
            // If this is not an limit/offset param, add it to the list of values
            if (!in_array($key, $limitParams, true)) {
                $vals[] = $value;
            // If this is a limit/offset param, add it to a separate array
            } else {
                $limitParamVals[] = $value;
            }
        }

        // Append the limit/offset-related params to the end of the values
        $vals = array_merge($vals, $limitParamVals);
    // If we don't use limit or offset, we can just get the positional array
    } else {
        $vals = $this->parameterContainer->getPositionalArray();
    }

    $position = 0;
    foreach ($vals as $value) {
        $this->parameterReferences[$position++][0] = $value;
    }

    // @todo bind errata
    //foreach ($this->parameterContainer as $name => &$value) {
    //    $p[$position][0] = $value;
    //    $position++;
    //    if ($this->parameterContainer->offsetHasErrata($name)) {
    //        $p[$position][3] = $this->parameterContainer->offsetGetErrata($name);
    //    }
    //}
}

[frank-mueller]

I have the same problem. I am using the PDO driver to connect to Microsoft SQL-Server.

This is my select:

    $select = $sql->select()
            ->from('tbl_pakete')
            ->where(array('CustomerID' => $this->customerId))
            ->offset(1)
            ->limit(10)
            ->order('PaketID');

An the dump of the PDOStatement is like this:

SELECT * FROM ( SELECT [tbl_pakete].*, ROW_NUMBER() OVER (ORDER BY [PaketID] ASC) AS [__ZEND_ROW_NUMBER] FROM [tbl_cug_pakete] WHERE [CustomerID] = :where1 ) AS [ZEND_SQL_SERVER_LIMIT_OFFSET_EMULATION] WHERE [ZEND_SQL_SERVER_LIMIT_OFFSET_EMULATION].[__ZEND_ROW_NUMBER] BETWEEN ?+1 AND ?+?

After this i get the following error:

SQLSTATE[HY093]: Invalid parameter number: mixed named and positional parameters

@tm8747a : The problem with "ROW_NUMBER() OVER (SELECT 1)" you can solve by adding an "order" to your select.

[tm8747a]

@frank-mueller Yeah, I'd figured out the ROW_NUMBER() issue. I'm surprised that you're getting that SQL with the PDO driver. I'm pretty sure when I used the PDO driver I was getting a query with LIMIT in it, not the query for SQL Server that you're getting. But maybe I messed up somewhere, will have to try it again. Just to make sure, what version are you running?

[faabiosr]

I created pull request about this problem.

Issue #3264

[ralphschindler]

Yeah, I see the problem, it appears that if you don't send a value with the original where(), then the placeholder is not created inside the ParameterContainer. This is why it appears that the LIMIT and OFFSET parameters are coming in before the where().

I think the answer to this is to ensure that when using the where() with a ?, that the ? is counted and if no value is provided, a placeholder is still created.

[tm8747a]

Just downloaded the latest code just to see if I was missing something for the PDO driver, but looks like the same error still occurs. @frank-mueller I'd be curious to know what you are running because when I try running a statement similar to yours with the PDO driver I get something like this:

SELECT "table_name".* FROM "table_name" WHERE "sub_id" = :where1 ORDER BY "order_field" DESC LIMIT :limit OFFSET :offset

In other words, it's not using the LIMIT/OFFSET style for SQL Server. Maybe I'm the one doing something wrong? I have my driver setup as follows:

array(
    'driver'        => 'Pdo=Pdo_Sqlsrv',
    'dsn'           => 'sqlsrv:server=localhost\SQLEXPRESS;database=db_name_here',
    'username'      => 'username_here',
    'password'      => 'pwd_here'
);

Anything wrong here?

[ralphschindler]

In your code above, try this:

$select = new \Zend\Db\Sql\Select(); 
$select->from('mytable')
        ->where(array('aField = ?' => array(null))
        ->order('myfield')
        ->limit(5)
        ->offset(0);

That should work for the current codebase.

To put it another way, Zend\Db\Sql\Select doesn't know from where('something = ?') that you're trying to do a placeholder b/c you did not pass in a set of parameters that it could use to know how many placeholder your expression string had it in. Since Zend\Db\Sql doesn't do any kind of parsing of your where string, it can't know how many you have in there.

Either way, I agree, this is not an ideal situation, and I am trying to get to the bottom of it with regards to what we should do when you have a situation where you want to provide parameters for the first time at execution time.

[ralphschindler]

Also note, the Sqlsrv driver for PDO is currently not supported.

[frank-mueller]

This is unfortunately a step backwards. With ZF1 Sqlsrv driver for PDO was still supported. And windows-based Sqlsrv support is not an option for me.

[tm8747a]

Good to know. May need to look at another framework, not being able to use named parameters may end up being a deal-breaker for me. Thanks for the clarification. Any specific technical reason why it's not supported? Are there features that it wouldn't be possible to get working? Or is it just a matter of manpower to get it done? If it's the latter, maybe I'll try to dig into it if I can find the time to make it work.

[ralphschindler]

Only reason it was not supported is because at the time of the writing of the SQL Server driver, the PDO version was not available yet. I will try to get this in for 2.1.

[frank-mueller]

In version 2.1 MSSQL PDO is not supported yet. Do you see the chance that this will change soon? This would be fine.

[ralphschindler]

I wanted to, but I had some problems with my integration testing suite and Sql Server, so I ran out of time.

Basically PDO Mssql does't support setting INSERT IDENTITY OFF. I need to get to the bottom of that.

[frank-mueller]

@ralphschindler Do you think, that version 2.2 includes the PDO support for MS SQL server? This would be very nice :-)

[chriskl]

I sent a pull request to fix all this:

#5092