Limit number of results returned by lookup and auto load additional results when user scrolls to bottom of list

[silkentrance]

@crucialfelix

In our app we have > 100k names of persons that we would like to be able to search for using ajax_selects.

The current implementation would, in worst case scenarios, result in a major performance drop as the lookup would return far too many items.

I am currently working on a PR that will

• load only a given number of items from the server with each request (limit)
• load additional items from the server as soon as the user scrolls down to the bottom of the list (offset)

I still need to flesh things out but I wonder whether such a PR would be appreciated and how soon it could be integrated into the master once all issues have been resolved.

Thanks!

[crucialfelix]

honestly I would build a custom interface for something like that. write a
django view for it, make an angular or react view to handle it.
I'm not sure how jquery-ui handles the scrolling and repopulating. a popup
like this isn't the best interface.

but also why do you say that it would return all the results ? it certainly
doesn't. I have 600k items on one site and the search works fine.
you will run into performance problems of course if the indexes aren't
efficient.
its the Lookup that is responsible for fetching results, nothing about
django-ajax-selects does this. so you can customize that query as you like.

On Fri Feb 06 2015 at 6:26:30 PM Carsten Klein [email protected]
wrote:

In our app we have > 100k names of persons that we would like to be able
to search for using ajax_selects.

The current implementation would, in worst case scenarios, result in a
major performance drop as the lookup would return far too many items.

I am currently working on a PR that will

• load only a given number of items from the server with each request
  (limit)
• load additional items from the server as soon as the user scrolls
  down to the bottom of the list (offset)

I still need to flesh things out but I wonder whether such a PR would be
appreciated and how soon it could be integrated into the master once all
issues have been resolved.

Thanks!

—
Reply to this email directly or view it on GitHub
#105.

[silkentrance]

Granted, however, entering a term of a min length of 1 and some common character like for example 'a' will return you most of your data from the server, especially when you look up multiple attributes in your model, e.g. first_name, last_name, country, city etc.

Regardless of how the lookup is being implemented or however optimized the indexes are, this would, in our scenario, return approximately 10k+ results, which is unacceptable performance wise.

The PR will eventually prevent such situations from ever occurring as it will incrementally load results from the server. Just have a look, the changes made are minimal and will not change the overall behaviour of the app/api.

The PR needs some more work, of course. Scrolling down the displayed menu will fetch additional results as expected, however, the overall UX needs some more work.

And using angular is not an option here, believe me, way too complicated for the folks responsible for implementing the application...

[crucialfelix]

https://github.com/crucialfelix/django-ajax-selects#plugin_options-property-default

plugin options: minLength

On Fri Feb 06 2015 at 11:12:18 PM Carsten Klein [email protected]
wrote:

Granted, however, entering a term of a min length of 1 and some common
character like for example 'a' will return you most of your data from the
server, especially when you look up multiple attributes in your model, e.g.
first_name, last_name, country, city etc.

The PR will eventually prevent such situations from ever occurring as it
will incrementally load results from the server. Just have a look, the
changes made are minimal and will not change the overall behaviour of the
app/api.

And using angular is not an option here, believe me, way too complicated
for the folks responsible for implementing the application...

—
Reply to this email directly or view it on GitHub
#105 (comment)
.

[silkentrance]

I know that, but not everyone likes to limit their searches by this option. In our use case, there is no restriction on minLength.

[crucialfelix]

then do it in your Lookup. if the query is only one char then just return YourModel.objects.none()

[silkentrance]

Doing it in just the lookup will limit the results to an artificial limit and will not allow the user to retrieve more results from the same query by simply scrolling down the list.

Alternatively you might want to google

https://www.google.de/search?client=safari&rls=en&q=limit+results+jquery+autocomplete&ie=UTF-8&oe=UTF-8&gfe_rd=cr&ei=_kTVVO3zIsyc-warsIDICA

especially the one http://bugs.jqueryui.com/ticket/9651

which had been closed and leaves it to the users to write plugins, something which ajax-selects is.

[silkentrance]

Doing it in the lookup requires an alternative implementation of the JQuery UI AutoComplete widget. I am currently working on this.

I think that this issue can be closed.