I am working on migrating a site from phpbb to djangobb. I have noticed that in some cases search can be very slow. I am using Haystack and whoosh. An example would be if a search matches ~10,000 posts. The page becomes unresponsive and eventually errors, probably due to server time out. I have narrowed this down to lines 179-191 of djangobb_forum.views.py

I think that the for loop takes an excessive amount of time. Does anyone have any ideas how to optimize this?

Thanks

I am with you Josh.
I fixed “Search by Author” now I continue to make searches faster to avoid server timeout etc.

greetings
scrapper

maybe problem in pagination?
trac imho better place for technical discussions)
http://djangobb.org/ticket/182 [djangobb.org]

slav0nic
maybe problem in pagination?
trac imho better place for technical discussions)
http://djangobb.org/ticket/182 [djangobb.org]

So here some info from me:
———————————
Reading more about whoosh it seems to be, that whoosh is mostly for testing and/or small amount of data to search in.
So the only thing which made the search speed up a bit was to change the haystack/constants.py settings a bit:
ITERATOR_LOAD_PER_QUERY defaults to 10, make this number greater something like 99999999.

But the truth is, its still slow. Right now we have about 25.000 posts to search and the number is increasing.
So to speed up the things we switched from WHOOSH to XAPIAN (C++).

Searches now do work fine now and perform fast.

@slav0nic:
you are right, pagination is an issue. To change from one result-page to the next you have to wait similar time as for the search.
I will look into this in the future. Must be a conceptional issue with pagination…

greetings
sc