It is hard to tell 5 to 6 sec might be long for 5k documents. If you are using EE
execute query in query work bench and goto Plan Text Tab and take look (This post has links how to analyze Filter taking too long - #4 by Marco_Greco)
If need help post that info.