GoCb seems to be leacking connection file descriptors

Go SDK
1

Hi

We have api server which uses gocb package version v2.3.1 and go verion 1.17

If we run following command for our api server pid it shows thousands of dead tcp connections and the number keeps increasing continuosly

lsof -p 12966

In below list there are TCP connections which have no state ( neither CLOSE_WAIT nor ESTABLISHED ). And connections with CLOSE_WAIT state eventually move to
no state

For example connection with id 708658515 which a connection to couchbase query server ( port 8093 ) will be found without state after sometime ( within second ) and will remain there till server is restarted.
That is causing system to starve for file descriptors.

For example connection with id 708656407 which has no state now, was earlier in CLOSE_WAIT state and the conenction was with couchbase server.

…
api-vidya 12966 adminuser 1763u sock 0,8 0t0 708654747 protocol: TCP
api-vidya 12966 adminuser 1764u sock 0,8 0t0 708654750 protocol: TCP
api-vidya 12966 adminuser 1765u sock 0,8 0t0 708654756 protocol: TCP
api-vidya 12966 adminuser 1766u sock 0,8 0t0 708657288 protocol: TCP
api-vidya 12966 adminuser 1767u sock 0,8 0t0 708654791 protocol: TCP
api-vidya 12966 adminuser 1768u sock 0,8 0t0 708654973 protocol: TCP
api-vidya 12966 adminuser 1769u sock 0,8 0t0 708656063 protocol: TCP
api-vidya 12966 adminuser 1770u sock 0,8 0t0 708656407 protocol: TCP
api-vidya 12966 adminuser 1771u IPv4 708659293 0t0 TCP localhost:36402->localhost:8093 (CLOSE_WAIT)
api-vidya 12966 adminuser 1772u IPv4 708655082 0t0 TCP localhost:52148->localhost:4002 (ESTABLISHED)
api-vidya 12966 adminuser 1773u sock 0,8 0t0 708659226 protocol: TCP
api-vidya 12966 adminuser 1774u IPv4 708658515 0t0 TCP localhost:36472->localhost:8093 (CLOSE_WAIT)
api-vidya 12966 adminuser 1775u IPv4 708658615 0t0 TCP localhost:51242->localhost:8094 (ESTABLISHED)
api-vidya 12966 adminuser 1776u IPv4 708657758 0t0 TCP localhost:36490->localhost:8093 (CLOSE_WAIT)
api-vidya 12966 adminuser 1777u IPv4 708657905 0t0 TCP localhost:36586->localhost:8093 (CLOSE_WAIT)
api-vidya 12966 adminuser 1778u IPv4 708659468 0t0 TCP localhost:36584->localhost:8093 (CLOSE_WAIT)
api-vidya 12966 adminuser 1779u IPv4 708657736 0t0 TCP localhost:36466->localhost:8093 (CLOSE_WAIT)
api-vidya 12966 adminuser 1780u IPv4 708656810 0t0 TCP localhost:36570->localhost:8093 (CLOSE_WAIT)
api-vidya 12966 adminuser 1782u IPv4 708658684 0t0 TCP localhost:36614->localhost:8093 (CLOSE_WAIT

Please suggest how to investigate it further.

While profiling it we found gocb is using 25MB

In api server we use global bucket. One thing we found in documentation that QueryResult has close function. Do we need to close Query result after each query function ?

2

Hi @Devesh_Mittal are you iterating all your query results to completion? When the SDK detects that there are no more rows to iterate then it will automatically close the stream, so you don’t need to call Close in that case.

3

Yes found this mistake in one of the cases where we are returning without completion of iteration and invoking that function is causing this leak.

Should we close QueryResult before returning ?

4

If you aren’t iterating then the results to completion then you will need to close the QueryResult to close the stream yes. Out of curiousity - are you doing a query which performs a mutation or something which is why you aren’t using the rows?

5

In that case I just need first entry not the complete list.