before the Indexerthread change partition by hash for meta().id was taking ~40 mins also .
This is unexpected. I was expecting that there will be a improvement in index build times with partition by hash(meta().id) vs partition by hash(11 fields). What is the number of partitions you have specified when building index with partition by hash(meta().id). If you have not specified any, can you build the index with 3 partitions (as you have specified for the index before).
Yes indexer CPU % is around >3700 % after thread change.
With 1024 threads, we are only utilising 37 cores of CPU while the node has up to 48 cores. So, there is CPU available but indexer is not able to make full use of it because it is being bottlenecked in the pipeline.
Since my index is not MOI and SGI based why memory throttle matters here .
Because, the index first goes to memory and then to disk. Also, mem_throttled is a transient phenomenon. What is the resident_percent of the index? Is it maintained at 100%?
IndexmemoryQuota per index node is 50 GB . Isn’t it sufficient for index on 180 M ?
How many indexes do you have on the bucket at the time of index build?