FTS giving different score for documents with the exact same field value. Why is that?

@abhinav I have a hard question for you:

I have multiple documents all of them have a field pp for all the documents this field has an identical value, in my case is ‘girl’

Array
(
    [0] => Array
        (
            [id] => 0ba7e858-5da8-4779-a598-9dc5f63f535f
            [index] => image_prompt_fts_4e8a00692c191b1f_18b18742
            [score] => 4.7508186293514
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [1] => Array
        (
            [id] => e6d9ee35-d9e6-4d02-9fba-b8fac0cc9548
            [index] => image_prompt_fts_4e8a00692c191b1f_18b18742
            [score] => 4.7508186293514
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [2] => Array
        (
            [id] => 28e60854-83f1-41f9-8ed4-d72744772b96
            [index] => image_prompt_fts_4e8a00692c191b1f_18b18742
            [score] => 4.7508186293514
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [3] => Array
        (
            [id] => 7cd89d0d-e90a-4d91-8005-896cd06e4cea
            [index] => image_prompt_fts_4e8a00692c191b1f_18b18742
            [score] => 4.7508186293514
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [4] => Array
        (
            [id] => 2c965a25-aa0e-444f-954a-5a8ce2a0d7e2
            [index] => image_prompt_fts_4e8a00692c191b1f_18b18742
            [score] => 4.7508186293514
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [5] => Array
        (
            [id] => 5d352c7c-468d-4881-902e-0a85eb0ed7bc
            [index] => image_prompt_fts_4e8a00692c191b1f_18b18742
            [score] => 4.7508186293514
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [6] => Array
        (
            [id] => 0a32ac1c-e2eb-44f1-bb8f-96b4b06669f7
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [7] => Array
        (
            [id] => 49d58e23-fbbd-4bc3-8de5-0a4f37fe7966
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [8] => Array
        (
            [id] => a638054b-9271-4d7d-9fd2-0e1a7398f876
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [9] => Array
        (
            [id] => 149e9c32-1a11-4784-8866-c429512aa34e
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [10] => Array
        (
            [id] => a89da735-6d4e-4c93-93f8-ab7b14140682
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [11] => Array
        (
            [id] => 1a4f91c9-868a-4e0d-9d30-9b2f260b9852
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [12] => Array
        (
            [id] => 9e9665de-ba18-4350-b680-c859ce63aa43
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [13] => Array
        (
            [id] => 76dd7f6b-ec40-42ad-9785-80d81caebedd
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [14] => Array
        (
            [id] => d0eb5c32-c408-4e47-acb8-9b410cf0dc5b
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [15] => Array
        (
            [id] => 84ce0450-0183-42f7-b912-a0170b0108d5
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [16] => Array
        (
            [id] => 6e44a146-43ff-418d-a5be-28c3af9c6b44
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [17] => Array
        (
            [id] => baaf9315-c868-486a-9e03-2759e6a3728e
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [18] => Array
        (
            [id] => afc416d0-660d-45eb-84cb-9de86542f434
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [19] => Array
        (
            [id] => 57aefd97-8480-4f52-9d50-105f0b189a32
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [20] => Array
        (
            [id] => 5b16e569-ef72-45ea-8226-42afbba4dea0
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [21] => Array
        (
            [id] => 3e15b763-a0b0-4021-906c-24907a30f4e9
            [index] => image_prompt_fts_4e8a00692c191b1f_50679776
            [score] => 4.7282418077371
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [22] => Array
        (
            [id] => d0e5bf48-5817-47ef-8f00-411bc7e5415e
            [index] => image_prompt_fts_4e8a00692c191b1f_11f47982
            [score] => 4.7267440373274
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [23] => Array
        (
            [id] => 45088e93-1c0c-4339-894a-9d06319e7942
            [index] => image_prompt_fts_4e8a00692c191b1f_11f47982
            [score] => 4.7267440373274
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [24] => Array
        (
            [id] => affba788-d4a4-4cf7-82ff-80e9e2a53971
            [index] => image_prompt_fts_4e8a00692c191b1f_11f47982
            [score] => 4.7267440373274
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [25] => Array
        (
            [id] => dbe5f2cc-fce5-4335-aefd-0a0107f932d1
            [index] => image_prompt_fts_4e8a00692c191b1f_11f47982
            [score] => 4.7267440373274
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [26] => Array
        (
            [id] => 351e9f3f-4c9c-490a-a00a-01584b42e879
            [index] => image_prompt_fts_4e8a00692c191b1f_11f47982
            [score] => 4.7267440373274
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [27] => Array
        (
            [id] => df98c3d8-706b-4011-ae74-4608cbba0f05
            [index] => image_prompt_fts_4e8a00692c191b1f_11f47982
            [score] => 4.7267440373274
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [28] => Array
        (
            [id] => 287b14fe-1fcd-4d69-9105-221a0d4e843c
            [index] => image_prompt_fts_4e8a00692c191b1f_11f47982
            [score] => 4.7267440373274
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [29] => Array
        (
            [id] => 4bf62944-ce9c-404f-b183-8472323717bd
            [index] => image_prompt_fts_4e8a00692c191b1f_11f47982
            [score] => 4.7267440373274
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [30] => Array
        (
            [id] => fcb9b6d6-c1ec-4960-9a69-d189f7f744c0
            [index] => image_prompt_fts_4e8a00692c191b1f_11f47982
            [score] => 4.7267440373274
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 1
                )

        )

    [31] => Array
        (
            [id] => 8cd8b105-a8c9-449b-a425-72be732e862f
            [index] => image_prompt_fts_4e8a00692c191b1f_11f47982
            [score] => 4.7267440373274
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

    [32] => Array
        (
            [id] => 7197a3dd-184b-4dbd-8088-da5d1f6c108e
            [index] => image_prompt_fts_4e8a00692c191b1f_11f47982
            [score] => 4.7267440373274
            [explanation] => Array
                (
                )

            [locations] => Array
                (
                    [0] => Array
                        (
                            [field] => pp
                            [term] => girl
                            [position] => 1
                            [startOffset] => 0
                            [endOffset] => 4
                        )

                )

            [fragments] => Array
                (
                )

            [fields] => Array
                (
                    [mt] => 2
                    [pi] => 
                )

        )

)

as you can se for all the documents the locations field is identical.

The problem I have is that _score is different for the documents, as you can see above.

Because my query is first sorting after the _score and then after other fields, none of these is working anymore as the _score is different.

Why is the _score different and how I can make sure the _score is identical for documents with identical field value?

Not really that hard a question :slight_smile:
This situation would typically occur in a partitioned index.

Firstly, worth mentioning here - the scoring algorithm that our search engine uses is tf-idf - which takes into account term frequency (that is rewarded) and number of documents the term occurs in (that is penalized).

Like I pointed out in your other ticket here, a search request that is received is individually applied to every partition and results are merged before responding to the user - so scores for the document hits generated are determined at a partition level.

=> Meaning regardless of the content within these documents being very alike, the score tends to differ based on the number of documents resident within each partition. Only if all the doc counts are equal among all the partitions would the scores end up alike.

This is a shortcoming with the tf-idf scoring algorithm, reason why we’re exploring other approaches like bm25 or perhaps even supporting a framework to support customized scoring (and not just simple boosting) in our future - hopefully sooner than later.

Until then if it is absolutely necessary to you for your query to generate the same _score for like documents - my only recommendation for now is to use a single partitioned index.

Thanks for the explanation, I will try to make the index with a single partition and see how bad it will be. I will report back tomorrow after the index will be rebuilt

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.