data mismatch when trying to run the indexer

Hi @AI DATA

Thank you for providing the details about the indexing error. This issue is fairly common with vector embeddings in Azure AI Search. It occurs because the embedding model generates values as doubles (Collection(Edm.Double)), while vector fields require singles (Collection(Edm.Single)) for search to work properly. This is not related to your free tier; it's a standard aspect of how the service manages type parsing across all plans.

The simplest solution is to use index projections in your skillset. This allows Azure AI Search to automatically convert doubles to singles during mapping, so you don't need to modify your model or field types. Here’s how you can set it up:

1. Please review your index schema: Ensure that your vector field, such as "contentVector," is configured as Collection(Edm.Single) with the correct dimensions. If it is already set up properly, no further adjustments are required.
2. Skillset Update: Please include an "indexProjections" section to manage the projection. For a standard configuration involving chunking and embeddings, the section may resemble the following example (ensure to modify the paths as appropriate):

      "indexProjections": {
        "selectors": [
          {
            "targetIndexName": "your-index-name",
            "parentKeyFieldName": "parent_id",
            "sourceContext": "/document/pages/*",
            "mappings": [
              { "name": "contentVector", "source": "/document/pages/*/vector" },
              { "name": "content", "source": "/document/pages/*" }
            ]
          }
        ],
        "parameters": {
          "projectionMode": "skipIndexingParentDocuments"
        }
      }
   

3. Please re-run your indexer after updating the skillset. Resetting and executing the indexer at this stage should ensure the vectors are populated accurately.

If projections aren't compatible with your pipeline, you might consider using a custom skill, such as an Azure Function, to explicitly convert the embeddings to float32 before indexing. However, projections tend to be a simpler and more reliable solution.

reference :

https://xtls-v4.hkg1.meaqua.org/en-us/rest/api/searchservice/supported-data-types

https://xtls-v4.hkg1.meaqua.org/en-us/azure/search/vector-search-how-to-create-index?tabs=push%2Cportal-check-index

https://xtls-v4.hkg1.meaqua.org/en-us/azure/search/search-how-to-define-index-projections?tabs=rest-create-index%2Crest-create-index-projection

https://xtls-v4.hkg1.meaqua.org/en-us/azure/search/vector-search-integrated-vectorization

Kindly let us know if the above helps or you need further assistance on this issue.

Please "upvote" if the information helped you. This will help us and others in the community as well.