Do Azure groups require direct access in Fabric's Lakehouse or Warehouse to access data from a semantic model via a Power BI report?

I have an issue with the Direct Lake semantic model that connects to Fabric's tables from either the Data Warehouse or Lakehouse environments.

Specifically, users in a specific Azure security group (Security Type: Cloud) are experiencing a "Missing References" error when loading data.

Currently, this group has been granted Read and Read ALL permissions to the underlying data, which allows them to access the Power BI report hosted in a Pro license workspace. However, this setup permits these users to access all tables within the Fabric's Data Warehouse and Lakehouse.

The primary concern here is how to enforce restrictions on access to the underlying data. It is essential to ensure that these users do not have access to the Fabric's Data Warehouse, Lakehouse, or onelake environments.