# Fabric Microsoft Fabric is a modern data warehouse solution from Microsoft that uses OneLake storage for data persistence. Sling provides optimized bulk loading for Fabric using OneLake staging. ## Setup The following credentials keys are accepted: * `host` **(required)** -> The hostname of the Fabric warehouse (e.g., `xxx.datawarehouse.fabric.microsoft.com`) * `database` **(required)** -> The warehouse or database name * `user` (optional) -> The username to access the warehouse (not required when using Azure AD auth) * `password` (optional) -> The password to access the warehouse (not required when using Azure AD auth) * `port` (optional) -> The port of the instance. Default is `1433`. * `schema` (optional) -> The default schema to use * `fedauth` (optional) -> The Azure Active Directory authentication string. See [here](https://github.com/microsoft/go-mssqldb?tab=readme-ov-file#azure-active-directory-authentication) for more details. Accepted values: `ActiveDirectoryDefault`, `ActiveDirectoryIntegrated`, `ActiveDirectoryPassword`, `ActiveDirectoryInteractive`, `ActiveDirectoryMSI`, `ActiveDirectoryManagedIdentity`, `ActiveDirectoryApplication`, `ActiveDirectoryServicePrincipal`, `ActiveDirectoryServicePrincipalAccessToken`, `ActiveDirectoryDeviceCode`, `ActiveDirectoryAzCli`. * `client_id` (optional) -> Application ID of an Azure service principal. Can also set via env var `AZURE_CLIENT_ID` * `tenant_id` (optional) -> ID of the application's Microsoft Entra tenant. Can also set via env var `AZURE_TENANT_ID` * `client_secret` (optional) -> A client secret generated for the App Registration. Can also set via env var `AZURE_CLIENT_SECRET` * `client_certificate_path` (optional) -> Path to a PEM or PKCS12 certificate file including private key. Can also set via env var `AZURE_CLIENT_CERTIFICATE_PATH` * `client_certificate_password` (optional) -> Password protecting the certificate file (PFX/PKCS12 only). Can also set via env var `AZURE_CLIENT_CERTIFICATE_PASSWORD` * `ssh_tunnel` (optional) -> The URL of the SSH server you would like to use as a tunnel (example `ssh://user:password@db.host:22`) * `ssh_private_key` (optional) -> The private key to use to access a SSH server (raw string or path to file). * `ssh_passphrase` (optional) -> The passphrase to use to access a SSH server. ### ABFS / OneLake Configuration For **bulk import operations**, Fabric uses OneLake staging via Azure Blob File System (ABFS). The following properties are required for bulk operations: * `abfs_endpoint` **(required for bulk)** -> The OneLake endpoint (e.g., `onelake.dfs.fabric.microsoft.com`) or Azure Data Lake Storage Gen2 endpoint (e.g., `myaccount.dfs.core.windows.net`) * `abfs_filesystem` **(required for bulk)** -> The workspace UUID identifier for staging files (OneLake) or container name (ADLS Gen2) * `abfs_parent` **(required for bulk)** -> The lakehouse UUID with the folder path for staging files. This is typically a UUID with the `/Files` path, e.g.: `a6682eca-b677-40d0-85a0-71665example/Files` * `format` (optional) -> File format for staging. Accepts `parquet` (default) or `csv` * `copy_into_endpoint` (optional) -> Override the endpoint used in the COPY INTO SQL command. Useful when your staging endpoint differs from what Fabric Warehouse expects for reading. See [Troubleshooting](#copy-into-endpoint-issues) for details. #### How to obtain ABFS values Create a Lakehouse, and then get the URL for the `Files` folder.
Click Properties under the `Files` menu
Copy the URL
You will get a URL such as: `https://onelake.dfs.fabric.microsoft.com/8e5f41f1-2677-4e95-78d8-7cd1eexample/a6682eca-b677-40d0-85a0-71665example/Files` The `abfs_filesystem` would be `8e5f41f1-2677-4e95-78d8-7cd1eexample`. The `abfs_parent` would be `a6682eca-b677-40d0-85a0-71665example/Files`. **Authentication for ABFS** (choose one): * `account_key` -> Storage account key for OneLake access * `sas_svc_url` -> Shared Access Signature URL for OneLake * `client_id`, `tenant_id`, `client_secret` -> Azure service principal with client secret * `client_id`, `tenant_id`, `client_certificate_path` -> Azure service principal with certificate {% hint style="info" %} If ABFS properties are not provided, Sling will fall back to standard INSERT statements, which are much slower for large datasets. For optimal performance with bulk data loading, configure OneLake staging. {% endhint %} ### Azure Environment Variables When using service principal authentication (for both the SQL connection via `fedauth` and for ABFS/OneLake staging), Sling maps connection properties to the standard Azure SDK environment variables at runtime. This is how the Azure `DefaultAzureCredential` chain picks up your credentials. | Connection Property | Azure Env Var | Purpose | | ----------------------------- | ----------------------------------- | ---------------------------------------------------- | | `client_id` | `AZURE_CLIENT_ID` | Application (client) ID of the App Registration | | `tenant_id` | `AZURE_TENANT_ID` | Microsoft Entra tenant (directory) ID | | `client_secret` | `AZURE_CLIENT_SECRET` | Client secret for the App Registration | | `client_certificate_path` | `AZURE_CLIENT_CERTIFICATE_PATH` | Path to PEM or PFX certificate file with private key | | `client_certificate_password` | `AZURE_CLIENT_CERTIFICATE_PASSWORD` | Password for PFX certificate file (optional) | You can either: 1. Set these as connection properties in `env.yaml` (recommended) — Sling maps them to `AZURE_*` env vars automatically 2. Set the `AZURE_*` env vars directly in your shell environment — they will be picked up by `ActiveDirectoryDefault` {% hint style="warning" %} When using `fedauth: ActiveDirectoryDefault`, the Azure SDK tries credentials in order: environment variables, workload identity, managed identity, Azure CLI, Azure Developer CLI. Make sure the appropriate credentials are available for your environment. {% endhint %} ### Additional Parameters Sling uses the `go-mssqldb` library and thus will accept any parameters listed [here](https://github.com/denisenkom/go-mssqldb#connection-parameters-and-dsn). Some parameters that may be of interest: * `encrypt` -> `strict`, `disable`, `false` or `true` - whether data between client and server is encrypted. For Fabric, typically set to `true`. * `log` -> logging level (accepts `1`, `2`, `4`, `8`, `16`, `32`). * `trusted_connection` -> `true` or `false` - whether to connect with a trusted connection using integrated security * `trust_server_certificate` -> `true` or `false` - whether the server certificate is checked. For Fabric, typically set to `false`. * `certificate` -> The file that contains the public key certificate of the CA that signed the server certificate. * `hostname_in_certificate` -> Specifies the Common Name (CN) in the server certificate. Default value is the server host. * `server_spn` -> The kerberos SPN (Service Principal Name) for the server. Default is MSSQLSvc/host:port. * `driver` -> A way to override the sql driver to connect with. Default is `sqlserver`. Other option is `azuresql`. ### Kerberos Parameters * `authenticator` - set this to `krb5` to enable kerberos authentication. If this is not present, the default provider would be `ntlm` for unix and `winsspi` for windows. * `krb5_config_file` (optional) - path to kerberos configuration file. Defaults to `/etc/krb5.conf`. Can also be set using `KRB5_CONFIG` environment variable. * `krb5_realm` (required with keytab and raw credentials) - Domain name for kerberos authentication. Omit this parameter if the realm is part of the user name like `username@REALM`. * `krb5_keytab_file` - path to Keytab file. Can also be set using environment variable `KRB5_KTNAME`. If no parameter or environment variable is set, the `DefaultClientKeytabName` value from the krb5 config file is used. * `krb5_cred_cache_file` - path to Credential cache. Can also be set using environment variable `KRB5CCNAME`. * `krb5_dns_lookup_kdc` - Optional parameter in all contexts. Set to lookup KDCs in DNS. Boolean. Default is true. * `krb5_udp_preference_limit` - Optional parameter in all contexts. 1 means to always use tcp. MIT krb5 has a default value of 1465, and it prevents user setting more than 32700. Integer. Default is 1. Sling supports authentication via 3 methods. See [here](https://github.com/microsoft/go-mssqldb?tab=readme-ov-file#kerberos-parameters) for more details. * Keytabs - Specify the username, keytab file, the krb5.conf file, and realm. ``` authenticator: krb5 krb5_realm: domain.com krb5_config_file: /etc/krb5.conf krb5_keytab_file: ~/user.keytab ``` * Credential Cache - Specify the krb5.conf file path and credential cache file path. ``` authenticator=krb5 krb5_config_file=/etc/krb5.conf krb5_cred_cache_file=~/user_cached_creds ``` * Raw credentials - Specify krb5.conf, Username, Password and Realm. ``` authenticator=krb5 krb5_realm=company.com krb5_config_file=/etc/krb5.conf ``` *** ## Connection Examples ### Using `sling conns` Here are examples of setting a connection named `FABRIC`. We must provide the `type=fabric` property: {% code overflow="wrap" %} ```bash # Basic connection with Azure AD authentication $ sling conns set FABRIC type=fabric host=myworkspace-uuid.datawarehouse.fabric.microsoft.com database=mywarehouse fedauth=ActiveDirectoryAzCli encrypt=true trust_server_certificate=false # With OneLake bulk staging $ sling conns set FABRIC type=fabric host=myworkspace-uuid.datawarehouse.fabric.microsoft.com database=mywarehouse fedauth=ActiveDirectoryAzCli encrypt=true trust_server_certificate=false abfs_endpoint=onelake.dfs.fabric.microsoft.com abfs_filesystem=filesystem-uuid abfs_parent=lakehouse-uuid/Files # Or use url $ sling conns set FABRIC url="fabric://myworkspace-uuid.datawarehouse.fabric.microsoft.com?database=mywarehouse&encrypt=true&trust_server_certificate=false&fedauth=ActiveDirectoryAzCli" ``` {% endcode %} ### Environment Variable See [here](https://docs.slingdata.io/sling-cli/environment#dot-env-file-.env.sling) to learn more about the `.env.sling` file. {% code overflow="wrap" %} ```bash # Basic connection export FABRIC='fabric://myworkspace-uuid.datawarehouse.fabric.microsoft.com?database=mywarehouse&encrypt=true&trust_server_certificate=false&fedauth=ActiveDirectoryAzCli' # With structured format export FABRIC='{ type: fabric, host: "myworkspace-uuid.datawarehouse.fabric.microsoft.com", database: "mywarehouse", trust_server_certificate: false, encrypt: true, fedauth: ActiveDirectoryAzCli }' # With OneLake bulk staging export FABRIC='{ type: fabric, host: "myworkspace-uuid.datawarehouse.fabric.microsoft.com", database: "mywarehouse", trust_server_certificate: false, encrypt: true, fedauth: ActiveDirectoryAzCli, account_key: "your_account_key_here", abfs_endpoint: "onelake.dfs.fabric.microsoft.com", abfs_filesystem: "filesystem-uuid", abfs_parent: "lakehouse-uuid/Files" }' ``` {% endcode %} ### Sling Env File YAML See [here](https://docs.slingdata.io/sling-cli/environment#sling-env-file-env.yaml) to learn more about the sling `env.yaml` file. ```yaml connections: FABRIC: type: fabric host: myworkspace-uuid.datawarehouse.fabric.microsoft.com database: mywarehouse encrypt: true trust_server_certificate: false fedauth: ActiveDirectoryAzCli # after running `az login` using Azure CLI Tool FABRIC_WITH_BULK_STAGING: type: fabric host: myworkspace-uuid.datawarehouse.fabric.microsoft.com database: mywarehouse encrypt: true trust_server_certificate: false fedauth: ActiveDirectoryAzCli # OneLake staging for bulk operations abfs_endpoint: onelake.dfs.fabric.microsoft.com abfs_filesystem: filesystem-uuid abfs_parent: lakehouse-uuid/Files format: parquet # Optional: parquet (default) or csv FABRIC_WITH_SERVICE_PRINCIPAL: type: fabric host: myworkspace-uuid.datawarehouse.fabric.microsoft.com database: mywarehouse encrypt: true trust_server_certificate: false fedauth: ActiveDirectoryServicePrincipal # Service principal credentials client_id: your-client-id tenant_id: your-tenant-id client_secret: your-client-secret # OneLake staging abfs_endpoint: onelake.dfs.fabric.microsoft.com abfs_filesystem: filesystem-uuid abfs_parent: lakehouse-uuid/Files FABRIC_WITH_CERTIFICATE: type: fabric host: myworkspace-uuid.datawarehouse.fabric.microsoft.com database: mywarehouse encrypt: true trust_server_certificate: false fedauth: ActiveDirectoryServicePrincipal # Service principal with certificate client_id: your-client-id tenant_id: your-tenant-id client_certificate_path: /path/to/cert.pem # OneLake staging abfs_endpoint: onelake.dfs.fabric.microsoft.com abfs_filesystem: filesystem-uuid abfs_parent: lakehouse-uuid/Files FABRIC_WITH_ACCOUNT_KEY: type: fabric host: myworkspace-uuid.datawarehouse.fabric.microsoft.com database: mywarehouse encrypt: true trust_server_certificate: false fedauth: ActiveDirectoryDefault account_key: your_account_key_here abfs_endpoint: onelake.dfs.fabric.microsoft.com abfs_filesystem: filesystem-uuid abfs_parent: lakehouse-uuid/Files FABRIC_WITH_SAS: type: fabric host: myworkspace-uuid.datawarehouse.fabric.microsoft.com database: mywarehouse encrypt: true trust_server_certificate: false fedauth: ActiveDirectoryDefault sas_svc_url: "https://account.dfs.fabric.microsoft.com/?sv=2021-06-08&ss=..." # OneLake staging abfs_endpoint: onelake.dfs.fabric.microsoft.com abfs_filesystem: filesystem-uuid abfs_parent: lakehouse-uuid/Files # Using Azure Data Lake Storage Gen2 instead of OneLake FABRIC_WITH_ADLS_GEN2: type: fabric host: myworkspace-uuid.datawarehouse.fabric.microsoft.com database: mywarehouse encrypt: true trust_server_certificate: false fedauth: ActiveDirectoryServicePrincipal client_id: your-client-id tenant_id: your-tenant-id client_secret: your-client-secret # ADLS Gen2 staging (Sling auto-converts DFS to Blob for COPY INTO) abfs_endpoint: myaccount.dfs.core.windows.net abfs_filesystem: my-container abfs_parent: sling-staging FABRIC_KERBEROS: type: fabric host: myworkspace-uuid.datawarehouse.fabric.microsoft.com database: mywarehouse user: myuser password: mypassword authenticator: krb5 krb5_config_file: /etc/krb5.conf krb5_realm: domain.com krb5_keytab_file: ~/MyUserName.keytab ``` ## Bulk Import Operations Fabric supports high-performance bulk loading using OneLake staging. When properly configured with ABFS properties, Sling will: 1. **Stage data to OneLake** - Write data files (Parquet or CSV) to your specified OneLake location 2. **Execute COPY INTO** - Load data from OneLake into Fabric warehouse using the optimized COPY INTO command 3. **Clean up** - Automatically remove staging files after successful load ### Performance Tips * **Use Parquet format** - Parquet provides better compression and faster loads (default) * **Configure OneLake staging** - Essential for large datasets (100K+ rows) * **Use Azure AD authentication** - Recommended for production environments * **Set appropriate file chunk sizes** - Use `file_max_rows` property to control staging file size (default: 500,000 rows) ### Example Replication with Bulk Loading ```yaml source: POSTGRES target: FABRIC defaults: mode: full-refresh streams: public.large_table: object: dbo.large_table ``` ## Important Notes * **Connection Type**: Use `type: fabric` (not `sqlserver`) for Fabric-specific optimizations * **BCP Not Supported**: Unlike SQL Server, Fabric does not use the BCP utility for bulk loading * **OneLake Staging**: Required for optimal bulk load performance * **Azure AD Recommended**: Most Fabric deployments use Azure AD authentication * **Endpoint Pattern**: Fabric warehouses use `.datawarehouse.fabric.microsoft.com` endpoints ## Troubleshooting ### Slow Bulk Loads If bulk loads are slow, ensure ABFS properties are configured: ```yaml abfs_endpoint: onelake.dfs.fabric.microsoft.com abfs_filesystem: filesystem-uuid ``` Without these, Sling falls back to row-by-row inserts. ### Authentication Issues For Azure AD authentication, ensure you've run: ```bash az login ``` Or provide service principal credentials if using `ActiveDirectoryServicePrincipal`. ### Connection Errors Verify your connection with: ```bash sling conns test FABRIC --debug ``` ### COPY INTO Endpoint Issues When using Azure Data Lake Storage Gen2 (ADLS Gen2) as staging instead of OneLake, you may encounter this error: ``` mssql: Content of directory on path 'https://xxx.dfs.core.windows.net/...' cannot be listed. ``` This occurs because: 1. **File uploads** via ABFS require the `.dfs.core.windows.net` endpoint 2. **COPY INTO command** often works better with the `.blob.core.windows.net` endpoint (especially with Service Principal authentication) **Manual Override**: If you need a different endpoint for COPY INTO, use the `copy_into_endpoint` property: ```yaml connections: FABRIC: type: fabric host: myworkspace-uuid.datawarehouse.fabric.microsoft.com database: mywarehouse fedauth: ActiveDirectoryServicePrincipal client_id: your-client-id tenant_id: your-tenant-id client_secret: your-client-secret # ADLS Gen2 staging abfs_endpoint: myaccount.dfs.core.windows.net abfs_filesystem: my-container abfs_parent: staging/folder # Override endpoint for COPY INTO command (optional) copy_into_endpoint: myaccount.blob.core.windows.net ``` If you are facing issues connecting, please reach out to us at , on [discord](https://discord.gg/q5xtaSNDvp) or open a Github Issue [here](https://github.com/slingdata-io/sling-cli/issues). --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.slingdata.io/connections/database-connections/fabric.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.